0.01
No commit activity in last 3 years
No release in over 3 years
A DSL to write web spider. Depend on capybara and capybara-webkit.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Development

>= 4.7.5, ~> 4.7.5
>= 0.10.3, ~> 0.10.3
>= 11.2.0, ~> 11.2.2
>= 1.4.7, ~> 1.4.7
>= 0.9.7, ~> 0.9.7
>= 0.8.7, ~> 0.8.7.6

Runtime

>= 2.7, ~> 2.7
>= 1.5.0, ~> 1.5.0
>= 3.0.0, ~> 3.0.0
>= 3.4.0, ~> 3.4.4
 Project Readme

micro-spider

A DSL to write web spider. Depend on capybara and capybara-webkit.

Example

require 'micro_spider'
spider = MicroSpider.new

spider.learn do
  site 'http://www.bbc.com'
  entrance '/news'
  fields :top_stories, 'a.title-link'
end

spider.crawl

spider.get('top_stories')
# or
spider.excretion['/news']['top_stories']