Botch
Botch is a simple DSL for quickly creating web crawlers.
Inspired by Sinatra.
Installation
add this line to your Gemfile.
gem 'botch'
or
$ gem install botch
Usage
require 'lib/botch'
require 'kconv'
require 'rack'
class SampleBotch < Botch::Base
set :user_agent, "SampleBotch"
helpers do
def h(str)
Rack::Utils.escape_html(str)
end
end
filter :example, :map => "example.com" do
status == 200
end
rule :example, :map => /example\.com/ do
h(body.toutf8)
end
end
if $0 == __FILE__
SampleBotch.run("http://example.com/") do |response|
puts response
end
endTODO
- RSpec
- Documentation
Contributing to Botch
- fork the project.
- create your feature branch. (
git checkout -b my-feature) - commit your changes. (
git commit -am 'commit message.') - push to the branch. (
git push origin my-feature) - send pull request.
License
MIT