A client to handle robots.txt
Installation
Add this line to your application's Gemfile:
gem 'morito'
And then execute:
$ bundle
Or install it yourself as:
$ gem install morito
Usage
client = Morito::Client.new('some user agent')
client.allowed?('http://example.com/some/path') # => true / false
client.allowed?('http://example.com/some/path', cache: false) # => true / false without cache
Contributing
- Fork it ( https://github.com/negipo/morito/fork )
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request