Masque
Masque is a browser emulated crawler builder wrapping capybara-webkit and poltergeist.
Installation
You need Xvfb and QT libraries before installing Masque gem.
- https://github.com/leonid-shevtsov/headless#installation
- https://github.com/thoughtbot/capybara-webkit/wiki/Installing-Qt-and-compiling-capybara-webkit
On Debian/Ubuntu:
$ sudo apt-get install libqt4-dev xvfb libicu48 ttf-ubuntu-font-family
$ gem install masque
On Mac OS X (assumes you have Homebrew installed):
$ brew install qt
$ gem install masque
Usage
Easy crawling websites they required JavaScript.
require "masque"
m = Masque.new(:display => 99, :driver => :webkit) # or :driver => :poltergeist
m.run do
# Capybara::DSL syntax
# https://github.com/jnicklas/capybara#the-dsl
visit "http://www.google.com/"
fill_in("q", :with => "capybara")
find('*[name="btnG"]').click
titles = evaluate_script <<-JS
(function(){
var titles = Array.prototype.map.call(
document.querySelectorAll('h3 a'),
function(a) {
return a.innerText;
}
);
return titles;
})();
JS
puts titles.join("\n")
end
more examples are available in examples/ directory.
TODO
- Impl useful DSL
- Respect driver specific features/options
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request