smoke

smoke is a Ruby based DSL that allows you to query web services such as YQL, RSS / Atom and JSON or XML in an elegant manner.

These "services" can then be re-represented, sorted and filtered. Data can be collected from multiple sources, sorted via a common property, chopped up and refried.

Then you can output as a plain ruby object or one of your other favourites (JSON, YAML, XML)

• The examples directory has something to get you running straight away
• I powered my entire site using smoke, until further documentation exists, this is probably a good place to start.
• Read further details in the rdoc documentation or the wiki

• Presentation from Webjam11 in Perth
• Presentation from Melbourne #roro
• Early screencast to get developer / peer feedback

gem install smoke

The concept comes from using Yahoo Pipes to make web based mash ups: Get a list of tv shows for my torrent client, compile a recipe book or make tools to give me a list of albums that artists in my music library are about to be released.

Smoke has its own special rack based library to expose your sources web services. Checkout rack-smoke for more details

• Test everything you do
• Add a way to output (XML, anyone?)
• Examples of queries you'd like to be able to do (email / github message them to me)

# This will use yahoo search to get an array of search results about Ruby
Smoke.yql(:ruby) do
  select  :all
  from    "search.web"
  where   :query, "ruby"
  
  discard :title, /tuesday/i
end

Smoke.yql(:python) do
  select  :all
  from    "search.web"
  where   :query, "python"
end

Smoke.join(:ruby, :python)

or even

Smoke.join(:python, :ruby) do
  emit do
    sort :title
    rename :shit_name => :title
  end
end

Source definition:

Smoke.feed :delicious do
  prepare do
    url "http://feeds.delicious.com/v2/rss/#{username}?count=15"
  end
end

Execution:

Smoke.delicious.username("bschwarz").output

• Items returned from smoke to be Hashie "rich" objects
• Output as a web feed (atom, rss)
  • How to push through values like "author"?
  • Passing modified time headers through to the atom formatter

• Document all sources with their irrespective differential methods
• How to use path
• YQL Definitions
• Tranformations
• Insert
• Joining
• Variable injection
• Sort, Reverse
• Keep, Discard
• Truncate
• Manually setting the content type for a url

Copyright (c) 2010 Ben Schwarz. See LICENSE for details.