smoke
smoke is a Ruby based DSL that allows you to query web services such as YQL, RSS / Atom and JSON or XML in an elegant manner.
These "services" can then be re-represented, sorted and filtered. Data can be collected from multiple sources, sorted via a common property, chopped up and refried.
Then you can output as a plain ruby object or one of your other favourites (JSON, YAML, XML)
Examples of use
- The
examples
directory has something to get you running straight away - I powered my entire site using smoke, until further documentation exists, this is probably a good place to start.
- Read further details in the rdoc documentation or the wiki
Media
- Presentation from Webjam11 in Perth
- Presentation from Melbourne #roro
- Early screencast to get developer / peer feedback
Install
gem install smoke
The concept
The concept comes from using Yahoo Pipes to make web based mash ups: Get a list of tv shows for my torrent client, compile a recipe book or make tools to give me a list of albums that artists in my music library are about to be released.
Using with web services
Smoke has its own special rack based library to expose your sources web services. Checkout rack-smoke for more details
How or what to contribute
- Test everything you do
- Add a way to output (XML, anyone?)
- Examples of queries you'd like to be able to do (email / github message them to me)
API Examples
YQL
# This will use yahoo search to get an array of search results about Ruby
Smoke.yql(:ruby) do
select :all
from "search.web"
where :query, "ruby"
discard :title, /tuesday/i
end
Smoke.yql(:python) do
select :all
from "search.web"
where :query, "python"
end
Join sources and use them together
Smoke.join(:ruby, :python)
or even
Smoke.join(:python, :ruby) do
emit do
sort :title
rename :shit_name => :title
end
end
Define a source allowing for variables to be injected later
Source definition:
Smoke.feed :delicious do
prepare do
url "http://feeds.delicious.com/v2/rss/#{username}?count=15"
end
end
Execution:
Smoke.delicious.username("bschwarz").output
TODO (working on, just mental notes)
- Items returned from smoke to be Hashie "rich" objects
- Output as a web feed (atom, rss)
- How to push through values like "author"?
- Passing modified time headers through to the atom formatter
For wiki pages (docs, later)
- Document all sources with their irrespective differential methods
- How to use
path
- YQL Definitions
- Tranformations
- Insert
- Joining
- Variable injection
- Sort, Reverse
- Keep, Discard
- Truncate
- Manually setting the content type for a url
Copyright
Copyright (c) 2010 Ben Schwarz. See LICENSE for details.