Edamame combines the Beanstalk priority queue with a Tokyo Tyrant database and God monitoring to produce a persistent distributed priority job queue system. Like beanstalk, it is fast, lightweight, distributed, priority queuing, reliable scheduling; it adds persistence, named jobs and job querying/enumeration.

A simple scraper for directed scrapes of APIs, feed or structured HTML. Plays nicely with wuclan and wukong.

The Infochimps Platform is an end-to-end, managed solution for building Big Data applications. It integrates best-of-breed technologies like Hadoop, Storm, Kafka, MongoDB, ElasticSearch, HBase, &c. and provides simple interfaces for accessing these powerful tools. Computation, analytics, scripting, &c. are all handled by Wukong within the platform. Wukong is an abstract framework for defining computations on data. Wukong processors and flows can run in many different execution contexts including: * locally on the command-line for testing or development purposes * as a Hadoop mapper or reducer for batch analytics or ETL * within Storm as part of a real-time data flow The Infochimps Platform uses the concept of a deploy pack for developers to develop all their processors, flows, and jobs within.

Treat your dataset like a: * stream of lines when it's efficient to process by lines * stream of field arrays when it's efficient to deal directly with fields * stream of lightweight objects when it's efficient to deal with objects Wukong is friends with Hadoop the elephant, Pig the query language, and the cat on your command line.

Lets you load data from the command-line into data stores like * Elasticsearch * MongoDB * HBase * MySQL * Kafka and others.

Storm processing for Ruby

Clusta is a Ruby library that implements network algorithms using Wukong. This means you can use and extend these algorithms on your laptop and seamlessly lift them into a Hadoop cluster when you're ready.

Facade adapting apis to activity streams spec

Massive-scale social network analysis. Nothing to f with.