Edamame combines the Beanstalk priority queue with a Tokyo Tyrant database and God monitoring to produce a persistent distributed priority job queue system. Like beanstalk, it is fast, lightweight, distributed, priority queuing, reliable scheduling; it adds persistence, named jobs and job querying/enumeration.

A simple scraper for directed scrapes of APIs, feed or structured HTML. Plays nicely with wuclan and wukong.

The Infochimps Platform is an end-to-end, managed solution for building Big Data applications. It integrates best-of-breed technologies like Hadoop, Storm, Kafka, MongoDB, ElasticSearch, HBase, &c. and provides simple interfaces for accessing these powerful tools. Computation, analytics, scripting, &c. are all handled by Wukong within the platform. Wukong is an abstract framework for defining computations on data. Wukong processors and flows can run in many different execution contexts including: * locally on the command-line for testing or development purposes * as a Hadoop mapper or reducer for batch analytics or ETL * within Storm as part of a real-time data flow The Infochimps Platform uses the concept of a deploy pack for developers to develop all their processors, flows, and jobs within.

Lets you load data from the command-line into data stores like * Elasticsearch * MongoDB * HBase * MySQL * Kafka and others.

Storm processing for Ruby

Clusta is a Ruby library that implements network algorithms using Wukong. This means you can use and extend these algorithms on your laptop and seamlessly lift them into a Hadoop cluster when you're ready.

Facade adapting apis to activity streams spec

Massive-scale social network analysis. Nothing to f with.