Project

clandestined

0.01
No commit activity in last 3 years
No release in over 3 years
clandestinedewdurbin/clandestined-rubyHomepageDocumentationSource CodeBug TrackerWiki
rendezvous hashing implementation based on murmur3 hash
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Popularity
Downloads
37,432
Stars
8
Forks
4
Watchers
3
 Releases
Current version
1.0.0
Total releases
4
First release
2014-07-06
Latest release
2014-12-18
 Issues
Open Issues
1
Closed Issues
0
Total Issues
1
Issue Closure Rate
0%
 Pull Requests
Open Pull Requests
1
Closed Pull Requests
1
Merged Pull Requests
7
Pull Request Acceptance Rate
77%
 Development
Primary Language
Ruby
Licenses
MIT
Average date of last 50 commits
2014-08-26
Reverse Dependencies
0

 Dependencies

Development

rake
>= 0
rake-compiler
>= 0
rubydoctest
>= 0
 Project Readme

clandestined

rendezvous hashing implementation based on murmur3 hash

motivation

in distributed systems, the need often arises to locate objects amongst a cluster of machines. consistent hashing and rendezvous hashing are methods of performing this task, while minimizing data movement on cluster topology changes.

clandestined is a library for rendezvous hashing which has the goal of simple clients and ease of use.

Currently supported Rubies:

  • Ruby 1.8.6 through Ruby 2.2
  • JRuby 1.8 and 1.9

Build Status

example usage

>> require 'clandestined/cluster'
>>
>> nodes = Hash[
     '1' => Hash['name' => 'node1.example.com', 'zone' => 'us-east-1a'],
     '2' => Hash['name' => 'node2.example.com', 'zone' => 'us-east-1a'],
     '3' => Hash['name' => 'node3.example.com', 'zone' => 'us-east-1a'],
     '4' => Hash['name' => 'node4.example.com', 'zone' => 'us-east-1b'],
     '5' => Hash['name' => 'node5.example.com', 'zone' => 'us-east-1b'],
     '6' => Hash['name' => 'node6.example.com', 'zone' => 'us-east-1b'],
     '7' => Hash['name' => 'node7.example.com', 'zone' => 'us-east-1c'],
     '8' => Hash['name' => 'node8.example.com', 'zone' => 'us-east-1c'],
     '9' => Hash['name' => 'node9.example.com', 'zone' => 'us-east-1c'],
     ]
>>
>> cluster = Clandestined::Cluster.new(nodes)
>> cluster.find_nodes('mykey')
=> ["4", "8"]

by default, Cluster will place 2 replicas around the cluster taking care to place the second replica in a separate zone from the first.

in the event that your cluster doesn't need zone awareness, you can either invoke the RendezvousHash class directly, or use a Cluster with replicas set to 1

>> require 'clandestined/cluster'
>> require 'clandestined/rendezvous_hash'
>>
>> nodes = Hash[
     '1' => Hash['name' => 'node1.example.com'],
     '2' => Hash['name' => 'node2.example.com'],
     '3' => Hash['name' => 'node3.example.com'],
     '4' => Hash['name' => 'node4.example.com'],
     '5' => Hash['name' => 'node5.example.com'],
     '6' => Hash['name' => 'node6.example.com'],
     '7' => Hash['name' => 'node7.example.com'],
     '8' => Hash['name' => 'node8.example.com'],
     '9' => Hash['name' => 'node9.example.com'],
     ]
>>
>> cluster = Clandestined::Cluster.new(nodes, 1)
>> rendezvous = Clandestined::RendezvousHash.new(nodes.keys)
>>
>> cluster.find_nodes('mykey')
=> ["4"]
>> rendezvous.find_node('mykey')
=> "4"

advanced usage

murmur3 seeding

if you plan to use keys based on untrusted input (supported, but go ahead), it would be best to use a custom seed for hashing. although this technique is by no means a way to fully mitigate a DoS attack using crafted keys, it may help you sleep better at night.

DISCLAIMER

clandestined was not designed with consideration for untrusted input, please see LICENSE.

END DISCLAIMER

>> require 'clandestined/cluster'
>> require 'clandestined/rendezvous_hash'
>>
>> nodes = Hash[
     '1' => Hash['name' => 'node1.example.com'],
     '2' => Hash['name' => 'node2.example.com'],
     '3' => Hash['name' => 'node3.example.com'],
     '4' => Hash['name' => 'node4.example.com'],
     '5' => Hash['name' => 'node5.example.com'],
     '6' => Hash['name' => 'node6.example.com'],
     '7' => Hash['name' => 'node7.example.com'],
     '8' => Hash['name' => 'node8.example.com'],
     '9' => Hash['name' => 'node9.example.com'],
     ]
>>
>> cluster = Clandestined::Cluster.new(nodes, 1, 1337)
>> rendezvous = Clandestined::RendezvousHash.new(nodes.keys, 1337)
>>
>> cluster.find_nodes('mykey')
=> ["7"]
>> rendezvous.find_node('mykey')
=> "7"

contributors

Thanks to the following contributors for their aid in making this project great: