Project

glove

0.01
No commit activity in last 3 years
No release in over 3 years
GloVe is an unsupervised learning algorithm for obtaining vector representations for words. Training is performed on aggregated global word-word co-occurrence statistics from a corpus, and the resulting representations showcase interesting linear substructures of the word vector space. This is a pure Ruby implementation of GloVe utilizing GSL.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Development

~> 1.7
~> 10.0
~> 3.1

Runtime

~> 1.3
~> 1.16
 Project Readme

Ruby GloVe

Build Status Code Climate Test Coverage Inline docs

Ruby implementation of Global Vectors for Word Representations.

Overview

GloVe is an unsupervised learning algorithm for obtaining vector representations for words. Training is performed on aggregated global word-word co-occurrence statistics from a corpus, and the resulting representations showcase interesting linear substructures of the word vector space.

NOTE This is an early prototype.

Resources

Dependencies

This library relies on the rb-gsl gem for Matrix and Vector operations, therefore you need GNU Scientific Library installed.

Linux:

$ sudo apt-get install libgsl0-dev

OS X:

$ brew install gsl

Only compatible with MRI: tested in versions 2.0.x and 2.1.x

Installation

$ gem install glove

or add to your Gemfile

gem 'glove'

Usage

require 'glove'

# See documentation for all available options
model = Glove::Model.new

# Next feed it some text.
text = File.read('quantum-physics.txt')
model.fit(text)

# Or you can pass it a Glove::Corpus object as the text argument instead
corpus = Glove::Corpus.build(text)
model.fit(corpus)

# Finally, to query the model, we need to train it
model.train

# So far, word similarity and analogy task methods have been included:
# Most similar words to quantum
model.most_similar('quantum')
# => [["physic", 0.9974459436353388], ["mechan", 0.9971606266531394], ["theori", 0.9965966776283189]]

# What words relate to atom like quantum relates to physics?
model.analogy_words('quantum', 'physics', 'atom')
# => [["electron", 0.9858380292886947], ["energi", 0.9815122410243475], ["photon", 0.9665073849076669]]

# Save the trained matrices and vectors for later usage in binary formats
model.save('corpus.bin', 'cooc-matrix.bin', 'word-vec.bin', 'word-biases.bin')

# Later on create a new instance and call #load
model = Glove::Model.new
model.load('corpus.bin', 'cooc-matrix.bin', 'word-vec.bin', 'word-biases.bin')
# Now you can query the model again and get the same results as above

Performance

Thanks to the rb-gsl bindings for GSL, matrix/vector operations are fast. The glove algorythm itself, however, requires quite a bit of computational power, even the original C library. If you need speed, use smaller texts with vocabulaty size no more than 100K words. Processing text with 160K words (compilation of several books on quantum mechanics) on a late 2012 MBP (8GB RAM) with ruby-2.1.5 takes about 7 minutes:

 $ ruby -Ilib benchmark/benchmark.rb
                 user     system      total        real
Fit Text    11.320000   0.070000  11.390000 ( 11.387612)
Vocabulary size: 158323
Unique tokens: 2903
Co-occur     1.330000   0.250000 1107.720000 (300.738453)
Train      121.120000  12.960000  134.080000 (128.409034)
Similarity   0.010000   0.000000    0.010000 (  0.057423)
Give me the 3 most similar words to quantum
[["problem", 0.9977609386134489], ["mechan", 0.9977529272587808], ["classic", 0.9974759411408415]]
Analogy      0.010000   0.000000   0.010000 (  0.010674)
What 3 words relate to atom like quantum relates to mechanics?
[["particl", 0.9982711579369483], ["find", 0.9982303885530384], ["expect", 0.9982017117355527]]

TODO

  • Word Vector graphs

Contributing

  1. Fork it ( https://github.com/vesselinv/glove/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request