No commit activity in last 3 years
No release in over 3 years
bulk import file generator as well as nested document from MySQL for elasticsearch bulk api
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Development

~> 1.6
>= 0

Runtime

 Project Readme

elasticsearch_mysql_importer Build Status

It is importing from mysql table with SQL to elasticsearch. Not only that, it could generating nested documents.

Install

Add this line to your application's Gemfile:

gem 'elasticsearch_mysql_importer'

And then execute:

$ bundle

Or install it yourself as:

$ gem install elasticsearch_mysql_importer

Usage

# Clone repository
$ git clone https://github.com/y-ken/elasticsearch_mysql_importer.git
$ cd elasticsearch_mysql_importer
$ bundle install --path vendor/bundle

# Setup mysql connection and query
$ vim example.rb

# Execute script, then it outputs result into ./requests.json
$ bundle exec ruby example/example.rb 

# Index document for elasticsearch if you didn't call 'write_elasticsearch' in example.rb
$ curl -s -XPOST localhost:9200/_bulk --data-binary @example/requests.json

TODO

Pull requests are very welcome!!

  • support thread
  • support CLI command

Contributing

  1. Fork it ( https://github.com/y-ken/elasticsearch_mysql_importer/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Copyright

Copyright © 2014- Kentaro Yoshida (@yoshi_ken)

License

MIT License