TorchData Ruby
Composable data loading for Ruby
Installation
Add this line to your application’s Gemfile:
gem "torchdata"Getting Started
This library follows the Python API. Many methods and options are missing at the moment. PRs welcome!
folder = "path/to/csv/folder"
datapipe = TorchData::DataPipes::Iter::FileLister.new([folder]).filter { |filename| filename.end_with?(".csv") }
datapipe = TorchData::DataPipes::Iter::FileOpener.new(datapipe, mode: "rt")
datapipe = datapipe.parse_csv(delimiter: ",")
train, valid = datapipe.random_split(total_length: 10000, weights: {train: 0.5, valid: 0.5}, seed: 0)
train.each do |x|
# code
end
valid.each do |y|
# code
endHistory
View the changelog
Contributing
Everyone is encouraged to help improve this project. Here are a few ways you can help:
- Report bugs
- Fix bugs and submit pull requests
- Write, clarify, or fix documentation
- Suggest or add new features
To get started with development:
git clone https://github.com/ankane/torchdata-ruby.git
cd torchdata-ruby
bundle install
bundle exec rake test