Project

saxony

0.0
No commit activity in last 3 years
No release in over 3 years
Parse gigantic XML files with pleasure and a without running out of memory.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Runtime

 Project Readme

Saxony - 0.3

Parse gigantic XML files with pleasure and without running out of memory.

Example

sax = Saxony.new :SomeObject, 1000
sax.parse 'path/2/huge.xml' do
  xml             # => The XML containing 1000 SomeObjects
  doc             # => Nokogiri object for 1000 SomeObjects
  total_count     # => Total number of SomeObjects processed
  elapsed_time    # => time processing current batch
  path            # => Current file being processed
  fh              # => Output file handle
end

# Process multiple files in parallel using Kernel.proc.
# By default
Saxony.fork ['path/2/huge.xml', 'path/2/huger.xml'] do
  # Inside the block, everything is the  
  # same as calling sax.parse above. 
  doc.xpath('//Listing').each do |l
    type = listing.xpath("Type").first.text
    fh.puts listing if type == 'some_criteria'
  end
end

Credits

Thanks

License

See LICENSE.txt