Amazon S3 Ruby library that leverages EventMachine and em-http-request.
By using EventMachine Happening does not block on S3 downloads/uploads thus allowing for a higher concurrency.
Happening was developed by Peritor for usage inside Nanite/EventMachine. Alternatives like RightAws block during the HTTP calls thus blocking the Nanite-Agent.
For now it only supports GET, PUT and DELETE operations on S3 items. The PUT operations support S3 ACLs/permissions. Happening will handle redirects and retries on errors by default.
Installation
gem install happening
Usage
require 'happening'
EM.run do
item = Happening::S3::Item.new('bucket', 'item_id')
item.get # non-authenticated download, works only for public-read content
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret')
item.get # authenticated download
item.put("The new content")
item.delete
end
The above examples are a bit useless, as you never get any content back. You need to specify a callback that interacts with the http response:
EM.run do
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret')
item.get do |response|
puts "the response content is: #{response.response}"
EM.stop
end
end
This will enqueue your download and run it in the EventMachine event loop.
You can also react to errors:
EM.run do
on_error = Proc.new {|response| puts "An error occured: #{response.response_header.status}"; EM.stop }
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret')
item.get(:on_error => on_error) do |response|
puts "the response content is: #{response.response}"
EM.stop
end
end
If you don't supply an error handler yourself, Happening will be default raise an Exception.
Downloading many files could look like this:
EM.run do
count = 100
on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop if count <= 0}
on_success = Proc.new {|http| puts "the response is: #{http.response}"; EM.stop if count <= 0}
count.times do |i|
item = Happening::S3::Item.new('bucket', "item_#{i}", :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret')
item.get(:on_success => on_success, :on_error => on_error)
end
end
Upload
Happening supports the simple S3 PUT upload:
EM.run do
on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop }
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret', :on_success => on_success, :on_error => on_error)
item.put( File.read('/etc/passwd'), :on_error => on_error ) do |response|
puts "Upload finished!"; EM.stop
end
end
Setting permissions looks like this:
EM.run do
on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop }
on_success = Proc.new {|http| puts "the response is: #{http.response}"; EM.stop }
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret', :permissions => 'public-write')
item.get(:on_success => on_success, :on_error => on_error)
end
Custom headers:
EM.run do
on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop }
on_success = Proc.new {|http| puts "the response is: #{http.response}"; EM.stop }
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret', :permissions => 'public-write')
item.put(:on_success => on_success,
:on_error => on_error,
:headers => {
'Cache-Control' => "max-age=252460800",
'Content-Type' => 'text/html',
'Expires' => 'Fri, 16 Nov 2018 22:09:29 GMT',
'x-amz-meta-abc' => 'ABC'
})
end
Deleting
Happening support the simple S3 PUT upload:
EM.run do
on_error = Proc.new {|response| puts "An error occured: #{response.response_header.status}"; EM.stop }
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret')
item.delete(:on_error => on_error) do |response|
puts "Deleted!"
EM.stop
end
end
Amazon returns no content on delete, so having a success handler is usually not needed for delete operations.
Head
You can also just load the headers of an S3 item:
EM.run do
on_error = Proc.new {|response| puts "An error occured: #{response.response_header.status}"; EM.stop }
item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret')
item.head(:on_error => on_error) do |response|
puts "Headers: #{response.inspect}"
EM.stop
end
end
Streaming
The response data can also be streamed:
EM.run do
item = Happening::S3::Item.new( bucket...
item.get(:on_error => on_error, :on_success => on_success ).stream do |chunk|
# .. handle the individual chunk
end
end
SSL Support
Happening will use SSL/HTTPS by default. What it cannot do by default is verify the SSL certificate. This means that traffic is encrypted but nobody can say if the SSL-endpoint is the one you except. In order to verify the SSL certificate you need to provide Happening with the path to a certificate CA collection in PEM format:
Happening::S3.ssl_options[:cert_chain_file] = '/etc/ca-bundle.crt'
You can also set this option on each item:
Happening::S3::Item.new('bucket', 'item_id',
:aws_access_key_id => 'A',
:aws_secret_access_key => 'B',
:ssl => {
:cert_chain_file => '/etc/ca-bundle.crt'
}
Or even on the request:
item.get(:ssl => {:cert_chain_file => '/etc/ca-bundle.crt'})
The SSL options are directly passed to EventMachine, see the EventMachine documentation for more information on the SSL support.
Credits
The AWS signing and canonical request description is based on RightAws.
License
Happening is licensed under the Apache 2.0 license. See LICENSE.txt
About
Happening was written by Jonathan Weiss for Peritor.