ActiveEncode

Code:

Docs:

Jump in:

What is ActiveEncode?

ActiveEncode serves as the basis for the interface between a Ruby (Rails) application and a provider of encoding services such as FFmpeg, Amazon Elastic Transcoder, AWS Elemental MediaConvert, and Zencoder.

Help

The Samvera community is here to help. Please see our support guide.

Installation

Add this line to your application's Gemfile:

gem 'active_encode'

And then execute:

$ bundle

Or install it yourself as:

$ gem install active_encode

Prerequisites

FFmpeg (tested with version 4+) and mediainfo (version 17.10+) need to be installed to use the FFmpeg engine adapter.

Usage

Set the engine adapter (default: test), configure it (if neccessary), then submit encoding jobs. The outputs option specifies the output(s) to create in an adapter-specific way, see individual adapter documentation.

ActiveEncode::Base.engine_adapter = :ffmpeg
file = "file://#{File.absolute_path "spec/fixtures/fireworks.mp4"}"
ActiveEncode::Base.create(file, { outputs: [{ label: "low", ffmpeg_opt: "-s 640x480", extension: "mp4"}, { label: "high", ffmpeg_opt: "-s 1280x720", extension: "mp4"}] })

Create returns an encoding job (which we sometimes call "an encode object") that has been submitted to the adapter for processing. At this point it will have an id, a state, the input url, and possibly additional adapter-specific metadata.

encode.id  # "1e4a907a-ccff-494f-ad70-b1c5072c2465"
encode.state  # :running
encode.input.url

At this point the encode is not complete. You can check on status by looking up the encode by id, or by calling #reload on an existing encode object to refresh it:

encode = ActiveEncode::Base.find("1e4a907a-ccff-494f-ad70-b1c5072c2465")
# or
encode.reload

encode.percent_complete
encode.status # running, cancelled, failed, completed
encode.errors # array of errors in case of status `failed`

Progress of a running encode is shown with current operations (multiple are possible when outputs are generated in parallel) and percent complete.

Technical metadata about the input file may be added by some adapters, and may be available before completion. This should include a mime type, checksum, duration, and basic technical details of the audio and video content of the file (codec, audio channels, bitrate, frame rate, and dimensions).

encode.input.url
encode.input.height
encode.input.width
encode.input.checksum
# etc

Outputs are added once they are created and should include the same technical metadata along with an id, label, and url.

output = encode.outputs.first
output.url
output.id
output.width

If you want to stop the encoding job call cancel:

encode.cancel!
encode.cancelled?  # true

An encode object is meant to be the record of the work of the encoding engine and not the current state of the outputs. Therefore moved or deleted outputs will not be reflected in the encode object.

AWS ElasticTranscoder

To use active_encode with the AWS ElasticTransoder, the following are required:

An S3 bucket to store master files
An S3 bucket to store derivatives (recommended to be separate)
An ElasticTranscoder pipeline
Some transcoding presets for the pipeline

Set the adapter:

ActiveEncode::Base.engine_adapter = :elastic_transcoder

Construct the options hash:

outputs = [{ key: "quality-low/hls/fireworks", preset_id: '1494429796844-aza6zh', segment_duration: '2' },
           { key: "quality-medium/hls/fireworks", preset_id: '1494429797061-kvg9ki', segment_duration: '2' },
           { key: "quality-high/hls/fireworks", preset_id: '1494429797265-9xi831', segment_duration: '2' }]
options = {pipeline_id: 'my-pipeline-id', masterfile_bucket: 'my-master-files', outputs: outputs}

Create the job:

file = 'file:///path/to/file/fireworks.mp4' # or 's3://my-bucket/fireworks.mp4'
encode = ActiveEncode::Base.create(file, options)

AWS Elemental MediaCovert

MediaConvert is a newer AWS service than Elastic Transcoder. The MediaConvert adapter works using output presets defined in the MediaConvert service for your account. Some additional dependencies will need to be added to your project, see Guide.

ActiveEncode::Base.engine_adapter = :media_convert
ActiveEncode::Base.engine_adapter.role = 'arn:aws:iam::111111111111:role/name-of-role'
ActiveEncode::Base.engine_adapter.output_bucket = 'name-of-bucket'

# will create CloudWatch/EventBridge resources necessary to capture outputs,
# only needs to be called once although is safe to call redundantly.
ActiveEncode::Base.engine_adapter.setup!

encode = ActiveEncode::Base.create(
  "file://path/to/file.mp4",
  {
    masterfile_bucket: "name-of-my-masterfile_bucket"
    output_prefix: "path/to/output/base_name_of_outputs",
    use_original_url: true,
    outputs: [
      { preset: "my-hls-preset-high", modifier: "_high" },
      { preset: "my-hls-preset-medium", modifier: "_medium" },
      { preset: "my-hls-preset-low", modifier: "_low" },
    ]
  }
)

See more details and guidance in our longer guide, or in comment docs in adapter class.

Custom jobs

Subclass ActiveEncode::Base to add custom callbacks or default options. Available callbacks are before, after, and around the create and cancel actions.

class CustomEncode < ActiveEncode::Base
  after_create do
    logger.info "Created encode with id #{self.reload.id}"
  end

  def self.default_options(input)
    {preset: 'avalon-skip-transcoding'}
  end
end

Engine Adapters

Engine adapters are shims between ActiveEncode and the back end encoding service. You can add an additional engine by creating an engine adapter class that implements :create, :find, and :cancel and passes the shared specs.

For example:

# In your application at:
# lib/active_encode/engine_adapters/my_custom_adapter.rb
module ActiveEncode
  module EngineAdapters
    class MyCustomAdapter
      def create(input_url, options = {})
        # Start a new encoding job. This may be an external service, or a
        # locally queued job.

        # Return an instance ActiveEncode::Base (or subclass) that represents
        # the encoding job that was just started.
      end

      def find(id, opts = {})
        # Find the encoding job for the given parameters.

        # Return an instance of ActiveEncode::Base (or subclass) that represents
        # the found encoding job.
      end

      def cancel(id)
        # Cancel the encoding job for the given id.

        # Return an instance of ActiveEncode::Base (or subclass) that represents
        # the canceled job.
      end
    end
  end
end

Then, use the shared specs...

# In your application at...
# spec/lib/active_encode/engine_adapters/my_custom_adapter_spec.rb
require 'spec_helper'
require 'active_encode/spec/shared_specs'
RSpec.describe MyCustomAdapter do
  let(:created_job) {
    # an instance of ActiveEncode::Base represented a newly created encode job
  }
  let(:running_job) {
    # an instance of ActiveEncode::Base represented a running encode job
  }
  let(:canceled_job) {
    # an instance of ActiveEncode::Base represented a canceled encode job
  }
  let(:completed_job) {
    # an instance of ActiveEncode::Base represented a completed encode job
  }
  let(:failed_job) {
    # an instance of ActiveEncode::Base represented a failed encode job
  }
  let(:completed_tech_metadata) {
    # a hash representing completed technical metadata
  }
  let(:completed_output) {
    # data representing completed output
  }
  let(:failed_tech_metadata) {
    # a hash representing failed technical metadata
  }

  # Run the shared specs.
  it_behaves_like 'an ActiveEncode::EngineAdapter'
end

Acknowledgments

This software has been developed by and is brought to you by the Samvera community. Learn more at the Samvera website.

active_encode

Development

Runtime