LSST's Git LFS S3

A Git LFS server that stores your large Git files on S3.

It works by generating a presigned URL that the Git LFS client can use to upload directly to S3. It also provides download URLs that allow Git clients to download directly from S3. No data is proxied through the Git LFS server.

Git LFS S3 is available on RubyGems.

gem install lsst-git-lfs-s3

Or add it to your Gemfile if you wish to bundle it as a part of another application.

gem 'git-lfs-s3'

All configuration is done via environment variables. All of these configuration variables must be set.

• AWS_REGION - the region where your S3 bucket is.
• AWS_ACCESS_KEY_ID - your AWS access key.
• AWS_SECRET_ACCESS_KEY - your AWS secret key.
• S3_BUCKET - the bucket you wish to use for LFS storage. While not required, I recommend using a dedicated bucket for this.
• LFS_SERVER_URL - the URL where this server can be reached; needed to fetch download URLs.

You can (and should) also set authentication information. When you push for the first time from git, you will be prompted to enter a username and password when authentication is enabled. You can configure these with environment variables as well.

• USERNAME - the login username.
• PASSWORD - the login password.

If you are bundling the service inside of another application, such as Rails, or a different server of your choosing, you can set the configuration directly on GitLfsS3::Application. See bin/git-lfs-s3 for an example.

You can also hook the authentication into your own service this way. For example:

GitLfsS3::Application.on_authenticate do |username, password, is_safe|
  if is_safe
    true
  else
    user = User.find(username: username)
    user.verify_password(password)
  end
end

The logger can be configured as well. This is especially handy if you want to hook it into your Rails logger:

GitLfsS3::Application.set :logger, Rails.logger

If you are new to Git LFS, make sure you read the Getting Started guide first. Once that's done, you can configure your Git client to use this server by creating a .gitconfig file in the root of your repository and adding this config, but with your server address:

[lfs]
    url = "http://yourserver.com"

Once that is done, you can tell Git LFS to track files with git lfs track "*.psd", for example.

This repository contains an executable that will run a basic WEBrick server. Since this service is so lightweight, that's likely all you'll need. The port will default to 8080, but can be configured with an environment variable.

PORT=4000 git-lfs-s3

However, because this is a Sinatra application, it can also be mounted within other Rack based projects or other Rack-based servers such as Unicorn or Puma. For example, if you wanted to add this to your Rails project, configure GitLfsS3 in an initializer, and add this your routes:

mount GitLfsS3::Application => '/lfs'

• Cloudfront support