ENV variables contract

Keywords: #p20220707a #env #variable #contract #ruby #gem

Ruby approach in creating contracts (manifests) for ENV variables.

Contract is a list of all ENV variables your app reads along with their validity criteria. If any of the requirements is not met, a negative scenario will be performed. For example, your application won't be able to start and/or you'll be notified that some ENV variables contain an invalid value. (more on this: Why are contracts necessary?)

Highlights:

• This gem does not force you to change the way you work with your ENV vars. It does not coerse/change the values of ENV variables.
• Not opinionated, not Rails-specific, can be applied to any [non-]ruby project.
• Customizable and well-tested.

In case you're not sure EnvControl is what you need, consider alternative gems.

After installing this gem, create the Contract:

EnvControl.configuration.contract = {
  DB_PASSWORD: :string, # any non-empty string
  MY_UUID_VAR: :uuid,   # an UUID string
  ...
}

Then validate the ENV variables:

EnvControl.validate(ENV)

#validate method ensures that contract format is valid, and then validates ENV variables against the contract.

In case the contract is breached, on_validation_error handler will be called with violated parts of contract passed as details. You can customize this handler to suit your needs.

Note: run validate() only after ENV variables are all set (after dotenv➚ or Figaro➚)

Consider the following example:

EnvControl.configuration.contract = {
  ADMIN_EMAIL: :email,
  DEBUG: ["true", nil],
  LC_CTYPE: "UTF-8",
  MY_VAR1: :string,
  MY_VAR2: :bool,
  MYSQL_DEBUG: :not_set, # same as nil
  MYSQL_PWD: :deprecated,
  RAILS_ENV: ["production", "development", "test"],
  TMPDIR: :existing_file_path,
  ...
}

A contract is a list of ENV variables and validators you have attached to them.

Validators can be:

• String literals that are exact values to compare a value with.

  Examples

  EnvControl.configuration.contract = {
    MY_SWITCH: "on",
    MY_BOOL: ["true", "false"], # same as :bool
  }

• nil, which literally means "we expect this variable to be unset".

  Examples

  EnvControl.configuration.contract = {
    DATABASE_HOST: nil,     # must be unset in preference to DATABASE_URL
    MY_SWITCH: ["on", nil], # "on" OR not set
  }

• Symbols, that are essentially names of predefined validators.

  Examples

  EnvControl.configuration.contract = {
    MY_VAR2: :bool, # "true" OR "false"
    MYSQL_DEBUG: :not_set,  # same as nil
    MYSQL_PWD: :deprecated, # same as nil
  }

• Regular expressions ➚ in case strings are not enough.

  Examples

  EnvControl.configuration.contract = {
    # same as :integer
    TIME_SHIFT_SEC: /\A[-]{,1}\d+\z/,
    # should not contain "beta" or "dev"
    APP_VERSION: /^(?!.*(beta|dev))/,
  }

• Custom callables (procs, lambdas, any objects that respond to #call method) in case regexps are not enough.

  Examples

  EnvControl.configuration.contract = {
    FILENAME_FORMAT: CustomFilenameFormatValidator.new,
    DATABASE_NAME: -> { _1.start_with?("production") }
  }

  Learn more about creating custom callable validators.

• a combination of the above as an Array. Contract will be considered satisfied if at least one of the listed validators is satisfied (logical OR).

  Example

  EnvControl.configuration.contract = {
    DLD_RETRY: [:bool, "weekly", "daily", "hourly", /\A\d+H\z/, nil],
  }

  This example combines validators of different types, allowing only: "true" OR "false" OR "weekly" OR "daily" OR "hourly" OR number of hours (e.g. "12H") OR nil

• environment-specific validations.

The EnvControl gem contains several built-in validators that you can use in your contracts.

Built-in validators are simply method names specified as symbols, e.g. :string, :uuid, :email etc.

These methods take ENV variable as input argument and return true or false depending on its value. Named validators (with some exceptions) only work with non-nil ENV variables.

List of built-in validators:

You can create your own named validators if needed.

The requirements for ENV variables may be different when you run your application in different environments➚.

For example, it is important in the development environment to prevent calls to production resources and storages. On the other hand, it makes sense to prohibit the enabling of variables that are responsible for debugging tools in production.

EnvControl allows you to specify environment-specific sets of validators for any of ENV variables.

EnvControl.configure do |config|
  config.environment_name = ENV.fetch('RAILS_ENV')
  config.contract = {
    S3_BUCKET: {
      "production" => :string,  # any non-empty name
      "test" => /test/,         # any name containing 'test' substring
      "default" => :not_set,    # by default the bucket should not be defined
    },
    FILTER_SENSITIVE: {
      "production" => "true",
      "default" => :bool,
    },
    UPLOADS: :existing_folder_path,
    ...
  }
end

You don't have to redefine the whole contract for each environment. It is enough to specify options for a particular variable.

Note that environment names must be strings.

"default" is a special reserved name used to define a fallback value.

You can create your own validators. There are two approaches available.

Validators of this kind must respond to the #call method, so they can be Procs, Lambdas or custom objects.

class StrongPasswordValidator
  def self.call(string)
    string.match? A_STRONG_PASSWORD_REGEX
  end
end

EnvControl.configuration.contract = {
  DB_PASSWORD: [StrongPasswordValidator, :not_set],
}

Custom methods to extend EnvControl::Validators module. These methods can reuse existing validators, making "AND" logic available to you:

module MyContractValidators
  def irc_uri(string)
    uri(string) && URI(string).scheme.eql?("irc")
  end
end

EnvControl::Validators.extend(MyContractValidators)

EnvControl.configuration.contract = {
  IRC_CHANNEL: :irc_uri,
  ...
}

gem install env_control

or add the gem to your Gemfile and then run bundle install:

# Gemfile
gem "env_control"

EnvControl.configuration is a global configuration object for EnvControl. You can set its attributes directly or within configure method's block:

require "env_control"

EnvControl.configure do |config|
  config.environment_name = ...
  config.contract = {...}
  config.on_validation_error = MyContractErrorHander
end

EnvControl.validate(ENV)

Global configuration settings are not mandatory as you can rely on corresponding keyword attributes in #validate method.

contract = {
  ...
}

EnvControl.validate(
  ENV,
  environment_name: "review",
  contract: contract,
  on_validation_error: MyContractErrorHander,
)

Configuration settings you can read and write:

• #environment_name
• #contract
• #on_validation_error
• #validators_allowing_nil

Sets the current environment name for environment-specific validations.

EnvControl.configure do |config|
  config.environment_name = ENV.fetch('RAILS_ENV')
end

A Hash (or a Hash-like structure) that defines the contract. The keys are variable names, the values are the corresponding validators.

EnvControl.configure do |config|
  config.environment_name = ENV.fetch('RAILS_ENV')
  config.contract = {
    # ...
  }
end

This configuration setting contains a handler that validate() method calls as the contract gets breached.

There is a default implementation that raises EnvControl::BreachOfContractError exception. You can customize this behavior by assigning a new callable handler:

EnvControl.configure do |config|
  config.on_validation_error = lambda do |report|
    error = BreachOfContractError.new(context: { report: report })
    Rollbar.critical(error)
  end
end

Or, in case you need to get report without raising an error:

EnvControl.configuration.on_validation_error = ->(report) { report }
EnvControl.validate(ENV) # returns report as a Hash with no error raised

Named validators work only with strings - they usually return false when an attempt is made to validate nil.

However, in rare cases you may need some validators to return true in response to nil. There is a list of such validators, which you can extend as needed.

EnvControl.configuration.validators_allowing_nil
=> [:deprecated, :empty, :ignore, :irrelevant, :not_set]

EnvControl.configuration.validators_allowing_nil << :custom_optional_validator

EnvControl.configuration.validators_allowing_nil
=> [:deprecated, :empty, :ignore, :irrelevant, :not_set, :custom_optional_validator]

As you can see, listed validators are mostly just aliases for nil with extra meanings.

In most scenarios it is better to allow nil explicitly:

EnvControl.configuration.contract = {
  DLD_RETRY: [:my_custom_validator, nil],
}

Having a contract for ENV vars gives you a number of new benefits:

• You explicitly list all requirements, so both your developers and devops know exactly which values are acceptable and which are not.
• You prevent your app from starting if there is something wrong with the ENV variables. E.g., you never misuse a production adapter or database in a staging environment (see best practices).
• You bring out the implicitly used ENV variables, revealing the hidden expectations of third-party gems. (As we often cannot change the logic of third-party gems, we're supposed to put up with inconsistency assigning, say, "on"/"off" to some ENV variables they require, "true"/"false" or "true"/nil to others, which makes working with ENV vars a poorly documented mess unless we have exposed it all in our contract).
• You explicitly declare unused variables as :deprecated, :irrelevant or :ignore, leaving developers no question about the applicability of a particular variable.

The larger your application, the more useful the ENV contract gets.

1. Keep the contract as permissive as possible. Avoid putting sensitive string literals.

2. List variables that are set but not related, marking them as :irrelevant. This will remove questions about their applicability.

3. Disallow unused variables that could potentially affect your apps, marking them as :not_set. This may require you to search for ENVs throughout your code base.

4. Maintain the ENV contract up to date so that other developers can use it as a source of truth about the ENV variables requirements. Feel free to add comments to the contract.

5. Keep the contract keys alphabetically sorted or group the keys by sub-systems of your application.

6. Some validators like :deprecated are effectively equivalent to nil. Give them preference when you need to accompany a requirement to have a variable unset with an appropriate reason.

7. Add :deprecated to existing validators before proceeding to remove code that uses the variable., e.g.:

   MY_VAR: [:deprecated, :string]

8. You may benefit from a contract environment that differs from your app environment. This may be useful if you, say, need to run your review app in "production"-like but restricted environment.

   Example

   EnvControl.configure do |config|
     config.environment_name = \
       if [ENV['RAILS_ENV'], ENV['REVIEW']] == ['production', 'true']
         'review' # virtual production-like environment
       else
         ENV.fetch('RAILS_ENV')
       end
   
     config.contract = {
       S3_BUCKET: {
         "production" => /prod/,
         "review" => /review/, # safe bucket
         "default" => :not_set
       }
     }
   end

• envied gem

• env_bang and env_bang-rails

• require_env

• envi gem

• valid-env gem

• envvar

• env-dependencies gem

• envforcer gem

• env_enforcer gem

• env_lint gem

• env_vars gem

• env_inspector gem

• environment_config gem

• env-checker gem

• envdocs-ruby gem more