lightweight_attributes

lightweight_attributes is a tiny monkey-patch for making Active Record model objects ultimately lightweight.

Benchmarks

Here's a result of a simple benchmark measuring memory usage, number of method calls, and execution time with and without this gem, for fetching 10,000 records from a MySQL database and iterating over each of them and reading each attribute. The benchmark is included in this repo so everyone can try.

Allocated Memory and Allocated Objects

lightweight_attributes halves the memory allocation and reduces the object creation to be 25%!

$ bundle e ruby benchmark.rb memory
****************************** ActiveModel::AttributeSet ******************************
Total allocated: 26969712 bytes (270001 objects)
Total retained:  24649712 bytes (260001 objects)

allocated memory by class
-----------------------------------
  12800000  Hash
   8000000  ActiveModel::Attribute::FromDatabase
   3600000  String
   1200000  Model
    880000  ActiveModel::LazyAttributeHash
    400000  ActiveModel::AttributeSet
     89712  Array

allocated objects by class
-----------------------------------
    100000  ActiveModel::Attribute::FromDatabase
     90000  String
     50000  Hash
     10000  ActiveModel::AttributeSet
     10000  ActiveModel::LazyAttributeHash
     10000  Model
         1  Array


****************************** LightweightAttributes ******************************
Total allocated: 14889712 bytes (70001 objects)
Total retained:  12569712 bytes (60001 objects)

allocated memory by class
-----------------------------------
  12800000  Hash
   1200000  Model
    800000  LightweightAttributes::AttributeSet
     89712  Array

allocated objects by class
-----------------------------------
     50000  Hash
     10000  LightweightAttributes::AttributeSet
     10000  Model
         1  Array

Number of Method Calls

lightweight_attributes reduces method calls to be less than 40%!

$ bundle e ruby benchmark.rb methods
****************************** ActiveModel::AttributeSet ******************************
{"ActiveRecord::Result#each"=>1,
 "ActiveRecord::Result#hash_rows"=>1,
 "ActiveRecord::Persistence::ClassMethods#instantiate"=>10000,
 "ActiveRecord::Inheritance::ClassMethods#discriminate_class_for_record"=>10000,
 "ActiveRecord::Inheritance::ClassMethods#using_single_table_inheritance?"=>10000,
 "ActiveRecord::ModelSchema::ClassMethods#inheritance_column"=>20000,
 "Object#present?"=>10000,
 "NilClass#blank?"=>10000,
 "ActiveRecord::Persistence::ClassMethods#discriminate_class_for_record"=>10000,
 "ActiveRecord::ModelSchema::ClassMethods#attributes_builder"=>10000,
 "ActiveModel::AttributeSet::Builder#build_from_database"=>10000,
 "ActiveModel::LazyAttributeHash#initialize"=>10000,
 "ActiveModel::AttributeSet#initialize"=>10000,
 "ActiveRecord::Core::ClassMethods#allocate"=>10000,
 "ActiveRecord::AttributeMethods::ClassMethods#define_attribute_methods"=>20000,
 "ActiveRecord::Core#init_with"=>10000,
 "#<Class:ActiveRecord::LegacyYamlAdapter>#convert"=>10000,
 "ActiveRecord::ModelSchema::ClassMethods#yaml_encoder"=>10000,
 "ActiveModel::AttributeSet::YAMLEncoder#decode"=>10000,
 "ActiveRecord::Aggregations#init_internals"=>10000,
 "ActiveRecord::Associations#init_internals"=>10000,
 "ActiveRecord::Core#init_internals"=>10000,
 "ActiveRecord::Base#_run_find_callbacks"=>10000,
 "ActiveSupport::Callbacks#run_callbacks"=>20000,
 "ActiveRecord::Base#__callbacks"=>20000,
 "#<Class:ActiveRecord::Base>#__callbacks"=>20000,
 "ActiveSupport::Callbacks::CallbackChain#empty?"=>20000,
 "ActiveRecord::Base#_run_initialize_callbacks"=>10000,
 "ActiveRecord::AttributeMethods::PrimaryKey#id"=>10000,
 "ActiveRecord::Transactions#sync_with_transaction_state"=>10000,
 "ActiveRecord::Transactions#update_attributes_from_transaction_state"=>10000,
 "ActiveRecord::AttributeMethods::PrimaryKey::ClassMethods#primary_key"=>10000,
 "ActiveRecord::AttributeMethods::Read#_read_attribute"=>100000,
 "ActiveModel::AttributeSet#fetch_value"=>100000,
 "ActiveModel::AttributeSet#[]"=>100000,
 "ActiveModel::LazyAttributeHash#[]"=>100000,
 "ActiveModel::LazyAttributeHash#assign_default_value"=>100000,
 "#<Class:ActiveModel::Attribute>#from_database"=>100000,
 "ActiveModel::Attribute#initialize"=>100000,
 "ActiveModel::Attribute#value"=>100000,
 "ActiveModel::Attribute::FromDatabase#type_cast"=>100000,
 "ActiveModel::Type::Integer#deserialize"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c613"=>10000,
 "ActiveModel::Type::Value#deserialize"=>90000,
 "ActiveModel::Type::Value#cast"=>90000,
 "ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter::MysqlString#cast_value"=>90000,
 "ActiveModel::Type::String#cast_value"=>90000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c623"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c633"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c643"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c653"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c663"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c673"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c683"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c693"=>10000}


****************************** LightweightAttributes ******************************
{"ActiveRecord::Result#each"=>1,
 "ActiveRecord::Result#hash_rows"=>1,
 "ActiveRecord::Persistence::ClassMethods#instantiate"=>10000,
 "ActiveRecord::Inheritance::ClassMethods#discriminate_class_for_record"=>10000,
 "ActiveRecord::Inheritance::ClassMethods#using_single_table_inheritance?"=>10000,
 "ActiveRecord::ModelSchema::ClassMethods#inheritance_column"=>20000,
 "Object#present?"=>10000,
 "NilClass#blank?"=>10000,
 "ActiveRecord::Persistence::ClassMethods#discriminate_class_for_record"=>10000,
 "ActiveRecord::ModelSchema::ClassMethods#attributes_builder"=>10000,
 "LightweightAttributes::AttributeSet::Builder#build_from_database"=>10000,
 "LightweightAttributes::AttributeSet#initialize"=>10000,
 "ActiveRecord::Core::ClassMethods#allocate"=>10000,
 "ActiveRecord::AttributeMethods::ClassMethods#define_attribute_methods"=>20000,
 "ActiveRecord::Core#init_with"=>10000,
 "#<Class:ActiveRecord::LegacyYamlAdapter>#convert"=>10000,
 "ActiveRecord::ModelSchema::ClassMethods#yaml_encoder"=>10000,
 "ActiveModel::AttributeSet::YAMLEncoder#decode"=>10000,
 "ActiveRecord::Aggregations#init_internals"=>10000,
 "ActiveRecord::Associations#init_internals"=>10000,
 "ActiveRecord::Core#init_internals"=>10000,
 "ActiveRecord::Base#_run_find_callbacks"=>10000,
 "ActiveSupport::Callbacks#run_callbacks"=>20000,
 "ActiveRecord::Base#__callbacks"=>20000,
 "#<Class:ActiveRecord::Base>#__callbacks"=>20000,
 "ActiveSupport::Callbacks::CallbackChain#empty?"=>20000,
 "ActiveRecord::Base#_run_initialize_callbacks"=>10000,
 "ActiveRecord::AttributeMethods::PrimaryKey#id"=>10000,
 "ActiveRecord::Transactions#sync_with_transaction_state"=>10000,
 "ActiveRecord::Transactions#update_attributes_from_transaction_state"=>10000,
 "ActiveRecord::AttributeMethods::PrimaryKey::ClassMethods#primary_key"=>10000,
 "ActiveRecord::AttributeMethods::Read#_read_attribute"=>100000,
 "LightweightAttributes::AttributeSet#fetch_value"=>100000,
 "ActiveModel::Type::Integer#deserialize"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c613"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c623"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c633"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c643"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c653"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c663"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c673"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c683"=>10000,
 "#<ActiveRecord::AttributeMethods::GeneratedAttributeMethods:0x00007fcb5c4e06b8>#__temp__36f6c693"=>10000}

Elapsed Time

lightweight_attributes makes it 4x faster!

$ bundle e ruby benchmark.rb time
****************************** ActiveModel::AttributeSet ******************************
1.590478

****************************** LightweightAttributes ******************************
0.384524

Installation

Just bundle 'lightweight_attributes' gem on your Rails apps, and you're all set. No configurations!

Usage

It just works. I said, no configurations. This gem changes Active Record's data structure without changing any public API, so you don't need to change any single line of code in your application. If bundling this gem introduces any incompatibility besides making things fast and less memory-consuming, that should be a bug.

Why ActiveRecord::Base Is Slow and Heavy, and How This Gem Makes It Fast and Lightweight?

The Active Record Object

Active Record model object is a super hero that plays multiple roles. One is "form object". For this use case, we usually create just one object per request, so we don't have to care about performance of this object.

Another use case is "data transfer object". For this purpose, we would create hundreds of objects for rendering one HTML page. We would even create millions of model objects for APIs or batch systems. In this case, size of each object deeply impacts the whole system performance.

The Attribute API

As a version 4.0 feature, Active Record eqiupped a new API named "attributes API" that handles type casting between user, application, and database. However, this new feature caused a significant performance regression. For that purpose, each attribute per each model instance holds an instance of Attribute object that handles all the heavy lifting works around type casting.

Because of that data structure, ActiveRecord::Base became a super fat and heavy object. For example, when instantiating 1,000 records having 10 columns, it internally creates 10,000 instances ofAttribute objects. This of course causes a massive GC pressure.

The Solution

This gem defers the creation of attribute objects in a particular use case.

For the "data transfer object" use case, what we really need is a read-only Struct like object. We don't need interactive type casting feature. We don't need dirty tracking feature. lightweight_attributes focuses on this use case. It's a set of monkey-patches that holds the set of data from DB in a single Hash object rather than an Array of Attribute objects.

This behavior is base on an assumption that AR model objects from database are "read only" in most cases. There could still be some cases where a model from database receives attribute writes (e.g. for update action). And for such cases, our attributes object takes "deoptimization" approach. When this Hash-based model object receives any attribute write, it metamorphses to be a normal attribute-based model object, then falls back to the original Active Record attributes API.

Contributing

Pull requests are welcome on GitHub at https://github.com/amatsuda/lightweight_attributes.

License

The gem is available as open source under the terms of the MIT License.

Sponsor

This gem has been developed under total financial support from Akatsuki Inc..