What is perf check
perf_check is a quick-n-dirty way to benchmark git branches of your rails app.
Imagine a rails-aware apache ab.
We typically run it locally or on staging, to get a general idea of how our branches might have affected app performance. Often, certain pages render differently if logged in, or as an admin, so perf_check provides an easy way to deal with that.
How to install
Add it to your Gemfile, stick it just in the :development group
gem 'perf_check'
You will actually have to commit this to git. Preferably to master, this will make life easiest. However, as long as the gem exists on whatever reference branch you are benchmarking against, you are good to go.
How to use
In it's simplest incarnation, just feed it an url:
$ bundle exec perf_check /notes/browse
=============================================================================
PERRRRF CHERRRK! Grab a coffee and don't touch your working tree (we automate git)
=============================================================================
Benchmarking /notes/browse:
Request 1: 774.8ms 66MB
Request 2: 773.4ms 66MB
Request 3: 771.1ms 66MB
Request 4: 774.1ms 66MB
Request 5: 773.7ms 66MB
Request 6: 774.8ms 66MB
Request 7: 773.4ms 66MB
Request 8: 771.1ms 66MB
Request 9: 774.1ms 66MB
Request 10: 773.7ms 67MB
Benchmarking /notes/browse:
Request 1: 20.2ms 68MB
Request 2: 23.0ms 68MB
Request 3: 19.9ms 68MB
Request 4: 19.5ms 68MB
Request 5: 19.4ms 68MB
Request 6: 20.2ms 68MB
Request 7: 23.0ms 68MB
Request 8: 19.9ms 68MB
Request 9: 19.5ms 69MB
Request 10: 19.4ms 69MB
==== Results ====
/notes/browse
master: 20.4ms
your branch: 773.4ms
change: +753.0ms (yours is 37.9x slower!!!)
How does it work
In the above example, perf_check assumes you are already on a feature branch. It then:
- Launches its own rails server on a custom port (with fragment caching enabled)
- Hits /user/45/posts 11 times (throws away the first 2 requests)
- Git stashes in case you have uncommitted stuff. Checks out master. Restarts server
- Hits /user/45/posts 11 times (throws away the first 2 requests)
- Applies the stash if it existed
- Prints results
Caveats of DOOOOM
We automate git
Do not edit the working tree while perf_check is running!
This program performs git checkouts and stashes, which are undone after the benchmark completes. If the working tree changes after the reference commit is checked out, numerous problems may arise.
Caching is forced on (by default)
Perf check start ups its rails server with cache_classes=true and perform_caching=true regardless of what's in your development.rb
You can pass --clear-cache which will run Rails.cache.clear before each batch of requests. This is useful when testing caching.
All options
$ bundle exec perf_check
Usage: perf_check [options] [route ...]
Benchmark options:
-n, --requests N Use N requests in benchmark, defaults to 10
-r, --reference COMMIT Benchmark against COMMIT instead of master
-q, --quick Fire off 5 requests just on this branch, no comparison with master
--clear-cache Call Rails.cache.clear before running benchmark
--302-success Consider HTTP 302 code a successful request
--302-failure Consider HTTP 302 code an unsuccessful request
--run-migrations Run migrations up and down with branch
--compare-paths Compare two paths against each other on the same branch
Usage examples:
Benchmark PostController#index against master
perf_check /user/45/posts
perf_check /user/45/posts -n5
Benchmark against a specific commit
perf_check /user/45/posts -r 0123abcdefg
perf_check /user/45/posts -r HEAD~2
Benchmark the changes in the working tree
perf_check /user/45/posts -r HEAD
Benchmark and diff the output against master
perf_check /user/45/posts --verify-no-diff
Diff the output on your branch with master
perf_check /user/45/posts --diff
Diff a bunch of urls listed in a file (newline seperated)
perf_check --diff --input FILE
Troubleshooting
Perf Check
Setting the PERF_CHECK env variable will start up your app with the middleware which does things like capture backtraces and count sql queries
PERF_CHECK=true bundle exec rails s
Redis
Certain versions of redis might need the following snippet to fork properly:
# Circumvent redis forking issues with our version of redis
# Will be fixed when we update redis https://github.com/redis/redis-rb/issues/364
class Redis
class Client
def ensure_connected
tries = 0
begin
if connected?
if Process.pid != @pid
reconnect
end
else
connect
end
tries += 1
yield
rescue ConnectionError
disconnect
if tries < 2 && @reconnect
retry
else
raise
end
rescue Exception
disconnect
raise
end
end
end
end