AllMyCircuits
AllMyCircuits is intended to be a threadsafe-ish circuit breaker implementation for Ruby. See Martin Fowler's article on the Circuit Breaker pattern for more info.
Usage
class MyService
include Singleton
def initialize
@circuit_breaker = AllMyCircuits::Breaker.new(
name: "my_service",
watch_errors: [Timeout::Error], # defaults to AllMyCircuits::Breaker.net_errors,
# which includes typical net/http and socket errors
sleep_seconds: 10, # leave circuit open for 10 seconds, than try to call the service again
strategy: AllMyCircuits::Strategies::PercentageOverWindowStrategy.new(
requests_window: 100, # number of requests to calculate the average failure rate for
failure_rate_percent_threshold: 25 # open circuit if 25% or more requests within 100-request window fail
# must trip open again if the first request fails
}
)
end
def run
begin
@breaker.run do
Timeout.timeout(1.0) { my_risky_call }
end
rescue AllMyCircuits::BreakerOpen => e
# log me somewhere
rescue
# uh-oh, risky call failed once
end
end
end
Testing
So, what have we got:
- Time-sensitive code: ...check
- Concurrent code: ...check
Dude, that's a real headache for someone who's not confortable enough with concurrent code. I haven't figured any awesome way of automated testing in this case. So, in the script folder, there are:
fake_service.rb
the Fake Service has 3 modes of operation: up
(default), die
and slowdown
.
-
up
(default) - http://localhost:8081/up - normal mode of operation, latency up to 50ms -
die
- http://localhost:8081/die - exceptions are raised left and right, slight delay in response -
slowdown
- http://localhost:8081/slowdown - successful responses with a significant delay.
Graphing Stress Test
runs WORKERS
number of workers which continuously hit http://localhost:8081. Graphs are served at http://localhost:8080.
This app allows to catch incorrect circuit breaker behavior visually.
Logging
Logger can be configured by setting AllMyCircuits.logger
.
Default log device is STDERR, and default level is ERROR (can be overridden with ALL_MY_CIRCUITS_LOG_LEVEL
environment variable).
TODO
- global controls through redis?