The project is in a healthy, maintained state
Extends Yabeda to collect Karafka metrics
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
 Dependencies

Development

~> 2.0
~> 13.0
~> 3.0

Runtime

>= 1.3, < 3
~> 2.0
~> 0.8
 Project Readme

⚠️ Project still in it's beginning, API is not stable yet.
       More metrics will probably be added, some metrics might need to change.
       If you find a problem, want a feature or want to help, just reach out on the issues.

Yabeda::Karafka

Yabeda plugin for Karafka events.

Installation

Add this line to your application's Gemfile:

gem 'yabeda-karafka-2', require: 'yabeda/karafka'
# Then add monitoring system adapter, e.g.:
# gem 'yabeda-prometheus'

And then install it and it's dependencies:

$ bundle

If you're not on Rails then configure Yabeda after your application was initialized:

Yabeda.configure!

If you're using Ruby on Rails then Yabeda will configure itself automatically!

Metrics

Consumer metrics

Metrics from events generated during the consumption of kafka messages

  • Total number of batches received: karafka_consumer_received_batches_total_count (segmented by topic, partition and consumer)
  • Total number of messages received: karafka_consumer_received_messages_total_count (segmented by topic, partition and consumer)
  • Total number of batches processed: karafka_consumer_processed_batches_total_count (segmented by topic, partition and consumer)
  • Total number of messages processed: karafka_consumer_processed_messages_total_count (segmented by topic, partition and consumer)
  • Quantity of messages received per batch of messages: karafka_consumer_messages_per_batch (segmented by topic, partition and consumer)
  • Time that took to process a batch of messages (ms): karafka_consumer_batch_processing_time (segmented by topic, partition and consumer)
  • Time that took to process message (ms): karafka_consumer_message_processing_time (segmented by topic, partition and consumer)

Producer metrics

Metrics from events generated during the producing of new kafka messages

  • Total number of kafka messages produced: karafka_producer_sent_messages_total_count (segmented by topic and type)
  • Time that took to send a message (ms): karafka_producer_message_send_time (segmented by topic and type)

Error metrics

Error metrics for error that occurred on a Karafka Producer and Consumer

  • Total number of error: karafka_errors_total_count (segmented by base_type, type and error)

Configuration

Configuration is handled by anyway_config gem. With it you can load settings from environment variables (upcased and prefixed with YABEDA_KARAFKA_), YAML files, and other sources. See anyway_config docs for details.

Config key Type Default Description
consumer_metrics boolean true Defines whether this process should collect consumer metrics
producer_metrics boolean true Defines whether this process should collect producer metrics

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/hbontempo-br/yabeda-karafka.

License

The gem is available as open source under the terms of the MIT License.