fluent-plugin-latency

Fluentd plugin to measure latency until receiving the messages.

This plugin is to investigate the network latency, in addition, the blocking situation of input plugins.

In the Fluentd mechanism, input plugins usually blocks and will not receive a new data until the previous data processing finishes.

By seeing the latency, you can easily find how long the blocking situation is occuring.

Fluentd messages include the time attribute which expresses the time of when the message is created, or when it is written in application logs.

This plugin takes the difference between the current time and the time attribute to obtain the latency.

Use RubyGems:

gem install fluent-plugin-latency

Following example measures the max and average latency until receiving messages.

<source>
  type forward
  port 24224
</source>

# Latency plugin output comes here
<match latency>
  type stdout
</match>

# Whatever you want to do
<match rewrite.**>
  type stdout
</match>

# All messages come here once. 
<match **>
  type copy
  <store>
    type latency
    tag latency
    interval 60
  </store>
  <store>
    type rewrite
    add_prefix rewrite
  </store>
</match>

Output will be like

latency: {"max":1.011,"avg":0.002","num":10}

where max and avg are the maximum and average latency, and num is the number of messages.

• interval

  The time interval to emit measurement results

• tag

  The output tag name. Default is latency

See CHANGELOG.md for details.

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

Copyright (c) 2014 Naotoshi Seo. See LICENSE for details.