Fluent::Plugin::Syslog

This repository contains the Fluentd Syslog Output Plugin.

Install this gem when setting up fluentd:

gem install fluent-plugin-syslog

This is a buffered output plugin for Fluentd that's configured to send logs to Syslog.

Each log line will arrive in Loggly with 2 payloads: the json representation of the fluent record and the data from the syslog wrapper.

Data from the syslog wrapper includes:

appName - this defaults to the fluent tag
hostname - this can be optionally configured as syslog_hostname (see below)
timestamp - this defaults to the timestamp associated with the record and falls back to the current time at the time it reaches the plugin

You're also able to (optionally) tag syslog records with any string you want, see syslog_tag below.

To configure this in fluentd:

<match whatever.*>
  @type syslog
  syslog_token <your_syslog_token>
  syslog_tag <your_syslog_tag>
  syslog_hostname "#{ENV['HOST']}"
  parse_json true
</match>

Also, if you set parse_json to true, as is shown in the example above, then the plugin will attempt to parse the message field from each fluentd record.

This plugin inherits a few useful config parameters from Fluent's BufferedOutput class.

Parameters for flushing the buffer, based on size and time, are buffer_chunk_limit and flush_interval, respectively. This plugin overrides the inherited default flush_interval to 1, causing the fluent buffer to flush to Loggly every second.

If the plugin fails to write to Loggly for any reason, the log message will be put back in Fluent's buffer and retried. Retrying can be tuned and inherits a default configuration where retry_wait is set to 1 second and retry_limit is set to 17 attempts.

If you want to change any of these parameters simply add them to the match stanza. For example, to flush the buffer every 60 seconds and stop retrying after 2 attempts, set something like:

<match whatever.*>
  @type syslog
  syslog_token <your_syslog_token>
  flush_interval 60
  retry_limit 2
</match>

BufferedOutput also allows you to keep the buffer stored on disk, where it can persist through process restarts. This is great for avoiding dropping logs during outages, see:

<match whatever.*>
  @type syslog
  syslog_token <your_syslog_token>
  buffer_type file
  buffer_path /var/log/fluentd-buffer
</match>

This plugin is targeting Ruby 2.4 and Fluentd v1.0, although it should work with older versions of both.

We have a Makefile to wrap common functions and make life easier.

make bundle

make test

To release a new version, update the version number in the GemSpec and then, run:

make release

Bug reports and pull requests are welcome on GitHub at: https://github.com/analogrithems/fluent-plugin-syslog

The gem is available as open source under the terms of the Apache License.