fluent-plugin-map

fluent-plugin-map(out_map) is the non-buffered plugin that can convert an event log to different event log(s)

Requirements

fluent-plugin-map	fluentd	ruby
>= 0.2.0	>= v0.14.0	>= 2.1
< 0.2.0	>= v0.12.0	>= 1.9

MapFilter

Configuration

parameter	description	default
map	Specify rule to convert an event log	nil
time	Specify rule to convert a time. The format of time is an integer number of seconds since the Epoch	nil
record	Specify rule to convert a record.The format of record is hash.	nil
multi	Enable to output multi logs	`false`
timeout	Specify timeout	`1`
format	Specify format. If `map` is specified, the format is `map`. If `time` and `record` is specified, the format is `record`. (`map`, `record`, `multimap`)	nil

Example

This sample config filter code file and time file.

<source>
  @type tail
  format apache
  path /var/log/httpd-access.log
  tag tag
  @label @raw
</source>
<label @raw>
  <match **>
    @type copy
    <store>
      @type relabel
      @label @code
    </store>
    <store>
      @type relabel
      @label @time
    </store>
  </match>
</label>
<label @code>
  <filter **>
    @type map
    map ([time, {"code" => record["code"].to_i}])
  </filter>
  <match **>
    @type file
    path code.log
  </match>
</label>
<label @time>
  <filter **>
    @type map
    map ([time, {"time" => record["time"].to_i}])
  </filter>
  <match **>
    @type file
    path time.log
  </match>
</label>

The parameter "map" can use 2 variables in event log; time, record. The format of time is an integer number of seconds since the Epoch. The format of record is hash. The config file parses # as the begin of comment. So the "map" value cannot use #{tag} operation. This plugin can output multi logs by seting multi to true.

If you don't use multi option, you can use time, record parameter. The 2 following filter directive is same:

<filter tag>
  @type map
  map ([time, {"code" => record["code"].to_i}])
</filter>
<filter tag>
  @type map
  time time
  record ({"code" => record["code"].to_i})
</filter>

MapOutput

Configuration

parameter	description	default
map	Specify rule to convert an event log	nil
time	Specify rule to convert a time. The format of time is an integer number of seconds since the Epoch	nil
record	Specify rule to convert a record.The format of record is hash.	nil
multi	Enable to output multi logs	`false`
timeout	Specify timeout	`1`
format	Specify format. If `map` is specified, the format is `map`. If `time` and `record` and (`tag` or `key`) are specified, the format is `record`. (`map`, `record`)	nil
key	Specify the key in record. This parameter is deprecated.	nil
tag	Specify the tag in record	nil

Example

This sample config output code file and time file.

<source>
  @type tail
  format apache
  path /var/log/httpd-access.log
  tag tag
</source>
<match tag>
  @type map
  map ([["code." + tag, time, {"code" => record["code"].to_i}], ["time." + tag, time, {"time" => record["time"].to_i}]])
  multi true
</match>
<match code.tag>
  @type file
  path code.log
</match>
<match time.tag>
  @type file
  path time.log
</match>

The parameter "map" can use 3 variables in event log; tag, time, record. The format of time is an integer number of seconds since the Epoch. The format of record is hash. The config file parses # as the begin of comment. So the "map" value cannot use #{tag} operation. This plugin can output multi logs by seting multi to true.

If you don't use multi option, you can use key, time, record parameter. The 2 following match directive is same:

<match tag>
  @type map
  map (["code." + tag, time, {"code" => record["code"].to_i}])
</match>
<match tag>
  @type map
  tag ("code." + tag)
  time time
  record ({"code" => record["code"].to_i})
</match>

Note

you have to wrap some configuration values with parenthesis like ("code." + tag), to avoid parsing by Fluentd itself. See also: Config File Syntax - Fluentd

Copyright

Apache License, Version 2.0