fluent-plugin-better-timestamp
fluentd plugin to store timestamp with millisecond to elasticsearch
fluent-plugin-elasticsearch
fluentd uses unixtime as 'time' and fluent-plugin-elasticsearch inserts the 'time' to elasticsearch as @timestamp. Therefore, it's not easy to store more detailed time data like millisec. But fluentd-plugin-elasticsearch provides another way to store timestamp to elasticsearch. if record has '@timestamp', that will be stored in elasticsearh.
fluent-plugin-better-timestamp
this plugin works as a filter that creates '@timestamp' with proper format from 'time' and one more field which have 'msec' data
install
gem install fluent-plugin-better-timestamp
configuration
'better-timestamp' plugin has only 3 config parameters
- tag
- msec_key - specifies the key name for msec (default: msec)
- timestamp_key - specifies the key name for output (default: @timestamp)
example
<source>
type tail
format
format /^(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}).(?<msec>\d{3}) .....
...
tag log
</source>
<match log>
type better_timestamp
tag log.with_msec
</match>
<match tag log.with_msec>
type elasticsearch
...
</match