fluent-plugin-filter-split
Fluentd filter plugin to split specified field.
Compared to existing plugins:
-
https://github.com/wshihadeh/fluent-plugin-record-splitter
- This is filter plugin. It splits the specified field into multiple records. The target field must be lines.
-
https://github.com/unquietwiki/fluent-plugin-split_record
- This is filter plugin. It splits the specified field into key/value pairs. The target field must contain "key=value" pairs.
-
https://github.com/SNakano/fluent-plugin-split-array
- This is filter plugin. It splits the specified field into multiple records. The target field must be always array.
-
https://github.com/toyama0919/fluent-plugin-split
- This is output plugin. It splits the specified field into multiple records. The target field must contains specific separator.
-
https://github.com/activeshadow/fluent-plugin-split-event
- This is filter plugin. It splits the specified field into multiple records. The target field must contains specific separator.
-
https://github.com/bitpatty/fluent-plugin-filter-split-message
-
- The only gem exists. https://rubygems.org/gems/fluent-plugin-filter-split-message
- This is filter plugin. It splits the specified field into multiple records. The target field must contains specific separator.
-
Thus, about above plugins, there are limitation of type of field or missing feature to control keep/remove other fields.
Installation
RubyGems
$ gem install fluent-plugin-filter-split
Bundler
Add following line to your Gemfile:
gem "fluent-plugin-filter-split"
And then execute:
$ bundle
Configuration
parameter | type | description | default |
---|---|---|---|
split_key | string (required) | Specify a target key to split | |
keep_other_key | bool (optional) | Specify a flag whether other key must be kept or not | false |
keep_keys | array (optional) | Specify keys to be kept in filtered record | [] |
remove_keys | array (optional) | Specify keys to be removed in filtered record | [] |
Example
<filter test>
@type filter_split
split_key target_field
keep_other_key true
</filter>
If following record is passed:
{'foo:'bar', 'target_field':[ {'k1':'v1'}, {'k2':'v2'} ] }
Then, you got:
{'foo:'bar', 'k1':'v1'}
{'foo:'bar', 'k2':'v2'}
If following record is passed:
{'foo:'bar', 'target_field':[ 'v1', 'v2' ] }
Then, you got:
{'foo:'bar', 'target_field':'v1'}
{'foo:'bar', 'target_field':'v2'}
Copyright
- Copyright(c) 2023- Kentaro Hayashi
- License
- Apache License, Version 2.0