fluent-plugin-split-by-size
Fluentd filter plugin to split events based on a max size.
Splits event into multiple events based on a max size using a field id in the original message as an id to associate parts of the original event. This is mostly designed for use with kinesis to allow larger records to be used than 1mb by splitting the records going into kinesis and then re-joining them later.
Installation
RubyGems
$ gem install fluent-plugin-split-by-size
Bundler
Add following line to your Gemfile:
gem "fluent-plugin-split-by-size"And then execute:
$ bundle
Configuration
max_event_size (integer) (optional)
Max size an event can be in bytes
Default value: 1048576.
id_field (string) (required)
Field that contains a unique id that will be added to all created events
Copyright
- Copyright(c) 2018- Timothy Schroeder
- License
- Apache License, Version 2.0