Project

fluent-plugin-redis-store

0.05
No commit activity in last 3 years
No release in over 3 years
fluent-plugin-redis-storepokehanai/fluent-plugin-redis-storeHomepageDocumentationSource CodeBug TrackerWiki
Redis(zset/set/list/string/publish) output plugin for Fluentd...
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Popularity
Downloads
1,233,020
Stars
17
Forks
17
Watchers
2
 Releases
Current version
0.2.0
Total releases
4
First release
2014-05-24
Latest release
2017-09-07
 Issues
Open Issues
0
Closed Issues
1
Total Issues
1
Issue Closure Rate
100%
 Pull Requests
Open Pull Requests
2
Closed Pull Requests
3
Merged Pull Requests
6
Pull Request Acceptance Rate
54%
 Development
Primary Language
Ruby
Licenses
Apache License Version 2.0
Average date of last 50 commits
2015-07-21
Reverse Dependencies
0

 Dependencies

Development

rake
>= 0
test-unit
>= 0

Runtime

fluentd
< 2, >= 0.14.15
redis
>= 0
 Project Readme

Redis Output Plugin For fluentd

Fluentd output plugin to upload/publish event data to Redis storage.

Requirements

fluent-plugin-redis-store fluentd ruby
>= 0.2.0 >= v0.14.15 >= 2.1
< 0.2.0 >= v0.12.0 >= 1.9

Background

This is folked project from fluent-plugin-redisstore.

Features

Supported Redis commands

Currently the plugin supports following Redis commands:

  • set by string type (of the plugin)
  • lpush/rpush by list type
  • sadd by set type
  • zadd by zset type
  • publish by publish type

Supported value format

key string for Redis storage

Redis commands require key and value.
For key, the plugin supports either way;

  1. Specify a fixed key.
    You can do this simply using key option in td-agent configuration file.

    type redis_store
key userdata

  2. Lookup a key string in every event data by a lookup path.
    If event data have structured data like

    { "user": { "name": "Kei" } }

    and you want to use each name of user, you can use key_path option.

    type redis_store
key_path user.name

    With the above data, Kei will be a key.

In addition, key_prefix and key_suffix are useful in some cases. Both are available either key and key_path

type redis_store
key_path user.name
key_prefix ouruser.
key_suffix .accesslog

With the previous data, key will be outuser.Kei.accesslog.

value data for Redis storage

To determine what value in every event data to be srtored, you have two options;

  1. Store extracted data in event data, by a lookup path with value_path option.
    It works like key_path.
  2. Store whole data.
    This is default behavior. To do it, simply omit value_path option.

Installation

fluent-gem install fluent-plugin-redis-store

# or if you are using td-agent:
td-agent-gem install fluent-plugin-redis-store

Configuration

Redis connection

Key Type Required? Default Description
host string Optional 127.0.0.1 host name of Redis server
port int Optional 6379 port number of Redis server
password string Optional password for Redis connection
path string Optional To connect via Unix socket, try '/tmp/redis.sock'
db int Optional 0 DB number of Redis
timeout float Optional 5.0 connection timeout in seconds

common options for storages

Key Type Default Description
key string Fixed key used to store(publish) in Redis
key_path string path to lookup for key in the event data
key_prefix string prefix of key
key_suffix string suffix of key
value_path string (whole event data) path to lookup for value in the event data
store_type string zset string/list/set/zset/publish
format_type string plain format type for value (plain/json/msgpack)
key_expire int -1 If set, the key will be expired in specified seconds
flush_interval time 1 Time interval which events will be flushed to Redis

Note: either key or key_path is required.

string storage specific options

Key Type Default Description
type string Fixed key used to store(publish) in Redis
No more options than common options.

list storage specific options

Key Type Default Description
order string asc asc: rpush, desc: lpush

set storage specific options

No more options than common options.

zset storage specific options

Key Type Default Description
score_path string (time of log event) path to lookup for score in the event data
collision_policy string (nil) Only update elements that already exist (XX) or add a new element (NX)
value_expire int value expiration in seconds

If value_expire is set, the plugin assumes that the score in the SortedSet is based on timestamp and it deletes expired members every after new event data arrives.

publish storage specific options

No more options than common options.

Contributors

moaikids
HANAI tohru
Mohit Khanna
yamada-shinji
Heitor de Souza
Hiroshi Hatake
Gabriel Bordeaux

Copyright

Copyright (c) 2013 moaikids
Copyright (c) 2014 HANAI Tohru

License

Apache License, Version 2.0