fluent-plugin-loki
Fluentd output plugin to Grafana Loki.
- Can be used to ship docker logs to Loki (using Fluentd docker logging driver)
- Enable easier transtion to Loki - an alternative to Loki's Promtail
Installation
RubyGems
$ gem install fluent-plugin-loki
Configuration
sample configuration:
<match *>
@type loki
endpoint_url http://127.0.0.1:3100
labels {"env":"prod","farm":"a"} # default: nil
tenant abcd # default: nil
rate_limit_msec 100 # default: 0 = no rate limiting
raise_on_error false # default: true
authentication basic # default: none
username alice # default: ''
password bobpop # default: '', secret: true
buffered true # default: false. Switch non-buffered/buffered mode
cacert_file /etc/ssl/endpoint1.cert # default: ''
token tokent # default: ''
custom_headers {"token":"arbitrary"} # default: nil
</match>
- endpoint_url - Loki's endpoint
- tenant - Loki tenant id
- labels - Labels for filtering in Grafana (currently they are static)
(generated by running: fluent-plugin-config-format output loki -p lib/fluent/plugin)
Usage examples for common fluentd inputs:
syslog
Setting up rsyslog
Open /etc/rsyslog.d/50-default.conf and append the following line:
*.* @127.0.0.1:5140
Then restart the rsyslogd service:
sudo systemctl restart syslog
sample fluentd config:
<source>
@type syslog
port 5140
bind 0.0.0.0
tag system
</source>
simulate data:
logger "came from syslog"
file
sample fluentd config:
## File input
## read apache logs with tag=apache.access
#<source>
# @type tail
# format apache
# path /var/log/httpd-access.log
# tag apache.access
#</source>
HTTP
sample fluentd config:
# HTTP input
# http://localhost:8888/<tag>?json=<json>
# for ex: http://localhost:8888/baz?json={"src":"http"}
<source>
@type http
@id http_input
port 8888
</source>
simulate data:
curl -X POST -d 'json={"src":"http"}' http://localhost:8888tcp
sample fluentd config:
## built-in TCP input
## $ echo <json> | fluent-cat <tag>
<source>
@type forward
@id forward_input
</source>
simulate data:
echo '{"src":"tcp"}' | fluent-cat tcpDevelopment (using Docker)
- Set up Loki and Grafana (can be done by running their docker-compose)
- Add loki data source to grafana
- Run Fluentd container with volume mapping to the dummy configuration and to the plugin:
docker run -d \
-p 9880:9880 \
-v $(pwd)/development/conf:/fluentd/etc \
-v $(pwd)/lib/fluent/plugin:/etc/fluent/plugin -e FLUENTD_CONF=fluentd.conf \
fluent/fluentd- Execute to send log:
curl -X POST -d 'json={"foo":"baz"}' http://localhost:9880
Copyright
- Copyright(c) 2018- Edan Shahmoon
- License
- Apache License, Version 2.0
Heavily based on fluent-plugin-out-http