Project

fluent-plugin-sflow

0.01

No commit activity in last 3 years

No release in over 3 years

fluent-plugin-sflow codeout/fluent-plugin-sflow Homepage Documentation Source Code Bug Tracker Wiki

sFlow v2 / v4 / v5 input plugin for Fluentd supporting many packet formats

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

Popularity

19,826

11

3

2

Releases

Current version

0.3.3

7

2017-03-24

2018-02-12

Issues

1

2

3

Issue Closure Rate

66%

Pull Requests

Open Pull Requests

0

Closed Pull Requests

0

Merged Pull Requests

2

Pull Request Acceptance Rate

100%

Development

Primary Language

C

Average date of last 50 commits

2018-05-03

Reverse Dependencies

0

Dependencies

Development

~> 1.12

rake

~> 10.0

rake-compiler

~> 1.0

~> 3.0

Runtime

< 2, >= 0.14.0

Project Readme

fluent-plugin-sflow

This branch is valid for Fluentd 0.14.x or later

See 0.12.x branch for Fluentd 0.12.x.

Overview

Fluentd input plugin that acts as sFlow v2/v4/v5 collector.

Including a ruby wrapper of sflowtool which processes sFlow datagrams to report in JSON format.

Features

fluent-plugin-sflow supports many packet formats below. Basically, it can process anything sflowtool can do. ✨

See sflowtool document for more details.

Sample Type

Flow Sample
Counter Sample

Address Family

IPv4
IPv6

Extended Data Type

Switch
Router
Gateway_v2
Gateway
User
Url
mplsLabelStack
Mpls
Nat
NatPort
MplsTunnel
MplsVC
MplsFTN
MplsLDP_FEC
VlanTunnel
WifiPayload
WifiRx
WifiTx
Aggregation
Socket4
ProxySocket4
Socket6
ProxySocket6
Decap
VNI
TCPInfo

Counter Data Type

Requirement

Fluentd: 0.14.x or lator
- See fluentd-0.12 branch for Fluentd 0.12.x.
gcc and make
Ruby dev package

Install

You need to install development tools beforehand.

📝 If you see any problem on other platforms rather than Debian, Ubuntu, or CentOS, please open a new issue.

Debian Linux or Ubuntu Linux

$ sudo apt install -y gcc make

If you're using ruby installed by apt, install ruby-dev.
( Skip if your ruby is installed by rbenv )

$ sudo apt install -y ruby-dev

CentOS Linux

$ sudo dnf install -y redhat-rpm-config gcc make

If you're using ruby installed by dnf ( or yum ), install ruby-devel.
( Skip if your ruby is installed by rbenv )

$ sudo dnf install -y ruby-devel

fluent-plugin-sflow

$ gem install fluent-plugin-sflow

You can also use fluent-gem or td-agent-gem instead of gem command.

Configuration

<source>
  @type sflow
  tag example.sflow

  bind 0.0.0.0
  port 6343
</source>

bind

IP address on which this plugin will accept sFlow. (Default: '0.0.0.0')

port

UDP port number on which this plugin will accept sFlow. (Default: 6343)

Record Example

Flow Sample

{
  "datagram_source_ip": "10.1.2.1",
  "datagram_size": 240,
  "unix_seconds_utc": 1502113217,
  "datagram_version": 5,
  "agent_sub_id": 0,
  "agent": "10.1.2.1",
  "packet_sequence_no": 53,
  "sys_up_time": 3808877,
  "samples_in_packet": 1,
  "sample_type_tag": "0:1",
  "sample_type": "flow_sample",
  "sample_sequence_no": 44,
  "source_id": "0:513",
  "mean_skip_count": 5,
  "sample_pool": 225,
  "drop_events": 0,
  "input_port": 513,
  "output_port": 512,
  "flow_block_tag": "0:1002",
  "header_protocol": 1,
  "sampled_packet_size": 102,
  "stripped_bytes": 4,
  "header_len": 98,
  "header_bytes": "02-05-86-71-7D-03-08-00-27-12-99-09-08-00-45-00-00-54-B0-61-40-00-40-01-79-44-0A-01-02-02-01-00-04-01-08-00-DF-97-04-12-03-AD-D2-1A-87-59-00-00-00-00-F6-61-02-00-00-00-00-00-10-11-12-13-14-15-16-17-18-19-1A-1B-1C-1D-1E-1F-20-21-22-23-24-25-26-27-28-29-2A-2B-2C-2D-2E-2F-30-31-32-33-34-35-36-37",
  "dst_mac": "020586717d03",
  "src_mac": "080027129909",
  "ip_size": 84,
  "ip_tot_len": 84,
  "src_ip": "10.1.2.2",
  "dst_ip": "1.0.4.1",
  "ip_protocol": 1,
  "ip_tos": 0,
  "ip_ttl": 64,
  "ip_id": 25008,
  "icmp_type": 8,
  "icmp_code": 0,
  "in_vlan": 0,
  "in_priority": 0,
  "out_vlan": 0,
  "out_priority": 0,
  "next_hop": "202.249.2.169",
  "src_subnet_mask": 32,
  "dst_subnet_mask": 22
}

Counters Sample

{
  "datagram_source_ip": "10.1.2.1",
  "datagram_size": 204,
  "unix_seconds_utc": 1502113415,
  "datagram_version": 5,
  "agent_sub_id": 0,
  "agent": "10.1.2.1",
  "packet_sequence_no": 54,
  "sys_up_time": 3810403,
  "samples_in_packet": 1,
  "sample_type_tag": "0:2",
  "sample_type": "counters_sample",
  "sample_sequence_no": 5,
  "source_id": "0:512",
  "counter_block_tag": "0:2",
  "if_index": 512,
  "network_type": 6,
  "if_speed": 10000000000,
  "if_direction": 1,
  "if_status": 3,
  "if_in_octets": 0,
  "if_in_ucast_pkts": 8,
  "if_in_multicast_pkts": 121,
  "if_in_broadcast_pkts": 0,
  "if_in_discards": 0,
  "if_in_errors": 0,
  "if_in_unknown_protos": 0,
  "if_out_octets": 130454,
  "if_out_ucast_pkts": 1060,
  "if_out_multicast_pkts": 0,
  "if_out_broadcast_pkts": 95,
  "if_out_discards": 0,
  "if_out_errors": 0,
  "if_promiscuous_mode": 0,
  "dot3_stats_alignment_errors": 0,
  "dot3_stats_fcserrors": 0,
  "dot3_stats_single_collision_frames": 0,
  "dot3_stats_multiple_collision_frames": 0,
  "dot3_stats_sqetest_errors": 0,
  "dot3_stats_deferred_transmissions": 0,
  "dot3_stats_late_collisions": 0,
  "dot3_stats_excessive_collisions": 0,
  "dot3_stats_internal_mac_transmit_errors": 0,
  "dot3_stats_carrier_sense_errors": 0,
  "dot3_stats_frame_too_longs": 0,
  "dot3_stats_internal_mac_receive_errors": 0,
  "dot3_stats_symbol_errors": 0
}

Benchmark

Here is a quick benchmark on Macbook Pro Mid 2015:

sFlow v5 flow samples, 5.88 samples per packet in average
Fluentd v0.14.17
13161 sFlow v5 records/s

TODO

Add more tests
- Please send me the sFlow pcap file if your device is not supported. 🌀
  1. Archive your pcap in .zip or .gz
  2. Open a new issue
  3. Attach the archive

Contributing

Please report issues or enhancement requests to GitHub issues. For questions or feedbacks write to my twitter @codeout.

Or send a pull request to fix.

Copyright and License

Copyright (c) 2017-2021 Shintaro Kojima. Code released under the MIT license.

Code includes a part of sflowtool which is distributed in the InMon sFlow License.