fluent-plugin-github-activities

Provides ability to watch public activities on GitHub. This crawls GitHub activities of specified users and forward each activity as a record.

Requirements

fluent-plugin-github-activities	fluentd	ruby
>= 0.7.0	>= v0.14.0	>= 2.1
< 0.7.0	>= v0.12.0	>= 1.9

Installation

Add this line to your application's Gemfile:

gem 'fluent-plugin-github-activities'

And then execute:

$ bundle

Or install it yourself as:

$ gem install fluent-plugin-github-activities

Supported activity types

Activities related to commits
- push
- commit (See also following notes)
- commit-comment
Activities related to repositories
- fork
- branch
- tag
Activities related to issues
- issue-open
- issue-close
- issue-reopen
- issue-assign
- issue-unassign
- issue-label
- issue-unlabel
Activities related to pull requests
- pull-request
- pull-request-merged
- pull-request-cancelled
- pull-request-reopen
- pull-request-comment

Forwarded message is same to an activity provided by GitHub. See also the API documentations of GitHub activity events.

Notes:

Because a "push" activity doesn't include full information of each commit, commits are separately forwarded commits as pseudo commit activities.
All forwarded records have an extra property $github-activities-related-avatar. It will be useful to get the URI of the avatar image easily, for both activity events and commits.
All forwarded records have an extra property $github-activities-related-organization-icon. It will be useful to get the URI of the logo image of the organization easily, for both activity events and commits.
Unsupported activities are also forwarded with their raw event type like StatusEvent.

Authentication

The configuration item access_token is optional but strongly recommended to be configured, because there is a rate limit: 60requests/hour by default. By an authenticated crawler, you can crawl 5000requests/hour (means about 80requests/minute). See also the guide to create new access token.

A new access token for your instance can be generated by a simple BASIC authentication, like:

$ curl -u 'your-account' -d '{"scopes":[],"note":"Sharetary"}' https://api.github.com/authorizations
Enter host password for user 'your-account':
{
  "id": 1234567,
  "url": "https://api.github.com/authorizations/1234567",
  "app": {
    "name": "Sharetary (API)",
    "url": "https://developer.github.com/v3/oauth_authorizations/",
    "client_id": "00000000000000000000"
  },
  "token": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "hashed_token": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "token_last_eight": "xxxxxxxx",
  "note": "Sharetary",
  "note_url": null,
  "created_at": "2015-06-02T03:17:46Z",
  "updated_at": "2015-06-02T03:17:46Z",
  "scopes": [

  ],
  "fingerprint": null
}

fluent-plugin-github-activities crawls only public activities, so you don't have to give any extra permission. Then the value of the token field is the access key to be written to the configuration file.

Configurations

<source>
  @type github-activities

  # Access token for this instance. See also the previous section.
  access_token access-token-for-this-instance

  # Interval seconds for requests. This is `1` by default.
  interval 1

  # Number of clients. This is `4` by default.
  clients 1

  # Base tag of forwarded records. It will be used as
  # <base_tag>.<activity type>, like: "github-activity.push",
  # "github-activity.StatusEvent", etc.
  base_tag github-activity.

  # The lisf of target users' account IDs on the GitHub to be crawled.
  users ashie,co-me,cosmo0920,hayamiz,hhatto,kenhys,kou
  # External list is also available.
  #users_list /path/to/list/of/users

  # Merged pull requests will provide push and commit activities,
  # so you possibly see same commits twice when a pull request by
  # a known user (in the list above) is merged by another known user.
  # To avoid such annoying duplicated records, they are ignored by
  # default. If you hope those records are also forwarded, set this
  # option `true` manually.
  #include_commits_from_pull_request true

  # Pull requests can include commits by unknown users (out of the
  # list above) and the crawler ignores such users' commits by default.
  # To include those commit activities, set this option `true` manually.
  #include_foreign_commits true
</source>

Breaking changes

pos_file is obsolete since 0.7.0. Use storage instead as following:

<source>
  @type github-activities
  <storage>
    @type local
    persistent true
    # or if you want to take over old pos_file,
    # path parameter must be end with ".json".
    # path /path/to/pos_file.json
  </storage>
</source>