Github to Slack

Create an end-to-end event pipeline that detects changes in github stars & forks and publishes the result to Slack. This guide uses two connectors:

http-source: to read periodically from a github, parse the fields from the json output, and publishes the result to a topic.
http-sink: to listen to the same topic, detect changes, and publish the result to Slack.

Let’s get started.

Prerequsites

Fluvio CLI running locally
Account on InfinyOn Cloud

Create http-source configuration file

Create an HTTP source connector configuration file called github.yaml :

apiVersion: 0.1.0
meta:
  version: 0.2.4
  name: github-stars-in
  type: http-source
  topic: stars-forks
  secrets:
    - name: GITHUB_TOKEN
http:
  endpoint: 'https://api.github.com/repos/infinyon/fluvio'
  method: GET
  headers: 
    - 'Authorization: token ${{ secrets.GITHUB_TOKEN }}'
  interval: 30s
transforms:
  - uses: infinyon/[email protected]
    with:
      spec:
        - operation: shift
          spec:
            "stargazers_count": "stars"
            "forks_count": "forks"

Github rate-limits API requests to 60 per hour, which you an extend to 5000 by creating an application token. Check out github documentation on how to create an Access Tokens.

Add the access token secret in InfinyOn Cloud :

fluvio cloud secret set GITHUB_TOKEN <access-token>

Create http-sink configuration file

Create an HTTP source connector configuration file called slack.yaml :

apiVersion: 0.1.0
meta:
  version: 0.2.4
  name: slack-stars-out
  type: http-sink
  topic: stars-forks
  secrets:
    - name: SLACK_TOKEN
http:
  endpoint: "https://hooks.slack.com/services/${{ secrets.SLACK_TOKEN }}"
  headers:
    - "Content-Type: application/json"
transforms:
  - uses: infinyon-labs/[email protected]
    lookback:
      last: 1
  - uses: infinyon/[email protected]
    with:
      spec:
        - operation: shift
          spec:
            "result": "text"

Check out Slack Webhooks on how to create a channel webhook token.

Add the access token secret in InfinyOn Cloud :

fluvio cloud secret set SLACK_TOKEN <webhook-token>

Download Smartmodules

Download the smartmodules used by the connectors to your cluster:

fluvio hub download infinyon/[email protected]
fluvio hub download infinyon-labs/[email protected]

Start Connectors

Start source & sink connectors:

fluvio cloud connector create -c github.yaml
fluvio cloud connector create -c slack.yaml

Check fluvio cloud connector log to ensure they have been successfully provisioned.

Test Data Pipeline

Check the last values generated by the github connector:

$ fluvio consume -dT 1 stars-forks
{"stars":1770,"forks":138}

Produce a new value

$ fluvio produce stars-forks
> {"stars":1769,"forks":138}
OK

An alert with 🌟 1769 will show-up in your slack channel.

Company

Resources