Sheriff: Permissions Bot

This bot, when deployed as a Heroku app and configured correctly, is capable of controlling permissions across GitHub, Slack, Heroku, and GSuite. It also actively monitors and alerts you to suspicious or unexpected activity on GitHub.

How It Works

Using a combination of webhooks and a YAML configuration file, Sheriff will automatically control your permissions and access controls across GitHub, Slack, Heroku, and GSuite. (Slack and GSuite plugins are optional and disabled by default).

It will post to a designated Slack channel every time it updates any permission setting or any time it detects potentially suspect actions including new deploy keys with write access, tag deletion or release branch deletion.

If you have an organization with a lot of repositories and/or org members using Sheriff can help ensure your organization remains secure and transparent.

Deployment

We recommend deploying this as a Heroku app (this is how Electron has deployed it), although you can use another deployment strategy if you want. There are three core components to Sheriff, all of which need to be configured for it to work:

The Webhook

Deploy the webhook to Heroku with this button ➡️

To the run the webhook server, you need to start the main Sheriff entry point.

npm start

You then need to create a webhook for your entire organization; you can do this on your organization's GitHub webhooks page:

https://github.com/organizations/{orgname}/settings/hooks/new

You want to specify the following options:

• Payload URL - The deployed URL of your webhook server, e.g. https://my-sheriff.mysite.com
• Content type - application/json
• Secret - Generate a random and secure secret here and save it for later in the configuration
• Which events? - Choose "Send me everything"

Finally, click "Add webhook".

The GitHub App

To manage GitHub instances, Sheriff requires you to create a GitHub App that gets installed in the desired Org.

The app needs the following OAuth scopes permitted:

Org:
administration:write
contents:read
metadata:read

Repo:
members:write
actions:write        # only if you use `vouched_ci`
pull_requests:read   # only if you use `vouched_ci`

Once created, you can generate and download a Private Key for the app, and supply it to Sheriff.

Before setting it as SHERIFF_GITHUB_APP_CREDS, you must pass it through a utility to change the format to what Octokit is expecting:

npx @electron/github-app-auth --creds={path-to-downloaded-private-key} --app-id={id-from-created-github-app}

The Cron Job

The actual permissions controller should be triggered every 10 minutes as a cron job. You can run this job with:

node lib/permissions/run.js --do-it-for-real-this-time

If you leave off the --do-it-for-real-this-time Sheriff will "dry run" and tell you what it would have done if you had let it run.

On Heroku you use the "Heroku Scheduler" addon to configure this cron job.

The Slack App

In order to provide realtime information on the actions Sheriff takes, we use a Slack app that sends messages to a channel. You'll need to create your own Slack App by following the instructions below.

1. Create a new Slack app on https://api.slack.com/apps - Name it whatever you like and choose your workspace as the development workspace
2. Go to "Incoming Webhooks" and enable it
3. Click "Add New Webhook to Workspace" and choose the channel you want Sheriff to post in to
4. Keep a note of the newly created Webhook URL as you'll need it later for configuration purposes.
5. Go to "OAuth & Permissions" and add the following OAuth scopes. usergroups:read, usergroups:write, users:read and users:read:email.
6. Follow the prompt to reinstall your app for the new OAuth scopes to take effect
7. Keep a note of the OAuth Access Token at the top of the page as you'll need it later for configuration purposes.

Configuration

Service Config

The following environment variables represent the configuration of the actual Sheriff deployment. For the permissions.yaml reference see the Permissions File section.

Permissions File

Your organization permissions are controlled through a config.yaml file stored in a .permissions repository in your GitHub organization. We keep that .permissions repository private but you can choose to keep it public if you wish. That repository needs a config.yaml file at the top level that is in the following format:

organization: <name of github org>
repository_defaults:
  # Whether repositories should have wikis enabled by default or not
  # For security reasons, you should consider defaulting this to false
  has_wiki: <boolean>
# Teams are not specific to a single platform; they are shared across GitHub, Slack and GSuite
teams:
  - name: <team name>
    # A list of members / maintainers of this GitHub team
    # Maintainer in GitHub conveys some extra permissions over the team (set description, avatar, etc.)
    members:
      - list
      - of
      - gh_usernames
    maintainers:
      - list
      - of
      - gh_usernames
    # Or don't provide members/maintainers and instead provide a list of other
    # teams to draw users from.  This doesn't set any parent/child relationship
    # rather it simply says:
    # for team of formation:
    #   self.members += team.members
    #   self.maintainers += team.maintainers
    # i.e. doing a union of members/maintainers of the formation teams to create
    # a new member list
    formation:
      - list
      - of
      - other
      - teams
    # Optional team properties
    # Human friendly display name for GSuite and Slack groups
    displayName: <string>
    # Hidden GitHub team? true=yes, false=no
    secret: <boolean>
    # Create a slack user group for this team
    # false=no, true=use name of team, string=custom_name
    # Used by the `slack` plugin
    slack: <boolean> | <string>
    # Create a GSuite group for this team
    # Leave undefined for "no"
    # Used by the `gsuite` plugin
    gsuite:
      # internal = only visible to other GSuite members
      # external = public facing group email address
      privacy: internal | external
repositories:
  - name: <repo name>
    teams:
      <team_name>: read | triage | write | maintain | admin
    external_collaborators:
      <gh_username>: read | triage | write | maintain | admin
    # Optional repository settings
    settings:
      # Wiki enabled? true=yes, false=no
      has_wiki: <boolean>
    # Public vs Private repository, no value is assumed to mean public
    visibility: public | private
    rulesets:
      - name-of-common-ruleset
      # Ruleset names must be unique for a given repository
      - name: <ruleset name>
        target: branch | tag
        # Optional enforcement status, defaults to "active"
        enforcement: disabled | active | evaluate
        # Optional bypass configuration
        bypass:
          # Team names that can bypass this ruleset
          teams: Array<string>
          # App ids that can bypass this ruleset
          apps: Array<number>
        ref_name:
          # Targeting ref for this ruleset, "~DEFAULT_BRANCH" is a magic
          # string that GitHub translates to the current default branch
          include: ["~DEFAULT_BRANCH"]
          # Optional targeting ref for this ruleset that is _excluded_ from the
          # include matching
          exclude: []
        # Which basic rules to apply as part of this ruleset
        # Possible / supported options are included and documented below
        rules:
          # Prevent refs matching this ruleset from being created
          - restrict_creation
          # Prevent refs matching this ruleset from being updated
          - restrict_update
          # Prevent refs matching this ruleset from being deleted
          - restrict_deletion
          # Prevent refs matching this ruleset from receiving force pushes
          - restrict_force_push
          # Prevent refs matching this ruleset from receiving merge commits
          - require_linear_history
          # Require refs matching this ruleset only receive signed commits
          - require_signed_commits
        # Can be simply "true" to require pull requests before changes are pushed to
        # matching refs. Advanced configuration can be provided via an object as documented
        # below
        require_pull_request:
          # All params are optional and map exactly to the github API
          # They are also all quite literal :)
          dismiss_stale_reviews_on_push: <boolean>
          require_code_owner_review: <boolean>
          require_last_push_approval: <boolean>
          required_approving_review_count: <number>
          required_review_thread_resolution: <boolean>
        # Status checks that are required to make changes to the matching ref
        require_status_checks:
            # The literal check name that is required, this is what shows up in the GitHub UI
          - context: <string>
            # The app ID that makes this check, this is optional in the GitHub API but required
            # in sheriff for good security practices
            app_id: <number>
common_rulesets:
  # Same structure as the object in `repositories[name].rulesets`
  # Used to deduplicate rulesets that you want to apply to multiple repos
  - <object>
# Optional, see "Vouched CI" below. Users are pinned by numeric GitHub user id,
# the login is cross-checked so a typo in the id can not vouch for someone else
vouched_ci:
  - login: <gh_username>
    id: <gh_user_id>

Vouched CI

GitHub can require a maintainer to approve GitHub Actions runs for pull requests from forks (Sheriff's forks_need_actions_approval repository setting turns that on for all external contributors). The vouched_ci list lets Sheriff approve those runs automatically for a small set of trusted people who are not collaborators on the repository, for example release engineers who work out of forks.

When a fork pull request run is created and gated on "Approve and run", GitHub emits a workflow_run requested event for it (reported as completed / action_required). Sheriff handles that event for pull_request runs from forks: because workflow_run.pull_requests is always empty for fork runs, it looks up the open pull requests for the run's owner:branch head and keeps the one whose head commit and head repository match the run. Sheriff then approves that single run only when all of the following hold:

• The user whose push created the run (the run's triggering_actor) is in vouched_ci with a matching id and login.
• Exactly one open pull request is pinned to the run's head commit and head repository, it comes from a fork, and it has at most 250 commits.
• Every commit in the pull request (not just the newest one) was authored and committed by that same user, and GitHub reports its signature as verified with reason valid, i.e. it was signed with a GPG/SSH/S-MIME key registered on that user's account.
• The pull request head still is the run's head commit after the commits were listed.

If any check fails Sheriff does nothing and logs why. Each workflow file gets its own run and its own event, so every run is verified and approved individually; each new push creates new runs and gets its own decision. Sheriff never approves "the pull request", only individual runs pinned to the commit it verified. Every approval is posted to Slack.

To be vouched, you must sign every commit you push with a key on your GitHub account (commits made through the GitHub web UI are signed by GitHub, not by you, and are not accepted), push your own commits yourself (nobody else's, including "apply suggestion" or cherry-picked patches from other people), and rebase rather than merge branches other than the pull request's base into your branch. Your numeric user id is available at https://api.github.com/users/<login>.

Vouched users can run arbitrary code in your fork pull request CI, treat the list like write access to the repository's Actions runners. Using this feature requires the GitHub App to have the actions:write and pull_requests:read repository permissions in addition to the ones listed above; the org-wide webhook already delivers workflow_run events when configured to send everything.

Generating your initial configuration

You can generate a permissions file for the current state of your org using the generate helper script.

node lib/permissions/generate.js

Please note you may want to edit this generated YAML file:

• All org owners are considered maintainers of the teams they are in, this may be semantically incorrect
• No GSuite or slack configuration is included in the generated file
• You may want to use the formation property to declare larger teams instead of listing all members individually

However in theory running Sheriff immediately on this generated file should result in a no-op run.

Deployment Recommendations

You should have alerting set up in case the cron job fails. Occasionally, it will fail due to an unexpected state on GitHub or an incorrect/incomplete permissions file.

Known Limitations

• Sheriff is not currently capable of inviting people to your org
  • Before adding them to the permissions file, ensure you've added them to the org.
• Sheriff will not remove people from your org, if your has "default member permissions" you should ensure users are manually removed when appropriate