octocov is a toolkit for collecting code metrics (code coverage, code to test ratio, test execution time and your own custom metrics).

Key features of octocov are:

• Useful both as a CI tool and as a CLI tool
• Support multiple coverage report formats.
• Support multiple code metrics.
• Support for even generating coverage report badges.
• Have a mechanism to aggregate reports from multiple repositories.

Getting Started

On GitHub Actions

:octocat: GitHub Actions for octocov is here !!

First, run test with coverage report output.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

And generete .octocov.yml to your repository.

$ octocov init
.octocov.yml is generated

And set up a workflow file as follows and run octocov on GitHub Actions.

# .github/workflows/ci.yml
name: Test

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-go@v4
        with:
          go-version-file: go.mod
      - name: Run tests with coverage report output
        run: go test ./... -coverprofile=coverage.out
      - uses: k1LoW/octocov-action@v1

Then, octocov comment the report of the code metrics to the pull request.

It is also possible to add reports to GitHub Actions Job Summaries by editing .octocov.yml.

It can also be inserted into the body of a pull request.

Note that only pull requests from the same repository can be commented on (Reporting to GitHub Actions Job Summaries is permitted). This is because the workflow token of a forked pull request does not have write permission.

On Terminal

octocov acts as a code metrics viewer on the terminal.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

And run octocov ls-files , octocov view [FILE...] and octocov diff [REPORT_A] [REPORT_B]

Usage example

Comment report to pull request

By setting comment:, comment the reports to pull request.

# .octocov.yml
comment:
  hideFooterLink: false # hide octocov link

octocov checks for "Code Coverage" by default. If it is running on GitHub Actions, it will also measure "Test Execution Time".

If you want to measure "Code to Test Ratio", set codeToTestRatio:.

comment:
codeToTestRatio:
  code:
    - "**/*.go"
    - "!**/*_test.go"
  test:
    - "**/*_test.go"

By setting report: ( report.path: or report.datastores ) and diff: ( diff.path: or diff.datastores ) additionally, it is possible to show differences from previous reports as well.

comment:
  updatePrevious: true
report:
  datastores:
    - artifact://${GITHUB_REPOSITORY}
diff:
  datastores:
    - artifact://${GITHUB_REPOSITORY}

The breakdown behind that comparison is folded into a Details section. Set comment.expandDetails: ( or the same key under summary: or body: ) to render it, and the Metadata of each custom metric set, already open.

The coverage values of the comment, the job summary and the pull request body link to a page where the report can be read. viewer: says which page that is. The file names keep pointing at the source on GitHub.

Check for acceptable score

By setting coverage.acceptable:, the condition of acceptable coverage is specified.

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
coverage:
  acceptable: 60%

$ octocov
Error: code coverage is 54.9%. the condition in the `coverage.acceptable:` section is not met (`60%`)

By setting codeToTestRatio.acceptable:, the condition of acceptable "Code to Test Ratio" is specified.

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
codeToTestRatio:
  acceptable: 1:1.2
  code:
    - "**/*.go"
    - "!**/*_test.go"
  test:
    - "**/*_test.go"

$ octocov
Error: code to test ratio is 1:1.1, the condition in the `codeToTestRatio.acceptable:` section is not met (`1:1.2`)

By setting testExecutionTime.acceptable:, the condition of acceptable "Test Execution Time" is specified (on GitHub Actions only) .

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
testExecutionTime:
  acceptable: 1 min

$ octocov
Error: test execution time is 1m15s, the condition in the `testExecutionTime.acceptable:` section is not met (`1 min`)

Generate report badges self.

By setting *.badge.path:, generate badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg

# .octocov.yml
codeToTestRatio:
  badge:
    path: docs/ratio.svg

# .octocov.yml
testExecutionTime:
  badge:
    path: docs/time.svg

You can display the coverage badge without external communication by setting a link to this badge image in README.md, etc.

# mytool

![coverage](docs/coverage.svg) ![coverage](docs/ratio.svg) ![coverage](docs/time.svg)

Push report badges self.

By setting push:, git push report badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg
push:

Store report to datastores

By setting report:, store the reports to datastores and local path.

# .octocov.yml
report:
  datastores:
    - github://owner/coverages/reports
    - s3://bucket/reports

The report of the default branch is stored under the repository itself, and the report of any other ref is stored beside it under a key naming that ref. A report is taken as the report of a pull request only on a pull request event, so a push to a branch that is the head of an open pull request is stored as the report of that branch. GitHub Actions Artifacts are named differently, see GitHub Actions Artifacts.

owner/repo/report.json                  # the default branch
owner/repo/refs/pull/123/report.json    # pull request #123
owner/repo/refs/heads/feat/x/report.json

diff.datastores: reads the report of the base branch on a pull request, so the comparison shown is against the branch the pull request merges into and leaves out whatever that branch and the default branch differ by. Where no datastore holds a report of the base branch, as when only the default branch reports, the report of the default branch is read instead. Any other run is compared against the default branch. The pull requests reporting as well changes none of this, so reporting from every run is the default, and report.if: is_default_branch is only needed to keep the reports of the other refs out of a datastore, which is worth doing for github://, where each report is a commit that stays.

# .octocov.yml
report:
  path: path/to/report.json

Supported datastores

• GitHub repository
• GitHub Actions Artifacts
• Amazon S3
• Google Cloud Storage (GCS)
• BigQuery
• Local

Central mode

By enabling central:, octocov acts as a central repository for collecting reports ( example ).

# .octocov.yml for central mode
central:
  root: . # root directory or index file path of collected coverage reports pages. default: .
  reports:
    datastores:
      - bq://my-project/my-dataset/reports # datastore paths (URLs) where reports are stored. default: local://reports
  badges:
    datastores:
      - local://badges # directory where badges are generated.
  push: # enable self git push

Where a report is collected from a GitHub Actions artifact ( artifact:// ), the badges of its row link to octocov.dev, and the markdown offered under Copy status badge markdown carries the same link. The pages there read a report out of the artifacts of the repository it describes, so collecting from one is what says the page is there to open. A report collected from any other datastore, and every report of a central repository running on GitHub Enterprise Server, gets plain images instead.

Merging reports from multiple GitHub Action jobs

This example illustrates how to merge coverage reports from multiple jobs. It is useful when you are testing code branches for multiple targets (e.g. test sharding, runtime version, dependency version, OS, etc).

# .github/workflows/ci.yml
name: merge coverages from multiple jobs

on:
  push:
    branches:
      - main
      - master
  pull_request:

jobs:
  test:
    strategy:
      fail-fast: true
      matrix:
        shard: [1, 2, 3]
    runs-on: ubuntu-latest
    permissions:
      contents: read # to checkout
    steps:
      - uses: actions/checkout@v4

      - run: YOUR TEST HERE

      - name: Upload coverage to workspace
        uses: actions/upload-artifact@v4
        with:
          path: lcov.info # adjust this path to your coverage file
          name: octocov-${{ matrix.shard }}
          if-no-files-found: error

  coverage-aggregation:
    runs-on: ubuntu-latest
    permissions:
      contents: read # to checkout
      pull-requests: write # for octocov to post comments
      actions: read # for octocov to parse test execution time
    needs:
      - test
    steps:
      - uses: actions/checkout@v4

      - uses: actions/download-artifact@v5
        with:
          pattern: octocov-*

      - uses: k1LoW/octocov-action@v1

# .octocov.yml
codeToTestRatio:
  code:
    - "**/*.go"
    - "!**/*_test.go"
  test:
    - "**/*_test.go"
coverage:
  paths:
    # Name of artifacts uploaded by the test-running jobs.
    # lcov.info (or equivalent) will be downloaded in the directory named after the below.
    - "octocov-1"
    - "octocov-2"
    - "octocov-3"
comment:
  if: is_pull_request
  hideFooterLink: true
  deletePrevious: true
summary:
  if: true
report:
  datastores:
    - artifact://${GITHUB_REPOSITORY}
diff:
  datastores:
    - artifact://${GITHUB_REPOSITORY}

Supported datastores

• GitHub repository
• GitHub Actions Artifacts
• Amazon S3
• Google Cloud Storage (GCS)
• BigQuery
• Local

View code coverage report of file

octocov ls-files command can be used to list files logged in code coverage report.

octocov view (alias: octocov cat) command can be used to view the file coverage report.

Configuration

repository:

The name of the repository.

It should be in the format owner/repo.

By default, the value of the environment variable GITHUB_REPOSITORY is set.

In case of monorepo, code metrics can be reported to datastore separately by specifying owner/repo/project-a or owner/repo@project-a.

repository: k1LoW/octocov

timeout:

Timeout for octocov execution. (default: 30sec)

timeout: 5min

coverage:

Configuration for code coverage.

coverage.path:

coverage.path: has been deprecated. Please use coverage.paths: instead.

coverage.paths:

The path to the coverage report file.

If no path is specified, the default path for each coverage format will be scanned.

coverage:
  paths:
    - tests/coverage.xml

coverage.exclude:

Exclude files from the coverage report. Patterns are matched using doublestar glob syntax.

octocov normalizes coverage file paths to git-root-relative paths before matching, so you can write patterns based on the repository directory structure regardless of coverage format.

coverage:
  exclude:
    - "internal/database/db/*.go"
    - "proto/**/*.pb.ts"

For backward compatibility, patterns are also matched against the original paths produced by the coverage tool (e.g., Go module paths like github.com/owner/repo/pkg/*.go).

coverage.acceptable:

acceptable coverage condition.

coverage:
  acceptable: 60%

coverage:
  acceptable: current >= 60% && diff >= 0.5%

The variables that can be used are as follows.

coverage:
  acceptable: patch >= 80%

patch is measured over the changed lines that the coverage report instruments, so changed lines that no coverage format instruments (blank lines, comments, declarations, ...) are not counted as uncovered.

patch is measured per changed line, while current and prev are measured in whatever unit the coverage format instruments. These coincide for line-based formats (LCOV, SimpleCov, Clover, Cobertura, JaCoCo), but not for Go coverage, which is statement-based. A single statement spanning several changed lines therefore weighs more in patch than it does in current/prev.

patch requires pull request context: octocov fetches the list of changed files and lines for the current pull request via the GitHub API. If that context is not available (e.g. not running against a pull request, or the GitHub API is not accessible), or if the pull request changed no lines that the coverage report instruments, patch is treated as 100% rather than failing the build.

Only the patch part of the condition is relaxed this way. If you combine patch with current/prev/diff in one expression (e.g. current >= 80% && patch >= 70%), the current >= 80% part keeps being enforced even when patch coverage cannot be measured.

Files that the GitHub API does not return a diff for are excluded from patch coverage, because the changed lines of those files cannot be determined. This happens for binary files, for files whose diff is too large for the API to include, and for files beyond the limit of the API response (up to 3000 files for a pull request, and up to 300 files when comparing against the default branch outside of a pull request). Such files lower neither the covered lines nor the total lines of patch.

The changed lines are numbered as the lines of the commit the coverage was measured on. On a pull_request run that is the merge commit GitHub creates for the pull request, so they are taken from the diff of that merge commit against its first parent rather than from the diff of the pull request itself, whose line numbers differ wherever the base branch has changed a file above the lines the pull request changed. When the commit is neither the head of the pull request nor that merge commit, as on pull_request_target, the lines of the pull request are used as they are. The same goes for the files the diff of the merge commit leaves out once it reaches its limit of 300 files: the list of changed files still comes from the pull request, so such files stay in patch, numbered as the lines of the pull request. Whenever changed lines are left numbered that way, octocov says so and why in the job log.

The file coverage table that octocov reports for a pull request (in the pull request comment, the job summary, and the pull request body) includes a Patch Coverage column, and its heading shows the overall patch coverage. This needs no configuration and is independent of coverage.acceptable:, so the column appears even when the condition does not reference patch. When the pull request changed no line that the coverage report instruments, the column is dropped instead of showing - for every file.

octocov diff [REPORT_A] [REPORT_B] --patch also prints the file coverage table of the comparison, including its patch coverage column, fetching the changed files and lines via the GitHub API. Since --patch asks for that data explicitly, it fails when the data cannot be fetched. A pull request that touches no instrumented file is not a failure, so that case reports why the table is missing and still exits successfully. Without --patch, octocov diff makes no GitHub API calls, as before.

It is also possible to omit the expression as follows

coverage.badge:

Set this if want to generate the badge self.

coverage.badge.path:

The path to the badge.

coverage:
  badge:
    path: docs/coverage.svg

coverage.badge.label: coverage.badge.labelColor:

The text and the background color of the left side of the badge. default: coverage and #24292E.

A color is a hex RGB notation ( #97CA00 ) or one of the shields.io color names ( brightgreen green yellowgreen yellow orange red blue grey lightgrey ).

coverage:
  badge:
    path: docs/coverage.svg
    label: cov
    labelColor: "#24292E"

coverage.badge.icon:

The icon image of the badge. A path relative to the config file, or a base64 encoded data: URI. none renders the badge without an icon. default: the octocov icon.

coverage:
  badge:
    path: docs/coverage.svg
    icon: .github/badge-icon.svg

coverage.badge.colors:

The color of the right side of the badge, as the conditions it is picked by. The color of the first met condition is used, and an entry without if: is met by anything.

The condition is written the same way as coverage.acceptable:, with the measured coverage as current.

coverage:
  badge:
    path: docs/coverage.svg
    colors:
      - if: current >= 80%
        color: green
      - if: 60%           # `current >= 60%`
        color: yellowgreen
      - color: red

When colors: is not set, or when no condition is met, the built-in thresholds are used ( >= 80% green, >= 60% yellowgreen, >= 40% yellow, >= 20% orange, otherwise red ).

coverage.if:

Conditions for measuring code coverage.

coverage:
  if: is_default_branch

codeToTestRatio:

Configuration for code to test ratio.

codeToTestRatio.code: codeToTestRatio.test:

Files to count.

codeToTestRatio:
  code: # files to count as "Code"
    - "**/*.go"
    - "!**/*_test.go"
  test: # files to count as "Test"
    - "**/*_test.go"

codeToTestRatio.acceptable:

acceptable ratio condition.

codeToTestRatio:
  acceptable: 1:1.2

codeToTestRatio:
  acceptable: current >= 1.2 && diff >= 0.0

The variables that can be used are as follows.

It is also possible to omit the expression as follows

codeToTestRatio.badge:

Set this if want to generate the badge self.

codeToTestRatio.badge.path:

The path to the badge.

codeToTestRatio:
  badge:
    path: docs/ratio.svg

codeToTestRatio.badge.label: codeToTestRatio.badge.labelColor: codeToTestRatio.badge.icon: codeToTestRatio.badge.colors:

The appearance of the badge, written the same way as coverage.badge:. The condition of colors: takes the measured ratio as current, written the same way as codeToTestRatio.acceptable:.

codeToTestRatio:
  badge:
    path: docs/ratio.svg
    colors:
      - if: 1:1.2
        color: green
      - color: red

The built-in thresholds are used when colors: is not set or when no condition is met ( >= 1.2 green, >= 1.0 yellowgreen, >= 0.8 yellow, >= 0.6 orange, otherwise red ).

codeToTestRatio.if:

Conditions for measuring code to test ratio.

codeToTestRatio:
  if: is_default_branch

testExecutionTime:

Configuration for test execution time.

testExecutionTime.acceptable

acceptable time condition.

testExecutionTime:
  acceptable: 1min

testExecutionTime:
  acceptable: current <= 1min && diff <= 1sec

The variables that can be used are as follows.

It is also possible to omit the expression as follows

testExecutionTime.steps

The name of the step to measure the execution time.

testExecutionTime:
  steps:
    - Run test
    - Run slow test

Steps with a matching name are collected from every job in the workflow run, and their execution times are merged (overlapping time ranges are counted only once). This makes it possible to measure the total test execution time when the tests are split across multiple jobs, for example by a GitHub Actions matrix. See Merging reports from multiple GitHub Action jobs.

If not specified, the step where the coverage report file is generated is used as the measurement target.

testExecutionTime.badge

Set this if want to generate the badge self.

testExecutionTime.badge.path

The path to the badge.

testExecutionTime:
  badge:
    path: docs/time.svg

testExecutionTime.badge.label: testExecutionTime.badge.labelColor: testExecutionTime.badge.icon: testExecutionTime.badge.colors:

The appearance of the badge, written the same way as coverage.badge:. The condition of colors: takes the measured time as current, written the same way as testExecutionTime.acceptable:.

testExecutionTime:
  badge:
    path: docs/time.svg
    colors:
      - if: 1min
        color: green
      - color: red

The built-in thresholds are used when colors: is not set or when no condition is met ( < 5min green, < 10min yellowgreen, < 15min yellow, < 20min orange, otherwise red ).

testExecutionTime.if:

Conditions for measuring test execution time.

testExecutionTime:
  if: is_pull_request

push:

Configuration for git push files self.

push.if:

Conditions for pushing files.

# .octocov.yml
push:
  if: is_default_branch

The variables available in the if section are here.

push.message:

message for commit.

# .octocov.yml
push:
  message: Update by octocov [skip ci]

comment:

Set this if want to comment report to pull request

comment.hideFooterLink:

Hide footer octocov link.

comment:
  hideFooterLink: true

comment.expandDetails:

Expand the collapsible sections of the report ( the Details of the comparison with the previous report, and the Metadata of each custom metric set ) instead of rendering them folded.

The Details section is rendered only when a previous report was found to compare against. The file coverage table folds itself once the pull request touches enough files, which this does not change, since that fold is what keeps the comment inside what GitHub will render.

comment:
  expandDetails: true

comment.deletePrevious:

Delete previous code metrics report comments instead of hiding them

comment:
  deletePrevious: true

comment.updatePrevious:

Update the previous code metrics report comment instead of creating a new one.

comment:
  updatePrevious: true

This is useful when you want octocov to keep a single managed pull request comment up to date.

comment.message:

Add message to code metrics report comments.

comment:
  message: See [coverage html](https://github.com/k1LoW/octocov).

comment.if:

Conditions for commenting report.

# .octocov.yml
comment:
  if: is_pull_request

The variables available in the if section are here.

summary:

Set this if want to add report to job summary page.

summary.hideFooterLink:

Hide footer octocov link.

summary:
  hideFooterLink: true

summary.expandDetails:

Expand the collapsible sections of the report ( the Details of the comparison with the previous report, and the Metadata of each custom metric set ) instead of rendering them folded.

summary:
  expandDetails: true

summary.message:

Add message to report.

summary:
  message: See [coverage html](https://github.com/k1LoW/octocov).

summary.if:

Conditions for adding report to job summary page.

# .octocov.yml
summary:
  if: true

The variables available in the if section are here.

body:

Set this if want to insert report to body of pull request.

body.hideFooterLink:

Hide footer octocov link.

body:
  hideFooterLink: true

body.expandDetails:

Expand the collapsible sections of the report ( the Details of the comparison with the previous report, and the Metadata of each custom metric set ) instead of rendering them folded.

body:
  expandDetails: true

body.message:

Add message to report.

body:
  message: See [coverage html](https://github.com/k1LoW/octocov).

body.if:

Conditions for inserting report body of pull request.

# .octocov.yml
body:
  if: is_pull_request

The variables available in the if section are here.

viewer:

Where the values in the tables of the comment, the job summary and the pull request body link to. It is one setting for the whole run, since the page a report can be read on does not depend on which of them the table is written to.

viewer: octocov.dev   # the octocov.dev pages
viewer: artifact      # a single page of HTML octocov renders and uploads as an artifact
viewer: none          # no links

Each of these is shorthand for a map with only type in it.

viewer:
  type: artifact

When viewer: is not set, a public repository on github.com whose report.datastores: has artifact:// links to octocov.dev, and everything else ( a private repository, a report stored in no artifact, GitHub Enterprise Server ) links to artifact. A run that stores no report of its own in an artifact, as a pull request run where report.if: holds the storing back, links to artifact too, since the page needs only the report it is compared against to be stored. none is only ever set explicitly.

octocov.dev and artifact link the overall coverage and the coverage of each file, and nothing else.

viewer: octocov.dev

The values link to octocov.dev, where the report stored in a GitHub Actions artifact can be browsed. Each overall coverage opens the report of the ref it describes, so the compared column opens the base branch or the default branch it was read for and the current one opens the pull request, and the coverage of each file opens that file. The compared column is linked only when diff.datastores: read it out of an artifact, since that is what the page it would open serves, and no column is linked at all unless this run stores its own report in one.

Opening the report of a private repository there means granting octocov.dev access to that repository.

viewer: artifact

On a pull request run, octocov renders the changes of the pull request as a single page of HTML ( the changed hunks with the coverage of the base and the head beside them, plus the files whose coverage moved while their code did not ) and uploads it as an artifact that opens in the browser. Anyone signed in to GitHub who can read the repository can open it, and signing in is required even on a public repository, as it is for every artifact.

• The overall coverage links to the page, and the coverage of each file links to the card of that file on it. A file the page drew no card for, since it stops drawing cards once a page holds enough of them, links to the page itself. The compared column gets no link, since the page shows both reports side by side.
• The hunks are taken the way the changed lines of patch are, from the diff of the merge commit the coverage was measured on against its first parent, so the coverage beside each line is that line's. The base column is drawn when the compared report was taken at the commit the old side of that diff is numbered as. Where the changed lines are left numbered as the pull request's, as on pull_request_target, the page draws no coverage of the head beside the hunks and says why.
• The page is rendered only when the comment, the job summary or the pull request body is written, and only when there is a previous report to compare against ( see diff: ). A run on a branch gets no page and no links.
• The artifact is named after the report, octocov-report[-<key>]@refs_pull_<number>.html, and a re-run of the workflow names its page octocov-report[-<key>]@refs_pull_<number>@attempt_<attempt>.html, since a run holds one artifact of a name. Once the comment, the job summary and the body are all written, the pages earlier attempts of this run and the first attempts of earlier runs of the same pull request uploaded are deleted, which needs actions: write. Without it they are left until their retention runs out, and so is a page an earlier run uploaded on a re-run of its own, since its name carries an attempt the next run cannot know.
• On GitHub Enterprise Server, where an artifact is always a zip archive, the page is uploaded zipped.
• The badges of the central mode get no link, since the page has no URL that stays the same from one run to the next.

viewer.type: custom

A viewer of your own. Each key under viewer.links: is an expr-lang expression evaluated for every value of that kind, with the same variables as *.if:.

A key left out links nothing. The variables of *.if: are joined by the following ones, and the pathEscape and queryEscape functions are available.

A string is the link, nil or an empty string is no link, and anything else is an error. The compared column is evaluated with the ref and commit of the compared report.

viewer:
  type: custom
  links:
    coverage: >-
      report.is_base ? nil
      : "https://coverage.example.com/" + env.GITHUB_REPOSITORY + "/pr/" + string(github.event.pull_request.number) + "/"
    coverageFile: >-
      report.is_base || file.path startsWith "gen/" ? nil
      : "https://coverage.example.com/" + env.GITHUB_REPOSITORY + "/pr/" + string(github.event.pull_request.number) + "/" + pathEscape(file.path) + ".html"

The links are written into the comment, which anyone who can read the pull request can read, so do not build them from env values that are not meant to be public.

The badges of the central mode link by the same expressions, coverage for the coverage badge, codeToTestRatio for the code to test ratio badge and testExecutionTime for the test execution time badge.

diff:

Configuration for comparing reports.

diff.path:

Path of the report to compare.

diff:
  path: path/to/coverage.yml

diff:
  path: path/to/report.json

diff.datastores:

Datastores where the report to be compared is stored. The report of the base branch of a pull request is read where there is one, and the report of the default branch otherwise ( see Store report to datastores ).

diff:
  datastores:
    - local://.octocov # Use .octocov/owner/repo/report.json
    - s3://my-bucket/reports # Use s3://my-bucket/reports/owner/repo/report.json

diff.if:

Conditions for comparing reports

# .octocov.yml
diff:
  if: is_pull_request
  path: path/to/report.json

The variables available in the if section are here.

report:

Configuration for reporting to datastores.

report.path:

Path to save the report.

report:
  path: path/to/report.json

report.datastores:

Datastores where the reports are stored.

report:
  datastores:
    - github://owner/coverages/reports
    - s3://bucket/reports

GitHub repository

Use github:// scheme.

github://[owner]/[repo]@[branch]/[prefix]

Required environment variables:

• GITHUB_TOKEN or OCTOCOV_GITHUB_TOKEN
• GITHUB_REPOSITORY or OCTOCOV_GITHUB_REPOSITORY
• GITHUB_API_URL or OCTOCOV_GITHUB_API_URL (optional)

GitHub Actions Artifacts

Use artifact:// or artifacts:// scheme.

artifact://[owner]/[repo]/[artifactName]

• artifact://[owner]/[repo]/[artifactName]
• artifact://[owner]/[repo] ( default artifactName: octocov-report )

The report of every ref is stored under the artifact name as it is configured. Beside it, each run stores an artifact of metadata that points, by ID, at the artifact holding the report of the ref the run is on, and records the commit that ref was at, which on a pull request is its head rather than the merge commit measured. It is named octocov-metadata-, the artifact name, @ and the ref, with the characters an artifact name may not hold replaced by _ ( e.g. octocov-metadata-octocov-report@refs_heads_main, octocov-metadata-octocov-report@refs_pull_123 ).

diff.datastores: reads the report the metadata of the base branch points at, or of the default branch where the base branch has no report, and the central mode the one of the default branch. Where that metadata is not there, as for a report stored by an octocov that did not write it, the newest artifact of the name that a run on that branch uploaded, other than a run of a pull request whose head is that branch, is read instead.

Note that reporting to the artifact can only be sent from the GitHub Actions of the same repository.

Required environment variables:

• GITHUB_TOKEN or OCTOCOV_GITHUB_TOKEN
• GITHUB_REPOSITORY or OCTOCOV_GITHUB_REPOSITORY
• GITHUB_API_URL or OCTOCOV_GITHUB_API_URL (optional)

Amazon S3

Use s3:// scheme.

s3://[bucket]/[prefix]

Required permission:

• s3:PutObject

Required environment variables:

• AWS_ACCESS_KEY_ID or OCTOCOV_AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY or OCTOCOV_AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN or OCTOCOV_AWS_SESSION_TOKEN (optional)

Google Cloud Storage

Use gs:// scheme.

gs://[bucket]/[prefix]

Required permission:

• storage.objects.create
• storage.objects.delete

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

BigQuery

Use bq:// scheme.

bq://[project ID]/[dataset ID]/[table]

Required permission:

• bigquery.datasets.get
• bigquery.tables.get
• bigquery.tables.updateData

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

Datastore schema:

Datastore schema

If you want to create a table, execute the following command ( require bigquery.datasets.create ).

$ octocov migrate-bq-table

Mackerel

Note: Only works with report.datastores or central.reReport.datastores

Use mackerel:// or mkr:// scheme.

mackerel://[Service Name]

Required permission:

• read
• write

Required environment variables:

• MACKEREL_API_KEY or OCTOCOV_MACKEREL_API_KEY

Local

Use local:// or file:// scheme.

local://[path]

Example:

If the absolute path of .octocov.yml is /path/to/.octocov.yml

• local://reports ... /path/to/reports directory
• local://./reports ... /path/to/reports directory
• local://../reports ... /path/reports directory
• local:///reports ... /reports directory.

report.if:

Conditions for storing a report.

# .octocov.yml
report:
  if: env.GITHUB_REF == 'refs/heads/main'
  datastores:
    - github://owner/coverages/reports

The variables available in the if section are here.

*.if:

Note: It supports expr-lang/expr expressions.

The variables available in the if section are as follows

central:

Note: When central mode is enabled, other functions are automatically turned off.

central.root:

The root directory or index file ( index file example ) path of collected coverage reports pages. default: .

central:
  root: path/to

central.reports:

central.reports.datastores:

Datastore paths (URLs) where reports are stored. default: local://reports

central:
  reports:
    datastores:
      - local://reports
      - gs://my-gcs-bucket/reports

Use GitHub Actions Artifacts as datastore

When using GitHub Actions Artifacts as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - artifact://${GITHUB_REPOSITORY}

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - artifact://owner/repo
      - artifact://owner/other-repo
      - artifact://owner/another-repo
      [...]
  push:

Code metrics and badges of my open source projects using octocov central mode is here.

Template repositoty is here.

Use GitHub repository as datastore

When using the central repository as a datastore, perform badge generation via on.push.

# .octocov.yml
report:
  datastores:
    - github://owner/central-repo/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - github://owner/central-repo/reports
  push:

or

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - local://reports
  push:

Use Amazon S3 bucket as datastore

When using the S3 bucket as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - s3://my-s3-bucket/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - s3://my-s3-bucket/reports
  push:

Required permission (Central Repo):

• s3:GetObject
• s3:ListObject

Required environment variables (Central Repo):

• AWS_ACCESS_KEY_ID or OCTOCOV_AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY or OCTOCOV_AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN or OCTOCOV_AWS_SESSION_TOKEN (optional)

Use GCS bucket as datastore

When using the GCS bucket as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - gs://my-gcs-bucket/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - gs://my-gcs-bucket/reports
  push:

Required permission (Central Repo):

• storage.objects.get
• storage.objects.list
• storage.buckets.get

Required environment variables (Central Repo):

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

Use BigQuery table as datastore

When using the BigQuery table as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - bq://my-project/my-dataset/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - bq://my-project/my-dataset/reports
  push:

Required permission (Central Repo):

• bigquery.jobs.create
• bigquery.tables.getData

Required environment variables (Central Repo):

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

central.badges:

central.badges.datastores:

Datastore paths (URLs) where badges are generated. default: local://badges

central:
  badges:
    datastores:
      - local://badges
      - s3://my-s3-buckets/badges

central.badges.coverage: central.badges.codeToTestRatio: central.badges.testExecutionTime:

The appearance of the badges generated in central mode, written the same way as coverage.badge: except for path:, which central mode decides.

A report does not carry the configuration of the repository it was collected from, so the badges of every repository are generated as configured here.

central:
  badges:
    datastores:
      - local://badges
    coverage:
      icon: none
      colors:
        - if: 80%
          color: green
        - color: red

central.push:

Configuration for git push index file and badges self.

central.if:

Conditions for central mode.

# .octocov.yml
central:
  if: env.GITHUB_REF == 'refs/heads/main'
  reports:
    datastores:
      - s3://my-s3-bucket/reports

The variables available in the if section are here.

central.reReport:

Store collected reports in yet another datastores.

central.reReport.if:

Conditions for re storing reports.

central.reReport.datastores:

Datastores where the reports are re-stored.

Supported coverage report formats

octocov supports multiple coverage report formats.

And octocov searches for the default path for each format.

If you want to specify the path of the report file, set coverage.path

coverage:
  paths:
    - /path/to/coverage.txt

Go coverage

Default path: coverage.out

LCOV

Default path: coverage/lcov.info

Support SF DA only

SimpleCov

Default path: coverage/.resultset.json

Clover

Default path: coverage.xml

Cobertura

Default path: coverage.xml

JaCoCo

Default path: build/reports/jacoco/test/jacocoTestReport.xml

Supported code metrics

• Code Coverage
• Code to Test Ratio
• Test Execution Time (on GitHub Actions only)

Custom metrics

octocov accepts custom metrics in addition to the three supporting metrics.

Specify the path to the custom metrics JSON file in an environment variable prefixed with OCTOCOV_CUSTOM_METRICS_ to collect the code metrics at the same time.

The JSON schema for custom metrics can be found here.

If there are multiple custom metrics JSON files, specify each file path in a separate environment variable (example here) or combine the JSONs that satisfy the JSON schema into an array.

Custom metrics acceptable conditions

You can set acceptable conditions for custom metrics using the acceptables array within your JSON file.

JSON file format with acceptables:

{
  "key": "performance_metrics",
  "name": "Performance Metrics",
  "metrics": [
    {
      "key": "response_time",
      "name": "Response Time",
      "value": 250.0,
      "unit": "ms"
    },
    {
      "key": "throughput",
      "name": "Throughput",
      "value": 1500.0,
      "unit": "req/s"
    },
    {
      "key": "error_rate",
      "name": "Error Rate",
      "value": 0.5,
      "unit": "%"
    }
  ],
  "acceptables": [
    "current.response_time < 300",
    "current.throughput > 1000",
    "current.error_rate < 1.0",
    "current.response_time <= prev.response_time"
  ]
}

Available variables in acceptables conditions:

If conditions are not met:

$ octocov
Error: custom metrics "performance_metrics" not acceptable condition (`current.response_time < 300`)

Detecting pull request number

octocov detect pull request number following order.

1. Get pull request number from GITHUB_PULL_REQUEST_NUMBER or OCTOCOV_GITHUB_PULL_REQUEST_NUMBER.
2. Get pull request number from GITHUB_REF ( e.g. refs/pull/1/merge ).
3. Get branch name from GITHUB_REF ( e.g. refs/heads/branch/branch/name ) and detect pull request number using GitHub API.

Override environment variables

If an environment variable with prefix OCTOCOV_ is set, it is used as an unprefixed environment variable in octocov.

For example, if OCTOCOV_GITHUB_REF is set, it is handled as GITHUB_REF in octocov.

This feature allows environment variables that cannot normally be overridden to be changed on octocov.

Install

deb:

$ export OCTOCOV_VERSION=X.X.X
$ curl -o octocov.deb -L https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.deb
$ dpkg -i octocov.deb

RPM:

$ export OCTOCOV_VERSION=X.X.X
$ yum install https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.rpm

apk:

$ export OCTOCOV_VERSION=X.X.X
$ curl -o octocov.apk -L https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.apk
$ apk add octocov.apk

homebrew tap:

$ brew install k1LoW/tap/octocov

aqua:

$ aqua g -i k1LoW/octocov

manually:

Download binary from releases page

go install:

$ go install github.com/k1LoW/octocov@latest

docker:

$ docker pull ghcr.io/k1low/octocov:latest

---