pr-sage

An AI PR reviewer built to eliminate review noise — not add to it.

Most AI reviewers re-review the whole PR on every push and repeat themselves until the team mutes them. pr-sage is designed around the opposite goal: say each thing once, follow your team's rules, and stay silent when there is nothing new to say.

• 🔇 Zero duplicate comments. Findings carry content fingerprints — a line shift won't make the same comment appear twice, and re-runs post nothing when nothing changed.
• ✅ Finding lifecycle. Follow-up reviews report which findings remain unresolved and which were fixed.
• ⏩ Incremental by default. After the first review, only the commits you pushed since get reviewed. Less noise, fewer tokens.
• 📏 Your rules, not generic advice. .pr-sage.json instructions plus automatic CLAUDE.md/CONTRIBUTING.md injection make reviews follow team conventions.
• 🚦 A quality gate, not just commentary. --fail-on critical blocks merges; --event auto approves clean PRs and requests changes on real problems.
• 🖥️ Reviews before the PR exists. pr-sage local reviews your git diff pre-push — no server, no PR, no GitHub token.
• 🔐 Your keys, your data path. No server, nothing stored; code goes only to the provider you choose — Claude, OpenAI, or Gemini — or never leaves your machine at all with a self-hosted OpenAI-compatible endpoint (Ollama, vLLM, LM Studio). See SECURITY.md.

Ships as a CLI, a GitHub Action, and a TypeScript library.

Quick start

The fastest path — an interactive wizard that writes your config and the GitHub Action workflow, and tells you exactly which secret to register:

npx pr-sage init
npx pr-sage doctor

Or by hand (CLI):

export GITHUB_TOKEN=ghp_...
export ANTHROPIC_API_KEY=sk-ant-...

npx pr-sage review --repo owner/name --pr 123

Preview without posting anything:

npx pr-sage review --repo owner/name --pr 123 --dry-run

Review your local changes before pushing (no PR, no GitHub token needed):

npx pr-sage local --base main            # diff vs main
npx pr-sage local --staged --fail-on critical   # gate staged changes

Review in Korean with a different provider:

export OPENAI_API_KEY=sk-...
npx pr-sage review --repo owner/name --pr 123 --provider openai --locale Korean

Options

Required environment variables: GITHUB_TOKEN (with pull_requests: write), plus the API key for your provider (ANTHROPIC_API_KEY, OPENAI_API_KEY, or GEMINI_API_KEY).

On repeat runs (e.g. new commits pushed to the PR), pr-sage reviews only the commits pushed since its last review (incremental mode), skips findings it has already commented, and posts nothing when there is nothing new — no duplicate-comment spam, no wasted tokens. If your repo has a CLAUDE.md or CONTRIBUTING.md, it is automatically injected as review context (disable with "repoContext": false). GitHub Enterprise works out of the box via $GITHUB_API_URL or the githubApiUrl config field.

Each run prints its token usage to stderr (LLM usage: N call(s), X input / Y output tokens) so cost stays visible. Every summary also reports review coverage. Partial reviews never auto-approve a PR. Use --fail-on-incomplete when incomplete coverage must fail the CI quality gate.

Self-hosted / local models

Point the OpenAI provider at any OpenAI-compatible server and private code never leaves your machine — no API key required:

ollama pull qwen2.5-coder:14b
OPENAI_BASE_URL=http://localhost:11434/v1 \
  npx pr-sage review --repo owner/repo --pr 123 --provider openai --model qwen2.5-coder:14b

Works the same with vLLM, LM Studio, or any gateway that speaks the OpenAI chat completions API. For GitHub Actions, init --provider self-hosted generates a workflow for a self-hosted runner; localhost must refer to that runner, not a GitHub-hosted VM.

Configuration file

Put a .pr-sage.json in the directory you run from (CLI flags override it):

{
  "provider": "anthropic",
  "locale": "auto",
  "exclude": ["src/generated/**", "**/*.snap"],
  "paths": ["packages/web/**"],
  "minSeverity": "suggestion",
  "failOn": "critical",
  "context": "full",
  "maxTokensPerRun": 200000,
  "failOnIncomplete": true,
  "skipLabels": ["skip-review"],
  "verify": true,
  "verifyProvider": "gemini",
  "verifyModel": "gemini-flash-latest",
  "verifyFailure": "abort",
  "checkRun": true,
  "pathRules": [
    {
      "paths": ["packages/api/**"],
      "instructions": "Check public API backward compatibility.",
      "minSeverity": "suggestion",
      "failOn": "warning"
    }
  ],
  "instructions": "We use Result<T, E> for error handling — flag thrown exceptions in domain code. Prefer early returns over nested conditionals."
}

instructions is injected into the review prompt — use it for team conventions the reviewer should enforce.

GitHub Action

# .github/workflows/pr-sage.yml
name: AI Review
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write
  checks: write

concurrency:
  group: pr-sage-${{ github.event.pull_request.number }}
  cancel-in-progress: true

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      # Read .pr-sage.json from trusted base code, never PR-controlled code.
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.base.sha }}
          persist-credentials: false
      - uses: Kyeom1997/pr-sage@v1
        with:
          provider: anthropic
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
          locale: Korean
          fail-on: critical   # optional: block merge on critical findings
          fail-on-incomplete: "true"
          check-run: "true"

To upload SARIF, add security-events: write and set sarif: "true". The Action also exposes first-class paths, max-tokens, verify-provider, verify-model, verify-failure, and openai-base-url inputs.

The base-SHA checkout is deliberate: configuration and repository guidelines must come from trusted base code. The PR diff itself is always fetched through the GitHub API, and pr-sage rechecks the head SHA immediately before posting so a slow review cannot comment on a superseded commit.

Measuring it

scripts/bench.mjs runs pr-sage (dry, nothing posted) over recent merged PRs of any public repo and records findings, severity mix, latency, and token usage, plus a labeling sheet for computing the valid-review rate:

node scripts/bench.mjs --repos fastify/fastify --per-repo 5 --provider gemini

--mode recall measures the other axis: it picks merged PRs that received human review comments, reviews each PR's first commit (the state humans reviewed), and reports how many human-flagged locations pr-sage also flags — with a side-by-side sheet for manual verification.

The Quality Benchmark workflow can run either benchmark manually and uploads the generated JSON and labeling sheet as workflow artifacts.

Measured (2026-07, 28 PRs)

28 recently merged PRs across fastify, hono, GitHub CLI, and Vite (gemini-flash-lite, zero run failures):

• 25/28 (89%) produced zero comments — quiet on code that had already passed human review. That silence is the point: no noise on clean diffs.
• The other 3 PRs got 7 findings. Verifying each claim against the actual diff: 3/7 valid overall, 2/3 for warning severity — the noise concentrated in the suggestion tier.
• Re-running those PRs with --verify kept exactly the 2 diff-confirmed-valid findings (a real GPG-signing regression question in a deployment workflow) and rejected every invalid one.
• Median 2.3 s and ~3.5 k input tokens per PR (≈ $0.01 for all 28 PRs at flash-lite list pricing).

Raw results and the per-finding verification notes live in bench-results/. Caveats: small sample, and merged-PR sampling measures noise, not recall — a detection benchmark (reviewing pre-review commits of PRs that later got human fixes) is future work.

How it works

1. Fetches the PR metadata and per-file patches from the GitHub API.
2. Annotates both sides of the diff — added/context lines with new-file numbers, deleted lines with old-file numbers — so findings can anchor to removed code too (e.g. "this deleted validation was still needed"). Lockfiles/build artifacts are filtered out.
3. Asks the LLM for a structured review (JSON schema — no parsing heuristics): summary + findings with path, line, severity, and an optional single-line suggestion.
4. Validates every finding at runtime (zod) and against the diff (GitHub rejects reviews that comment on lines outside the diff), demotes unsafe multi-line suggestions, skips findings already posted by a previous run, retries on provider rate limits, and posts one review: inline comments + summary.

Severities: 🔴 critical · 🟡 warning · 🔵 suggestion · ⚪ nitpick. Safe single-line fixes are posted as GitHub suggestion blocks you can commit with one click.

Programmatic use

import { GitHubClient, createProvider, runReview } from "pr-sage";

const github = new GitHubClient(process.env.GITHUB_TOKEN!, "owner", "repo");
const pr = await github.fetchPullRequest(123);
const provider = createProvider("anthropic");
const { result } = await runReview(provider, pr, {
  locale: "English",
  exclude: [],
  batchCharBudget: 80_000,
  log: console.error,
});

Security & privacy

Reviewing code means sending diffs (and optionally full files) to the LLM provider you choose — read SECURITY.md for the exact data flow, provider policy links, prompt-injection mitigations, and token scope guidance before enabling this on private repositories. The GitHub Action executes the bundled code committed at the tag you pin (no install step), and npm releases carry provenance.

License

MIT