Circuit Breaker

A GitHub Action that scores a pull request's review-effort / abandonment risk at creation time, before a human ever looks at it.

It's a direct reimplementation of the Gated Triage Policy proposed in:

Dao Sy Duy Minh et al., "Early-Stage Prediction of Review Effort in AI-Generated Pull Requests", MSR '26.

The finding this is built on

The paper analyzes 33,707 agent-authored PRs and finds a two-regime outcome: 28.3% merge almost instantly (agents are good at narrow, scoped tasks), while the rest enter iterative review and frequently get abandoned by the agent ("ghosting") once a maintainer pushes back with subjective feedback. This creates a hidden attention tax: maintainers can't tell which regime a PR is in until they've already sunk review time into it.

Their headline result is that this is predictable from structural signals available the instant the PR is opened — patch size, file count, and whether the PR states a plan — with a LightGBM model reaching AUC 0.957. Critically, a size-only baseline already reaches AUC 0.933, and CI-file touches and semantic PR-description content turn out to be non-causal confounds once agent identity is controlled for. Their practical recommendation (Section 7) isn't "deploy our classifier" — it's a lightweight Gated Triage Policy: flag PRs over 500 additions, fast-fail ones with no stated plan, enforce a 14-day abandonment timeout.

This repo implements that policy directly, as rule-based scoring — no model training, no dependency on their dataset, works on any repo immediately.

What it scores

At PR-open time, using only GET /pulls/{n} and GET /pulls/{n}/files:

Output is a transparent 0–100 score, a Low / Medium / High tier, a recommendation, and the list of rules that fired — so a maintainer reading the bot comment can see exactly why, not just a black-box probability.

Usage on another repo

You don't clone or copy anything from this repo. In the repo you want to triage, add a workflow file at .github/workflows/circuit-breaker.yml that references this action by owner/repo@ref:

name: Circuit Breaker
on:
  pull_request_target:
    types: [opened, synchronize, reopened]

permissions:
  pull-requests: write   # required: the action comments + labels PRs
  contents: read

jobs:
  triage:
    runs-on: ubuntu-latest
    steps:
      - uses: glassrun/circuit-breaker@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          fail-on-high-risk: "false"   # set "true" to hard-gate instead of just labeling

That's the whole integration. A few things to know:

• Token: secrets.GITHUB_TOKEN is the repo's own auto-generated Actions token — no PAT, no secret to create. It only needs the pull-requests: write permission declared above; GitHub scopes it to that single workflow run.
• Pinning: @v1 tracks the latest v1.x fix (the tag gets moved forward as bugs are fixed, same convention as actions/checkout@v4). For a fully reproducible pin, use a commit SHA instead: glassrun/circuit-breaker@<sha>.
• No actions/checkout needed. Unlike a local uses: ./ action (which is how this repo's own dogfood workflow references itself, and does need checkout — see below), a remote uses: owner/repo@ref action is fetched by the runner directly. This action never reads consuming-repo files anyway; it only calls the PR/files API and posts comments.
• Fork PRs work correctly, including getting a risk comment/label, via pull_request_target instead of pull_request. pull_request_target normally carries a well-known risk (a write-scoped token combined with checking out and running the fork's own code — a "pwn request"), but that risk doesn't apply here: this workflow never checks out or executes anything from the PR branch, only structural metadata read over the API. If you add steps to this job later, don't check out or run github.event.pull_request.head.sha — doing so would reintroduce that risk. If you'd rather avoid pull_request_target entirely, the safe fallback is the two-workflow pull_request + workflow_run artifact pattern from GitHub's own docs — more moving parts, not needed for what this action does.

See .github/workflows/circuit-breaker.yml for the version that dogfoods this action on itself.

Progressive rollout (CI/deps files first)

The paper's Ethical Implications section pairs the issue-linked exception above with a second mitigation: roll out enforcement narrow, starting with high-risk file types (CI configs, dependency manifests), before applying it repo-wide. Those paths are where a false positive is cheapest to catch and an unreviewed regression is most expensive, so they're a reasonable place to trial fail-on-high-risk: true before trusting it broadly. Scope the workflow trigger itself with paths::

on:
  pull_request_target:
    types: [opened, synchronize, reopened]
    paths:
      - '.github/workflows/**'
      - '.gitlab-ci.yml'
      - '.circleci/**'
      - 'Dockerfile'
      - 'docker-compose*.yml'
      - '**/package-lock.json'
      - '**/requirements*.txt'
      - '**/go.sum'

Drop the paths: filter once you've validated the gate's behavior and want it running on every PR.

Local / offline scoring

echo '{"pr": {...}, "files": [...]}' | python src/score_pr.py --local

pr and files are the raw shapes of GET /pulls/{n} and GET /pulls/{n}/files.

Development

python3 -m venv .venv && .venv/bin/pip install -r requirements.txt pytest
.venv/bin/python -m pytest

Known limitations (from the paper's own caveats)

• Correlational, not causal. The gate flags structural risk, not code quality. A well-planned 600-line refactor will still trip the size gate.
• Silent abandonment is a blind spot. The paper notes small PRs that look safe can still stall on subjective feedback — this policy, like theirs, only reliably catches the "explosive" high-footprint failures at creation time, not slow silent ones after review starts.
• Fairness. A size-based gate can penalize legitimate large refactors. As the paper suggests, the size gate (and the >500-addition flag) is waived when the PR body uses a GitHub closing keyword (fixes #N, closes #N, resolves #N) — but the unplanned-complexity and entropy gates still apply, since an issue link doesn't by itself mean the PR states a plan. The paper's other suggested mitigation — progressive rollout starting with high-risk file types (CI/deps) rather than repo-wide enforcement — is a deployment choice, not a code change; see Progressive rollout above.
• Not the trained model. This is the cheap heuristic policy the paper recommends for deployment, not a reproduction of their 0.957-AUC LightGBM classifier. If you want that, you'd train against their public replication package (AIDev v1.0, Zenodo 17993901) instead.