slop-scan

Deterministic CLI for finding AI-associated slop patterns in JavaScript and TypeScript repositories.

Scan a repo, surface the hotspots, and compare codebases using normalized slop metrics.

Why use it

• Find the hotspots fast — see which files and directories concentrate the most suspicious patterns
• Understand why something was flagged — every finding includes a rule ID and evidence
• Compare repos fairly — normalize by file count, logical KLOC, and function count
• Benchmark heuristics over time — rerun the pinned benchmark set and watch movement

Good fit for

• checking third-party repos that feel vibe-coded
• comparing known AI-generated repos to mature OSS baselines
• finding low-judgment boilerplate in your own codebase
• iterating on deterministic slop heuristics

Why can you trust it?

Every rule is tested and benchmarked against popular, mature OSS repos pinned to exact commit SHAs from before AI coding was common. See Benchmarks.

Install

Install globally with npm:

npm install -g slop-scan

Install it in a project and run it with npm tools:

npm install --save-dev slop-scan
npx slop-scan scan .

Quick start

Scan the current repo:

slop-scan scan .

Scan the current repo in lint mode:

slop-scan scan . --lint

Scan another repo and get JSON:

slop-scan scan /path/to/repo --json

Use it like a linter

Use --lint when you want human-readable findings in local runs, CI logs, or PR checks.

slop-scan scan . --lint

Example output:

medium  Found 3 duplicated function signatures  structure.duplicate-function-signatures
  at src/users/normalize.ts:1:1
  at src/teams/normalize.ts:1:1
  at src/accounts/normalize.ts:1:1

JSON output

Use --json when you want full-fidelity output for scripts, CI, or post-processing.

slop-scan scan . --json

Example CI check:

slop-scan scan . --json | jq -e '.summary.findingCount == 0'

The CLI currently exits non-zero for CLI/runtime errors, not for findings.

Delta comparisons

Use delta when you want a machine-readable comparison between two scans.

Compare two paths directly:

slop-scan delta ../main .
slop-scan delta --base ../main --head . --json

Compare two saved reports:

slop-scan scan ../main --json > base.json
slop-scan scan . --json > head.json
slop-scan delta --base-report base.json --head-report head.json

Fail CI only when new or worse occurrence-level findings show up:

slop-scan delta --base ../main --fail-on added,worsened

delta --json emits a generic report format with:

• base/head scan summaries
• occurrence-level change classification (added, resolved, worsened, improved)
• per-path score deltas
• report metadata and config hashes so downstream tools can detect mismatched scan conditions
• stable per-occurrence fingerprints for built-in rules, so grouped findings can match across rescans without relying on rendered message text

What it catches

Current default-pack checks focus on patterns that often show up in unreviewed generated code:

• log-and-continue catch blocks
• error-obscuring catch blocks (default-return or generic replacement error)
• empty catch blocks
• promise .catch() default fallbacks
• generic status envelopes
• generic record casts
• stringified unknown errors
• pass-through wrappers
• duplicated test mock/setup patterns

scan reports raw + normalized scores, hotspot tables, and grouped findings. Use --json when you want the full evidence payload.

Supported files

Current language support:

• .ts
• .tsx
• .js
• .jsx
• .mjs
• .cjs

Benchmarks

The repo ships with a pinned, recreatable benchmark set comparing known AI-generated repos against well-regarded OSS repos, with the mature-OSS cohort pinned to the latest default-branch commit on or before 2025-01-01.

Why before Jan 1, 2025? Because this cutoff aims to catch mature OSS before AI coding had materially changed mainstream repository shape and review norms.

Blended score = geometric mean of the six normalized-metric ratios versus the mature OSS cohort medians, then rescaled so the mature OSS cohort median is 1.00. Higher means a repo is consistently noisier across the benchmark dimensions.

Cohort medians

Rolling benchmark snapshot

Latest default-branch history, still normalized against the frozen pinned baseline. Ordered by latest pinned score.

Legend:

• Current blended = latest repo score vs the current mature-OSS medians from the same rolling run
• Latest pinned = latest repo score vs the frozen pinned mature-OSS baseline snapshot
• Highest pinned = highest stored repo score on that same pinned baseline
• Δ prev = latest pinned - previous week's pinned score
• Δ peak = latest pinned - highest pinned score, so more negative means the repo is below its own historical high

For exact pinned SHAs, the current default-pack medians, and the full per-metric breakdowns, see the saved snapshot and pinned benchmark report.

Full benchmark assets:

• manifest: benchmarks/sets/known-ai-vs-solid-oss.json
• pinned snapshot: benchmarks/results/known-ai-vs-solid-oss.json
• pinned report: reports/known-ai-vs-solid-oss-benchmark.md
• rolling latest summary: benchmarks/history/known-ai-vs-solid-oss/latest.json
• rolling history report: reports/known-ai-vs-solid-oss-history.md

Configuration

The analyzer reads slop-scan.config.ts, slop-scan.config.js, slop-scan.config.mjs, slop-scan.config.cjs, or slop-scan.config.json from the scan root. Root .gitignore entries are also respected.

{
  "ignores": ["dist/**", "coverage/**", "**/*.generated.ts"],
  "plugins": {
    "acme": "slop-scan-plugin-acme"
  },
  "extends": ["plugin:acme/recommended"],
  "rules": {
    "structure.over-fragmentation": { "enabled": true, "weight": 1.2 },
    "comments.placeholder-comments": { "enabled": false },
    "acme/no-generated-wrapper": { "enabled": true, "options": { "threshold": 3 } }
  },
  "overrides": [
    {
      "files": ["src/rules/**"],
      "rules": {
        "structure.over-fragmentation": { "enabled": false }
      }
    }
  ]
}

Supported today:

• ignores
• plugins.<namespace> as either a package/path string or a plugin object in module configs
• extends: ["plugin:<namespace>/<config>"]
• rules.<id>.enabled
• rules.<id>.weight
• rules.<id>.options
• overrides[].files
• overrides[].rules.<id>.enabled
• overrides[].rules.<id>.weight
• overrides[].rules.<id>.options

Plugins

slop-scan can load third-party rule plugins and plugin preset configs from JSON or module configs.

For plugin setup, naming rules, and authoring examples, see docs/plugins.md. Simple plugin rules can now declare stable delta behavior with helpers like delta.byPath() / delta.byLocations(), and clustered rules can attach lightweight deltaKeys instead of building fingerprints manually.

See also:

• examples/local-plugin/contains-word-plugin.mjs
• examples/local-plugin/slop-scan.config.ts

This repo also commits a root slop-scan.config.json for self-scans and local development. It keeps the scan focused on the tool itself by excluding heavyweight benchmark checkouts and intentionally disables directory-structure rules under src/rules/**.

Docs

• plugin guide: docs/plugins.md
• built-in rule docs: browse src/rules/
• benchmark guide: benchmarks/README.md
• pinned benchmark report: reports/known-ai-vs-solid-oss-benchmark.md
• rolling benchmark history: reports/known-ai-vs-solid-oss-history.md
• exploratory note on non-JS/TS candidates: reports/exploratory-vite-astro-openclaw-beads.md
• contributing guide: CONTRIBUTING.md

Contributing

Issues and pull requests are welcome.

For local development, validation, and benchmark reproduction, see CONTRIBUTING.md.

Sponsor

Sponsored by Modem.

License

MIT