AI Shipcheck

Your AI says it's done. Shipcheck tells you if it's ready to ship.

npx ai-shipcheck .

No signup · No API key · No source-code upload · Runs locally

---

   NOT READY   61/100 across 41 assessed checks                        my-app

  Detected: Next.js, React, Supabase, Vitest, Next.js App Router · TypeScript · tests present
  Assessed: 41 of 63 checks run · 7/9 categories scored · 9 files · 3 not assessed

  Security                        ██████████░░░░░░   60  1 finding
  Authentication & Authorization  ████████░░░░░░░░   52  2 findings
  Database & Data Safety          ░░░░░░░░░░░░░░░░    0  7 findings
  Reliability                     ████████████████   98  1 finding
  Testing                         ███████████░░░░░   71  3 findings
  Observability                   ███████████████░   92  1 finding
  Performance                     ███████████████░   92  3 findings
  Accessibility                   ················  not assessed
  AI Cost & Abuse Controls        ················  n/a

  Findings

  CRITICAL BLOCKER Service-role key read through a browser-visible environment variable
      lib/supabase-admin.ts:5:27
      │ process.env.NEXT_PUBLIC_SUPABASE_SERVICE_ROLE_KEY!,
      lib/supabase-admin.ts reads the Supabase service-role key from a variable with a public
      build-time prefix. The value is inlined into the browser bundle, granting every visitor full,
      RLS-bypassing access to the database.
      Fix: Use the anon key in the browser and rely on row-level security for access control. Keep
           the service-role key in server-only code - a route handler, server action, or edge
           function - and rotate it immediately if it has ever been in a client bundle.
      auth/supabase-service-role-exposure · confidence high · ai-shipcheck explain auth/…

  Verdict

  · 4 blocking issues must be fixed before deploying. A blocker forces NOT READY regardless of score.
  · Weakest category: Database & Data Safety at 0/100 (7 findings).

  Scanned 9 files in 28 ms · 41 checks run, 19 not applicable, 3 not assessed · ai-shipcheck v1.0.1
  Static analysis of source code - not a security certification.

_{Real output from fixtures/vulnerable-supabase, trimmed for length.}

Why

AI coding tools are very good at producing code that runs, and much less good at producing code that survives production. They report the work as finished either way.

The gaps are consistent and boring: the Supabase table nobody enabled RLS on, the route handler that writes without checking who is calling, the NEXT_PUBLIC_ variable holding a secret, the LLM endpoint with no rate limit and no token cap. Shipcheck looks for exactly those, statically, on your machine.

What it checks

Nine categories, scored independently. 63 rules, each with documentation, a vulnerable fixture, a secure fixture and tests.

Full catalogue: docs/rules — or run ai-shipcheck rules.

Stacks it understands: Next.js (both routers), React, Vite, Express, Fastify, Hono, NestJS, Remix, Astro, SvelteKit, Nuxt, Supabase, Firebase, Prisma, Drizzle, Mongoose, Stripe, OpenAI, Anthropic, Vercel AI SDK, LangChain, tRPC, and the common test runners. Framework-specific rules run only when the framework is detected, monorepos included.

Use it

npx ai-shipcheck .                       # scan the current directory
npx ai-shipcheck . --fail-on critical    # exit 1 when a critical finding exists
npx ai-shipcheck . --format sarif        # also: json, markdown
npx ai-shipcheck explain <rule-id>       # why a rule exists, and how to fix it

Findings alone never fail the command — you opt in with --fail-on or --min-score, so adding this to an existing pipeline is a reversible step. Exit codes: 0 thresholds met, 1 not met, 2 usage error, 3 internal error.

Requires Node.js 22 or newer. Configuration is optional; a shipcheck.config.json can exclude paths, disable rules or set thresholds.

Complete reference: docs/cli.md · docs/configuration.md

GitHub Action

- uses: sinceaihq/ai-shipcheck@v1
  with:
    fail-on: critical
    min-score: 80

Annotates findings inline on the diff, writes a Markdown report to the job summary, and produces SARIF for code scanning. Outputs score, verdict, critical-count and high-count. It is bundled into a single committed file, so a workflow pinned to a tag runs exactly that code with nothing fetched at run time.

Inputs, outputs and recipes: docs/github-action.md

Trust

• Nothing leaves your machine. No telemetry, no network calls, no upload.
• Nothing in the scanned repository is executed — no import, no eval, no install scripts. Files are read as bytes and analysed lexically.
• Secrets are masked everywhere they could be printed.
• Scans are bounded, and a truncated scan says so rather than quietly reporting on a fraction of the project.
• One runtime dependency.

Every finding cites a file, a line, a rule, a severity and a confidence — kept deliberately separate, so a heuristic finding is still described as severe while counting for less against the score. Categories that cannot be assessed are excluded rather than awarded a free 100.

The rules were validated against 20 real public repositories pinned by commit SHA. Triaging what they reported cut findings from 5,710 to 2,819 — a 51% reduction, none of it from suppressing projects by name — and surfaced a lexer bug that had been reporting wrong line numbers in any file containing a multi-line comment. Every fix carries a regression test.

docs/trust.md · corpus/TRIAGE.md · docs/THREAT_MODEL.md

Scope

Worth knowing before you rely on it:

• JavaScript and TypeScript only. Other languages are walked, not analysed.
• Lexical, not semantic. No cross-file reasoning, no type information, and taint tracking that follows a value one hop. A custom auth wrapper it does not recognise can produce a false positive.
• Express and Fastify routes are not covered by the auth rules. Those understand Next.js conventions today. This is the largest known gap.
• It cannot see your infrastructure. A table with RLS enabled in a dashboard but absent from migrations is reported as unassessed, not safe.

A clean report means the checks it knows how to make found nothing — not that the code is correct. This is static analysis, not a security certification.

docs/LIMITATIONS.md

Documentation

Contributing

False positives are the most serious class of bug here. If a rule fires on correct code, report it — the fix ships with a regression test.

Adding a rule is meant to be a single afternoon: write it, add a vulnerable fixture and a secure fixture, run npm run docs:rules, run npm run check. The build fails if a rule never fires on a vulnerable fixture, and fails if any rule fires on a secure one.

CONTRIBUTING.md · good first issues · SUPPORT.md · Discussions

License

MIT — see LICENSE.

---

Built by Since AI · builders@sinceai.fi