PaperTrail

The evidence-verification & provenance platform for regulated life-sciences work.
Claude reads the literature; a deterministic engine proves every number.

▶ Live app · ▶ Watch the demo · Next.js 16 · Postgres/pgvector · Anthropic Claude · Apache-2.0

Built for Built with Claude: Life Sciences (Anthropic × Gladstone Institutes) — Builder Track.

---

Table of contents

• Demo
• What it is
• The moat: deterministic verification, not vibes
• Evaluation & limitations (read this)
• Capabilities
  • Claude-powered features
  • Deterministic evidence synthesis
  • Biomedical evidence engines
  • Evidence Intelligence + enterprise governance
• Open source, assimilated
• Documents: upload anything
• Tech stack
• Quickstart (local)
• Deploy (Vercel + Neon)
• API reference
• Architecture & docs
• License & attribution

---

Demo

A ~45-second walkthrough of the live app: catching an overstated lecanemab claim (grounded to the primary source), turning dictated bench notes into a reproducible record, matching a de-identified patient to clinical trials with per-criterion eligibility, and the Mixture-of-Agents verdict refuting a population-overgeneralized claim.

If the player doesn't render (some markdown viewers strip <video>), watch / download the MP4 here.

Every screen is the real pipeline running against live PubMed / ClinicalTrials.gov and the Claude API — not a mockup. Deterministic math decides each verdict; Claude only writes the grounded narrative.

---

What it is

The user (Builder Track): a pharma medical-affairs / regulatory reviewer, a translational lab, a systematic-review team, or a research coordinator who must produce and defend the evidence behind a claim — and cannot afford to be wrong. Today that is hours of PhD time per claim, and an unsubstantiated claim is a legal and scientific liability.

PaperTrail is a multi-tenant platform where those teams verify scientific and efficacy claims against primary sources, ingest and mine their own document corpora, run auditable agentic research workflows, and produce a defensible provenance trail for every cited number — software they can use without the builder in the room, built to outlast the hackathon.

At its center is a differentiator no chatbot can match: a deterministic verification engine that checks a claim against a trial's own registered statistical results — with no LLM in the numeric loop.

The moat: deterministic verification, not vibes

Generic claim verifiers stop at LLM judgment or textual entailment over prose — they assess whether the literature agrees with a claim. PaperTrail independently recomputes the number the trial actually registered:

• Structured registry verification (lib/structuredVerification.ts) — parses ClinicalTrials.gov's resultsSection: registered outcome analyses (paramType/paramValue/CI/ p-value) and raw per-arm event counts.
• Biostatistics recomputed from raw counts (lib/biostats.ts) — absolute risk reduction, NNT, risk ratio + 95% CI (log-RR delta method), oracle-tested against epitools/OpenEpi.
• Endpoint-switch detection — flags a claim whose magnitude matches a secondary outcome, not the trial's primary result.
• Code-enforced exact-span grounding (lib/grounding.ts) — every flagged span is a verbatim substring of the cached source; any span the model can't locate is dropped. PaperTrail structurally cannot make an unsourced claim about a source.
• Multi-source cross-verification — corroborated / conflicting / single-source, with a deterministic trust-score adjustment.

Example, from SPRINT's registry: a claim of "cut cardiovascular risk in half" is flagged overstates_registry — the registered primary result is HR 0.75 (95% CI 0.64–0.89), a ~25% reduction; raw counts give ARR 1.6 points, NNT 62. No LLM produced that number.

Evaluation & limitations (read this)

We would rather be honest than impressive.

Benchmark (balanced 60-case SciFact, 20/20/20, claude-sonnet-4-6): PaperTrail scored 58.3% accuracy vs a Claude-alone baseline at 70.0%. On SciFact, PaperTrail loses. We are not hiding it. (This supersedes an earlier invalid 10-case smoke that had sliced a single gold label.)

Three things are true about that result:

1. It's a task mismatch — the own-it limitation. SciFact tests general scientific-claim entailment (mechanisms, associations). PaperTrail's engine is tuned for the opposite task — clinical-trial efficacy-magnitude verification (recompute "reduced events by 30%" against a registry). The confusion matrix is unambiguous: SUPPORT recall 10%, CONTRADICT recall 90% — it aggressively over-flags and maps SUPPORT → CONTRADICT. Outside its design envelope, it over-flags. That's a genuine limitation.
2. Its honest-abstention holds up: NEI F1 73.2 (precision 71.4) — when it can't verify, it says so rather than guessing.
3. It's far more reliable than the raw baseline: PaperTrail errored on 2/60 cases; Claude-alone errored on 16/60 (raw Claude often returned prose instead of valid JSON — PaperTrail's Zod-validated pipeline did not).

On the design task, it measurably beats the LLM. The fair benchmark — 20 clinical-efficacy claims verified against a source that reports the registered effect size, including subtle magnitude drift (npm run bench -- --clinical, docs/benchmark-clinical.md):

A 20-point margin. The gap is the deterministic recompute: on claims like "reduced by 37%" against a source reporting HR 0.75 (a 25% reduction), reconcile() flags magnitude_overstated whenever the implied ratio crosses the CI lower bound — catching all 11 overstatements with no LLM in the decision. Claude-alone often computed the right number in prose but broke its JSON contract (5/20 errors); PaperTrail's Zod-validated pipeline errored on 0/20. We own the one imperfection: PaperTrail over-flagged 1 accurate case (a false positive). SciFact remains the wrong yardstick; we do not cite 58.3% as a capability number. Full confusion matrices in docs/benchmark.md + docs/benchmark-clinical.md.

Capabilities

Claude-powered features

Each uses Claude for the genuinely hard natural-language work, then hands every factual/numeric claim to the deterministic engine to ground or recompute it. Structured Claude output is Zod-validated before use; no flagged span survives that isn't a verbatim substring of the source.

Deterministic evidence synthesis

Public, rate-limited, no LLM in the numeric path — reproducible from the request body, oracle-tested against reference tools (metafor / epitools).

meta-analysis (fixed + DerSimonian–Laird random effects, I²/τ²/Q, prediction interval) · continuous-meta (MD + Hedges g) · network-meta (Bucher indirect) · meta-regression (WLS) · subgroup (Q-between interaction) · survival (Kaplan–Meier, log-rank, Cox) · dose-response · trial-sequential (RIS + O'Brien–Fleming) · publication bias (Egger's + trim-and-fill) · GRADE + risk-of-bias · absolute effects (ARR/NNT) · evidence-report (+ batch, HTML/text/PDF export, org-scoped persistence).

Biomedical evidence engines

Deterministic verdicts on the canonical open bio-data sources — the stack bio-AI companies (Causaly, Open Targets, Aetion) monetize — no proprietary EHR or wet lab required.

Evidence Intelligence + enterprise governance

The layer that makes it sellable to regulated pharma: a biomedical knowledge graph (/api/kg, Biolink typing + graph link-prediction), an evidence-dossier orchestrator (/api/dossier — target/drug/disease/claim → complete, verified, cited, trust-scored dossier), hash-chained provenance + submission export, real-world-evidence signals, and enterprise governance: data-source provenance registry, versioned /api/v1 gateway with API-key quotas, per-engine usage metering, validation/compliance status, a tamper-evident evidence audit chain (21 CFR Part 11-style), SLA observability, evidence webhooks, and data-governance (retention + DSAR export).

Open source, assimilated

We didn't just call libraries — we ported the real algorithms of 17 open-source projects into native PaperTrail TypeScript, running on our own Postgres + API routes + console + grounding layer. Deterministic math is native TS; Claude is used only where the original relied on a trained model. The upstream source is retained under backend/engines/ (excluded from the build); every project is honored in NOTICE.

Documents: upload anything

Upload PDF, DOCX, XLSX/XLS, CSV, Markdown, or plain text — a drag-and-drop uploader at /console/documents/upload reads each file to text (mammoth for DOCX, SheetJS for spreadsheets, unpdf/Docling for PDF), with server-side format sniffing (it won't trust a lying MIME type), org-scoped and audited. Extracted text feeds the same claim/evidence pipeline as everything else.

Tech stack

• Next.js 16 (App Router, TypeScript strict), React 19, Tailwind
• Postgres + pgvector (Neon), 58 ordered migrations under db/migrations/
• Anthropic Claude (extraction, verification, agentic work) · Voyage embeddings
• Auth: jose (HS256 session JWT) + bcryptjs, org-scoped API contract (lib/api/*)
• Deterministic math in native TS (no external stats service); simple-statistics for a few quantiles
• 1,000+ tests (Vitest), CI (GitHub Actions), deployed on Vercel

Quickstart (local)

Prerequisites: Node 20+, a Postgres/pgvector DB (Neon), an Anthropic key, a Voyage key.

cp .env.example .env.local     # fill in real values
npm install
npm run db:migrate             # applies db/migrations.sql then db/migrations/*.sql in order
npm run dev                    # http://localhost:3000 → register at /register, app at /console

Verify the build:

npx tsc --noEmit      # type-check (backend/ and reference are excluded)
npm test              # unit + component tests (live-API tests skip without ANTHROPIC_API_KEY)
npm run build         # production build
npm run bench         # accuracy benchmark (spends Claude tokens — see docs/benchmark.md)

Deploy (Vercel + Neon)

Live at https://papertrail-topaz-phi.vercel.app.

# 1. Provision a Neon Postgres, then apply all migrations:
DATABASE_URL="postgresql://…/neondb?sslmode=require" npm run db:migrate

# 2. Set env on Vercel (production):
vercel env add DATABASE_URL production
vercel env add ANTHROPIC_API_KEY production
vercel env add VOYAGE_API_KEY production
vercel env add AUTH_SECRET production      # openssl rand -base64 32
vercel env add CRON_SECRET production      # openssl rand -base64 32

# 3. Deploy:
vercel --prod

GET /api/health returns { status, checks: { db, anthropic_key, voyage_key }, version } and never 500s. A Vercel Cron (vercel.json) hits /api/cron/tick (all-org job sweep, CRON_SECRET- authed). On the Hobby plan crons run daily; restore */5 * * * * on Pro. backend/, python/, and reference/ are excluded from the upload via .vercelignore.

API reference

All public compute endpoints POST a JSON body and return { success, data, error }.

curl -sX POST "$BASE_URL/api/synthesis" \
  -H 'content-type: application/json' \
  -d '{
    "claim": "Drug X reduced major cardiovascular events by 30%",
    "studies": [
      { "label": "Trial A", "measure": "RR", "point": 0.72, "ci_lower": 0.60, "ci_upper": 0.86 },
      { "label": "Trial B", "measure": "RR", "point": 0.68, "ci_lower": 0.51, "ci_upper": 0.90 }
    ]
  }'

Architecture & docs

• docs/enterprise-evidence-platform.md — the platform architecture
• docs/enterprise-architecture.md — module / page / API blueprint
• docs/benchmark.md — benchmark methodology & the honest smoke-test caveat
• docs/bio-commercial-landscape.md · docs/bio-roadmap.md — bio-AI landscape & backlog
• PRD.md — product requirements · ARCHITECTURE.md — technical design · CLAUDE.md — conventions

License & attribution

Apache License 2.0 — see LICENSE. Third-party attributions for the assimilated open-source projects and consumed open bio-data sources are in NOTICE. GPL/AGPL projects were deliberately not vendored; where their published approach informed a method, only the public algorithm was reimplemented independently. Not affiliated with any other product named "PaperTrail".