SecretSource

Goal: provide the factually correct reference for every fact mentioned in a piece of research or an interview.

Problem statement: interviews, news articles, and research papers routinely make claims that need fact-checking or a correct citation, and doing this by hand is slow and inconsistent.

SecretSource is a pipeline that takes a document (PDF, references, media), extracts its claims, finds and retrieves the evidence those claims should cite, and has a panel of verifier agents judge whether each claim is actually supported — producing an annotated report a human reviewer can adjudicate.

Pipeline

1. Input — a PDF (with references/media) is uploaded.
2. Parse + quality — parsed with GROBID + Docling, then checked by a quality gate.
3. Claims + citations — a claim-extractor agent pulls out atomic, typed, cite-worthy claims; citation scope maps each claim to its reference(s) and flags missing citations.
4. Evidence flow — references are resolved via source tools (Crossref, OpenAlex, Semantic Scholar, PubMed, arXiv), gated on whether full text or only an abstract is available, then retrieved with BM25 + ColBERTv2 (text) and ColPali/ColQwen (visual).
5. Specialist review — three verifier agents (evidence text+visual, quantitative, validity/overclaim) judge each claim against the retrieved evidence.
6. Decide, explain, learn — an aggregator fuses and calibrates the verdicts and routes uncertain cases; a report is generated (annotated PDF, verdict + evidence, safer rewrite); a human reviewer adjudicates, corrects false flags, and feeds labels back into the loop.

Repository layout

Researcher team