Local CLI that verifies citations in agent/deep-research Markdown reports
0
stars
7
commits
Python
primary language
Sep 7, 2026
updated
Local CLI that verifies citations in agent / deep-research Markdown reports.
citeguard check report.md → per-citation verdicts: URL resolve, title/host soft-match, optional claim–source overlap.
--fixtures mode for CIExploration bet for The Lord (0 SEK). Not a merchant product.
| CiteGuard | Ask ChatGPT / another LLM | |
|---|---|---|
| Reproducible | Deterministic resolve + scores | Non-deterministic prose |
| Offline CI | --fixtures planted cases | Needs network + model |
| Cost / keys | Free local HTTP | API key / subscription |
| Audit trail | JSON verdicts per URL | Chat transcript |
| Hallucination check on the checker | No LLM in the loop | Can invent “looks fine” |
CiteGuard does not claim to prove a source supports a legal/scientific conclusion. It flags dead links, title bait, weak overlap, and shady redirects — the failure modes that show up in agent research dumps.
Portable SKILL.md (Cursor / Claude / Codex / agentskills.io format):
skill/SKILL.md.cursor/skills/citeguard/SKILL.mdskills/citeguard/SKILL.mdnpx skills add larrylot/citeguard -s citeguard -y
cd citeguard
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
# Offline CI-style check on a clean planted report
citeguard check fixtures/clean.md --fixtures
# A bad report (dead links) — exits 1
citeguard check fixtures/dead_link.md --fixtures
# Title-bait SEO farm planted as "asyncio docs"
citeguard check fixtures/title_mismatch.md --fixtures
# Machine-readable
citeguard check fixtures/mixed.md --fixtures --json
Example human output:
CiteGuard — fixtures/mixed.md
Summary: {'clean': 3, 'dead': 1, 'title_mismatch': 1, 'total': 5}
------------------------------------------------------------
[OK] L3 https://docs.github.com/en/actions
...
[DEAD] L7 https://example.invalid/dead-citation-404
...
[TITLE] L9 https://example.invalid/python-asyncio-guide
Live network check (no fixtures):
citeguard check path/to/agent-report.md
citeguard check path/to/agent-report.md --json --no-overlap
| Verdict | Meaning |
|---|---|
clean | Resolved; title/claim checks passed |
dead | 404 or network failure |
http_error | Non-success HTTP (e.g. 500) |
title_mismatch | Link text soft-match vs <title> too low |
claim_weak | Claim sentence tokens barely appear in page text |
redirect_suspect | Cross-host redirect without strong title match |
unresolved | URL missing from fixtures catalog (fixtures mode only) |
Synthetic agent-style Markdown under examples/realworld/ (dead links, DNS failures, title bait on example.com, plus working RFC / Example Domain controls).
examples/realworld/REPORT.mdexamples/realworld/out/corpus/ — 20 short public-domain-style fake agent-research Markdown snippets with planted failures (example.invalid, title bait via example.com / httpbin.org). Documented for benchmarks. See corpus/README.md.
One-page JSON output demo: docs/index.html (GitHub Pages: https://larrylot.github.io/citeguard/).
docker build -t citeguard .
docker run --rm -v "$PWD":/data -w /data citeguard check fixtures/mixed.md --fixtures
fixtures/ ships 9 Markdown reports + *.expected.json + HTML pages + catalog.json for offline resolve:
pytest -q
# accuracy gate: ≥80% resolve+classify on fixtures
Free form — HUMAN SETUP ≤10 min: see WAITLIST.md. Paste the public URL below when ready:
Waitlist:
<!-- HUMAN: paste Tally or Google Form URL -->
SHOW_HN.md — Show HN draftPOSTS.md — Reddit / LinkedIn templatesAgent does not post or outreach.
See STATUS.md. Kill if <10 stars and <10 waitlist after 7 days with ≥1 human public post, or fixture accuracy <80%, or HN reads it as a chatbot wrapper.
MIT — see LICENSE.
Python
88.8%
HTML
9.9%
Dockerfile
1.3%
Local CLI that verifies citations in agent/deep-research Markdown reports
0
stars
7
commits
Python
primary language
Sep 7, 2026
updated
Local CLI that verifies citations in agent / deep-research Markdown reports.
citeguard check report.md → per-citation verdicts: URL resolve, title/host soft-match, optional claim–source overlap.
--fixtures mode for CIExploration bet for The Lord (0 SEK). Not a merchant product.
| CiteGuard | Ask ChatGPT / another LLM | |
|---|---|---|
| Reproducible | Deterministic resolve + scores | Non-deterministic prose |
| Offline CI | --fixtures planted cases | Needs network + model |
| Cost / keys | Free local HTTP | API key / subscription |
| Audit trail | JSON verdicts per URL | Chat transcript |
| Hallucination check on the checker | No LLM in the loop | Can invent “looks fine” |
CiteGuard does not claim to prove a source supports a legal/scientific conclusion. It flags dead links, title bait, weak overlap, and shady redirects — the failure modes that show up in agent research dumps.
Portable SKILL.md (Cursor / Claude / Codex / agentskills.io format):
skill/SKILL.md.cursor/skills/citeguard/SKILL.mdskills/citeguard/SKILL.mdnpx skills add larrylot/citeguard -s citeguard -y
cd citeguard
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
# Offline CI-style check on a clean planted report
citeguard check fixtures/clean.md --fixtures
# A bad report (dead links) — exits 1
citeguard check fixtures/dead_link.md --fixtures
# Title-bait SEO farm planted as "asyncio docs"
citeguard check fixtures/title_mismatch.md --fixtures
# Machine-readable
citeguard check fixtures/mixed.md --fixtures --json
Example human output:
CiteGuard — fixtures/mixed.md
Summary: {'clean': 3, 'dead': 1, 'title_mismatch': 1, 'total': 5}
------------------------------------------------------------
[OK] L3 https://docs.github.com/en/actions
...
[DEAD] L7 https://example.invalid/dead-citation-404
...
[TITLE] L9 https://example.invalid/python-asyncio-guide
Live network check (no fixtures):
citeguard check path/to/agent-report.md
citeguard check path/to/agent-report.md --json --no-overlap
| Verdict | Meaning |
|---|---|
clean | Resolved; title/claim checks passed |
dead | 404 or network failure |
http_error | Non-success HTTP (e.g. 500) |
title_mismatch | Link text soft-match vs <title> too low |
claim_weak | Claim sentence tokens barely appear in page text |
redirect_suspect | Cross-host redirect without strong title match |
unresolved | URL missing from fixtures catalog (fixtures mode only) |
Synthetic agent-style Markdown under examples/realworld/ (dead links, DNS failures, title bait on example.com, plus working RFC / Example Domain controls).
examples/realworld/REPORT.mdexamples/realworld/out/corpus/ — 20 short public-domain-style fake agent-research Markdown snippets with planted failures (example.invalid, title bait via example.com / httpbin.org). Documented for benchmarks. See corpus/README.md.
One-page JSON output demo: docs/index.html (GitHub Pages: https://larrylot.github.io/citeguard/).
docker build -t citeguard .
docker run --rm -v "$PWD":/data -w /data citeguard check fixtures/mixed.md --fixtures
fixtures/ ships 9 Markdown reports + *.expected.json + HTML pages + catalog.json for offline resolve:
pytest -q
# accuracy gate: ≥80% resolve+classify on fixtures
Free form — HUMAN SETUP ≤10 min: see WAITLIST.md. Paste the public URL below when ready:
Waitlist:
<!-- HUMAN: paste Tally or Google Form URL -->
SHOW_HN.md — Show HN draftPOSTS.md — Reddit / LinkedIn templatesAgent does not post or outreach.
See STATUS.md. Kill if <10 stars and <10 waitlist after 7 days with ≥1 human public post, or fixture accuracy <80%, or HN reads it as a chatbot wrapper.
MIT — see LICENSE.
Python
88.8%
HTML
9.9%
Dockerfile
1.3%