larrylot/citeguard

Local CLI that verifies citations in agent/deep-research Markdown reports

0

stars

7

commits

Python

primary language

Sep 7, 2026

updated

larrylot.github.io/citeguard/
agents
citation
cli
hallucination
llm
markdown
python
research
validation

README

CiteGuard

CI

Local CLI that verifies citations in agent / deep-research Markdown reports.

citeguard check report.md → per-citation verdicts: URL resolve, title/host soft-match, optional claim–source overlap.

  • No API keys for core checks
  • No telemetry, no SaaS, no account
  • Offline --fixtures mode for CI
  • MIT licensed

Exploration bet for The Lord (0 SEK). Not a merchant product.

Why not “just ask ChatGPT”?

CiteGuardAsk ChatGPT / another LLM
ReproducibleDeterministic resolve + scoresNon-deterministic prose
Offline CI--fixtures planted casesNeeds network + model
Cost / keysFree local HTTPAPI key / subscription
Audit trailJSON verdicts per URLChat transcript
Hallucination check on the checkerNo LLM in the loopCan invent “looks fine”

CiteGuard does not claim to prove a source supports a legal/scientific conclusion. It flags dead links, title bait, weak overlap, and shady redirects — the failure modes that show up in agent research dumps.

Agent skill

Portable SKILL.md (Cursor / Claude / Codex / agentskills.io format):

npx skills add larrylot/citeguard -s citeguard -y

Install

cd citeguard
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Demo (copy-paste)

# Offline CI-style check on a clean planted report
citeguard check fixtures/clean.md --fixtures

# A bad report (dead links) — exits 1
citeguard check fixtures/dead_link.md --fixtures

# Title-bait SEO farm planted as "asyncio docs"
citeguard check fixtures/title_mismatch.md --fixtures

# Machine-readable
citeguard check fixtures/mixed.md --fixtures --json

Example human output:

CiteGuard — fixtures/mixed.md
Summary: {'clean': 3, 'dead': 1, 'title_mismatch': 1, 'total': 5}
------------------------------------------------------------
[OK] L3 https://docs.github.com/en/actions
...
[DEAD] L7 https://example.invalid/dead-citation-404
...
[TITLE] L9 https://example.invalid/python-asyncio-guide

Live network check (no fixtures):

citeguard check path/to/agent-report.md
citeguard check path/to/agent-report.md --json --no-overlap

Verdicts

VerdictMeaning
cleanResolved; title/claim checks passed
dead404 or network failure
http_errorNon-success HTTP (e.g. 500)
title_mismatchLink text soft-match vs <title> too low
claim_weakClaim sentence tokens barely appear in page text
redirect_suspectCross-host redirect without strong title match
unresolvedURL missing from fixtures catalog (fixtures mode only)

Real-world evidence dumps

Synthetic agent-style Markdown under examples/realworld/ (dead links, DNS failures, title bait on example.com, plus working RFC / Example Domain controls).

Citation Failure Corpus

corpus/ — 20 short public-domain-style fake agent-research Markdown snippets with planted failures (example.invalid, title bait via example.com / httpbin.org). Documented for benchmarks. See corpus/README.md.

Docs / demo page

One-page JSON output demo: docs/index.html (GitHub Pages: https://larrylot.github.io/citeguard/).

  • CiteGuard vs alternatives — honest comparison vs “ask ChatGPT”, LinkChecker, html-proofer, ReportBench (FACT / ASSUMPTION labeled)

Docker

docker build -t citeguard .
docker run --rm -v "$PWD":/data -w /data citeguard check fixtures/mixed.md --fixtures

Fixtures

fixtures/ ships 9 Markdown reports + *.expected.json + HTML pages + catalog.json for offline resolve:

  • clean cites, dead links, title mismatch, claim–source mismatch, redirect suspect, HTTP error, footnotes, mixed, bare URLs
pytest -q
# accuracy gate: ≥80% resolve+classify on fixtures

Waitlist (hosted / batch API later)

Free form — HUMAN SETUP ≤10 min: see WAITLIST.md. Paste the public URL below when ready:

Waitlist: <!-- HUMAN: paste Tally or Google Form URL -->

Posts (human sends)

Agent does not post or outreach.

Status / kill

See STATUS.md. Kill if <10 stars and <10 waitlist after 7 days with ≥1 human public post, or fixture accuracy <80%, or HN reads it as a chatbot wrapper.

License

MIT — see LICENSE.

Contributors

the-lord

5 commits

larrylot

2 commits

larrylot/citeguard

Local CLI that verifies citations in agent/deep-research Markdown reports

0

stars

7

commits

Python

primary language

Sep 7, 2026

updated

larrylot.github.io/citeguard/
agents
citation
cli
hallucination
llm
markdown
python
research
validation

README

CiteGuard

CI

Local CLI that verifies citations in agent / deep-research Markdown reports.

citeguard check report.md → per-citation verdicts: URL resolve, title/host soft-match, optional claim–source overlap.

  • No API keys for core checks
  • No telemetry, no SaaS, no account
  • Offline --fixtures mode for CI
  • MIT licensed

Exploration bet for The Lord (0 SEK). Not a merchant product.

Why not “just ask ChatGPT”?

CiteGuardAsk ChatGPT / another LLM
ReproducibleDeterministic resolve + scoresNon-deterministic prose
Offline CI--fixtures planted casesNeeds network + model
Cost / keysFree local HTTPAPI key / subscription
Audit trailJSON verdicts per URLChat transcript
Hallucination check on the checkerNo LLM in the loopCan invent “looks fine”

CiteGuard does not claim to prove a source supports a legal/scientific conclusion. It flags dead links, title bait, weak overlap, and shady redirects — the failure modes that show up in agent research dumps.

Agent skill

Portable SKILL.md (Cursor / Claude / Codex / agentskills.io format):

npx skills add larrylot/citeguard -s citeguard -y

Install

cd citeguard
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Demo (copy-paste)

# Offline CI-style check on a clean planted report
citeguard check fixtures/clean.md --fixtures

# A bad report (dead links) — exits 1
citeguard check fixtures/dead_link.md --fixtures

# Title-bait SEO farm planted as "asyncio docs"
citeguard check fixtures/title_mismatch.md --fixtures

# Machine-readable
citeguard check fixtures/mixed.md --fixtures --json

Example human output:

CiteGuard — fixtures/mixed.md
Summary: {'clean': 3, 'dead': 1, 'title_mismatch': 1, 'total': 5}
------------------------------------------------------------
[OK] L3 https://docs.github.com/en/actions
...
[DEAD] L7 https://example.invalid/dead-citation-404
...
[TITLE] L9 https://example.invalid/python-asyncio-guide

Live network check (no fixtures):

citeguard check path/to/agent-report.md
citeguard check path/to/agent-report.md --json --no-overlap

Verdicts

VerdictMeaning
cleanResolved; title/claim checks passed
dead404 or network failure
http_errorNon-success HTTP (e.g. 500)
title_mismatchLink text soft-match vs <title> too low
claim_weakClaim sentence tokens barely appear in page text
redirect_suspectCross-host redirect without strong title match
unresolvedURL missing from fixtures catalog (fixtures mode only)

Real-world evidence dumps

Synthetic agent-style Markdown under examples/realworld/ (dead links, DNS failures, title bait on example.com, plus working RFC / Example Domain controls).

Citation Failure Corpus

corpus/ — 20 short public-domain-style fake agent-research Markdown snippets with planted failures (example.invalid, title bait via example.com / httpbin.org). Documented for benchmarks. See corpus/README.md.

Docs / demo page

One-page JSON output demo: docs/index.html (GitHub Pages: https://larrylot.github.io/citeguard/).

  • CiteGuard vs alternatives — honest comparison vs “ask ChatGPT”, LinkChecker, html-proofer, ReportBench (FACT / ASSUMPTION labeled)

Docker

docker build -t citeguard .
docker run --rm -v "$PWD":/data -w /data citeguard check fixtures/mixed.md --fixtures

Fixtures

fixtures/ ships 9 Markdown reports + *.expected.json + HTML pages + catalog.json for offline resolve:

  • clean cites, dead links, title mismatch, claim–source mismatch, redirect suspect, HTTP error, footnotes, mixed, bare URLs
pytest -q
# accuracy gate: ≥80% resolve+classify on fixtures

Waitlist (hosted / batch API later)

Free form — HUMAN SETUP ≤10 min: see WAITLIST.md. Paste the public URL below when ready:

Waitlist: <!-- HUMAN: paste Tally or Google Form URL -->

Posts (human sends)

Agent does not post or outreach.

Status / kill

See STATUS.md. Kill if <10 stars and <10 waitlist after 7 days with ≥1 human public post, or fixture accuracy <80%, or HN reads it as a chatbot wrapper.

License

MIT — see LICENSE.

Contributors

the-lord

5 commits

larrylot

2 commits

Languages

Python

88.8%

HTML

9.9%

Dockerfile

1.3%