edgextract

Turn markdown into a knowledge graph with an ontology you write and a cutoff you own.

Your code proposes names. Your ontology says which kinds and links are legal. A small local decision model answers one closed yes-or-no question at a time. Anything it is unsure about waits for a person instead of silently entering your graph.

_{Real output from a local tev1 run on 13_northwind.md: open the interactive page · full report · read the article (PDF). One of the five links in that graph is wrong (Acme ACQUIRED Northwind; the text says "invested in"). We left it in. That is what the review queue and your cutoff are for.}

60-second start

Rust (native): cargo add edgextract or cargo install edgextract.

Browser WASM: npm i @raphael.mansuy/edgextract (demo: GitHub Pages · Hugging Face Space).

git clone https://github.com/raphaelmansuy/edgextract && cd edgextract
uv sync --all-extras
ollama pull tev1                       # a local decision model (Ollama 0.35+)
uv run edgextract --model tev1 report data/golden/docs/13_northwind.md \
    --ontology company_news --out graph.html
open graph.html

No model yet? Everything below runs offline against a stand-in model:

make test examples

Or from Python, in five lines:

from edgextract import extract_text, load_ontology_named, write_report

text = open("note.md").read()
ontology = load_ontology_named("company_news")      # or "tech_docs", "conll04", or a YAML path
result = extract_text(text, ontology, model="tev1")
write_report("graph.html", text, result, ontology, title="My note")

Try it in your browser

Hosted demo: GitHub Pages · Hugging Face Space (direct app) — Chromium with WebGPU; first Load Tev1 downloads ~1 GB from Hugging Face into the browser cache (no server of ours sees your text). The hosted page does not call 127.0.0.1:11434. Optional Ollama: ?backend=ollama&host=https://your-host plus OLLAMA_ORIGINS (HTTPS required; the browser blocks mixed-content http://). Open the Space on *.hf.space, not only the Hub iframe.

The Rust crate compiles to WebAssembly, and a small Vite + TypeScript page runs the whole pipeline in a tab. Your text goes only to the decision model you name (by default Tev1 on WebGPU in this tab; Ollama is one click away), never to a server of ours.

ollama pull tev1                  # Ollama already accepts pages served from localhost;
                                  # for another origin set OLLAMA_ORIGINS before `ollama serve`

make demo                # builds the wasm, then serves http://localhost:5273
# WebGPU default loads raphaelmansuy/tev1-0.8b-onnx-webgpu (~1 GB, browser cache).
# Optional: make demo-webgpu-model / make demo-webgpu-publish / make demo-webgpu-export-tev1
# Attribution: docs/THIRD_PARTY_NOTICES.md
# GitHub Pages: push to master runs .github/workflows/pages.yml (lean dist, Hub weights).

• Bring any document. Upload or drop a .md / .txt file, or paste text.
• Bring any ontology. Pick one of seven samples (company news, tech docs, research papers, film, medicine, history, CoNLL04), start from a template, upload your own YAML, and edit it with live validation.
• Names no list knows are still read. Every capitalized name in your text is put to the model as a closed question ("is this a name this ontology can hold?"), then typed. You can also pin names to an ontology with one click.
• Own the cutoff. Two sliders move what is kept, sent to review, or dropped. The decision cache means moving them asks the model nothing new.
• The model does the extracting. Step 3 defaults to WebGPU in the tab: the header button is Load Tev1 until weights are ready, then Extract graph. The default graph is raphaelmansuy/tev1-0.8b-onnx-webgpu (Together Tev1 ONNX; attribution in docs/THIRD_PARTY_NOTICES.md). Ollama is the other option (http://localhost:11434, model tev1; set OLLAMA_ORIGINS) and remains the Metal/CUDA speed ceiling. Same closed-question JSON either way. There is no rule-based mode in the page. If the backend cannot answer, the graph stays empty.

make demo-e2e runs the Playwright suite against a deterministic test double of the host (34 tests, with screenshots in docs/img/demo). Details in web/README.md.

Why not just ask a chat model for JSON?

A chat model writes the next token. A JSON schema makes the output well-formed. It does not make it true, and it does not tell you how sure the model was. You parse, repair, and hope.

edgextract asks a different question. For every candidate pair it sends a closed yes-or-no to Ollama's POST /v1/systemone and gets back a probability. Your cutoff then decides: keep, send to a person, or drop.

We measured it on CoNLL04, a standard news benchmark (288 test sentences, exact-span scoring, none of the names pre-listed). Same scorer for every row. The first run used a smaller name finder and scored slightly higher on links; we publish both and use the final, shipped configuration as the number of record.

Read this table honestly. A model trained on the benchmark's own training sentences still wins by a wide margin. The comparison that matters is the zero-shot rows. There, the closed decision keeps far fewer wrong links than the chat model (links precision 0.41 against 0.27; 218 wrong links kept against 504) and gives up some recall for it. Name scores on that test are close (0.690 against 0.643). On the twelve tech notes they are not: tev1 scores 0.97, mostly because the names are already on the list, and Mistral Small scored 0.68 on the saved run (0.80 on an earlier run the same day). See the full evaluation.

How it works

1. Split markdown into sentences with character offsets.
2. Propose names: the ontology's listed names, markdown cues, and optionally a span finder (GLiNER).
3. Type each name by lookup. For an unknown span, ask yes-or-no ("is this a name?"), then pick one kind from your list.
4. Drop pairs your ontology does not allow. Illegal links are never asked.
5. Ask one yes-or-no per legal link type, so "uses" and "depends on" can both be true.
6. Keep, review, or drop using cutoffs in your code.

The model never finds a span and never invents a type.

Write your own ontology

uv run edgextract init-ontology my_domain.yaml       # a commented starter
uv run edgextract validate-ontology my_domain.yaml   # prints kinds and legal links in plain words

id: company_news
types:
  - {id: PERSON,  description: A named person (founder, executive, investor).}
  - {id: COMPANY, description: A named company, startup, or fund.}
relations:
  - id: FOUNDED
    description: The person founded the company.
    domain: [PERSON]
    range: [COMPANY]
gazetteer:            # optional: names you already know
  Ada Lovelace: PERSON

Learn by example

Nine short scripts, each one idea. See examples/README.md.

Output

EdgeQuake-shaped JSON: entities with canonical UPPER_SNAKE names and kinds, relationships with the evidence sentence, a review list for uncertain items, and a rejected list. edgextract export-cypher prints Cypher; edgextract graph writes the interactive page.

Honest limits

• The cutoffs are not calibrated. GateConfig.fitted starts false. Fit them on your labels with edgextract calibrate. We do not claim anything about hosted-Jev calibration.
• Pronouns are skipped. There is no coreference and links do not cross sentences in this version.
• Direction can flip. On CoNLL04 the final run kept 16 reversed links (9 in the first run). On the EdgeQuake sample the model kept "Nimble uses Ollama" at 0.85, and the answer key says the other direction.
• weight is the model's probability. The cutoff compares that number. It is not a rescaled score.
• It needs Ollama 0.35+ with a decision model. On one seven-sentence note, an earlier comparison took 7.9 s for tev1 and 77.8 s for gemma4 on the same computer, and 3.9 s for hosted Mistral. A later tev1 pass with the model already loaded took 4.3 s. The library default model is nimble, which scored 0.38 on links for that note. The published CoNLL04 run used tev1 and the larger name finder, which edgextract report does not load unless you run eval-benchmark.
• A trained joint model beats it if you have labeled sentences in your domain.
• format/JSON-schema on a chat host is not a decision. Do not call it one.

Roadmap

• Fit and ship per-ontology cutoffs from labeled data
• Cross-sentence links with a bounded window
• Optional pronoun resolution
• A Rust binding for EdgeQuake ingest

Learn more

• The article (PDF): the problem, the idea, and the measured results in plain English
• Spec pack: requirements, contract, evaluation
• Contributing · Code of conduct · Apache-2.0

EdgeQuake was the inspiration for the output shape. edgextract is an independent Python library.

By Raphael MANSUY.