idealens

idealens detects whether a document's ideas came from a person or an AI model, whoever wrote the words. It runs the IdeaLens pipeline end to end:

1. Classify the document into one of eight long-form formats (an LLM with WebOrganizer's annotation prompt, or WebOrganizer's encoder).
2. Extract an outline: an LLM writes a role-labelled list of the document's ideas, using the same prompt and six worked examples the detector was trained with.
3. Score the outline with a detector, which returns P(human) and a verdict at a calibrated false-positive rate.

The same package also scores the prose-level detector ProseLens and the baselines released with the paper.

Models: IdeaLens and the other detectors listed under Models. Data: WildOutlines (training corpus), IdeaShift, IdeaShift-X and TwiceTold (evaluation sets). All of them are in the IdeaLens collection.

Try it in your browser: the IdeaLens & ProseLens demo scores your own text, with no installation or keys.

Paper: IdeaLens: Detecting AI Ideas in Long-form Writing (arXiv:2610.06778).

Install

pip install "idealens[vllm]"          # the default scoring backend (Nemotron models, one 80 GB GPU)
pip install "idealens[hf]"            # transformers backend; needed for the ModernBERT and Qwen models
pip install "idealens[openai]"        # OpenAI, OpenRouter or any OpenAI-compatible endpoint (also the logistic models)
pip install "idealens[anthropic]"     # Claude
pip install "idealens[vertex]"        # Gemini on Vertex AI (batch mode also needs a GCS bucket)
pip install "idealens[weborganizer]"  # WebOrganizer's encoder for format classification
pip install "idealens[tinker]"        # score IdeaLens and ProseLens on Tinker's servers: no GPU needed
pip install "idealens[all]"

Python 3.10 or later. The vLLM extra pins vllm==0.21.*, transformers>=5.15 and xgrammar==0.2.1.

The models and datasets are public: no access request and no Hugging Face login are needed to download them.

Credentials

The logistic models embed outlines with OpenAI's text-embedding-3-large, so they also need OPENAI_API_KEY. Tinker scoring (--backend tinker) needs TINKER_API_KEY.

Quick start

Input is JSONL with a text field and, optionally, id, url, format and topic.

idealens run docs.jsonl -o scores.jsonl --dry-run   # estimate tokens and cost; sends nothing
idealens run docs.jsonl -o scores.jsonl             # classify, extract and score with IdeaLens

Or one step at a time, which lets you extract once and score with several models:

idealens classify docs.jsonl     -o formats.jsonl
idealens extract  formats.jsonl  -o outlines.jsonl
idealens score    outlines.jsonl -o scores.jsonl --model IdeaLens
idealens score    outlines.jsonl -o scores_np.jsonl --model IdeaLens-NoParaphrase

Every command keeps the input record's fields and appends to its output. Re-running a command resumes it: records already in the output are skipped, and records whose provider call failed are retried (the failed attempts are moved to <output>.failed.jsonl).

In Python:

import idealens as il

texts = ["...", "..."]
formats = il.classify(texts)                   # gemini-3.7-flash by default
outlines = il.extract(texts, formats)
with il.Detector("IdeaLens") as det:           # vLLM; the repo's published thresholds
    records = det.score_outlines(outlines, format=formats)

for r in records:
    print(r["p_human"], r["verdict"]["ai"])

il.run(texts, det) does all three steps. Any provider can be passed as an object:

from idealens.providers import make
prov = make("openai", "gpt-6-sol")
outlines = il.extract(texts, formats, provider=prov, mode="batch")

Ways to use idealens

Step-by-step scenarios (laptop without a GPU, your own GPU server or Slurm cluster, large collections, keeping documents on your own machines, other languages, ideas vs. prose, calibrating on your own domain, notebooks, and troubleshooting) are in the usage guide. The summary below lists the building blocks.

Without a GPU: Tinker

IdeaLens and ProseLens are also published on Tinker, which runs the models on its own servers, so any laptop works. Tinker bills your account per token: scoring with IdeaLens costs about $0.50 per 1,000 documents at list price (the outline extraction costs more; --dry-run prices both).

1. Sign up at auth.thinkingmachines.ai/sign-up and create an API key in the Tinker console.
2. Install and run:

pip install "idealens[tinker]"
export TINKER_API_KEY=...       # your Tinker key
export GEMINI_API_KEY=...       # for outline extraction (or another provider, see Credentials)
idealens run docs.jsonl -o scores.jsonl --backend tinker

In Python, il.Detector("IdeaLens", backend="tinker"). ProseLens works the same way (--model ProseLens).

Which detector?

1. From documents, end to end

idealens run classifies each document's format, extracts its outline and scores it. This is the route the published thresholds assume when the extractor is gemini-3.7-flash (the default).

idealens run docs.jsonl -o scores.jsonl --model IdeaLens

2. Extract once, score with several detectors

Extraction is the expensive step. Keep its output and score it with as many outline models as you like:

idealens classify docs.jsonl     -o formats.jsonl
idealens extract  formats.jsonl  -o outlines.jsonl
idealens score outlines.jsonl -o idealens.jsonl  --model IdeaLens
idealens score outlines.jsonl -o modernbert.jsonl --model IdeaLens-ModernBERT-L
idealens score outlines.jsonl -o per_item.jsonl   --model IdeaLens-ModernBERT-L-PerItem

3. Score outlines you already have

The outline field (or the field named by --input-field) can hold the extractor's JSON object or plain text with one [Role] content line per item:

{"id": "doc1", "format": "News Article", "outline": "[Central Development] The council approved the new pipeline.\n[Background Context] The reservoir has been shrinking for three summers."}

with il.Detector("IdeaLens-NoParaphrase") as det:
    records = det.score_outlines(["[Central Development] ...\n[Background Context] ..."], format=["News Article"])

Roles must come from the format's role vocabulary for the scores to mean what they meant in training; the extractor in this package uses it.

4. Score documents directly

The prose detectors read the document itself, so no outline and no LLM call is needed:

idealens score docs.jsonl -o prose.jsonl --model ProseLens

5. Choose the LLM that classifies and extracts

Any provider in Credentials works for classify, extract, run and calibrate, online or as a batch job (--mode batch, half price where the provider has a batch API). That includes a model you serve yourself behind an OpenAI-compatible endpoint (vLLM, Ollama, ...):

idealens run docs.jsonl -o scores.jsonl --provider openai   --llm-model gpt-6-sol --mode batch
idealens run docs.jsonl -o scores.jsonl --provider compatible --base-url http://localhost:8000/v1 --llm-model <served model>

The published thresholds were fitted on gemini-3.7-flash outlines. With another extractor every record carries a warning, and calibrating on your own human documents (step 8) restores a known false-positive rate.

6. Give formats yourself, or classify locally

A record's format field, or --format for the whole file, skips classification. A format outside the eight is refused unless you pass --force-fit, which maps it to the closest one. --method weborganizer classifies with WebOrganizer's encoder on your own machine instead of an LLM (pip install "idealens[weborganizer]").

idealens run reviews.jsonl -o scores.jsonl --format "User Reviews"
idealens classify docs.jsonl -o formats.jsonl --method weborganizer --device cuda

7. Choose how the model runs

--weights PATH scores with a local copy of a model's weights instead of downloading them (a directory, or the .npz file for the logistic models). In Python, Detector passes extra keyword arguments to the backend, e.g. il.Detector("IdeaLens", tensor_parallel_size=2).

8. Choose the operating point, or calibrate your own

Every record carries verdicts at all calibrated false-positive rates and schemes (see Verdicts and thresholds); --fpr and --scheme pick the default one. For a new domain, fit cuts on human documents from it and score against them:

idealens calibrate my_humans.jsonl --save-as my_domain --model IdeaLens --group-by source
idealens score outlines.jsonl -o scores.jsonl --thresholds my_domain --group-by source --scheme group:source

9. Large jobs

--dry-run prices a run before anything is sent; idealens cost out.jsonl totals a finished one. Every command appends to its output and can be re-run after an interruption: finished records are skipped and failed provider calls are retried. --chunk sets how many records are scored per write, --workers how many online requests run at once.

10. Without this package

Each model card on Hugging Face shows how to load the weights with transformers or vLLM and read P(human) directly, and each model repo holds its thresholds.json.

Models

idealens models lists them. All read the output of the same extraction step, except the two ProseLens models, which read the document itself.

Per-item models score each item and pool the item scores by their mean log-odds. Each model uses its own backend unless you pass --backend.

The package does not paraphrase outlines. IdeaLens was trained and calibrated on paraphrased outlines; IdeaLens-NoParaphrase was trained on outlines as extracted, and is the closer match to what this package produces.

Verdicts and thresholds

A document is flagged as AI when its P(human) is strictly below the cut. The default verdict uses the 1% global cut: the score below which 1% of the human calibration documents fall. Each record carries:

Choose another operating point with --fpr 0.005 --scheme per_format. When a document has no cut under a scheme (its format or topic was not calibrated, or its format was force-fitted), that verdict is None with a reason. It never falls back to the global cut.

The published cuts were fitted on 80,000 human documents with outlines extracted by gemini-3.7-flash with the six worked examples. Other extractor models work, but their outlines can shift the score distribution, so the calibrated false-positive rate is no longer guaranteed. Every record says so in warnings.

Calibrating on your own data

If your documents differ from web text, fit cuts on human documents from your own domain:

idealens calibrate my_humans.jsonl --save-as my_domain --model IdeaLens
idealens score outlines.jsonl -o scores.jsonl --thresholds my_domain

The input can be scored records, outlines or documents; the missing steps are run first. --group-by FIELD fits a cut per value of a field, and --label-field / --human-value let you pass a mixed file and calibrate on its human rows. A cut is fitted only where there are enough documents to estimate it (at least 25 expected below the cut and 200 in the group). Profiles are saved under ~/.config/idealens/thresholds (or $IDEALENS_HOME/thresholds).

Formats

The detectors were trained on eight formats: Academic Writing, Creative Writing, Knowledge Article, News Article, Nonfiction Writing, Personal About Page, Personal Blog and User Reviews. A document in another format is assigned the closest of the eight by a second LLM pass (or by the encoder's probabilities). Such a document has forced_format: true, keeps its original label in original_format, and gets no per-format verdict.

If you know a document's format, give it in the record's format field or with --format; classification is then skipped. idealens run --check-format classifies anyway and reports disagreements.

Cost

--dry-run estimates the tokens and cost of a run before anything is sent, and idealens cost output.jsonl totals a finished one. Extraction dominates: the prompt carries six worked examples, and reasoning tokens make up most of the output. Batch mode (--mode batch) costs half as much where the provider has a batch API (Gemini on Vertex, OpenAI, Anthropic); not every model is served by it.

Hardware

The vLLM backend checks that the weights fit before it starts, and stops with a message otherwise. Use tensor_parallel_size (Detector(..., tensor_parallel_size=2)) to spread a model over two smaller GPUs.

The paper's code, prompts and results

paper/ holds what the paper's appendix points to: the full result table of every evaluation (paper/results/), the prompts, role vocabularies and worked examples (paper/prompts/), and the research code that built the corpus, trained the models and ran the evaluations (paper/code/). The package above is the way to run the detectors; paper/code/ documents exactly what ran for the paper.

Development

pip install -e ".[dev]"
pytest tests

The tests need no GPU, network or API key.

License

The code, prompts and results in this repository are released under CC BY-NC-SA 4.0: free to share and adapt for non-commercial purposes, with attribution and under the same license. Versions 0.1.0 to 0.1.3 were released under Apache 2.0. Each model and dataset has its own license, given on its page.

Citation

If you use IdeaLens, ProseLens or the datasets, please cite the paper (arXiv:2610.06778):

@article{idealens2026,
  title         = {IdeaLens: Detecting AI Ideas in Long-form Writing},
  author        = {Rajendhran, Rishanth and Choi, Minjoon and Russell, Jenna and Namuduri, Ramya and B{\"o}l{\"o}ni-Turgut, Deniz and Karpinska, Marzena and Wieting, John and Iyyer, Mohit},
  journal       = {arXiv preprint arXiv:2610.06778},
  year          = {2026},
  eprint        = {2610.06778},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.06778}
}