MELD

Reference implementation of MELD, an AI-vs-human text detector. Training, scoring, and evaluation. Self-contained.

Layout

meldv3/
  model.py       # encoder + style basis, human anchors, family/operation prototypes
  losses.py      # per-family one-vs-human BCE under a group-DRO objective
  data.py        # JSONL dataset, token-budget batching, collator
  train.py       # DDP trainer
  evaluate.py    # chunked scoring, AUROC / TPR@FPR
  calibrate.py   # human-quantile thresholds
  spans.py       # token-score curve -> AI spans
  families.py    # generator string -> family task
scripts/
  score_text.py       # score a document or a JSONL
  prepare_data.py     # build the training mixture
  eval_pools.py       # evaluate a checkpoint on held-out pools
  enroll.py           # add a new generator from a handful of samples
  finalize_release.py # bake thresholds into a checkpoint
  smoke_cpu_test.py   # CPU logic tests, no GPU
configs/

Install

pip install -r requirements.txt
python scripts/smoke_cpu_test.py       # optional: CPU logic tests

The default backbone is jhu-clsp/ettin-encoder-400m.

Score

Released weights are on the Hugging Face Hub as anon-review-meld-2026/meld; download that repository into a folder and point --model_dir at it.

# released model folder (model.safetensors + meld_config.json + config.json)
python scripts/score_text.py --model_dir path/to/model --text "..."

# or a checkpoint from training
python scripts/score_text.py --checkpoint runs/x/meldv3_release.pt --text "..."

# a whole file, one JSON object per line
python scripts/score_text.py --model_dir path/to/model --jsonl docs.jsonl

Thresholds ship inside the model. --fpr 1|5|10 picks the operating point, --stratum picks a per-domain one, --threshold overrides both. Give the model at least 100 words; scores on very short text run high regardless of who wrote it.

Train

Rows are JSONL: {"text": ..., "label": 0|1, "generator": ..., "domain": ...}, label 1 for AI. prepare_data.py turns raw corpora into that schema; configs/family_map_v3.json maps generator strings onto family tasks.

torchrun --nproc_per_node=4 -m meldv3.train \
    --data_dir data/v5 --output_dir runs/meld

Defaults are the released configuration; -h lists the rest. Training writes checkpoint_best.pt (selected on held-in validation only), checkpoint_last.pt, and an eval trajectory. Run finalize_release.py on the best checkpoint to bake thresholds in.

Spans and enrollment

python -m meldv3.spans --model_dir path/to/model --text_file doc.txt
python scripts/enroll.py --model_dir path/to/model --samples new_generator.jsonl \
    --family_name my_model --out ckpt_enrolled.pt

Spans need no span labels; enrollment needs no gradients. Every entry point takes --model_dir or --checkpoint. A .pt does not carry the encoder's architecture config, so add --backbone_dir path/to/model to load one without reaching the network.

License

MIT (see LICENSE).