WildAI models

License: CC BY-NC-SA 4.0. The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.

The language models released with How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text, from 19.9M to 973M parameters, trained on controlled mixtures of human and wild AI-generated web text. Each model is in the subfolder named after it, with its own model card; load one by passing that name as subfolder, which downloads only that model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("pangram/WildAI-models", subfolder="19.9m-g001-control", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pangram/WildAI-models", subfolder="19.9m-g001-control", trust_remote_code=True, dtype=torch.bfloat16)

inputs = tokenizer("The history of the printing press", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=40, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

The tokenizer prepends <|bos|> to every text, as in training; pass add_special_tokens=False to skip it. The model predicts <|bos|> where a document ends, and generation stops there. Weights are stored in bfloat16, the precision the paper's losses were computed in; dtype=torch.float32 computes in float32 instead.

Models

Names are <size>-<group>-<arm>[-r<ratio>]. A group g001-g099 is one human-only control (control) and every model trained on exactly its human documents plus added AI text (ai), fresh human text (human) or its own human text repeated (repeat), with r added tokens per human token. f01-f19 are filtering pairs: a 2026 web mix (web-mix) and the same mix with its AI documents removed (ai-removed). In the tables, r is the realized AI-to-human token ratio and C4 the held-out loss in bits per byte.

19.9M

35.8M

86.2M

135M

268M

477M

973M

License

License: CC BY-NC-SA 4.0. The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. The full text is in LICENSE.

Citation

@article{russell2026wildai,
  title   = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
  author  = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
  journal = {arXiv preprint arXiv:2609.40295},
  year    = {2026},
  url     = {https://arxiv.org/abs/2609.40295}
}