Scaling Automated Post-Training

1. Introduction

This repository accompanies the blog post Scaling Automated Post-Training. It contains the figures used in the post. Full solutions and relevant artifacts from the reported results to follow.

The artifacts/ directory contains released PostTrainBench training artifacts for the reported 1×H100 / 10-hour and unlimited-budget results. See artifacts/README.md for the layout and composite scores.

Locus is Intology's automated research system. In this work, we evaluate an updated version of Locus on automated LLM post-training and related long-horizon ML tasks. Locus sets a new state of the art on PostTrainBench, continues to improve under the larger-compute PostTrainBench+ setting past where frontier coding-agent baselines plateau, places ahead of human competitors across live prize-money Kaggle competitions, and post-trains a production model for Bubble with substantially lower error, latency, and cost.

2. Results

PostTrainBench

Under the official PostTrainBench setting, Locus with Opus 5 scores 44.7, ahead of Claude Code (Fable 5) at 41.8 and the remaining baselines.

Figure 1: Official Tier 1 PostTrainBench composite comparison.

PostTrainBench+

Given multiple orders of magnitude more compute, Locus reaches a composite score of 51.6%, surpassing the human-tuned official Qwen3-1.7B-Instruct checkpoint (49.4%) and every measured baseline.

Figure 2: PostTrainBench+ composite score against adjusted H100 GPU-hours.

Large-scale reasoning post-training

On AIME 2025 under PostTrainBench+, Locus reaches 20%, double the next-best baseline. Its solution trajectory also scales with training-token count more clearly than the coding-agent baselines.

Figure 3: AIME 2025 final-score comparison under PostTrainBench+.

Figure 4: AIME 2025 accuracy against per-solution training-token count.

Solution diversity

Under PostTrainBench+, Locus explores a larger number of unique approaches per benchmark than the baselines.

Figure 5: Unique approaches explored per benchmark under PostTrainBench+.

Live Kaggle competitions

Across all prize-money competitions with public leaderboards that were live at evaluation time, Locus achieved a higher peak average rank than all other competitors.

Figure 6: Percent of human teams beat by Locus on each competition at the pinned snapshot.

Production deployment

For Bubble, Locus discovered and trained a language model that runs in production at ~2.8× lower error, ~5.4× lower latency, and 105× lower cost than the legacy system.

Figure 7: Relative Bubble production metrics for the Locus-trained model versus legacy.

3. Citation

@article{intology2026scaling,
  title={Scaling Automated Post-Training},
  author={Intology},
  year={2026},
  url={https://www.intology.ai/blog/scaling-automated-post-training}
}

4. License

This repository is released under the MIT License. See the LICENSE file for more details.