SWE-ZERO 12M Trajectories

The largest agentic-coding trace dataset to date: 112 B tokens of execution-free agentic trajectories covering 122 K pull requests, 3 K repositories, and 16 programming languages.

Motivation

Agentic mid-training has become a standard ingredient for frontier coding models:

• Code World Model (Sept 2025) mid-trains on 5 T tokens, using 3 M agentic trajectories from 10.2 K Docker images across 3.15 K repos.
• Kimi-Dev (Dec 2025) mid-trains on ~150 B tokens of "high-quality and real-world data" to instill BugFixer and TestWriter priors.
• DeepSeek-V4 (May 2026) incorporates agentic data during mid-training to enhance coding capability.

All three rely on containerized execution to verify trajectories, and that is where the data ceiling sits. Kimi-Dev was first to call it out:

"while repository snapshots from GitHub are available, not all snapshots are equipped with an executable Docker environment."

The current largest containerized collection, SWE-rebench-V2, ships 32 K tasks with pre-built images, and concedes the ceiling by additionally releasing 120 K+ tasks that could not be containerized (4× more) with only install instructions and fail-to-pass metadata, because building those images is prohibitively expensive.

SWE-ZERO (prior work) was the first to propose an execution-free pipeline for agentic SWE training data, framing the two Docker-imposed bottlenecks:

• Data scalability: a large fraction of real-world repositories and PRs are discarded because intricate or legacy build configurations do not work in modern containerized environments.
• Training scalability: orchestrating thousands of task-specific Docker images creates massive infrastructure overhead that complicates large-scale optimization and RL.

SWE-ZERO sidesteps the Docker requirement by instructing the model not to run repo-specific commands like pytest, npm test, or build scripts that depend on the project's installed environment. The model is still free to explore and edit the repository using standard POSIX commands that need no setup: grep, find, cat, sed, ls, git, and so on. None of those require the repo to be built or its dependencies installed, so the pipeline can run against any GitHub PR snapshot and unlocks the long tail of PRs that prior pipelines must throw away.

SWE-ZERO released 9 B tokens, and we scale this 10× to 112 B tokens using the same execution-free recipe. Coverage:

• 12,290,800 rollouts
• 122,908 unique PRs
• 3,222 repositories
• 16 programming languages

To our knowledge it is now 5.7× larger than the previous largest agent-trace dataset, open-thoughts/AgentTrove (111.06 B vs 19.41 B tokens):

Experiment validation

Diversity check (MinHash-64). We follow the diversity protocol from the Code World Model paper. It measures the mean Jaccard similarity between rollouts in the same PR, on bash-command shingles. Anything under 0.5 is considered diverse. On our corpus, we measure 0.2730, well under the threshold.

Training check. To verify the trajectories actually transfer into model capability, we trained Marin-8B base on three subsets of an earlier snapshot of this dataset (@ab6295c3, 1.6 M rollouts / 18.4 B tokens) and evaluated on SWE-bench Verified (100 random tasks, mini-swe-agent v1 + Harbor, 3 runs per model, temperature 1.0):

Resolve rate scales consistently with training data through the 1 B-token point we tested. For methodological simplicity these runs use the standard SFT masked-loss recipe, but this corpus is intended as a mid-training dataset. Downstream models should be further post-trained on dedicated SFT datasets such as open-thoughts/AgentTrove, nvidia/SWE-Hero-openhands-trajectories, and ricdomolm/mini-coder-trajs-400k. Full methodology, per-task breakdown, and other experiment details: marin-community/marin#4898 (comment).

Methodology

Each rollout starts from a real GitHub PR snapshot in nebius/SWE-rebench-V2-PRs, which provides real-world software-engineering tasks spanning 3,222 repos. We sample 100 independent multi-turn rollouts per PR from ricdomolm/mini-coder-1.7b (a compact 1.7 B-parameter code model that scores 50.4 pass@100 on SWE-bench Verified) at temperature 1.0. Trajectories follow the mini-swe-agent v1 format: a messages list where the assistant issues one bash command per turn and the harness returns the command output.

Following SWE-ZERO, generation is fully execution-free: no container is built, no test is executed, no verifier is consulted. That is what unlocks the scale.

Schema

Sampling settings

Limitations

This is a mid-training corpus, not an SFT dataset. Treat it accordingly.

• Skill acquisition, not task completion. The goal of this corpus is to instill agentic tool-use priors in a base model at scale: how to drive a bash shell, navigate a repository, read and edit files, and interleave thinking with actions. It is not designed to teach the model to solve SWE-bench-style tasks end-to-end. That is what downstream RL and SFT stages are for.
• No verification. Because SWE-ZERO is execution-free, no trajectory has been checked against ground-truth tests. Patches are not known to compile, apply cleanly, or pass. Correctness was deliberately traded for scale and repo coverage.
• Low submission rate. Most rollouts do not reach a successful Submitted exit status; many exit incomplete or hit other terminal conditions. If you filter to clean submissions only, you will lose the bulk of the corpus.
• 15-turn cap. Trajectories are truncated at 15 model turns to keep p99 token cost bounded and throughput high. Many tasks are not solvable in 15 turns and the trajectory simply ends mid-investigation.

That said, prior work shows these limitations are forgiving in practice. Both SERA and NemotronTerminal report that downstream SFT performance is robust to incomplete, failed, and length-truncated traces. Concretely, NemotronTerminal's Table 7 shows that no filtering (12.4 %) significantly surpasses both complete-only (6.74 %) and success-only (5.06 %) filtering strategies. Our own experiments with Qwen3-1.7-Base corroborate this: the model reaches 9 % on SWE-bench Verified when trained on 100K samples from this corpus truncated to 8K tokens, even though most rollouts do not end with a patch submission. Full details in marin-community/marin#5611 (comment).

Acknowledgements

Thanks to TPU Research Cloud for sponsoring all the compute for this project!

Citation

@misc{li2026swezero12m,
  author       = {Xiang Li},
  title        = {{SWE-ZERO-12M-trajectories: 112B Tokens of Execution-Free Agentic Trajectories}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories}},
}