radixark/miles

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

2,779

stars

2,115

commits

Python

primary language

Sep 10, 2026

updated

miles.radixark.com Browse cluster: LLM Inference Optimization & Serving →

README

Enterprise-Grade Reinforcement Learning for Large-Scale Model Post-Training

News

[2026/08] 🔥 Miles v0.1 is released! Read the blog post here: Miles v0.1: Production-level Post-training.
[2026/07] Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles (blog).
[2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 (blog).
[2026/07] On-policy distillation lands in Miles (blog).
[2026/07] 🔥 SGLang and Miles add day-0 support for Inkling, a frontier multimodal model (blog).
[2026/07] DeepSeek-V4 Flash RL training comes to AMD Instinct MI355X with Miles (blog).
[2026/06] SGLang and Miles add day-0 support for NVIDIA Nemotron 3 Ultra (blog).
[2026/05] No token left behind: token-in-token-out in Miles (blog).
[2026/04] Updating 1 T parameters in seconds: P2P weight transfer in large-scale distributed RL (blog).
[2026/04] 🔥 DeepSeek-V4 on day 0: from fast inference to verified RL with SGLang and Miles (blog).

About

Miles is a high-performance, enterprise-ready reinforcement learning framework for large-scale model post-training. It pairs SGLang for high-throughput rollout with Megatron-LM for scalable training, and ships the precision, stability, and observability features an RL run needs at trillion-parameter scale. A PyTorch FSDP2 backend is available for runs that would rather train the HuggingFace implementation as-is, though the recipes, the parallelism, and the largest models all live on Megatron-LM. See Training Backends.

"A journey of a thousand miles begins with a single rollout."

Performance

Fully async RL. Rollout and training workers are decoupled, with configurable on- and off-policy schedules, a pipeline tuned for fewer bubbles, and customizable async rollout and eval modes. See Fully Async RL.
Fast agentic rollout. Generation runs on SGLang behind a router that spreads requests across engines, preserves per-request metadata, and health-checks the fleet. Tuned for multi-turn agentic workloads.
Fast weight updates. New weights reach the engines in-loop in seconds, even on a trillion-parameter model such as Kimi-K2.6, with P2P RDMA as the fast path for disaggregated setups.
Low-precision training. MXFP8 and NVFP4 training with a numerically stable RL recipe that reduces precision-induced divergence. FP8, INT4 QAT, BF16, and FP16 are also supported.
LoRA and multi-LoRA. Low-rank adapters train frontier-scale models on a fraction of the GPUs, and the same adapters load straight into SGLang for rollout.

Correctness and resilience

Token-in-token-out (TITO). Supported for every model and every black-box harness, with no detokenize/retokenize round-trip between rollout and training.
Rollout Routing Replay (R3). Expert routing recorded during rollout is replayed in the trainer's forward pass, removing the MoE routing mismatch that destabilizes large runs, with compute and communication overlapped to keep the cost down.
Fault tolerance. When an SGLang engine dies, Miles recovers it and resumes the run in place: no restart, no pause.

What Miles runs

Day-0 model support. DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling, and Nemotron landed on release day. Beyond day 0, nearly every frontier model runs on Miles, including Kimi-K2.6 and Qwen3.5. See Models.
Extensive hardware support. NVIDIA GB300, GB200, B300, B200, H200, H100, and A100, and AMD MI300X, MI325, MI350, and MI355X via ROCm. See Installation for per-GPU status and the container image for each.
Wide recipe support. GRPO, GSPO, PPO, and REINFORCE++ for RL, plus SFT and on-policy distillation.
Agentic environments. Train coding and computer-use agents through connectors for Harbor, HUD, NeMo Gym, OpenEnv, Verifiers, and more, each plugging into the rollout layer that fits it, with task sandboxes on AgentENV, Daytona, E2B, or Modal. See Agentic Environments.
Diffusion models. Flow-GRPO, DiffusionNFT and SFT on an sglang-diffusion rollout engine and an FSDP2 trainer, in Miles-diffusion.

Getting Started

Acknowledgment

Miles was forked from slime, and integrates SGLang, Megatron-LM, and torch_memory_saver.

Miles is shaped by the teams that build on it and support its development, from hardware and cloud to model labs, agent infrastructure, and academia:

Organizations building on, contributing to, and collaborating with Miles

Citation

If Miles is useful in your research or your product, please cite it:

@misc{miles2026,
  title        = {Miles: Enterprise-Grade Reinforcement Learning for Large-Scale Model Post-Training},
  author       = {Miles Team},
  year         = {2026},
  howpublished = {\url{https://github.com/radixark/miles}}
}

Contributors

(top 30 of 118)

788 commits

281 commits

200 commits

180 commits

See what people are saying

X (2)

🚀 Trending on GitHub miles by radixark ⭐ 2,592 (64 stars today) 🧠 An enterprise-grade reinforcement learning framework for post-training large language and vision-language models. https://t.co/fO4G2ipMcw #GenAI #OpenSource #AI

@TheArthiAI · 1 likes · Sep 5, 2026

Similar repos

radixark/miles_diffusion

Miles-diffusion is an post-training framework for large-scale diffusion model training and…

shared contributors

THUDM/slime

slime is an LLM post-training framework for RL Scaling.

8,435

shared contributors

volcengine/verl

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

23,379

shared contributors

verl-project/verl

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

23,389

shared contributors

PrimeIntellect-ai/prime-rl

Agentic RL Training at Scale

2,030

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

35,766

shared contributors

lkm2835/sglang

SGLang is a fast serving framework for large language models and vision language models.

shared contributors

alibaba/ROLL

An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models

3,391

Languages

Python

97.7%

JavaScript

1.2%

radixark/miles

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

2,779

stars

2,115

commits

Python

primary language

Sep 10, 2026

updated

miles.radixark.com Browse cluster: LLM Inference Optimization & Serving →

README

Enterprise-Grade Reinforcement Learning for Large-Scale Model Post-Training

News

[2026/08] 🔥 Miles v0.1 is released! Read the blog post here: Miles v0.1: Production-level Post-training.
[2026/07] Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles (blog).
[2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 (blog).
[2026/07] On-policy distillation lands in Miles (blog).
[2026/07] 🔥 SGLang and Miles add day-0 support for Inkling, a frontier multimodal model (blog).
[2026/07] DeepSeek-V4 Flash RL training comes to AMD Instinct MI355X with Miles (blog).
[2026/06] SGLang and Miles add day-0 support for NVIDIA Nemotron 3 Ultra (blog).
[2026/05] No token left behind: token-in-token-out in Miles (blog).
[2026/04] Updating 1 T parameters in seconds: P2P weight transfer in large-scale distributed RL (blog).
[2026/04] 🔥 DeepSeek-V4 on day 0: from fast inference to verified RL with SGLang and Miles (blog).

About

"A journey of a thousand miles begins with a single rollout."

Performance

Fully async RL. Rollout and training workers are decoupled, with configurable on- and off-policy schedules, a pipeline tuned for fewer bubbles, and customizable async rollout and eval modes. See Fully Async RL.
Fast agentic rollout. Generation runs on SGLang behind a router that spreads requests across engines, preserves per-request metadata, and health-checks the fleet. Tuned for multi-turn agentic workloads.
Fast weight updates. New weights reach the engines in-loop in seconds, even on a trillion-parameter model such as Kimi-K2.6, with P2P RDMA as the fast path for disaggregated setups.
Low-precision training. MXFP8 and NVFP4 training with a numerically stable RL recipe that reduces precision-induced divergence. FP8, INT4 QAT, BF16, and FP16 are also supported.
LoRA and multi-LoRA. Low-rank adapters train frontier-scale models on a fraction of the GPUs, and the same adapters load straight into SGLang for rollout.

Correctness and resilience

Token-in-token-out (TITO). Supported for every model and every black-box harness, with no detokenize/retokenize round-trip between rollout and training.
Rollout Routing Replay (R3). Expert routing recorded during rollout is replayed in the trainer's forward pass, removing the MoE routing mismatch that destabilizes large runs, with compute and communication overlapped to keep the cost down.
Fault tolerance. When an SGLang engine dies, Miles recovers it and resumes the run in place: no restart, no pause.

What Miles runs

Day-0 model support. DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling, and Nemotron landed on release day. Beyond day 0, nearly every frontier model runs on Miles, including Kimi-K2.6 and Qwen3.5. See Models.
Extensive hardware support. NVIDIA GB300, GB200, B300, B200, H200, H100, and A100, and AMD MI300X, MI325, MI350, and MI355X via ROCm. See Installation for per-GPU status and the container image for each.
Wide recipe support. GRPO, GSPO, PPO, and REINFORCE++ for RL, plus SFT and on-policy distillation.
Agentic environments. Train coding and computer-use agents through connectors for Harbor, HUD, NeMo Gym, OpenEnv, Verifiers, and more, each plugging into the rollout layer that fits it, with task sandboxes on AgentENV, Daytona, E2B, or Modal. See Agentic Environments.
Diffusion models. Flow-GRPO, DiffusionNFT and SFT on an sglang-diffusion rollout engine and an FSDP2 trainer, in Miles-diffusion.

Getting Started

Acknowledgment

Miles was forked from slime, and integrates SGLang, Megatron-LM, and torch_memory_saver.

Miles is shaped by the teams that build on it and support its development, from hardware and cloud to model labs, agent infrastructure, and academia:

Citation

If Miles is useful in your research or your product, please cite it:

@misc{miles2026,
  title        = {Miles: Enterprise-Grade Reinforcement Learning for Large-Scale Model Post-Training},
  author       = {Miles Team},
  year         = {2026},
  howpublished = {\url{https://github.com/radixark/miles}}
}