Miles is an open RL framework to make training simple, efficient, and stable.
Proven at frontier scale and in production at labs and enterprises.
Production Features
Built for agentic RL
Native SGLang rollout, with fully async, Token-in-Token-Out (TITO), P2P weight updates, and Rollout Routing Replay (R3).
Stable and fast training
New training features like NVFP4, disk offload, and zero-KL. Support multi training backend across Megatron-LM and FSDP.
Recipes for the whole post-training stack
SFT, RL, and on-policy distillation (OPD), full-parameter or LoRA.
Day 0 support for frontier models
Verified support for frontier open LLMs and diffusion models, from the day they ship.
Extensive hardware supports
NVIDIA Blackwell, Hopper, and Ampere, AMD Instinct via ROCm.
The agentic environments you already use
Integrations with Harbor, HUD, NeMo Gym, OpenEnv, and Verifiers, with task sandboxes on AgentENV, Daytona, E2B, or Modal.
How People Use Miles
“Miles has been a powerful framework for our agentic RL work. Its efficiency, robustness, and advanced features for large-scale models allowed us to iterate faster and scale experiments with ease.”Rameswar PandaIBM
Supported Models
Qwen
FAQ
Miles is an open-source framework for large-scale model post-training, with a particular focus on reinforcement learning and agentic workloads. It combines SGLang for high-throughput rollouts with Megatron-LM or FSDP2 for distributed training.
Miles manages complete trajectories containing model responses, tool calls, observations, and environment interactions. Its session server and token-in-token-out (TITO) data path preserve the exact tokens and metadata produced during rollout, even when the agent uses an external harness.
Yes. Miles provides integrations for frameworks such as Harbor, HUD, NeMo Gym, OpenEnv, and Verifiers, as well as sandbox providers including Daytona, E2B, Modal, and AgentENV. Custom rollout, reward, and data-processing functions can also be added without modifying the core training loop.
Miles supports both synchronous and fully asynchronous RL. Its sample-level scheduler replaces completed trajectories immediately, while a bounded data buffer allows rollout generation and optimizer steps to proceed concurrently — even when trajectory lengths vary widely.
No. Miles supports GRPO, GSPO, PPO, REINFORCE++, on-policy distillation, and supervised fine-tuning. Its reward functions, advantage computation, loss, data selection, and policy-lag controls are customizable for new research ideas.
Miles includes recipes for model families such as DeepSeek, Kimi, GLM, Qwen, and Nemotron, including dense, MoE, and multimodal architectures. It supports distributed execution across NVIDIA and AMD accelerators, with low-precision training options such as FP8, MXFP8, and NVFP4 where supported.
Miles runs on infrastructure you control, so your datasets, trajectories, checkpoints, and model weights remain within your deployment. You decide how artifacts are stored, evaluated, exported, and served.
The recommended starting point is the Docker-based quick start, which runs a complete GRPO loop on a supported model. From there, you can replace the model, dataset, reward function, or rollout implementation incrementally rather than building an agentic RL stack from scratch.