Skip to main content
Miles owns the training side of environment RL — batch orchestration, lossless token-in/token-out recording, reward hooks, filtering — and is agnostic about where the environment itself comes from:
  • No environment — single-turn RLVR: a prompt dataset scored by the built-in rule-based rewards (math, ifbench, …) or a custom reward function.
  • Your own environment — plug your code into one of the three rollout layers described in Integration shapes; most environments sit in the agent function, with the session server recording tokens (see Rollout Endpoints).
  • An external ecosystem — adopt a prebuilt connector from the table below; connectors occupy the same three layers.
Sandbox providers are a different axis: they provision the task containers inside a connector rather than occupying a rollout layer. All external ecosystem support is experimental.

​
Integration shapes

The rollout stack is three nested plug-in layers (see Customization): each column in the table below wraps the one to its left, so replacing an outer layer also takes over everything an inner one would. A connector replaces exactly one layer. ✓ = the external framework takes it over; ○ = stays in Miles. ¹ Typically by speaking SGLang’s native /generate (token IDs in and out) rather than the session-server chat endpoint Miles’ own recording uses. ² The environment may grade an episode itself (Harbor and τ-bench do); the score still enters training through Miles’ Sample.reward / RM hooks, and group-level reward handling stays in Miles.