Skip to main content
This example is an upgraded version of the original retool example, using the updated interfaces provided by the miles framework to implement multi-turn RL training with tool calls in a cleaner way.

​
Key Differences from v1

v1 (retool) requires manually implementing the full multi-turn conversation loop in generate_with_retool.py, directly depending on low-level GenerateState and sglang_rollout interfaces — resulting in verbose code tightly coupled to the framework internals. v2 (retool_v2) uses the framework’s standard plugin interfaces. Users only need to implement three functions and mount them via command-line arguments: Users only need to focus on business logic (tool definitions, tool execution, reward calculation). Multi-turn scheduling, token concatenation, loss masking, etc. are all handled by the framework.

​
Files

  • tool_sandbox.py: Tool definitions (tool_specs), tool execution (execute_tool), reward function (reward_func), and sandboxed safe execution environment
  • run_retool_multi_turn.py: Training launch script

​
Quick Start

The launch script prepares everything it needs on its own: it downloads the dapo-math-17k training set and the aime-2024 eval set, downloads the checkpoint, and converts it to torch_dist before training starts.