--multimodal-token-space-processing and --sglang-enable-token-space-processor, each request sends the earlier turns as SGLang already expanded them and the newest turn with one placeholder per image, which SGLang expands; this needs an SGLang build with the token-space processor, and --shared-media-dir (a directory every SGLang engine and trainer rank mounts) because geo3k stores its images as base64 data URLs.
In terms of the environment interaction, this example initializes a custom interactive environment with the APIs below.
Environment API (geo3k)
Environment API (geo3k)
build_env(sample: Sample | None = None, args: Any | None = None, **_) -> Geo3kEnv: constructs the env.reset() -> tuple[dict, dict]: clears internal state.step(response_text: str) -> tuple[dict, bool, dict]: parses the actor’s response text and update the state. Return new observation, a flag that marks whether the task is done, and step_info.format_observation(observation: dict) -> dict: converts an env observation into a chat message.
The reward model is the default math RM.
Reproduce
Qwen3.5-35B-A3B in token space
run_geo3k_vlm_multi_turn_token_space.py runs the same three-turn task on the plugin Qwen3.5 text model with its
frozen vision tower. SGLang expands each turn’s image placeholders in token space, runs the vision tower
unsharded (--sglang-mm-enable-dp-encoder) and caches the per-image features on disk; every trainer rank rebuilds
the same tensors from the shared image files in the background. Routing replay (R3) records each turn’s own rows,
so the trainer replays the experts the engine chose for every turn of the conversation.
What each file does
examples/geo3k_vlm/multi_turn/run_geo3k_vlm_multi_turn.py: downloads model, sets training/rollout args, and launches the run.examples/geo3k_vlm/multi_turn/run_geo3k_vlm_multi_turn_token_space.py: the same run for Qwen3.5-35B-A3B in token space.examples/geo3k_vlm/multi_turn/geo3k_vlm_multi_turn_config.yaml: specifiesmax_turnsandrollout_interaction_env_pathfor the multi-turn rollout.examples/geo3k_vlm/multi_turn/rollout.py: custom multi-turn rollout that calls SGLang for token generation, builds loss masks/log_probs, enforces max_turns, and early-stops on max_new_tokens.examples/geo3k_vlm/multi_turn/env_geo3k.py: geo3k tool-calling env that parses <tool_call>{…}</tool_call>, scores math answers, and returns tool feedback per turn.

