> ## Documentation Index
> Fetch the complete documentation index at: https://miles.radixark.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Hot Restart Example

> Replaces the orchestration script and rollout executor of a live run, keeping trainers and engines up.

Replace the orchestration script and the rollout executor of a running job without taking down
the trainers or the inference engines.

## Quick Start

Start the run under a name the same command can be relaunched with:

```bash theme={null}
python examples/infra_features/hot_restart/run_qwen3_0_6b_hot_restart.py \
    --cluster-backend kubernetes --namespace <your-namespace> --run-id my-run
```

While it trains, hot restart it with the same command plus one flag:

```bash theme={null}
python examples/infra_features/hot_restart/run_qwen3_0_6b_hot_restart.py \
    --cluster-backend kubernetes --namespace <your-namespace> --run-id my-run \
    --hot-restart orchestration,rollout_executor
```

Both invocations name the same W\&B run, so the relaunched orchestration script resumes reporting
into the run the original launch opened.

## TODO

* Show a hot restart that changes the rollout executor's args and resumes under them; today the
  command is repeated verbatim.