> ## Documentation Index
> Fetch the complete documentation index at: https://miles.radixark.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# MiniMax H3

> Text-to-audio-video Flow-GRPO + PickScore on MiniMax H3 — the verified 2-GPU recipe.

## 1. Model introduction

[MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) is MiniMax's omni-modal generative system: one transformer
denoises a packed text + video + audio sequence and emits video with native stereo audio. MiniMax distilled CFG
into the checkpoint, so the official setting has **no classifier-free guidance** — the recipe is unguided
(`--diffusion-guidance-scale 1.0`). The Flow-GRPO recipe trains the **t2va** (text-to-audio-video) workflow, video
branch only — the audio stream is rolled out but not trained.

## 2. Supported variants

| Model | HF ID | Notes |
| - | - | - |
| MiniMax-H3 | [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) | t2va video-only Flow-GRPO; t2va LoRA SFT (joint audio/video forward, the recipe supervises video only); ref2va joint audio/video LoRA SFT (`scripts/run_diffusion_sft_h3_ref2va.py`). `fl2va` is not trained yet, and ref2va has no RL path. |

## 3. Family config

From `miles/backends/fsdp_utils/configs/h3.py`:

| Property | Value | Why |
| - | - | - |
| CFG training | Off (asserted) | CFG is distilled into the checkpoint; the forward is unguided |
| Weight sync | Any mode | Rollouts load the native **FL2VA** partition; full weights are converted to its layout by the `h3` ParamTransform in `models/diffusers/h3/weight_bridge.py` (native names, fused QKV in per-head rows, `[gate, value]` FFN), LoRA adapters (IPC or NCCL) in sgl-d's `lora_merge`. Full-weight sync relies on the rollout DiT staying resident, which miles guarantees by pinning sgl-d's `performance_mode` to `manual` (sgl-d's layerwise offload does no TP slicing on tensor updates) |
| LoRA targets | `attn.to_{q,k,v}`, `attn.to_out.0`, `ff.net.0.proj`, `ff.net.2` | Trainer pushes PEFT/diffusers names; sglang-d `lora_merge` maps and fuses QKV / swaps FFN |
| Optimizer state | `audio` allowed missing | GRPO, and SFT with `--fsdp-supervised-streams visual`, never train the audio branch |
| SFT streams | `--fsdp-flow-shift visual=...,audio=...` (asserted) | Every H3 SFT pair carries an audio stream noised on its own schedule — the target's soundtrack, or zero latents for a silent clip, which never enter the loss; `--fsdp-supervised-streams visual` keeps it out of the loss |
| Sample micro-batch | 1 (asserted) | One packed sequence per forward |
| Forced sampling params | `task=t2va`, `short_edge=768`, `conditions=[]` | 768 is the only short edge MiniMax verifies (sgl-d warns on any other value), so miles pins it and does not expose it; `task` follows `--diffusion-task`, and H3 rollout serves only `t2va` (asserted), with no conditions |

## 4. Launch

### 4.1 LoRA SFT on (video, prompt) pairs (8 GPUs, no rollout engines)

Recipe: `scripts/run_diffusion_sft_h3_t2va.py`

**Status:** [📈 V — Verified](/docs/diffusion/user-guide/recipe-verification.md#v)

```bash theme={null}
python3 scripts/run_diffusion_sft_h3_t2va.py   # downloads the default dataset on first run
# custom data: --extra-args "--prompt-data /abs/train.jsonl"
```

### 4.2 Flow-GRPO + PickScore (2 GPUs)

Canonical recipe: `scripts/run_diffusion_grpo_h3_t2va_2gpu.py` — train, rollout, and PickScore
colocated on 2 GPUs; 16:9 / 4 s (with `short_edge=768` the canvas is 1344×768 / 107 frames).

**Status:** [🛡️ FG — Fully gated](/docs/diffusion/user-guide/recipe-verification.md#fg)

```bash theme={null}
# alignment diagnostic: freeze the weights, so log_prob_mean_abs_diff measures
# train-vs-rollout deviation rather than parameter drift
python3 scripts/run_diffusion_grpo_h3_t2va_2gpu.py \
  --num-rollout 2 --eval-interval 0 --extra-args "--debug-skip-optimizer-step"

# long run at the resource-limited scale
python3 scripts/run_diffusion_grpo_h3_t2va_2gpu.py \
  --rollout-batch-size 4 --n-samples-per-prompt 16 --num-rollout 200
```

## 5. Recipe notes

SFT: the encode pool (`miles/rollout/encoder_hub/h3/`) imports the engine's own recipes
(`minimax_h3_packed_sequence`, `minimax_h3_encode_reference_video_rows`,
`minimax_h3_text_only_ids`), so cached train pairs stay bit-compatible with the engine's
rollout conditioning. Dataset clips must already sit on the serving grid (short\_edge=768
canvas, 24 fps, 17n+5 frames); `--sft-offload-encoder` keeps the \~70GB encoders (text encoder + video/audio VAEs) in
host RAM outside encode bursts, and `--train-only` (required by `--loss-type sft_loss`) runs no rollout
engine, so no weights are synced.
`scripts/export_lora.py` converts any saved checkpoint into one safetensors file (+
`adapter_config.json` sidecar) directly loadable by sgl-diffusion's `--lora-path`.

H3 solves `x0 = x + sigma * v`, the opposite sign from the flow-matching convention the shared
train and rollout steps assume, so `compute_noise_pred` returns the negated velocity. Both halves
must agree: a flipped sign still produces plausible latents and trains on garbage.

`--diffusion-num-steps N` runs N DiT forwards on the engine (N + 1 sigma endpoints); `sglang-miles`
before its bump to sglang `main` (sgl-project/sglang#38671) ran N - 1.

The rollout engine needs sglang's `sglang-miles` branch, which
[Installation](/docs/diffusion/getting-started/installation.md) already builds from
(`SGLANG_DIFFUSION_BRANCH`).

## 6. Reference results

### 6.1 LoRA SFT (8 GPUs)

Recipe defaults plus `--num-epoch 10` (the default is 3) on a 254-window video dataset (70 rollouts, 280 steps): the
aggregate loss is
dominated by which sigmas each 8-sample step happened to draw, so read convergence from the
per-bucket curves (`--log-loss-sigma-bucket`) instead, which trend downward.

![MiniMax H3 SFT loss](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/h3/sft_loss.png?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=c1d58a1c4e80d88c9f5d700dd83ba9fb)

### 6.2 Flow-GRPO + PickScore (2 GPUs)

Long run (`rollout_batch_size=4`, `n_samples_per_prompt=16`):

* `train/log_prob_mean_abs_diff` stays in **1.4e-5 .. 7.1e-5** (under `1e-4`).
* Eval PickScore (`eval/pickscore_val`, every 10 rollouts): **0.806 → 0.814**, peak **0.816** at rollout 59.
* Train `rollout/reward/raw_mean` sits around **\~0.79** (64 samples / rollout, 32 per optimizer step; noisier than eval).

This is the batch-4 curve; it is not evidence for a larger topology, and no larger canonical H3
topology is committed yet.

![MiniMax H3 log\_prob\_mean\_abs\_diff](https://mintcdn.com/radixark/-43MEB1vZnsTAlRx/diffusion/assets/images/h3/log_prob_diff.png?fit=max\&auto=format\&n=-43MEB1vZnsTAlRx\&q=85\&s=bc2e443dea76d41c616d2d57b2cd9670)

![MiniMax H3 eval PickScore](https://mintcdn.com/radixark/-43MEB1vZnsTAlRx/diffusion/assets/images/h3/reward_eval.png?fit=max\&auto=format\&n=-43MEB1vZnsTAlRx\&q=85\&s=0e29cca9bbcc70f251a7c1f5d8afbd75)

![MiniMax H3 train rollout PickScore](https://mintcdn.com/radixark/-43MEB1vZnsTAlRx/diffusion/assets/images/h3/reward_rollout.png?fit=max\&auto=format\&n=-43MEB1vZnsTAlRx\&q=85\&s=51edbcb711009a19ec45693c09297218)

## 7. Pairs well with

* [LoRA Training and Weight Sync](/docs/diffusion/advanced/lora.md) — the IPC merge (`--lora-ipc-weight-sync` with `--colocate`) the Flow-GRPO recipe uses.
* [Rewards](/docs/diffusion/user-guide/rewards.md) — PickScore worker pool configuration.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.