> ## Documentation Index > Fetch the complete documentation index at: https://miles.radixark.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Training an H3 LoRA with SFT > Build a MiniMax H3 fire / explosion / smoke LoRA with the 8-GPU t2va SFT recipe — topic selection, data curation, preprocessing, training and results. Follow this guide to tune **an explosion / fire / smoke reality enhancer** for MiniMax-H3. The LoRA is always-on: no trigger word, just load it. Alpha is the strength slider (128 = full, 64 = half). ## 1. Topic selection The H3 base is already very strong on aesthetics and general quality (high-grade official data, CFG distilled), so **a purely aesthetic LoRA struggles to add value** — our first attempt (a lighting LoRA) ended as a negative result. One viable differentiation is physical dynamics, which the base is *relatively* weak at (though still strong enough compared with other open source models). Topic selection decides the outcome; verify the base model's weakness with a handful of comparison generations before investing in data work. ## 2. Data curation H3 is strong. Most stock footage looks *worse* than what it generates, and metadata filters can't tell (under 20% usable yield). So we built a data pipeline: **a VLM watches the frames for coarse screening, a human GSB blind test makes the fine cut**, and preprocessing (next section) lands the winners on H3's serving grid. ``` WISA-80K (79,480) → keyword recall for splash/smoke/explosion 4,563 clips → Qwen2.5-VL-7B judges 6 frames per clip top 500 (180/160/160) → H3 generations from the same prompt + seed 500 counterparts → human GSB blind test, 500 pairs 198 dataset wins → visual dedup (frame correlation > 0.9) 193 clips → 254 training windows ``` * **VLM rubric**: five-field JSON — `category_ok / spectacle / quality / watermark / mostly_static`; total = \`0.5*spectacle + 0.3*quality + 2*category\_ok - 4*watermark * 3\*mostly\_static\`. About 5.4 s/clip on 8 GPUs — the full pool screens overnight. * **The GSB blind test is the final quality gate**: randomized sides per pair, opaque video URLs (untraceable even from devtools), source revealed after each vote. It answers one question — "is this clip actually better than what H3 generates itself?" — and only winners enter the training set. Of the 500 pairs, 198 went to the dataset, 187 to H3, and 115 were draws; a draw does not enter either, so only \~40% of screened candidates survive (splash 40% / smoke 41% / explosion 38%). On the pairs that were decided the dataset only just edges H3 out, 51%. * Long clips yield up to two non-overlapping windows (one 10 s source = 2 training samples sharing a caption). ## 3. Preprocessing Data must land exactly on H3's serving grid (the encoder rejects anything off-grid): | Item | Value | | - | - | | Canvas | short\_edge=768, 16:9 → 1344×768 | | Frame rate | constant 24 fps (strictly validated: every frame within 2 ms of its 24 fps slot) | | Frame count | 17n+5 lattice; the recipe uses 107 frames ≈ 4.46 s (`--diffusion-output-num-frames 107`) | One ffmpeg pass: `fps=24, scale=...:flags=lanczos, crop=1344:768, setsar=1`, cut to 107 frames (the SFT reader rejects clips with non-square pixels). The training manifest is a jsonl, one line per sample: ```json theme={null} {"prompt": "...", "target": {"visual": "clips/clip.mp4"}} ``` Relative paths are anchored at the jsonl's own directory — the dataset works wherever it is downloaded. The published jsonl also repeats each path under `metadata.video`, the field releases before `target` read. The output of sections 2–3 is published as [rockdu/WISA-80K-Practical-Dynamics-254](https://huggingface.co/datasets/rockdu/WISA-80K-Practical-Dynamics-254) (254 windows + train.jsonl, fully curated **and already on the serving grid**) — to train on it you skip both sections and go straight to training below. ## 4. Training ```bash theme={null} export WANDB_API_KEY=... # without it the recipe submits no wandb flags at all python3 scripts/run_diffusion_sft_h3_t2va.py # zero arguments: downloads the dataset above on first run and trains # custom data: --extra-args "--prompt-data /abs/train.jsonl" # delivery config: --num-epoch 10 (recipe default is 3) ``` Watch the run's log in wandb (project `miles-diffusion-sft`). The recipe defaults are exactly the validated optimum: **lr 3e-5, weight decay 0.01, LoRA rank 64 / alpha 128, rollout batch 32 (1 sample per GPU per optimizer step)**, `--fsdp-flow-shift visual=12,audio=3`, `--fsdp-supervised-streams visual`, `--diffusion-guidance-scale 1.0` (H3 has no CFG), `--sft-offload-encoder`. miles-diffusion started with RL and later grew to cover SFT. Read a rollout here as whatever produces the next batch of training samples — RL generates them, SFT encodes them. Each batch is then consumed by `--num-steps-per-rollout` 4 optimizer steps. Both fit the same loop, which is why `iter_N` counts rollouts rather than steps. Performance reference (8×H200, 254 windows): | Stage | Cost | | - | - | | Encoding | \~15 s/clip/GPU; the first 8 rollouts (one dataset pass) each stall \~110 s on their own misses, 230 s for rollout 0 which builds the pool; warm from rollout 8 | | Cache | keyed by the prompt, conditions, target, encoder settings, and each media file's path, size, and mtime (editing a clip in place without changing those reuses the stale entry); ≈2× dataset size, lives in `.sft_cache/` next to the jsonl, reused across runs | | Rollout overlap | rollout `i+1` is produced while rollout `i` trains. A colocated encode burst still waits for the train step, so a cold epoch stalls on encoding while a warm one hides the cache load entirely; `--rollout-num-gpus` reserves encoder GPUs and overlaps the encoding too | | Encoder residency | `--sft-offload-encoder`: the \~70 GB encoder sleeps in host RAM and only occupies the GPU during encode bursts | | Training | \~31 s per optimizer step, `--num-steps-per-rollout` 4 of them per encoded batch | | Checkpoints | saved every `--save-interval` rollouts (the recipe passes 20), at every epoch boundary, and on the final rollout; iter\_N means N rollouts completed | End to end, the default 3 epochs (21 rollouts, 84 steps) take 66 min from a cold cache; 10 epochs (70 rollouts, 280 steps) take 2 h 50 min cold, 2 h 31 min on a warm one. Training leaves DCP checkpoints only. Export the LoRA with one single command: ```bash theme={null} python3 scripts/export_lora.py --ckpt-dir /ckpt/iter_0000070 \ --out practical_dynamics_fx.safetensors --lora-rank 64 --lora-alpha 128 ``` This writes the safetensors plus an `adapter_config.json` sidecar; sgl-diffusion loads it directly via `--lora-path`. Reference loss (8×H200, same figure as the repo docs): ![MiniMax H3 SFT loss](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/h3/sft_loss.png?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=c1d58a1c4e80d88c9f5d700dd83ba9fb) ## 5. Training tips **Learning rate is extremely sensitive.** Ablation on identical data and config (rank 64 / alpha 128, 254 windows × 10 epochs): | lr | Outcome | | - | - | | 3e-4 | collapses | | 1e-4 | visible frame degradation | | 5e-5 | overfitting signs within the first epoch | | **3e-5** | **best** ✅ | | 1e-5 | underfits, barely differs from base | Other tips for DiT LoRA SFT: * **Watch bucketed loss, not the aggregate**: DiT loss magnitude varies hugely with the noise level, so the aggregate curve mostly reflects which sigmas a step happened to draw — not training progress. The per-bucket curves from `--log-loss-sigma-bucket` (default 10 buckets) compare like with like. * **Quality beats quantity**: \~200 clips that survived the GSB gate converge in 10 epochs and outperform larger, looser pools. * **Don't train audio on noisy soundtracks**: real-footage audio tracks are noisy, so the t2va recipe passes `--fsdp-supervised-streams visual`. The soundtrack is still encoded, noised and fed to the DiT as its native joint input, but excluded from the loss; drop the flag to train audio. ## 6. Results ### Showcase ![forest fire: base vs LoRA](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/h3/lora_forest_fire.webp?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=43f18c727906785df0329df4723fb442) Beyond looking better, the LoRA also fixes physical badcases of the base model — below: incense burning upside down, and smoke passing straight through glass. ![incense: base vs LoRA](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/h3/lora_incense.webp?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=44443f3aa7e0629735d68d6b523f67c7) ![smoke clipping: base vs LoRA](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/h3/lora_smoke_clip.webp?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=5dc9bc8fd823d27d2cd5d5a57182eb8f) ### Known boundaries Generalization beyond the dataset's distribution is limited. On ten out-of-distribution detailed prompts related to fire / explosions / smoke, the LoRA scored 6 wins / 3 losses / 1 tie against base — the reliable wins stay close to the training distribution. ![powder keg: base vs LoRA](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/h3/lora_powder_keg.webp?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=d21c078cd6cd0918624dd9605480262c) Lower alpha or unload for: | Scenario | What goes wrong at full strength | | - | - | | Multi-beat scripted events (e.g. fuse burns, then explodes) | later beats get consumed to amplify the current dynamic | | Restrained macro shots | over-energizes what should stay subtle | ### Artifact `practical_dynamics_fx.safetensors` (rank 64, alpha 128, \~1.4 GB) + `adapter_config.json`. ```bash theme={null} sglang serve --model-path MiniMaxAI/MiniMax-H3 \ --lora-path practical_dynamics_fx.safetensors ``` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.