> ## Documentation Index
> Fetch the complete documentation index at: https://miles.radixark.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Qwen-Image

> Flow-GRPO with PickScore on Qwen-Image — the flow_grpo-aligned 5-GPU recipe.

## 1. Model introduction

[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) is Alibaba's MMDiT text-to-image model. In
miles-diffusion it is the **reference image recipe**: the launcher is aligned flag-for-flag with
flow\_grpo's `pickscore_qwenimage` config, so a run here is directly comparable to the published
baseline.

**Key highlights for RL training:**

* **flow\_grpo-aligned.** Per-prompt reward mean with a batch-wide std (`--globalize-reward-std`
  alone) is exactly flow\_grpo's `PerPromptStatTracker` with `global_std=True`; `gaussian` LoRA
  init and `adam-beta2 0.999` match likewise.
* **RoPE caches rebuilt on CUDA.** diffusers builds `QwenEmbedRope`'s frequency tables on CPU
  while sglang-d builds them on CUDA; the fp32 ULP difference drifts every block. In one frozen-weight comparison,
  `noise_pred` mean |Δ| was approximately `2e-2`. `postprocess_model_after_materialize` rebuilds the caches on device
  after FSDP wrapping.
* **Variable-length text.** Conditioning is padded per batch with the attention mask derived
  from `txt_seq_lens`; the trainer rebuilds it and does not use a mask the engine may send for batched requests.

## 2. Supported variants

| Model | HF ID | Notes |
| - | - | - |
| Qwen-Image | [`Qwen/Qwen-Image`](https://huggingface.co/Qwen/Qwen-Image) | Family key `qwen_image` |

A checkpoint name containing `qwen-image` resolves to this config. A longer match
such as `qwen-image-2.1` resolves to [Qwen-Image 2.1](/docs/diffusion/models/qwen-image/qwen-image-21.md). For a
differently-named local directory, add `--diffusion-model-family qwen_image`.

## 3. Family config

Registered in `miles/backends/fsdp_utils/configs/qwen_image.py`:

| Property | Value | Why |
| - | - | - |
| Timestep scaling | Trajectory timesteps ÷ 1000 before the DiT (`process_timestep_as_input`) | Matches how sglang-d's Qwen-Image pipeline rescales them |
| CFG combine | `uncond + scale·(cond − uncond)`, rescaled back to `‖cond‖` only when `true_cfg_scale > 1.0` | Mirrors sglang-d's `postprocess_cfg_noise`; the recipe's `true_cfg_scale 4.0` takes the rescaling branch |
| Cond collation | Pad `encoder_hidden_states` to batch max, mask from `txt_seq_lens`; honours `pad_to_len` | Legacy tiling path can reproduce whole-window padding bitwise |
| RoPE caches | Rebuilt on the model's CUDA device after materialize | CPU/CUDA `torch.pow` differ by fp32 ULPs |
| LoRA targets | `to_q to_k to_v to_out.0`, `add_q_proj add_k_proj add_v_proj to_add_out`, img/txt MLP proj | |
| Optimizer state | `transformer_blocks.59` text-branch params allowed missing | The last block's text outputs are discarded, so those params never receive gradient |

## 4. Launch

Canonical recipe: `scripts/run_diffusion_grpo_pickscore_5gpu_flowgrpo_aligned.py` — 4 colocate
GPUs + 1 PickScore GPU, 512×512, PickScore reward.

**Status:** [🛡️ FG — Fully gated](/docs/diffusion/user-guide/recipe-verification.md#fg)

```bash theme={null}
python3 scripts/run_diffusion_grpo_pickscore_5gpu_flowgrpo_aligned.py
```

Overrides go through the dataclass CLI, e.g.
`--num-rollout 50 --extra-args "--diffusion-kl-beta 0.02"`.

## 5. Recipe notes

* Of the 10 denoising steps, only steps 3-4 are trained: `--diffusion-sde-window-range 3,5` with
  `--diffusion-num-sde-steps 2` leaves the window nowhere to move, so every sample trains the
  same two mid-schedule steps as SDE while the rest run ODE.
* Batch arithmetic: 32 prompts × 16 samples = 512 samples per rollout, split into two optimizer
  steps of 256 samples; the 2 trained SDE steps expand the 512 samples to 1024 train pairs per rollout.
* `--rollout-patch-group qwen_image` applies the diffusers-op-parity patches on the engine. It costs a
  little rollout throughput and buys a much smaller `train/log_prob_mean_abs_diff`.

## 6. Reference results

The 5-GPU recipe raises PickScore (`rollout/reward/raw_mean`) from \~0.84 to \~0.90 over 400 rollouts;
held-out `eval/pickscore_test` goes from 0.859 at the first eval (after rollout 30; the recipe skips the
pre-training eval with `--skip-eval-before-train`) to 0.900:

![Qwen-Image PickScore reward mean](https://mintcdn.com/radixark/sL-lqXHDLHHpg3Xm/diffusion/assets/images/qwen-image/grpo-pickscore-raw-reward.png?fit=max\&auto=format\&n=sL-lqXHDLHHpg3Xm\&q=85\&s=4e4190634a5811e224b9476f19a409ce)

## 7. Pairs well with

* [LoRA weight sync](/docs/diffusion/advanced/lora.md) — `--lora-ipc-weight-sync` is on in this recipe.
* [Dtype Control](/docs/diffusion/advanced/dtype-control.md) — why fp32 master + bf16 forward.
* [Deterministic Training](/docs/diffusion/advanced/deterministic.md) — `--deterministic-mode` is on.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.