> ## Documentation Index
> Fetch the complete documentation index at: https://miles.radixark.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Inkling-Small

> Launch recipe for Inkling-Small (276 B), the compact sibling of Inkling — same architecture, 4-node H200 footprint.

## 1. Model Introduction

[Inkling-Small](https://huggingface.co/thinkingmachines/Inkling-Small) is the compact member of Thinking Machines Lab's Inkling family: a 276 B-total / 12 B-active-parameter, 42-layer multimodal MoE (256 routed + 2 shared experts, top-6 sigmoid routing) that matches — and on some benchmarks exceeds — the flagship 975 B model (e.g. SWEBench Verified 80.2 vs 77.6) at a fraction of the deployment footprint. The architecture is the same as [Inkling](/docs/models/thinkingmachines/inkling) — ShortConv, local/global relative attention, and the shared-expert-sink MoE — so everything on the Inkling page (architecture summary, attention backends, R3 routing replay, LoRA schema, multimodal RL) applies unchanged. This page only covers what differs: the model registry entry and the validated small-cluster launch profile.

## 2. Supported Variants

| Model         | Active / Total | Layers | HF ID                                                                                   | Recipe    |
| ------------- | -------------- | ------ | --------------------------------------------------------------------------------------- | --------- |
| Inkling-Small | 12 B / 276 B   | 42     | [thinkingmachines/Inkling-Small](https://huggingface.co/thinkingmachines/Inkling-Small) | this page |

## 3. Quick start

Validated on 4 nodes × 8 H200 (TP4 SP PP8 EP4, DP1):

```bash theme={null}
cd /root/miles

# Full-parameter GRPO. 276 B fits with the CPU-offloaded optimizer -
# no NVMe streaming needed (unlike the 975 B recipe).
python scripts/run_inkling_975b.py train \
   --model-name Inkling-Small --train-mode full --task dapo_math \
   --num-nodes 4 --num-gpus-per-node 8 \
   --sglang-context-length 4096 --rollout-max-response-len 2048 \
   --extra-args "--offload-train-target cpu --sglang-mem-fraction-static 0.65 \
      --optimizer-cpu-offload --overlap-cpu-optimizer-d2h-h2d --use-precision-aware-optimizer"

# LoRA GRPO (rank 32, all-linear), same cluster; adapter-only weight
# sync swaps in ~4 s per rollout.
python scripts/run_inkling_975b.py train \
   --model-name Inkling-Small --train-mode lora --task dapo_math \
   --num-nodes 4 --num-gpus-per-node 8 \
   --sglang-context-length 4096 --rollout-max-response-len 2048
```

The model definition lives in `scripts/models/inkling-small.sh` (`MODEL_ARGS_NUM_LAYERS` overrides the layer count for sliced smoke/parity checkpoints). HF → `torch_dist` conversion uses the same tool as Inkling with this recipe file — a single 8-GPU node (TP8 EP8) converts it in one pass.

## 4. Validated parallelism

| Hardware | GPUs | TP | SP | PP | EP | expert-TP | Notes                                                 |
| -------- | ---- | -- | -- | -- | -- | --------- | ----------------------------------------------------- |
| H200     | 32   | 4  | on | 8  | 4  | 1         | `--decoder-last-pipeline-num-layers 7` (42 = 7×5 + 7) |

Batch shape is configurable from the launcher (`--rollout-batch-size`, `--global-batch-size`; defaults 32/64). The validated Small runs used 64/128 with `--lr 6e-6`, producing a steadily rising dapo-math reward curve; LoRA uses the launcher's `lr 5e-6` default.