Inkling-Small

1. Model Introduction

Inkling-Small is the compact member of Thinking Machines Lab’s Inkling family: a 276 B-total / 12 B-active-parameter, 42-layer multimodal MoE (256 routed + 2 shared experts, top-6 sigmoid routing) that matches — and on some benchmarks exceeds — the flagship 975 B model (e.g. SWEBench Verified 80.2 vs 77.6) at a fraction of the deployment footprint. The architecture is the same as Inkling — ShortConv, local/global relative attention, and the shared-expert-sink MoE — so everything on the Inkling page (architecture summary, attention backends, R3 routing replay, LoRA schema, multimodal RL) applies unchanged. This page only covers what differs: the model registry entry and the validated small-cluster launch profile.

2. Supported Variants

Model	Active / Total	Layers	HF ID	Recipe
Inkling-Small	12 B / 276 B	42	thinkingmachines/Inkling-Small	this page

3. Quick start

Validated on 4 nodes × 8 H200 (TP4 SP PP8 EP4, DP1):

cd /root/miles

# Full-parameter GRPO. 276 B fits with the CPU-offloaded optimizer -
# no NVMe streaming needed (unlike the 975 B recipe).
python scripts/run_inkling_975b.py train \
   --model-name Inkling-Small --train-mode full --task dapo_math \
   --num-nodes 4 --num-gpus-per-node 8 \
   --sglang-context-length 4096 --rollout-max-response-len 2048 \
   --extra-args "--offload-train-target cpu --sglang-mem-fraction-static 0.65 \
      --optimizer-cpu-offload --overlap-cpu-optimizer-d2h-h2d --use-precision-aware-optimizer"

# LoRA GRPO (rank 32, all-linear), same cluster; adapter-only weight
# sync swaps in ~4 s per rollout.
python scripts/run_inkling_975b.py train \
   --model-name Inkling-Small --train-mode lora --task dapo_math \
   --num-nodes 4 --num-gpus-per-node 8 \
   --sglang-context-length 4096 --rollout-max-response-len 2048

The model definition lives in scripts/models/inkling-small.sh (MODEL_ARGS_NUM_LAYERS overrides the layer count for sliced smoke/parity checkpoints). HF → torch_dist conversion uses the same tool as Inkling with this recipe file — a single 8-GPU node (TP8 EP8) converts it in one pass.

4. Validated parallelism

Hardware	GPUs	TP	SP	PP	EP	expert-TP	Notes
H200	32	4	on	8	4	1	`--decoder-last-pipeline-num-layers 7` (42 = 7×5 + 7)

Batch shape is configurable from the launcher (--rollout-batch-size, --global-batch-size; defaults 32/64). The validated Small runs used 64/128 with --lr 6e-6, producing a steadily rising dapo-math reward curve; LoRA uses the launcher’s lr 5e-6 default.

​1. Model Introduction

​2. Supported Variants

​3. Quick start

​4. Validated parallelism

1. Model Introduction

2. Supported Variants

3. Quick start

4. Validated parallelism