Brief
vLLM v0.29.0 makes Model Runner V2 the default and adds per-request speculative decoding metrics
vLLM's v0.29.0 release completes the Model Runner V2 rollout that began with pooling models, and adds a flag that reports speculative decoding acceptance statistics per request.
vLLM v0.29.0 completes the Model Runner V2 rollout: the release notes state MRV2 is now the default for all models, after the rollout began with pooling models. The release carries 594 commits from 277 contributors, 91 of them new. MRV1 remains in use for a few ROCm models and for features MRV2 does not yet support.
Two items matter to operators. A flag, --per-request-spec-decode-metrics, returns per-request acceptance statistics in OpenAI API responses. Batch-sharded sampling is described as cutting per-step logits memory by 1/TP, and CUDA graph memory profiling was added for KV cache auto-sizing. New model support includes Hy4-preview, Qwen3.8-Flash-Next, GraniteSWA and GraniteMoeSWA, NemotronH_Omni_Reasoning_V3 and Kimi K3 NVFP4 checkpoints.
Our reading
The headline number is a default flip, not a new engine: anyone who pinned vLLM and has not retested on the ROCm paths still served by MRV1 now has two runtimes to reason about. The per-request acceptance metric is the more immediately useful piece for teams running speculative decoding, because it moves acceptance from a cluster-wide guess to a number attached to a request. Self-hosting teams tu…
What to do or watch
Check whether any model in your deployment is one of the ROCm models still on MRV1, then turn on --per-request-spec-decode-metrics on a canary and compare acceptance per request against your current aggregate view. Whether MRV2 matches MRV1's output on your models is something the release notes do not answer.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by github.com
- Model Runner V2 is now the default for all models.
- MRV1 remains in use for a few ROCm models and features MRV2 does not yet support.
- The release features 594 commits from 277 contributors (91 new).
- Per-request acceptance stats are exposed in OpenAI API responses via --per-request-spec-decode-metrics.
- Batch-sharded sampling cuts per-step logits memory by 1/TP.
- New models include Hy4-preview, Qwen3.8-Flash-Next, GraniteSWA and GraniteMoeSWA, NemotronH_Omni_Reasoning_V3 with MTP, and Kimi K3 NVFP4 checkpoints.
Sources
- vLLM releasesText stored 16 September 2026
How this story was checked. Written from the 1 page listed above, stored 16 September 2026; claims checked against that stored text on 16 September 2026.
What that means
- 6 of 6 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.