Brief
llama.cpp b10997 extends MoE tile heuristic to RDNA3.5 on ROCm
Release b10997 widens the HIP MoE ncols_opt tile condition so that RDNA3.5, not just RDNA3.0, is included, with reported matrix-multiply gains measured on a single AMD test machine. The numbers come from one configuration; no independent replication is part of this release.
The change is narrow. In ggml/src/ggml-cuda/mmq.cu the condition GGML_CUDA_CC_IS_RDNA3_0 becomes GGML_CUDA_CC_IS_RDNA3, so the MoE ncols_opt tile heuristic now also applies to RDNA3.5. The release states that dense dispatch logic is unchanged, which puts the effect on mixture-of-experts matmuls rather than on dense layers.
The release reports results on one machine: an AMD Radeon 8060S, gfx1151 (RDNA3.5), 20 CU, wave32, on an AMD Ryzen AI MAX+ 388, 8C/16T, 23.79 GB RAM. Correctness checks with test-backend-ops -b ROCm0 for MUL_MAT and MUL_MAT_ID are reported as all passing. Performance on that machine: LFM2.5-8B-A1B-UD-Q4_K_M +16.198%, Qwen1.5-MoE-A2.7B-Q2_K +6.189%, pooled +11.081%. Token generation tg128 is described as unchanged on the Q4_K MoE model and +2.188% on the Q2_K one.
Our reading
For readers running local MoE models on RDNA3.5 iGPUs under ROCm, this is a build-level change that arrives without new APIs or configuration: upgrading the build is the whole action. Anyone on RDNA3.0 hardware sees no change, and anyone on CUDA, Vulkan, Metal or CPU paths is unaffected. The reported spread between a Q4_K and a Q2_K MoE model suggests the benefit depends on quantisation and model…
What to do or watch
If you run MoE models on RDNA3.5 with ROCm, move to a build at or after b10997 and repeat the release's own check (test-backend-ops -b ROCm0 -o MUL_MAT and -o MUL_MAT_ID, then compare tg128) on your own model before assuming the published percentages transfer.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by github.com
- The MoE ncols_opt tile heuristic condition was changed from GGML_CUDA_CC_IS_RDNA3_0 to GGML_CUDA_CC_IS_RDNA3 in ggml/src/ggml-cuda/mmq.cu, and the dense dispatch logic remains unchanged.
- The test machine used was an AMD Radeon 8060S, gfx1151 (RDNA3.5), 20 CU, wave32 on an AMD Ryzen AI MAX+ 388, 8C/16T, with 23.79 GB RAM.
- Reported performance on the target machine: LFM2.5-8B-A1B-UD-Q4_K_M +16.198%, Qwen1.5-MoE-A2.7B-Q2_K +6.189%, pooled (16 pairs) +11.081%.
- Token generation (tg128) is unchanged on the Q4_K MoE model and +2.188% on the Q2_K one.
- Correctness verification reports MUL_MAT 64/64, 29/29, 48/48, 14/14 and MUL_MAT_ID 84/84, 3/3, 74/74, 3/3 as passing.
Sources
- llama.cpp releases (GitHub)Text stored 16 September 2026
How this story was checked. Written from the 1 page listed above, stored 16 September 2026; claims checked against that stored text on 16 September 2026.
What that means
- 5 of 5 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.