ROCm/vllm#630: MXFP4 kernel dispatch fix for garbage output

Fields

id

sources/src5-gotcha-rocm-vllm-630-mxfp4-dispatch

URL

https://github.com/ROCm/vllm/issues/630

Publisher

AMD ROCm

Kind

forum

Published

2025-08-12

Accessed

2026-10-04

notes

PR fixes MXFP4 quantization accuracy on ROCm. Issue: garbage output for Llama-3.1-405B-Instruct-MXFP4-Preview. Root cause: wrong kernel dispatch logic. Fix: shuffle weight, remove wrong bias, refine dispatch to use gemm_a4w4 for preshuffled scenario. Verified on MI355X. Demonstrates quantization accuracy loss at scale on ROCm.

Notes

PR fixes MXFP4 quantization accuracy on ROCm. Issue: garbage output for Llama-3.1-405B-Instruct-MXFP4-Preview. Root cause: wrong kernel dispatch logic. Fix: shuffle weight, remove wrong bias, refine dispatch to use gemm_a4w4 for preshuffled scenario. Verified on MI355X. Demonstrates quantization accuracy loss at scale on ROCm.

Cross-references

This record references no other records.