vLLM#56770: MXFP4 W4A16 misclassified as W4A4 on SM100+

Fields

id

sources/src5-gotcha-vllm-56770-mxfp4-w4a16-misclassified

URL

https://github.com/vllm-project/vllm/issues/56770

Publisher

vLLM

Kind

forum

Published

2026-09-14

Accessed

2026-10-04

notes

compressed-tensors MXFP4 W4A16 is misclassified as W4A4 and dispatched to W4A4 kernel on SM100+ (B200). Root cause: FlashInferMxfp4LinearKernel selected instead of MarlinMxfp4Kernel. Accuracy: AIME-24 51.25% (wrong) vs 66.67% (correct). Demonstrates quantization accuracy loss from kernel misdispatch on Blackwell.

Notes

compressed-tensors MXFP4 W4A16 is misclassified as W4A4 and dispatched to W4A4 kernel on SM100+ (B200). Root cause: FlashInferMxfp4LinearKernel selected instead of MarlinMxfp4Kernel. Accuracy: AIME-24 51.25% (wrong) vs 66.67% (correct). Demonstrates quantization accuracy loss from kernel misdispatch on Blackwell.

Cross-references

This record references no other records.