vLLM issue #56535: FlashInfer flashinfer_b12x MoE backend runs W4A16 checkpoints as W4A4

Fields

id

sources/qgotcha-vllm-56535-flashinfer-b12x-w4a16-as-w4a4

URL

https://github.com/vllm-project/vllm/issues/56535

Publisher

vLLM GitHub issues

Kind

forum

Accessed

2026-10-03

notes

Root cause: _normalize_quant_mode in flashinfer fused_moe/cute_dsl/blackwell_sm12x/moe_dispatch.py defaults quant_mode to 'nvfp4' when activation_precision defaults to 'fp4'. Includes a GSM8K number for the broken vs fixed path.

Notes

Root cause: _normalize_quant_mode in flashinfer fused_moe/cute_dsl/blackwell_sm12x/moe_dispatch.py defaults quant_mode to 'nvfp4' when activation_precision defaults to 'fp4'. Includes a GSM8K number for the broken vs fixed path.

Cross-references

This record references no other records.