vLLM issue #34694: NVFP4 checkpoints produce garbled output with --dtype bfloat16 on GPUs without native FP4 (SM 89 RTX 4090, SM 70 V100)
forum
Fields
id
sources/comm-github-vllm-34694-nvfp4-marlin-garbledPublisher
vLLM GitHub issues
Kind
forum
Accessed
2026-10-04
notes
Reporter supplied quantified evidence on GLM-4.7-Flash-NVFP4: 54.7% of output values zero, cos_sim 0.22 vs FP32 reference, GSM8K 5-shot garbled. Independent reproductions in-thread on DIFFERENT hardware and DIFFERENT suspected cause: naroam1 on NVIDIA L4 (Ada SM 8.9), vLLM v0.19.0 and v0.14.0; drew-chen on RTX 4080. linjiapro reproduced garbage output on SM120 but root-caused it to a different defect (modelopt->MARLIN config bridge dropping swiglu_alpha/beta), so SM120 garbage is NOT established as the same bug.
Notes
Reporter supplied quantified evidence on GLM-4.7-Flash-NVFP4: 54.7% of output values zero, cos_sim 0.22 vs FP32 reference, GSM8K 5-shot garbled. Independent reproductions in-thread on DIFFERENT hardware and DIFFERENT suspected cause: naroam1 on NVIDIA L4 (Ada SM 8.9), vLLM v0.19.0 and v0.14.0; drew-chen on RTX 4080. linjiapro reproduced garbage output on SM120 but root-caused it to a different defect (modelopt->MARLIN config bridge dropping swiglu_alpha/beta), so SM120 garbage is NOT established as the same bug.
Cross-references
This record references no other records.