inference/research
verifiedhigh · 0.85updated 2026-10-04

vLLM issue #34694: NVFP4 checkpoints produce garbled output with --dtype bfloat16 on GPUs without native FP4 (SM 89 RTX 4090, SM 70 V100)

forum

Fields

id
sources/comm-github-vllm-34694-nvfp4-marlin-garbled
Publisher
vLLM GitHub issues
Kind
forum
Accessed
2026-10-04
notes
Reporter supplied quantified evidence on GLM-4.7-Flash-NVFP4: 54.7% of output values zero, cos_sim 0.22 vs FP32 reference, GSM8K 5-shot garbled. Independent reproductions in-thread on DIFFERENT hardware and DIFFERENT suspected cause: naroam1 on NVIDIA L4 (Ada SM 8.9), vLLM v0.19.0 and v0.14.0; drew-chen on RTX 4080. linjiapro reproduced garbage output on SM120 but root-caused it to a different defect (modelopt->MARLIN config bridge dropping swiglu_alpha/beta), so SM120 garbage is NOT established as the same bug.

Notes

Reporter supplied quantified evidence on GLM-4.7-Flash-NVFP4: 54.7% of output values zero, cos_sim 0.22 vs FP32 reference, GSM8K 5-shot garbled. Independent reproductions in-thread on DIFFERENT hardware and DIFFERENT suspected cause: naroam1 on NVIDIA L4 (Ada SM 8.9), vLLM v0.19.0 and v0.14.0; drew-chen on RTX 4080. linjiapro reproduced garbage output on SM120 but root-caused it to a different defect (modelopt->MARLIN config bridge dropping swiglu_alpha/beta), so SM120 garbage is NOT established as the same bug.

Cross-references

This record references no other records.