vLLM#53328: FP4 warning incorrectly blames GPU when checkpoint is weight-only

Fields

id

sources/src5-gotcha-vllm-53328-fp4-warning-weight-only

URL

https://github.com/vllm-project/vllm/issues/53328

Publisher

vLLM

Kind

forum

Published

2026-08-21

Accessed

2026-10-04

notes

prepare_nvfp4_moe_layer_for_marlin() warns 'GPU does not have native FP4 support' even when GPU supports FP4 but checkpoint is weight-only (W4A16). Warning is false on Jetson Thor (SM110). PR gates warning on cutlass_fp4_supported(). Demonstrates misleading error messages in quantization.

Notes

prepare_nvfp4_moe_layer_for_marlin() warns 'GPU does not have native FP4 support' even when GPU supports FP4 but checkpoint is weight-only (W4A16). Warning is false on Jetson Thor (SM110). PR gates warning on cutlass_fp4_supported(). Demonstrates misleading error messages in quantization.

Cross-references

This record references no other records.