vLLM issue #49546: VLLM_MARLIN_INPUT_DTYPE=fp8 (Marlin W4A8-FP8) silently corrupts output on GB10/sm_121a

Fields

id

sources/qgotcha-vllm-49546-marlin-w4a8-fp8-sm121a

URL

https://github.com/vllm-project/vllm/issues/49546

Publisher

vLLM GitHub issues

Kind

forum

Accessed

2026-10-03

notes

Root cause not confirmed by a maintainer; reporter gives a code-reading analysis of the arch guard (is_device_capability_family(120) admits sm_121a), the unguarded e4m3 mma.sync, and an fp8-vs-int8 global-scale fold asymmetry. Reproduced by the reporter on GB10 / DGX Spark with Qwen3.5-122B-A10B WNA16 INT4 (AutoRound, group_size 128) MoE.

Notes

Root cause not confirmed by a maintainer; reporter gives a code-reading analysis of the arch guard (is_device_capability_family(120) admits sm_121a), the unguarded e4m3 mma.sync, and an fp8-vs-int8 global-scale fold asymmetry. Reproduced by the reporter on GB10 / DGX Spark with Qwen3.5-122B-A10B WNA16 INT4 (AutoRound, group_size 128) MoE.

Cross-references

This record references no other records.