inference/research
verifiedhigh · 0.90updated 2026-10-03

vLLM issue #59086: VLLM_BATCH_INVARIANT=1 is not batch-invariant for AWQ models on sm8x in the default mode

forum

Fields

id
sources/nrg2-gh-vllm-59086-awq-not-invariant
Publisher
vLLM project
Kind
forum
Published
2026-09-28
Accessed
2026-10-03
notes
OPEN as of 2026-10-03, five days old at time of access. Same shape as #57016 but for AWQ quantized models on sm8x (Ampere-class) in the DEFAULT mode. Together the two issues establish that batch invariance in vLLM is currently opt-in per kernel path and not yet universal across (architecture x quantization) combinations - so the correct test posture is to assert the invariant on the exact config you ship, and treat a pass on another config as no evidence.

Notes

OPEN as of 2026-10-03, five days old at time of access. Same shape as #57016 but for AWQ quantized models on sm8x (Ampere-class) in the DEFAULT mode. Together the two issues establish that batch invariance in vLLM is currently opt-in per kernel path and not yet universal across (architecture x quantization) combinations - so the correct test posture is to assert the invariant on the exact config you ship, and treat a pass on another config as no evidence.

Cross-references

This record references no other records.