inference/research
verifiedhigh · 0.90updated 2026-10-03

vLLM issue #57016: VLLM_BATCH_INVARIANT=1 is not batch-invariant for fused-MoE experts with no warning

forum

Fields

id
sources/nrg2-gh-vllm-57016-fused-moe-not-invariant
Publisher
vLLM project
Kind
forum
Published
2026-09-15
Accessed
2026-10-03
notes
OPEN as of 2026-10-03. A user asserting that the batch-invariance guarantee silently does not hold for fused-MoE expert kernels - the worst failure mode, because enabling the flag produces no warning. Environment in the report: PyTorch 2.10.0+cu128, driver 550.127.08, RHEL 8.10, GPU 'gpu-a6-7'. This is direct evidence that the oracle must be run per kernel family and per quantization, not assumed from a passing end-to-end test on a dense model.

Notes

OPEN as of 2026-10-03. A user asserting that the batch-invariance guarantee silently does not hold for fused-MoE expert kernels - the worst failure mode, because enabling the flag produces no warning. Environment in the report: PyTorch 2.10.0+cu128, driver 550.127.08, RHEL 8.10, GPU 'gpu-a6-7'. This is direct evidence that the oracle must be run per kernel family and per quantization, not assumed from a passing end-to-end test on a dense model.

Cross-references

This record references no other records.