vLLM issue #57016: VLLM_BATCH_INVARIANT=1 is not batch-invariant for fused-MoE experts with no warning
forum
Fields
id
sources/nrg2-gh-vllm-57016-fused-moe-not-invariantPublisher
vLLM project
Kind
forum
Published
2026-09-15
Accessed
2026-10-03
notes
OPEN as of 2026-10-03. A user asserting that the batch-invariance guarantee silently does not hold for fused-MoE expert kernels - the worst failure mode, because enabling the flag produces no warning. Environment in the report: PyTorch 2.10.0+cu128, driver 550.127.08, RHEL 8.10, GPU 'gpu-a6-7'. This is direct evidence that the oracle must be run per kernel family and per quantization, not assumed from a passing end-to-end test on a dense model.
Notes
OPEN as of 2026-10-03. A user asserting that the batch-invariance guarantee silently does not hold for fused-MoE expert kernels - the worst failure mode, because enabling the flag produces no warning. Environment in the report: PyTorch 2.10.0+cu128, driver 550.127.08, RHEL 8.10, GPU 'gpu-a6-7'. This is direct evidence that the oracle must be run per kernel family and per quantization, not assumed from a passing end-to-end test on a dense model.
Cross-references
This record references no other records.