inference/research
verifiedhigh · 0.85updated 2026-10-03

vLLM batch invariance is beta and its hardware support is bounded: NVIDIA compute capability 8.0+ and Intel XPU with Triton backend only - no ROCm/HIP, Metal, TPU or CPU claim

80.0 minimum NVIDIA compute capability (8.0) for which batch invariance is supported; the enum has no determinism-support metric so 'quality' is a placeholder

Fields

id
benchmarks/vllm-batch-invariance-hardware-support-scope-cc80-and-xpu-triton
Metric
quality
Value
80
Unit
minimum NVIDIA compute capability (8.0) for which batch invariance is supported; the enum has no determinism-support metric so 'quality' is a placeholder
Methodology
Documentation fact, not a measurement - recorded because it bounds how far the batch-invariance oracle can be applied, which is the practical constraint on differential testing across this repo's hardware. The vLLM feature page is marked beta with 'some features are still under active development' and tracks progress at vllm-project/vllm#27433. Stated support: NVIDIA GPUs of compute capability 8.0 or higher, and Intel XPUs with Triton support, where the Triton Attention backend is REQUIRED (not the FlashInfer backend). No AMD ROCm/HIP, no Apple Metal, no TPU and no CPU support is claimed. Upstream implements the oracle as a per-op test suite under tests/v1/determinism/ - 10 files covering batch invariance for VLM, CUTLASS matmul, NVFP4, RMSNorm, XPU, plus an online end-to-end test.
Measured by
vendor
Reproducible
yes
notes
THE SCOPE LIMIT THAT DETERMINES THE WHOLE TEST STRATEGY ON NON-NVIDIA PARTS. The strongest available oracle for silent numeric corruption - same prompt, different batch size, identical logits - is implemented upstream for NVIDIA and Intel XPU ONLY. So on the AMD, Apple, Gaudi, TPU and Ascend records in this repo there is no supported batch-invariance mode to test against, and the repo's own ROCm and Metal gotchas therefore cannot be cleared by the vLLM flag. For those parts the cross-backend and cold-vs-warm-cache oracles in docs/10-testing-methodology.md are the only available instruments, which is precisely why they need to be written down. Also note the per-op structure of the upstream suite: invariance is asserted per kernel family (RMSNorm, matmul/CUTLASS, NVFP4, attention, MoE-adjacent paths), not once end-to-end - consistent with open issues #57016 (fused-MoE not invariant, silently) and #59086 (AWQ on sm8x not invariant in default mode), which show an end-to-end pass on one config is no evidence for another.

Notes

THE SCOPE LIMIT THAT DETERMINES THE WHOLE TEST STRATEGY ON NON-NVIDIA PARTS. The strongest available oracle for silent numeric corruption - same prompt, different batch size, identical logits - is implemented upstream for NVIDIA and Intel XPU ONLY. So on the AMD, Apple, Gaudi, TPU and Ascend records in this repo there is no supported batch-invariance mode to test against, and the repo's own ROCm and Metal gotchas therefore cannot be cleared by the vLLM flag. For those parts the cross-backend and cold-vs-warm-cache oracles in docs/10-testing-methodology.md are the only available instruments, which is precisely why they need to be written down. Also note the per-op structure of the upstream suite: invariance is asserted per kernel family (RMSNorm, matmul/CUTLASS, NVFP4, attention, MoE-adjacent paths), not once end-to-end - consistent with open issues #57016 (fused-MoE not invariant, silently) and #59086 (AWQ on sm8x not invariant in default mode), which show an end-to-end pass on one config is no evidence for another.

Cross-references

Engine (1)