vLLM#58263: Batch-invariant mode not in KV-offload namespace

Fields

id

sources/src5-gotcha-vllm-58263-batch-invariant-namespace

URL

https://github.com/vllm-project/vllm/issues/58263

Publisher

vLLM

Kind

forum

Published

2026-09-23

Accessed

2026-10-04

notes

VLLM_BATCH_INVARIANT=1 is not part of any KV-offload namespace or KV-connector compatibility check. Instance in batch-invariant mode can load KV blocks produced under different numerical configuration. Output depends on whether block was hit and which instance wrote it. Undermines determinism guarantee. Demonstrates silent numerical provenance issue in KV cache sharing.

Notes

VLLM_BATCH_INVARIANT=1 is not part of any KV-offload namespace or KV-connector compatibility check. Instance in batch-invariant mode can load KV blocks produced under different numerical configuration. Output depends on whether block was hit and which instance wrote it. Undermines determinism guarantee. Demonstrates silent numerical provenance issue in KV cache sharing.

Cross-references

This record references no other records.