On GPUs in NVIDIA Confidential Computing mode, vLLM's V2 model runner serves silent garbage
[blocker] framework
Fields
id
gotchas/cc-mode-uva-view-garbage-outputClass
framework
Severity
blocker
Concepts
confidential-computingtdx
Symptom
On an H100 NVL in CC mode inside a TDX guest, vLLM 0.29.0 with the default V2 GPU model runner starts and serves, but every response is garbage - '!!!!!!!!' (token id 0 repeated) or '[][][][]'. No error or warning anywhere. The same model is correct under plain transformers on the same GPU, so it is easy to blame the GPU or the CC setup.
Root cause
The V2 runner feeds input_ids, positions and block tables to the GPU through UVA views of pinned host memory (vllm/v1/worker/gpu/buffer_utils.py -> UvaBuffer -> get_accelerator_view_from_cpu_tensor -> torch.ops._C.get_cuda_view_from_cpu_tensor). Under GPU CC the device cannot access guest memory directly; all host/device traffic goes through the driver's encrypted bounce buffer, so those views are stale.
Workaround
Set VLLM_USE_V2_MODEL_RUNNER=0 to fall back to the V1 runner, which produces correct output on the same setup.
notes
Diagnostic ordering that matters here: the same guest runs transformers, CuPy and plain torch correctly, so 'works under transformers' does NOT rule out the GPU or CC configuration. Test vLLM's runner version before suspecting hardware.
Notes
Diagnostic ordering that matters here: the same guest runs transformers, CuPy and plain torch correctly, so 'works under transformers' does NOT rule out the GPU or CC configuration. Test vLLM's runner version before suspecting hardware.