Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study

TEE overhead for LLM inference is minimal on-GPU; the penalty comes from CPU-GPU transfer over PCIe. For the majority of typical LLM queries overhead stays below 7%, and larger models and longer sequences approach zero overhead.

paper

2024-09-06

2026-10-04