TensorRT-LLM: Llama 2 70B FP8 on 8x H100 80GB, single-request latency at batch size 1

Engine: TensorRT-LLM v0.5.0 (published command line: build.py with --enable_fp8 --fp8_kv_cache --strongly_typed, then mpirun -n 8 gptSessionBenchmark --batch_size 14 --input_output_len 2048,1;2048,128, --warm_up 1 --num_runs 5). Hardware: DGX H100, 8x H100 80GB HBM3, TP=8, PP=1. Prompt length 2048, output length 128. Batch size 1 (one request at a time). Concurrency 1. Tensor-parallel size 8. Figure is PER-REQUEST: the source describes it as the full end-to-end time to serve one inference, so it covers prefill plus all 128 decode steps, not just first-token latency - the schema's closest honest metric is ttft_ms and the unit says so explicitly. NOT split into TTFT vs ITL by the source. Omitted by the source: clock/power settings, framework commit, container image, and whether the 1.7 s is mean or best-of-5.

NVIDIA published the exact build and benchmark command lines, so this is reproducible modulo the unspecified commit/container. IMPORTANT: this is a vendor-selected comparison against AMD's MI300X launch claim and NVIDIA explicitly says AMD's figures used unoptimized software - read the 'H100 is 2x faster' framing in that source as vendor positioning, not a neutral result. The paired throughput point under a 2.5 s fixed response-time budget is in the same figure but is a chart-only value (see the note on the NVIDIA H100 source record). LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: single-request latency - no load scenario applies. ABSENT CONDITIONS (1 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - warmup: unrecoverable from bench-nv-nvidia-h100-tensorrtllm-blog - would need the raw harness log's warmup phase or the benchmark command line (loadgen min_duration/min_query_count, or trtllm-bench --warm_up N / --num_runs N) None of these were filled in. A plausible value is worse than an absent one: it would be trusted.