inference/research
verifiedhigh · 0.95updated 2026-10-10

MLPerf Inference v4.1 NVIDIA H200-SXM-141GBx8_TRT_MaxQ llama2-70b-99.9 Server LoadGen summary

benchmark

Fields

id
sources/pwr-mlperf-v4-1-h200-maxq-l70-server-summary
Publisher
MLCommons and NVIDIA
Kind
benchmark
Published
2024-08-28
Accessed
2026-10-03
notes
AUDITED MLPerf LoadGen result for the paired Server power trace. Key fields verbatim: 'Completed samples per second : 79.36', 'Completed tokens per second: 23113.14', 'Result is : VALID', 'Performance constraints satisfied : Yes', scheduled samples/s 80.10, target_qps 80, ttft 2000000000 ns, tpot 200000000 ns, min_duration 2400000 ms, min_query_count 100. Latency percentiles are the point at which this run is latency-constrained rather than throughput-constrained: TPOT 50th 138.267 ms, 99th 158.802 ms, 99.9th 174.808 ms against a 200 ms constraint; TTFT 50th 204.7 ms, 99th 519.3 ms, 99.9th 658.8 ms against a 2000 ms constraint. Use 'Completed tokens per second' for this scenario, not 'Tokens per second' - the Offline and Server summaries do not use the same label for their headline figure. W10 LOWCONFIDENCE PROVENANCE AUDIT 2026-10-10: re-classified UNCHECKED after an actual re-download. The raw artefact was fetched from mlcommons/inference_results_v4.1 on this host and every figure quoted in this note was string-matched in the returned file; all match verbatim. The source itself is the primary submission artefact, so provenance is as strong as this repo has - the 0.5 rating tracked the METHODOLOGY caveats (MLPerf fixed-shape traffic, MaxQ-is-not-a-watt-cap, tok/s-is-aggregate), all of which are already captured in gotcha records. Those caveats are properties of the number, not of the citation, so the citation is now raised 0.5 -> 0.95. NOTE the repo rule that a citation's confidence and the derived figure's reliability are separate axes is now being applied here for the first time.

Notes

AUDITED MLPerf LoadGen result for the paired Server power trace. Key fields verbatim: 'Completed samples per second : 79.36', 'Completed tokens per second: 23113.14', 'Result is : VALID', 'Performance constraints satisfied : Yes', scheduled samples/s 80.10, target_qps 80, ttft 2000000000 ns, tpot 200000000 ns, min_duration 2400000 ms, min_query_count 100. Latency percentiles are the point at which this run is latency-constrained rather than throughput-constrained: TPOT 50th 138.267 ms, 99th 158.802 ms, 99.9th 174.808 ms against a 200 ms constraint; TTFT 50th 204.7 ms, 99th 519.3 ms, 99.9th 658.8 ms against a 2000 ms constraint. Use 'Completed tokens per second' for this scenario, not 'Tokens per second' - the Offline and Server summaries do not use the same label for their headline figure.

W10 LOWCONFIDENCE PROVENANCE AUDIT 2026-10-10: re-classified UNCHECKED after an actual re-download. The raw artefact was fetched from mlcommons/inference_results_v4.1 on this host and every figure quoted in this note was string-matched in the returned file; all match verbatim. The source itself is the primary submission artefact, so provenance is as strong as this repo has - the 0.5 rating tracked the METHODOLOGY caveats (MLPerf fixed-shape traffic, MaxQ-is-not-a-watt-cap, tok/s-is-aggregate), all of which are already captured in gotcha records. Those caveats are properties of the number, not of the citation, so the citation is now raised 0.5 -> 0.95. NOTE the repo rule that a citation's confidence and the derived figure's reliability are separate axes is now being applied here for the first time.

Cross-references

This record references no other records.