MLPerf v6.0 Jetson AGX Thor: llama3.1-8b-edge SingleStream TTFT p90 126.5 ms
126.477827 ms, 90th-percentile first-token latency, per query (SingleStream, concurrency 1)
Fields
id
benchmarks/mlperf-v6-0-thor-llama31-8b-edge-singlestream-ttftMetric
ttft_ms
Value
126.4778
Unit
ms, 90th-percentile first-token latency, per query (SingleStream, concurrency 1)
Model
llama3.1-8b-edge
Methodology
MLPerf Inference open-division submission, 1x NVIDIA Jetson AGX Thor Developer Kit 128GB (unified memory), TensorRT 10.8, CUDA 12.8. Quoted verbatim from performance/run_1/mlperf_log_summary.txt, Result is VALID. v6.0, llama3.1-8b-edge, FP4 weights, SingleStream, samples_per_query 1, max_async_queries 1. '90.00 percentile first token latency (ns): 126477827' = 126.48 ms. Per request. The 90th-percentile end-to-end query latency on the same log is 1357.55 ms and is not stored.
Measured by
vendor
Reproducible
yes
notes
Open division. Much lower TTFT than the v5.1 submission of the same scenario (1397 ms); the submission does not explain the change, so both are kept.
Notes
Open division. Much lower TTFT than the v5.1 submission of the same scenario (1397 ms); the submission does not explain the change, so both are kept.