MLPerf v6.0 Jetson AGX Thor: llama3.1-8b-edge Offline 350.605 tok/s aggregate
350.605 tok/s AGGREGATE, single Thor module, Offline scenario
Fields
id
benchmarks/mlperf-v6-0-thor-llama31-8b-edge-offline-tokensMetric
tps_aggregate
Value
350.605
Unit
tok/s AGGREGATE, single Thor module, Offline scenario
Model
llama3.1-8b-edge
Methodology
MLPerf Inference open-division submission, 1x NVIDIA Jetson AGX Thor Developer Kit 128GB (unified memory), TensorRT 10.8, CUDA 12.8. Quoted verbatim from performance/run_1/mlperf_log_summary.txt, Result is VALID. v6.0, llama3.1-8b-edge, FP4 weights, Offline. samples_per_query 5000, target_qps 2000, min_query_count 5000. 'Tokens per second: 350.605' (2.71603 samples/s). Aggregate system throughput, not a per-request decode rate.
Measured by
vendor
Reproducible
yes
notes
Open division. Lower than the v5.0 70B-FP16 Offline number because the workload, precision and scenario configuration differ, not because the hardware changed.
Notes
Open division. Lower than the v5.0 70B-FP16 Offline number because the workload, precision and scenario configuration differ, not because the hardware changed.