Argonne: GH200 x1, vLLM 0.6.1, Qwen1.5-7B FP16, 2048io, batch 1
249.5 tok/s aggregate (batch 1, single request)
Fields
id
benchmarks/argonne-gh200-vllm-qwen157b-fp16-b1-2048io-tpsMetric
tps_aggregate
Value
249.5
Unit
tok/s aggregate (batch 1, single request)
Model
Qwen/Qwen1.5-7B
Methodology
THIRD-PARTY MEASURED (Argonne LLM-Inference-Bench, arXiv:2411.00136, All_results.csv). vLLM 0.6.1, BF16/FP16, 2048 in/2048 out, batch 1, 1x GH200 (96GB HBM3, Grace+Hopper superchip). Throughput = batch*(in+out)/end-to-end latency = AGGREGATE (batch-1 == per-request e2e rate incl prefill+decode). Reported latency n/as.
Measured by
third_party
Reproducible
yes