Argonne LLM-Inference-Bench: Intel Max 1550 x1, vLLM 0.5.2-XPU, Mistral-7B FP16, 128in/128out, batch 1
126.4 tok/s aggregate (batch 1 = single request)
Fields
id
benchmarks/argonne-max1550-vllm-mistral-7b-fp16-b1-128io-tpsMetric
tps_aggregate
Value
126.4
Unit
tok/s aggregate (batch 1 = single request)
Model
mistralai/Mistral-7B-v0.1
Methodology
THIRD-PARTY MEASURED (Argonne LLM-Inference-Bench, arXiv:2411.00136, All_results.csv). FP16/BF16 weights, 128 in / 128 out. Throughput = batch*(in+out)/end-to-end latency = AGGREGATE over the batch, NOT per-request decode tok/s. vLLM built for XPU (VLLM_TARGET_DEVICE=xpu, v0.5.2 + intel-extension-for-pytorch 2.3.110). Batch 1, 1x Max 1550 (128GB HBM2e, 896 EU). Latency 2.026s.
Measured by
third_party
Reproducible
yes
notes
Aurora/ANL measurement. vLLM-XPU was an early port (v0.5.2, mid-2024); treat as a floor, not a ceiling.
Notes
Aurora/ANL measurement. vLLM-XPU was an early port (v0.5.2, mid-2024); treat as a floor, not a ceiling.