vLLM-XPU on Max 1550: Meta-Llama-3-8B FP16 b64 128io throughput
2123.6 tok/s (aggregate x64)
Fields
id
benchmarks/argonne-max1550-vllm-metallam-fp16-1x-b64-128io-tpsMetric
tps_aggregate
Value
2,123.6
Unit
tok/s (aggregate x64)
Model
Meta-Llama-3-8B
Methodology
Argonne LLM-Inference-Bench (arXiv:2411.00136), Intel Max 1550 (Ponte Vecchio). vLLM 0.5.2-XPU FP16, batch 64, 128in/128out. Throughput = batch*(in+out)/latency (Eq.2). AGGREGATE across batch.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.
Notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.