vLLM-XPU on Max 1550: Llama-2-7b-hf FP16 b64 128io throughput
2100.9 tok/s (aggregate x64)
Fields
id
benchmarks/argonne-max1550-vllm-llama27b-fp16-1x-b64-128io-tpsMetric
tps_aggregate
Value
2,100.9
Unit
tok/s (aggregate x64)
Model
Llama-2-7b-hf
Methodology
Argonne LLM-Inference-Bench (arXiv:2411.00136), Intel Max 1550 (Ponte Vecchio). vLLM 0.5.2-XPU FP16, batch 64, 128in/128out. Throughput = batch*(in+out)/latency (Eq.2). AGGREGATE across batch.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.
Notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.