vLLM-XPU on Max 1550: Llama-2-7b-hf FP16 b1 128io throughput
127.5 tok/s (per-request b1)
Fields
id
benchmarks/argonne-max1550-vllm-llama27b-fp16-1x-b1-128io-tpsMetric
tps_aggregate
Value
127.5
Unit
tok/s (per-request b1)
Model
Llama-2-7b-hf
Methodology
Argonne LLM-Inference-Bench (arXiv:2411.00136), Intel Max 1550 (Ponte Vecchio). vLLM 0.5.2-XPU FP16, batch 1, 128in/128out. Throughput = batch*(in+out)/latency (Eq.2). per-request batch-1.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.
Notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.