vLLM-XPU on Max 1550: Mistral-7B-v0.1 FP16 b1 128io throughput
126.4 tok/s (per-request b1)
Fields
id
benchmarks/argonne-max1550-vllm-mistral7-fp16-1x-b1-128io-tpsMetric
tps_aggregate
Value
126.4
Unit
tok/s (per-request b1)
Model
Mistral-7B-v0.1
Methodology
Argonne LLM-Inference-Bench (arXiv:2411.00136), Intel Max 1550 (Ponte Vecchio). vLLM 0.5.2-XPU FP16, batch 1, 128in/128out. Throughput = batch*(in+out)/latency (Eq.2). per-request batch-1.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.
Notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.