vLLM-XPU on Max 1550: Mistral-7B-v0.1 FP16 b64 128io throughput
2245.0 tok/s (aggregate x64)
Fields
id
benchmarks/argonne-max1550-vllm-mistral7-fp16-1x-b64-128io-tpsMetric
tps_aggregate
Value
2,245
Unit
tok/s (aggregate x64)
Model
Mistral-7B-v0.1
Methodology
Argonne LLM-Inference-Bench (arXiv:2411.00136), Intel Max 1550 (Ponte Vecchio). vLLM 0.5.2-XPU FP16, batch 64, 128in/128out. Throughput = batch*(in+out)/latency (Eq.2). AGGREGATE across batch.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.
Notes
Intel Data Center GPU Max 1550 (128GB HBM2e). First measured record on xpu-intel-dc-gpu-max-1550.