Arc Pro B70 Qwen3.5-27B token-gen (vLLM, single-req)
13.4 tok/s single-request floor, dynamic FP8
Fields
id
benchmarks/comm-arc-pro-b70-qwen35-27b-vllm-fp8-tgMetric
decode_tok_s
Value
13.4
Unit
tok/s single-request floor, dynamic FP8
Model
Qwen3.5-27B
Methodology
vLLM (Intel upstream), dynamic-FP8 BF16 path, single request; mem-util capped 0.8 for stability (wendell, Level1Techs)
Measured by
third_party
Thermal state
unknown
notes
Native FP8 model crashed - needed dynamic-quant BF16. 4x B70 TP=4 50-req: 370 t/s avg / 550 t/s peak output, TTFT ~11.5s; 8-req 2-card 97.8 t/s agg. vs 4x 3090 same test ~348 t/s.
Notes
Native FP8 model crashed - needed dynamic-quant BF16. 4x B70 TP=4 50-req: 370 t/s avg / 550 t/s peak output, TTFT ~11.5s; 8-req 2-card 97.8 t/s agg. vs 4x 3090 same test ~348 t/s.