vllm-ascend v0.17.0 on Huawei Atlas 300I Pro (Ascend 310P3, 24GB LPDDR4X): Qwen3-4B FP16, ~9 tok/s decode
9.0 ~tok/s decode (reporter's rounded figure)
Fields
id
benchmarks/vllm-ascend-atlas300ipro-qwen3-4b-fp16-decodeMetric
decode_tok_s
Value
9
Unit
~tok/s decode (reporter's rounded figure)
Model
Qwen/Qwen3-4B (FP16, ~8GB weights)
Methodology
FORUM REPORT (GitCode Ascend/MindIE-LLM issue #623, reporter jeffqiu, 2026-05-27). Hardware: Atlas 300I Pro card IT21PDDA01, single Ascend 310P3 SoC, 24GB LPDDR4X, on Kylin Linux V10 aarch64 (Phytium D2000 host). CANN 8.5.1, NPU driver 25.5.1, torch_npu 2.9.0.post1. Engine: vllm-ascend v0.17.0. Model: Qwen3-4B FP16 (~8GB). Batch/prompt/output lengths NOT stated — the number appears in a bug report as evidence the hardware works, not as a headline measurement. Treat ~9 tok/s as an order-of-magnitude single-request decode figure, not a tuned peak.
Measured by
third_party
Reproducible
yes
notes
First benchmark record for huawei-atlas-300i-pro. Context: the reporter ran vllm-ascend successfully to prove the card was functional after MindIE 3.0.0's 300I-Duo container image hard-faulted the NPU (see gotcha-mindie300-duo-image-on-300i-pro-npu-fault). The Atlas 300I Pro is a 24GB LPDDR4X inference card (~70W class); ~9 tok/s on a 4B FP16 model is consistent with its ~204 GB/s memory bandwidth being the binding constraint, not a software defect.
Notes
First benchmark record for huawei-atlas-300i-pro. Context: the reporter ran vllm-ascend successfully to prove the card was functional after MindIE 3.0.0's 300I-Duo container image hard-faulted the NPU (see gotcha-mindie300-duo-image-on-300i-pro-npu-fault). The Atlas 300I Pro is a 24GB LPDDR4X inference card (~70W class); ~9 tok/s on a 4B FP16 model is consistent with its ~204 GB/s memory bandwidth being the binding constraint, not a software defect.