id
engines/niche-vllm-ascend
Repo
vllm-project/vllm-ascend
Design goal
hardware-coverageproductioncommunity
Scheduling
continuous-batchingpaged-kvprefix-cache
Formats
safetensorsawqgptqfp8
Notable features
The canonical example of vLLM's hardware-pluggable strategy: rather than forking vLLM for each new accelerator, the community maintains out-of-tree plugins that implement the hardware interfaceCreated by the vLLM community itself in February 2025, following vLLM's [RFC] Hardware pluggable (vLLM issue 11162, December 2024). That lineage is the interesting part: this is vLLM's answer to how it will support non-CUDA hardware foreverTransformer, MoE, embedding and multimodal LLMs are all supported on Ascend NPU, per the READMERelease cadence tracks vLLM exactly, which is the maintenance mechanism: v0.7.3 (2025-05) through v0.23.0.post1 (2026-09-22) and an RC line ahead of itHosted documentation at docs.vllm.ai/projects/ascend, and user stories page documenting LLaMA-Factory, verl, TRL and GPUStack on top of it2918 stars, last push 2026-10-02, 4415 open issues
Hardware caveats
THE PINNED STACK IS THE RECORD. The README's Prerequisites: Linux only; Python >= 3.10 and < 3.13; CANN == 9.1.0; PyTorch == 2.10.0; TorchNPU == 2.10.0.post4; and vLLM at the SAME version as vllm-ascend. Six versions must move in lockstep. This is the cost of out-of-tree hardware support and the reason support matrices matterSpecific hardware, not a family: Atlas 800I A2 Inference, Atlas A2 Training, Atlas 800I A3 Inference, Atlas A3 Training, plus Atlas 300I Duo marked Experimental. If your SKU is not on that list it is not supportedThe dependency is Huawei's CANN toolkit and torch_npu, not CUDA. There is no Linux-ROCm or CPU path in the pluginLinux only - no Windows, and the surrounding CANN/torch_npu tooling is Linux-centricFeature coverage is tracked in an external support matrix at docs.vllm.ai rather than in the README, so 'vLLM works on Ascend' does not mean every vLLM feature works on Ascend. Check the matrix per quantisation, dtype and model4415 open issues against 2918 stars is an enormous backlog - partly an artifact of a fast-moving upstream and a large user base, partly a sign that support load exceeds what a community plugin absorbs easilyVersion lock means you are frequently on a plugin RC rather than a stable release; v0.26.0rc1 was current while v0.23.0 was the newest stable in the 2026-09 news itemsThe plugin is only as good as Huawei's kernels. A capability present in upstream vLLM that has no Ascend kernel is simply absent here
Best for
Running vLLM on Huawei Ascend NPUs, full stop - there is no realistic alternative at this feature levelStudying how a hardware vendor's ecosystem gets supported without forking the serving framework, and what that pattern costsAnyone deploying into Huawei-dominated regions where the hardware is available and the software stack is pinned anyway
Avoid for
Flexible version management - vLLM, vLLM-Ascend, torch_npu, PyTorch, Python and CANN must all be version-matchedHardware outside the five listed Atlas configurationsNon-Linux hosts, and for any assumption that upstream vLLM features are automatically present - consult the external support matrixA stable release train; expect to run release candidates
notes
The single most important record in the niche-hardware half of this slice. It documents vLLM's hardware-pluggable architecture in production, which is the mechanism by which every future non-CUDA accelerator will be supported - and the pinned-version cost that mechanism imposes. Related: [[niche-tt-metal]] does the same for Tenstorrent via a separate plugin, and [[niche-tensorrt-llm]] is the in-tree CUDA example. All metadata from the GitHub API on 2026-10-03. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release v0.23.0.post1 published 2026-09-22, 679 contributors (distinct committers including anonymous), open_issues_count 3745 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 2919 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set. TRACING VERIFIED NEGATIVELY 2026-10-05. This is an ACTIVE absence, checked against all 664 Python files under vllm_ascend/ (excluding tests): a word-boundary search for otel, opentelemetry, start_span and get_tracer returns ZERO hits. The near-misses are false friends - 'remotely loaded' in dyntra_lb_scheduler.py, recompute_scheduler.py, scheduler_profiling_chunk.py and mamba_hybrid.py, and 'span' only ever as substring of another word. vllm_ascend/observability/ is NOT a tracing surface: provider.py registers an MS Service Metric provider (imported lazily from ms_service_metric.provider_api so there is no hard Core dependency) over two YAML files, and handlers.py pushes five EPLB expert-hotness gauges through get_metric_recorder(). So on Ascend the tracing you get is upstream vLLM's, whatever it is - the plugin contributes metrics, not spans. Recorded as verified_negatively rather than left unknown because the search was exhaustive and the negative is the useful answer: a reader can now stop looking, and can rule out 'Ascend has better tracing than CUDA' as a selection criterion.