Repo
vllm-project/vllm-metal
Languages
PythonC++MetalMLX
Design goal
Apple-onlylow-latencyserve-vLLM-semantics-on-Mac
Scheduling
continuous-batchingpaged-kvprefix-cache
Formats
safetensorsmlx-quantized-4bitmlx-quantized-8bit
Notable features
A vLLM hardware plugin, not a fork: upstream vLLM supplies the API server, the scheduler and the paged block manager, mlx_lm supplies the token-wise model layers, and the plugin owns the request-aware attention pathUnified paged varlen Metal kernel is the default attention backend as of v0.2.0, which the project credits with 83x TTFT and 3.6x throughput versus v0.1.0Uses M5 NAX tensor units for MHA, GQA and MQA prefillSpeculative decoding support in the pluginOpenAI-compatible API server, installable via Homebrew or a curl installer that needs no compiler
Hardware caveats
Apple Silicon on macOS 15 (Sequoia) or newer only - there is no other backend and no non-Apple pathIt is community maintained, not upstream vLLM. vLLM's own install docs point at it as a separate package, so its feature set and stability track the plugin, not vLLMWorks best with models from the mlx-community HF org that were quantized for MLX; other checkpoints may not hit the tuned pathsModel coverage is a growing set with a separate supported-models matrix, so an architecture not on that list is unsupportedKernel gains are hardware-generation specific - the NAX prefill acceleration is an M5 feature and does not exist on earlier Apple Silicon
Best for
Running vLLM's serving semantics (paged KV, continuous batching, prefix caching) on a MacLocal development against a vLLM-compatible API on Apple SiliconServing a model that fits in unified memory with acceptable single-user latency
Avoid for
Multi-tenant datacenter serving - unified memory bandwidth does not compete with HBMNon-Apple Silicon, or macOS older than 15Production use where the plugin's release cadence and community-maintained status are unacceptable risk; upstream vLLM itself treats this as a community path
notes
Splitting from [[vllm]]: vLLM upstream has no Metal backend of its own and documents vLLM-Metal as the path for Apple Silicon. The 83x TTFT / 3.6x throughput figures in the plugin's news section are self-reported by the project for its own v0.1.0-to-v0.2.0 transition and were not independently reproduced. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release v0.30.0 published 2026-09-23, 85 contributors (distinct committers including anonymous), open_issues_count 29 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 1810 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.