Sonar (formerly Aphrodite Engine)
cuda, rocm, cpu, metal, tpu, xpu
Fields
id
engines/niche-aphrodite-sonarRepo
dphnAI/sonar
Languages
PythonC++RustCUDACMetal
License
AGPL-3.0
First release
2023
Design goal
throughput-firstaggressive-batchingmulti-modelhobbyist-serveproduction
Scheduling
continuous-batchingpaged-kvprefix-cache
Backends
cudarocmcpumetaltpuxpu
Formats
safetensorsawqgptqfp8
Notable features
vLLM fork, not a clean-room engine: the README states 'Sonar is based on vLLM' and adds model/quantization formats, sampling methods, kernels and deployment featuresContinuous batching plus paged KV-cache management, with prefix caching enabled by default rather than behind a flagTensor, pipeline, data and expert parallelism; multi-node multiprocessing without needing a Ray cluster, which is the practical difference from stock vLLMSpeculative decoding via MTP, EAGLE, DSpark, DFlash and n-gram methodsQuantized weights plus FP8 KV cachePrefill/decode disaggregation through NIXL and other KV connectorsServes OpenAI, Anthropic, pooling, scoring, reranking, transcription and Kobold APIs, not just OpenAI chatStill installed and invoked as `aphrodite serve` with default port 2242, so docs and Docker images written for Aphrodite keep working
Hardware caveats
The project was RENAMED. github.com/aphrodite-engine/aphrodite-engine HTTP 301-redirects to dphnAI/sonar; the aphrodite-engine GitHub org now contains only .github, aphrodite-loadbalancer and aphrodite-jax. Anything citing 'aphrodite-engine/aphrodite-engine' as a live repo is citing a redirectAGPL-3.0. This is the licensing fact that matters most for a vLLM fork - vLLM upstream is Apache-2.0, so embedding Sonar in a closed product is a different legal question from embedding vLLMThe repo still builds a Torch CUDA 13.0 wheel by default (APHRODITE_USE_PRECOMPILED=1, uv pip install --torch-backend=cu130), so CUDA version is coupled to the release, not to your clusterAutomatic installer covers Linux x86-64 (CUDA/ROCm/CPU), Linux arm64 CPU and Apple-silicon macOS on Metal; ROCm, Intel XPU, TPU, Docker, WSL2 and source builds are documented as a separate installation guide, i.e. second-classLast push 2026-09-11, latest release v0.24.1 (2026-09-11), 252 open issues - active, but a 0.x version line on top of a fast-moving upstreamBeing a vLLM fork means every upstream refactor lands as merge work here; vLLM's own hardware plugins (see [[niche-vllm-ascend]]) reduce the reason to exist on non-CUDA hardware
Best for
A self-hosted single-model or small-mixture server for a hobbyist or small team that wants vLLM-class features with a one-line installer and more model/quantization coverage than upstream shipsMulti-model serving without standing up RayReading what a well-run vLLM fork looks like - the README documents exactly which deployment features were added on top, which is a useful diff against upstream
Avoid for
Anything proprietary without a licence review - AGPL-3.0 with network use is exactly the case that clause was written forProduction at scale, where vLLM upstream plus [[niche-vllm-production-stack]] is the supported path and 0.x churn is a liabilityPinning to a specific vLLM behaviour - you inherit upstream's breakages through the fork
notes
CORRECTION TO THE COMMON STORY: the widely repeated claim that Aphrodite Engine was 'absorbed into SGLang' could not be verified. A GitHub commit search for 'aphrodite' in sgl-project/sglang returns 0 commits (4 issues mention it). What is verifiable is the 2023-06-23 creation date, the rename to Sonar, the surviving `aphrodite` CLI name, and the Apache-2.0 -> AGPL-3.0 licence change. Treat 'absorbed into SGLang' as folklore. All metadata figures from the GitHub API on 2026-10-03. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-09-11 (0.8 months before this record), latest release v0.24.1 published 2026-09-11, 43 contributors (distinct committers including anonymous), open_issues_count 57 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 1869 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.
Notes
CORRECTION TO THE COMMON STORY: the widely repeated claim that Aphrodite Engine was 'absorbed into SGLang' could not be verified. A GitHub commit search for 'aphrodite' in sgl-project/sglang returns 0 commits (4 issues mention it). What is verifiable is the 2023-06-23 creation date, the rename to Sonar, the surviving `aphrodite` CLI name, and the Apache-2.0 -> AGPL-3.0 licence change. Treat 'absorbed into SGLang' as folklore. All metadata figures from the GitHub API on 2026-10-03. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-09-11 (0.8 months before this record), latest release v0.24.1 published 2026-09-11, 43 contributors (distinct committers including anonymous), open_issues_count 57 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 1869 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.
Cross-references
This record references no other records — it is a leaf. It is cited by 0.