SGLang

Scheduling terms: 'radix-cache' is the distinguishing mechanism (radix tree over KV blocks); 'prefix-cache' is used here in the looser sense that the radix cache provides prefix reuse generally. AMD entry is Instinct-class only per the README's supported-hardware table. TPU and NPU paths are separate projects, so 'tpu' here means supported but not first-class in this repo. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0 months), latest release v0.5.21 published 2026-10-02 - a release two days before this record's date, which is a strong liveness signal - 1996 contributors, open_issues_count 5553 (issues PLUS pull requests), NOT archived, 36777 stars. DEPLOYMENT: the STRONGEST production evidence in the entire engine dataset, graded YES with numbers attached - see [[ev-production-evidence-is-scarce-and-nearly-always-single-site]]. SGLang issue #40877 (2026-09-23, open) self-describes as 'Field data from running DeepSeek-V4.1-Flash as a production SGLang backend on 8x RTX PRO 6000 Blackwell Max-Q (SM120, 96 GB, PCIe-only - no NVLink)' with the operator stating the numbers 'come from a working day of production use'. Published benchmarks with full conditions are in [[ev-sglang-dsv41-production-128k-prefill-tps]], [[ev-sglang-dsv41-production-decode-tok-s-per-user]], [[ev-sglang-dspark-spec-decoding-2x-decode-ab]] and [[ev-sglang-dsv41-prefix-cache-hit-prefill-speedup]]. The method quality is worth noting because it is the standard this repo holds numbers to: configuration verified server-side via /get_server_info rather than assumed, timing taken from the response `usage` field rather than SSE chunk counts, and a repeat leg (6,765 vs 6,853 tok/s, -1.3%) establishing run-to-run variance. The operator also documents the trap that corrupted their own earlier measurements - trusting container env vars produced 'a full sweep of false data' before they added a config gate. SELF-DECLARED LIMITS, which are why the numbers are citable as production evidence: 'No 24 h soak', block sizes 4/8/12 not re-measured, and >32K prefill/long-context degradation not characterised. Configuration traps documented on this model: DP2 x TP4, prefill context-parallel and --enable-dp-attention alone are all rejected or silently no-op, with the error 'V4.1 vision currently supports TP/EP without DP, CP, PP or MoE A2A'. BUILD COST: no published figure found for SGLang's source compile. Its issue #6830 ('On deepseek, it often takes tens of minutes') refers to torch.compile and CUDA-graph CAPTURE at server startup, which is a JIT/startup cost rather than a source build, and it is too thin to record as a number - see [[ev-build-cost-is-three-separate-builds]]. FEATURE AXES: chunked prefill and prefix caching were already recorded; disaggregated routing exists via the model gateway ('Add EPD (Encode-Prefill-Decode) Disaggregated Routing Support'); structured output is UNSTATED and was not confirmed from the pages read. tracing_support set to per-phase on 2026-10-04: SGLang is the only engine here whose trace is split by phase. Root span f'{role} Req {rid[:8]}' over per-thread and per-slice spans named from the RequestStage enum (tokenize, prefill_forward, decode_loop, decode_forward, and the disaggregation stages prefill_bootstrap, prefill_transfer_kv_cache, decode_prepare, decode_bootstrap, decode_waiting, decode_transferred), with nesting depth encoded as level 1-4. Evidence: obs-sglang-observability-trace-py, obs-sglang-request-stage-enum, and a 2-GPU CI test asserting a PD-mode request emits prefill_transfer_kv_cache, decode_transferred and mooncake_worker_send spans (obs-sglang-disaggregation-trace-test). Caveat: the OTel SDK is NOT bundled - process_tracing_init() raises RuntimeError unless opentelemetry is installed, so the surface exists in source but is absent from a default install.