NVIDIA Dynamo

KV transfer path: NIXL as the primary connector (NixlConnector), with UCX as the default transport and RDMA/GPUDirect for inter-node, plus Mooncake and LMCache as alternative connectors and KVBM for GPU/host tiering. UCX-specific env vars are required - the vLLM connector guide notes NCCL variables such as NCCL_IB_HCA do not apply to the NIXL path. Does Dynamo supersede or complement the Triton/TRT-LLM paths? Complement, and by design it does not try to replace them: it is an orchestration layer that drives them. The TRT-LLM maintainer response on issue #17665 is the clearest statement of the relationship - 'TensorRT-LLM disaggregated serving is now planned to fully adopt the Python transceiver', i.e. Dynamo supplies the runtime and TRT-LLM supplies the transceiver implementation. Handoff contract as implemented (components/src/dynamo/common/backend/disagg.py): prefill workers have max_tokens clamped to 1 because 'producing more than one token is wasted compute and breaks the prefill->decode handoff contract', and a decode-mode worker raises if the request arrives without a prefill_result, so a missing handoff is a hard error rather than a silent recompute. Its tracker also shows the operational surface is still young: open multi-node hang reports (#11914), a UCX_NET_DEVICES silent-garbage root cause (#12463, triaged urgent), an LMCache-on-vLLM silent cache corruption report (#14416), and an RFC to bind batch-invariance into KV namespaces (#58263 in vLLM). ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0 months), latest release v1.5.0 published 2026-09-21, 420 contributors, open_issues_count 1658 (issues PLUS pull requests), NOT archived, 8219 stars. Actively developed with a large team. DEPLOYMENT: NO EVIDENCE OF PRODUCTION DEPLOYMENT FOUND as of 2026-10-04 - see [[ev-production-evidence-is-scarce-and-nearly-always-single-site]]. Note that this is a weaker negative than for a small library: the README publishes an offline batch-inference throughput TUNING GUIDE with a worked example (a log line showing 'Decode batch. #running-req: 233, #token: 370959, token usage: 0.82, cuda graph: True, gen throughput (token/s): 4594.01, #queue-req: 317'), which is evidence that operators tune this engine seriously but is explicitly a configuration guide, not a report of anyone's deployment. FEATURE AXES: all seven UNSTATED as of 2026-10-04; nothing was confirmed from its public materials in this pass. Given that this engine exists to compose vLLM and SGLang with disaggregated serving, disaggregated P/D is the axis most likely to be YES and most worth checking first - but it is recorded as unstated, not assumed. TRACING VERIFIED 2026-10-05, read at main 2026-10-05. lib/backend-common/src/telemetry.rs is explicit about the convention in its own module doc: use tracing::info_span! for static span names, which nest under the framework's engine.generate parent via the runtime's .instrument chain, and start_span() for runtime-computed names, which goes through OTel directly 'while still inheriting the bridged parent context'. start_span returns a NO-OP SpanGuard when no valid parent span context exists, so instrumentation never panics and never emits an orphan root span - there is a test asserting exactly that (start_span_is_noop_without_bridge). Both paths land in one OTel tree. The frontend collector also records ttft_ms onto the request span, so a trace shows the latency that the dynamo_frontend_time_to_first_token_seconds histogram aggregates. per-request rather than per-phase: one span per generate call with TTFT as an attribute. The metrics story is the bigger story and is recorded in the metric_exposure set: Dynamo BRIDGES vendor engine registries (vllm:, sglang:, trtllm_, lmcache:) into one /metrics rather than reimplementing them, so a Dynamo endpoint serves metrics that belong to those engines' records, plus its own generated dynamo_* names.