OpenTelemetry support is real in four of eight engines and absent in the rest; a trace of one request is one span in vLLM and three spans in Triton, and neither is per-token

You enable OTel on four engines and get nothing on the other four, with no warning. Or you get a trace and expect a span per token or per prefill/decode step and find one span for the whole request, so the trace cannot answer 'which phase was slow'. Or you compare a vLLM trace with a Triton trace and the span counts differ by an order of magnitude for the same logical operation.

Per-engine status, verified in source: vLLM - full OTel, bundled packages, one span per request. SGLang - full OTel plus per-slice spans, but the SDK is an install-time extra. TensorRT-LLM - OTel via --otlp_traces_endpoint, packages installed separately. Triton - OTel as an alternative trace backend (--trace-config mode=opentelemetry) producing top-level/model/compute spans. TGI - OTel present but opt-in and generic: it emits gRPC SERVER spans named after the RPC method with rpc.* attributes, via a custom interceptor that exists to handle gRPC over Unix domain sockets. llama.cpp - no OTel at all (0 code hits in-repo). LMDeploy - no OTel at all (0 code hits). Ollama - no OTel at all; the single README hit points at a third-party tool. What a trace actually looks like: vLLM creates ONE span named `llm_request` of SpanKind.SERVER, started at the frontend arrival timestamp with the extracted parent trace context, carrying gen_ai.latency.time_to_first_token, gen_ai.latency.e2e, gen_ai.latency.time_in_queue, gen_ai.latency.time_in_model_prefill, gen_ai.latency.time_in_model_decode, gen_ai.latency.time_in_model_inference, gen_ai.usage.prompt_tokens, gen_ai.usage.completion_tokens and gen_ai.request.id - prefill and decode appear as attributes on a single span, not as child spans. Triton's minimal trace is 3 spans (top level, model, compute) with model_name/model_version/request_id/parent_id attributes. TGI's is 1 generic RPC span. Only SGLang produces a per-phase tree: a root span f'{role} Req {rid[:8]}' over the incoming traceparent, then per-thread and per-slice spans named by RequestStage.stage_name (tokenize, api_server_dispatch, request_process, prefill_waiting, prefill_forward, chunked_prefill, decode_loop, decode_forward, spec_draft, spec_verify, run_batch_cpu, and the disaggregation stages prefill_bootstrap, prefill_transfer_kv_cache, decode_prepare, decode_bootstrap, decode_waiting, decode_transferred), with level 1-4 encoding nesting depth and a runtime-adjustable level 0-3.

Pick the engine's tracing honestly rather than assuming uniformity. For a single-request latency breakdown on vLLM, read the attributes on the one llm_request span - that is sufficient and cheap. For prefill/decode/KV-transfer phase attribution, SGLang is the only engine here that gives you spans, and its level-2 setting ('Trace all slices except nested ones') is the practical default. Add --collect-detailed-traces (vLLM, model|worker|all) only when you need time_in_model_forward vs time_in_model_execute, and accept the documented cost - the project itself notes this 'involves use of possibly costly and or blocking operations'. On SGLang set SGLANG_TRACE_ASYNC=1 at high batch; on Triton do not mix OTel-mode and native-mode traces in one analysis because one uses the system clock and the other a steady clock.

Disaggregated prefill/decode is where the traces differ most and where they are most worth having. SGLang has a 2-GPU CI test that asserts a request through a load balancer in PD mode emits spans named prefill_transfer_kv_cache, decode_transferred and mooncake_worker_send - so the KV handoff across the wire is traceable, and the same request also feeds `sglang:per_stage_req_latency_seconds{stage=...}` because the metrics and the trace share one stage vocabulary. TensorRT-LLM's OTel guide documents proxy configuration for disaggregated serving, so context does propagate, but the spans are not broken down by phase. vLLM's disaggregated path has no documented per-phase span structure; the KV-transfer wait appears only in metrics (`vllm:num_requests_waiting_by_reason{reason="deferred"}` covers 'KV transfer' among the deferral causes, and `vllm:num_requests_kv_fetch_by_stage` splits async KV load into waiting_to_start / in_progress / completed_waiting) rather than as trace spans. That is the honest state: vLLM can tell you a request is waiting on a KV fetch; SGLang can show you the transfer itself in a timeline.