inference/research
verifiedhigh · 0.85updated 2026-10-03

Dynamo components/src/dynamo/common/backend/disagg.py

repo

Fields

id
sources/pd-dynamo-disagg-py
Publisher
NVIDIA
Kind
repo
Accessed
2026-10-03
notes
Source for the prefill-to-decode handoff contract as implemented. enforce_prefill_max_tokens(): 'clamp stop_conditions.max_tokens to 1 in PREFILL mode. Prefill workers only need the KV cache populated for the prompt; producing more than one token is wasted compute and breaks the prefill->decode handoff contract.' require_prefill_result(): a decode-mode worker raises if the request carries no prefill_result, i.e. the decode side treats the handoff payload as a hard precondition. extract_prefill_result() docstring: 'engines use the embedded disaggregated_params to resume from the KV cache the prefill peer already populated'. So the decoded-from-KV contract is explicit and engine-side, not implicit - which matters for the numerics gotcha.

Notes

Source for the prefill-to-decode handoff contract as implemented. enforce_prefill_max_tokens(): 'clamp stop_conditions.max_tokens to 1 in PREFILL mode. Prefill workers only need the KV cache populated for the prompt; producing more than one token is wasted compute and breaks the prefill->decode handoff contract.' require_prefill_result(): a decode-mode worker raises if the request carries no prefill_result, i.e. the decode side treats the handoff payload as a hard precondition. extract_prefill_result() docstring: 'engines use the embedded disaggregated_params to resume from the KV cache the prefill peer already populated'. So the decoded-from-KV contract is explicit and engine-side, not implicit - which matters for the numerics gotcha.

Cross-references

This record references no other records.