obs-sglang-schedule-batch-retract
repo
Fields
id
sources/obs-sglang-schedule-batch-retractPublisher
sgl-project/sglang
Kind
repo
Accessed
2026-10-04
notes
release_req(): release_kv_cache(req, tree_cache, is_insert=False) - 'don't insert into the tree because we need the space instantly' - then evict_from_tree_cache for the freed budget, then req.reset_for_retract() which increments retraction_count and zeroes num_matched_prefix_tokens. This is the recompute-on-resume path that makes retraction expensive.
Notes
release_req(): release_kv_cache(req, tree_cache, is_insert=False) - 'don't insert into the tree because we need the space instantly' - then evict_from_tree_cache for the freed budget, then req.reset_for_retract() which increments retraction_count and zeroes num_matched_prefix_tokens. This is the recompute-on-resume path that makes retraction expensive.
Cross-references
This record references no other records.