obs-sglang-scheduler-retraction · inference-research

Fields

id

sources/obs-sglang-scheduler-retraction

URL

https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/managers/scheduler.py

Publisher

sgl-project/sglang

Kind

repo

Accessed

2026-10-04

notes

_update_after_schedule: 'KV cache pool is full. Retract requests.' retract_decode() returns retracted_reqs plus reqs_to_abort; aborted requests get HTTPStatus.SERVICE_UNAVAILABLE ('Retracted under decode memory pressure without a KV backup. Retry later.') or INTERNAL_SERVER_ERROR. The warning line logs #retracted_reqs and #new_tokens_gained.

Notes

_update_after_schedule: 'KV cache pool is full. Retract requests.' retract_decode() returns retracted_reqs plus reqs_to_abort; aborted requests get HTTPStatus.SERVICE_UNAVAILABLE ('Retracted under decode memory pressure without a KV backup. Retry later.') or INTERNAL_SERVER_ERROR. The warning line logs #retracted_reqs and #new_tokens_gained.

Cross-references

This record references no other records.