obs-vllm-scheduler-preemption · inference-research

Fields

id

sources/obs-vllm-scheduler-preemption

URL

https://github.com/vllm-project/vllm/blob/main/vllm/v1/core/sched/scheduler.py

Publisher

vllm-project/vllm

Kind

repo

Accessed

2026-10-04

notes

_preempt_request(): sets request.num_computed_tokens = 0, clears spec_token_ids, marks output stale, increments request.num_preemptions, records EngineCoreEventType.PREEMPTED, and prepends the request back to the waiting queue. get_kv_cache_usage() delegates to kv_cache_manager.usage -> block_pool.get_usage().

Notes

_preempt_request(): sets request.num_computed_tokens = 0, clears spec_token_ids, marks output stale, increments request.num_preemptions, records EngineCoreEventType.PREEMPTED, and prepends the request back to the waiting queue. get_kv_cache_usage() delegates to kv_cache_manager.usage -> block_pool.get_usage().

Cross-references

This record references no other records.