SGLang server arguments reference (queueing, memory, scheduling, watchdog, retraction policy)

Verbatim rows: --max-running-requests 'The maximum number of running requests.' default None. --max-queued-requests 'The maximum number of queued requests. This option is ignored when using disaggregation-mode.' default None. --max-total-tokens 'The maximum number of tokens in the memory pool. If not specified, it will be automatically calculated based on the memory usage fraction. This option is typically used for development and debugging purposes.' --retraction-policy 'The decode retraction policy to use when the KV cache is full. length preserves the existing behavior and retracts short-output, long-input requests first. priority retracts lower-priority requests first, using the same priority direction as priority scheduling.' default length. --schedule-conservativeness 'How conservative the schedule policy is. A larger value means more conservative scheduling. Use a larger value if you see requests being retracted frequently.' default 1.0. --priority-scheduling-preemption-threshold 'Minimum difference in priorities for an incoming request to have to preempt running request(s).' default 10. --enable-priority-scheduling default False, and --abort-on-priority-when-disabled 'If set, abort requests that specify a priority when priority scheduling is disabled.' default False. --chunked-prefill-size 'The maximum number of tokens in a chunk for the chunked prefill. Setting this to -1 means disabling chunked prefill.' default None. --max-prefill-tokens default 16384. --watchdog-timeout 'If a forward batch takes longer than this, the server will crash to prevent hanging.' default 300 seconds. --soft-watchdog-timeout 'If a forward batch takes longer than this, the server will dump information for debugging.' default None. --asr-max-buffer-seconds 'Maximum seconds of PCM audio the streaming ASR WebSocket handler will accumulate before closing the session with a buffer_overflow error. Guards against OOM when a client streams audio faster than inference can consume it. Default 60s.' --admin-api-key 'Set admin API key for administrative/control endpoints (e.g., weights update, cache flush, /server_info). Endpoints marked as admin-only require Authorization: Bearer when this is set.' --load-balance-method 'The total_tokens algorithm can only be used when DP attention is applied. This algorithm performs load balancing based on the real-time token load of the DP workers.' --lora-eviction-policy default lru and --lora-drain-wait-threshold 'When any LoRA adapter request waits longer than this threshold (in seconds), the scheduler will selectively drain one running adapter to make room. This mitigates extreme tail latency under high or skewed workloads by preventing a small set of adapters from mo[nopolising].' default 0.0. --prefill-delayer-queue-min-ratio 'Defers prefill until the waiting queue reaches min(running_req * ratio, max_prefill_bs) so small fragments batch into a larger prefill', default None, typical 0.1-0.5, with --prefill-delayer-max-delay-ms as a wall-clock cap 'to bound worst-case TTFT', typical 1000-5000.