vLLM custom logits processors documentation (docs/features/custom_logitsprocs.md) and vllm/v1/sample/logits_processor/interface.py
repo
Fields
id
sources/sec-vllm-logits-processor-interfacePublisher
vLLM project
Kind
repo
Published
2026-10-04
Accessed
2026-10-04
notes
States the architectural facts that matter for any in-decode filter: logits processors operate at BATCH granularity on a (num_logits_rows) x vocab_size tensor; a logits ROW IS NOT A REQUEST - rows are reordered every step, and under speculative decoding one request owns one row per draft token. The interface exposes is_argmax_invariant(), which is the hook that tells the engine whether a processor can change greedy output. Legacy V1 runner uses vllm.v1.sample.logits_processor.LogitsProcessor; MRV2 uses vllm.v1.worker.gpu.sample.logits_processor.LogitsProcessor. Interface explicitly not yet stabilised.
Notes
States the architectural facts that matter for any in-decode filter: logits processors operate at BATCH granularity on a (num_logits_rows) x vocab_size tensor; a logits ROW IS NOT A REQUEST - rows are reordered every step, and under speculative decoding one request owns one row per draft token. The interface exposes is_argmax_invariant(), which is the hook that tells the engine whether a processor can change greedy output. Legacy V1 runner uses vllm.v1.sample.logits_processor.LogitsProcessor; MRV2 uses vllm.v1.worker.gpu.sample.logits_processor.LogitsProcessor. Interface explicitly not yet stabilised.
Cross-references
This record references no other records.