inference/research
verifiedhigh · 0.95updated 2026-10-04

vLLM custom logits processors documentation (docs/features/custom_logitsprocs.md) and vllm/v1/sample/logits_processor/interface.py

repo

Fields

id
sources/sec-vllm-logits-processor-interface
Publisher
vLLM project
Kind
repo
Published
2026-10-04
Accessed
2026-10-04
notes
States the architectural facts that matter for any in-decode filter: logits processors operate at BATCH granularity on a (num_logits_rows) x vocab_size tensor; a logits ROW IS NOT A REQUEST - rows are reordered every step, and under speculative decoding one request owns one row per draft token. The interface exposes is_argmax_invariant(), which is the hook that tells the engine whether a processor can change greedy output. Legacy V1 runner uses vllm.v1.sample.logits_processor.LogitsProcessor; MRV2 uses vllm.v1.worker.gpu.sample.logits_processor.LogitsProcessor. Interface explicitly not yet stabilised.

Notes

States the architectural facts that matter for any in-decode filter: logits processors operate at BATCH granularity on a (num_logits_rows) x vocab_size tensor; a logits ROW IS NOT A REQUEST - rows are reordered every step, and under speculative decoding one request owns one row per draft token. The interface exposes is_argmax_invariant(), which is the hook that tells the engine whether a processor can change greedy output. Legacy V1 runner uses vllm.v1.sample.logits_processor.LogitsProcessor; MRV2 uses vllm.v1.worker.gpu.sample.logits_processor.LogitsProcessor. Interface explicitly not yet stabilised.

Cross-references

This record references no other records.