vLLM vllm/v1/sample/sampler.py (Sampler.forward / Sampler.sample) source
repo
Fields
id
sources/sec-vllm-sampler-pipeline-sourcePublisher
vLLM project
Kind
repo
Published
2026-10-04
Accessed
2026-10-04
notes
The exact order a safety transform must survive in: allowed-token-ids whitelist -> bad-words exclusion -> non-argmax-invariant logit processors (min_tokens, logit_bias) -> penalties -> sample (greedy or temperature -> argmax-invariant processors such as min_p -> top-k/top-p). Two things this makes concrete: (1) with all_greedy the engine returns BEFORE the argmax-invariant processor stage, so any safety transform declaring is_argmax_invariant() is silently skipped under greedy decoding; (2) if a request's transform is skipped, the sampler mixes argmax and non-argmax rows in one batch via torch.where, which is exactly the batch-invariant-batching hazard documented elsewhere in this repo.
Notes
The exact order a safety transform must survive in: allowed-token-ids whitelist -> bad-words exclusion -> non-argmax-invariant logit processors (min_tokens, logit_bias) -> penalties -> sample (greedy or temperature -> argmax-invariant processors such as min_p -> top-k/top-p). Two things this makes concrete: (1) with all_greedy the engine returns BEFORE the argmax-invariant processor stage, so any safety transform declaring is_argmax_invariant() is silently skipped under greedy decoding; (2) if a request's transform is skipped, the sampler mixes argmax and non-argmax rows in one batch via torch.where, which is exactly the batch-invariant-batching hazard documented elsewhere in this repo.
Cross-references
This record references no other records.