vLLM issue #47847: disable_flashinfer_q_quantization + fp8 KV + spec decode silently corrupts sliding-window models on SM100
forum
Fields
id
sources/qgotcha-vllm-47847-flashinfer-bf16q-fp8kv-spec-swaPublisher
vLLM GitHub issues
Kind
forum
Accessed
2026-10-03
notes
Reporter root-caused the kernel defect in flashinfer-ai/flashinfer#3864 (trtllm_batch_decode_with_kv_cache with BF16-Q + FP8-KV mis-computes sliding-window attention past the window at certain seq_len/page alignments) and separately identified a q_scale double-application in vllm/v1/attention/backends/flashinfer.py that is inert when all scales are 1.0.
Notes
Reporter root-caused the kernel defect in flashinfer-ai/flashinfer#3864 (trtllm_batch_decode_with_kv_cache with BF16-Q + FP8-KV mis-computes sliding-window attention past the window at certain seq_len/page alignments) and separately identified a q_scale double-application in vllm/v1/attention/backends/flashinfer.py that is inert when all scales are 1.0.
Cross-references
This record references no other records.