inference/research
verifiedhigh · 0.90updated 2026-10-03

vLLM issue #47847: disable_flashinfer_q_quantization + fp8 KV + spec decode silently corrupts sliding-window models on SM100

forum

Fields

id
sources/qgotcha-vllm-47847-flashinfer-bf16q-fp8kv-spec-swa
Publisher
vLLM GitHub issues
Kind
forum
Accessed
2026-10-03
notes
Reporter root-caused the kernel defect in flashinfer-ai/flashinfer#3864 (trtllm_batch_decode_with_kv_cache with BF16-Q + FP8-KV mis-computes sliding-window attention past the window at certain seq_len/page alignments) and separately identified a q_scale double-application in vllm/v1/attention/backends/flashinfer.py that is inert when all scales are 1.0.

Notes

Reporter root-caused the kernel defect in flashinfer-ai/flashinfer#3864 (trtllm_batch_decode_with_kv_cache with BF16-Q + FP8-KV mis-computes sliding-window attention past the window at certain seq_len/page alignments) and separately identified a q_scale double-application in vllm/v1/attention/backends/flashinfer.py that is inert when all scales are 1.0.

Cross-references

This record references no other records.