inference/research
verifiedhigh · 0.90blockerupdated 2026-10-03

SM120 FlashInfer sparse-MLA only instantiates page_block_size=64, so mixed-compress-ratio DeepSeek checkpoints cannot start

[blocker] kernel

Fields

id
gotchas/sm120-sparse-mla-block-size
Class
kernel
Severity
blocker
Concepts
blackwell-sm120deepseek-v4-1-flash
Symptom
On 8x RTX PRO 6000 Blackwell (SM120) with TP8, deepseek-ai/DeepSeek-V4.1-Flash fails to start with: ValueError: SM120 sparse-MLA has no decode kernel for this shape: num_tokens=8, num_heads=8, topk=128, d_qk=512, page_block_size=32, model_type=dsv4, extra_topk=0.
Root cause
V4.1's two-tier sparse attention gives each layer a compress_ratio (this checkpoint: [0,0,2,2,...,2,1,1,1,1,1]). vLLM computes compressed-layer KV page block size as block_size // compress_ratio, so ratio-2 layers get pbs = 64//2 = 32. FlashInfer's SM120 dispatch planner hard-codes _PAGE_BLOCK_SIZE = 64 and instantiates the dsv4 decode kernels only at pbs=64. There is no global block size that works: 64 produces pbs=32 and crashes, while 128 gives 'No common block size for 64 (FLASHINFER_MLA_SPARSE_DSV41: [128])' because no pbs=128 instantiation exists either.
Workaround
None available on SM120 with this checkpoint and FlashInfer 0.7.0 - the mixed-ratio architecture is incompatible with the current kernel set. Run the model on SM100 datacenter Blackwell (where the required instantiations exist), or wait for a FlashInfer planner change that instantiates pbs=32.
notes
The reporter walked the other SM120 gates first: indexer_kv_dtype='mxfp4' is rejected on SM120 ('requires Blackwell datacenter GPUs sm_10x') and had to be switched to 'fp8'. So SM120 consumer/server parts hit a cluster of independent kernel-set gaps for this model family, not one bug.

Notes

The reporter walked the other SM120 gates first: indexer_kv_dtype='mxfp4' is rejected on SM120 ('requires Blackwell datacenter GPUs sm_10x') and had to be switched to 'fp8'. So SM120 consumer/server parts hit a cluster of independent kernel-set gaps for this model family, not one bug.

Cross-references