vLLM CUTLASS capability gates (csrc/.../w8a8/cutlass/scaled_mm_entry.cu and fp4/nvfp4_scaled_mm_entry.cu): cutlass_scaled_mm_supports_fp8 / _block_fp8 / _fp4
Fields
sources/w4q-vllm-cutlass-capability-gatesNotes
Read from the vLLM main branch 2026-10-04. These C++ predicates are the ground truth behind the Python cutlass_fp8_supported()/cutlass_block_fp8_supported()/cutlass_fp4_supported() helpers and therefore behind which FP8 and FP4 block-scaled kernels get selected per architecture. Three functions, quoted. (1) cutlass_scaled_mm_supports_fp8(cc): 'CUTLASS FP8 kernels need at least CUDA 12.0 on SM90 systems (Hopper) and CUDA 12.4 on SM89 systems (Lovelace)'; returns CUDA_VERSION>=12000 for cc>=90, CUDA_VERSION>=12040 for 89<=cc<90, else false. There is no branch for cc>90 other than the >=90 one, so SM100/103/107/120 all take the SM90 rule - the FP8 per-tensor CUTLASS path is not arch-gated above Hopper, only CUDA-version-gated. (2) cutlass_scaled_mm_supports_block_fp8(cc): 'CUTLASS block-quantized FP8 kernels need at least CUDA 12.0 and at least SM90 (Hopper)'; returns CUDA_VERSION>=12080 for cc>=100, CUDA_VERSION>=12000 for 90<=cc<100, else false. So block-scaled FP8 via CUTLASS is FALSE on Ampere, Ada and sm120, and TRUE on Hopper with CUDA 12.0+ and on Blackwell sm100/103 with CUDA 12.8+ - a CUDA-version gate on Blackwell that the docs table does not express at all. (3) cutlass_scaled_mm_supports_fp4(cc) in csrc/libtorch_stable/quantization/fp4/nvfp4_scaled_mm_entry.cu: returns false unless the CUDA runtime is >= 12080, then returns true only for 100<=cc<120 when ENABLE_NVFP4_SM100 was compiled in, or 120<=cc<130 when ENABLE_NVFP4_SM120 was compiled in, else false. The in-code comment states the reason: 'Only report support when the SM-specific kernel was actually compiled in, so the Python-side backend selector does not choose CUTLASS and then hit TORCH_CHECK_NOT_IMPLEMENTED (or worse, fall through to Marlin).' The matching dispatch function cutlass_scaled_fp4_mm has no branch outside 100<=sm<120 and 120<=sm<130 and otherwise raises STD_TORCH_CHECK_NOT_IMPLEMENTED(false, 'No compiled nvfp4 mm kernel for SM ', sm, '. Recompile with CUDA >= 12.8 and CC >= 100.'). Consequences worth carrying: native NVFP4 CUTLASS compute exists ONLY on sm100/103 and sm120/121 - not Hopper, not Ada, not Ampere - and sm107 (Rubin, cc=107) falls in the 100<=cc<120 branch and is therefore covered by the sm100 kernel; anything at cc 130+ would also raise. Also present and arch-dispatched in this file family: cutlass_group_gemm_supported(cc) (CUDA 12.3+ for SM90, CUDA 12.8+ for SM100), and per-arch dispatch on ENABLE_SCALED_MM_SM100 / SM120 for the tensor-scaled FP8 path, with separate blockwise translation units per architecture (scaled_mm_blockwise_sm90_fp8, _sm100_fp8, _sm120_fp8) - which is direct evidence that sm120 has a BLOCKWISE FP8 CUTLASS kernel even though the vendor matrix marks FP8(block scaling) '.' for Blackwell(sm120) in TensorRT-LLM. The two are different engines and this is not a contradiction, but a reader comparing the TRT-LLM matrix against vLLM must not treat it as one. | 2026-10-05 PATH SWEEP: THREE STALE PATHS FIXED, ALL CONFIRMED MOVES. vLLM relocated its whole csrc quantization tree under csrc/libtorch_stable/quantization/. Verified with the authenticated contents API, not inferred: csrc/libtorch_stable/quantization now contains marlin, fp4, w8a8, awq, gptq, machete, hadamard, cutlass_w4a8, fused_kernels, vectorization.cuh and activation_kernels.cu, and an authenticated code search for the old prefix csrc/quantization returns only 5 hits, i.e. the old tree is largely gone. So the correct paths are csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_entry.cu, csrc/libtorch_stable/quantization/marlin/dequant.h and csrc/libtorch_stable/quantization/fp4/nvfp4_scaled_mm_entry.cu. A reader following the pre-2026-10-05 text would have hit a 404 that looks first-party, which is the failure mode this sweep exists to close.
Cross-references
This record references no other records.