TensorRT-LLM Marlin NVFP4 W4A16 kernel and its own arch predicate isMarlinNvfp4DenseSmSupported (SM89, SM90-99, SM120/121 only)

Read 2026-10-05 from NVIDIA/TensorRT-LLM main. This CORRECTS A WAVE-4 RECORD. docs/23-quantization-selection.md §0 and §1.1 state that TensorRT-LLM 'ships no kernel for any of the four AWQ/GPTQ cells on sm120' and that on consumer Blackwell the newest 4-bit format is available 'and the most popular 4-bit format is not'. That remains true for AWQ/GPTQ. It is no longer the whole NVFP4 story: TensorRT-LLM now ships a Marlin NVFP4 kernel. cpp/tensorrt_llm/kernels/marlin/marlin_nvfp4.h, header comment 'Host-side header for the Marlin NVFP4 W4A16 kernels', contains two inline predicates, quoted: `inline bool isMarlinNvfp4SmSupported(int sm) { return sm >= 89 && sm < 100; }` and `inline bool isMarlinNvfp4DenseSmSupported(int sm) { return isMarlinNvfp4SmSupported(sm) || sm == 120 || sm == 121; }`. And cpp/tensorrt_llm/kernels/marlin/marlin_nvfp4_gemm.cu gates the launch on it: `int const sm = tensorrt_llm::common::getSMVersion(); TLLM_CHECK_WITH_INFO(isMarlinNvfp4DenseSmSupported(sm), 'Marlin NVFP4 GEMM is only supported on SM89, SM90-99, and SM120/121; current SM = %d', sm);`. The arch set is the point: Ada (89), the whole 90-99 band, and consumer Blackwell (120/121) — but NOT sm100/103/107 (sm >= 100 is excluded by the sm < 100 half of isMarlinNvfp4SmSupported and is not re-added by the 120/121 clause). So Marlin NVFP4 in TensorRT-LLM is precisely the complement of native FP4 on sm100/103: on the parts where the CUTLASS NVFP4 kernels exist, Marlin NVFP4 is compiled out, and on the parts where it does not, Marlin NVFP4 is the route. A MoE sibling exists in the same directory (marlin_nvfp4_moe_gemm.cu / marlin_nvfp4_moe_template.h). It is a W4A16 kernel — the header comment says so — so selecting it reproduces the same W4A4-checkpoint-served-as-W4A16 degradation the repo already records for vLLM's MarlinNvFp4LinearKernel and SGLang's flashinfer_trtllm/marlin backend, but on a third engine and on a DIFFERENT arch set. Corroborating the native side: cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp enforces 'if ((config_type_param & CutlassGemmConfig::FP4_ONLY) && !(config_type_param & CutlassGemmConfig::BLACKWELL)) { // FP4 is only supported on blackwell return {}; }' and only routes to the sm120-specific candidate list when 'sm >= 120 && (config_type_param & BLACKWELL) && ((config_type_param & FP4_ONLY) || (config_type_param & FP8FP4_MIXED))'. The instantiated kernel templates are correspondingly narrow: cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/ instantiates Sm100, Sm103 and SM120 launchers only, with template classes CutlassFp4GemmRunner<..., FP4GemmType::W4A4_NVFP4_NVFP4>, <W4A8_MXFP4_MXFP8> and <W8A8_MXFP8_MXFP8> under #ifdef ENABLE_BF16. Note what is absent: no Sm90 FP4 template instantiation at all, which is an independent code-level confirmation that native MX/NVFP4 has no sm90 CUTLASS path in this engine either.