MLC-LLM quantization scheme definitions (GroupQuantize, BlockScaleQuantize) and their compile-time kernel availability conditions

Read 2026-10-05 from mlc-ai/mlc-llm main. MLC-LLM's quantized matmul is a TIR/TVM compiled operator rather than a hand-written CUDA kernel, so its 'kernel matrix' is expressed as scheme dataclasses plus a runtime availability check on an extern cutlass kernel — a third distinct dispatch shape after vLLM's ordered registries and SGLang's enum ladder. Two schemes carry gates. (1) BlockScaleQuantize, dataclass fields quoted: `kind: str = 'block-scale'`, `weight_dtype: Literal['float8_e4m3fn', 'float8_e5m2'] = 'float8_e4m3fn'`, `model_dtype: Literal['float16', 'bfloat16'] = 'bfloat16'`, `weight_block_size: Optional[Tuple[int, int]] = None`, `use_activation_scale: bool = False`, and __post_init__ asserts `self.kind == 'block-scale-quant'` and that weight_dtype.type_code is in [DataTypeCode.Float8E4M3FN, DataTypeCode.Float8E5M2]. That is DeepSeek-style block-scaled FP8, both FP8 dtypes selectable. Its forward() computes `shape_supported_by_cutlass = (self.weight.shape[0] % 128 == 0 and self.weight.shape[1] % 128 == 0)` and then gates the whole fast path on a runtime kernel probe: `if extern.get_store().cutlass_gemm and tvm.get_global_func('cutlass.groupwise_scaled_gemm_e4m3fn_e4m3fn', allow_missing=True) is not None:` — otherwise it falls to dequantize-then-BF16. Two things to carry: the cutlass groupwise GEMM is a COMPILED-IN TVM FUNCTION, so it exists only if the user's TVM build enabled cutlass_gemm, which is an opt-in at build time and not a package install; and the source itself flags the arch coverage as unverified, carrying the literal comment '# Todo: check "shape supported by cutlass" for Hopper'. The repo has no record of MLC-LLM's block-scale FP8 availability, and this is it. (2) GroupQuantize, dataclass fields quoted: `kind: str`, `group_size: int`, `quantize_dtype: Literal['int3', 'int4', 'int8']`, `storage_dtype: Literal['uint32']`, `model_dtype: Literal['float16', 'float32', 'bfloat16']`, `linear_weight_layout: Literal['KN', 'NK']`, with __post_init__ asserting quantize_dtype.type_code == DataTypeCode.INT. So MLC's W4A16 group-quant is a closed enum over {int3, int4, int8} only — there is no FP8 or FP4 member of the group-quant family at all, which is a verified negative against any assumption that MLC can serve a GGUF-equivalent fp4/fp8 group format: those are BlockScaleQuantize's job, and BlockScaleQuantize's own weight_dtype is restricted to the two FP8 types. GroupQuantize also validates storage: 'if storage_dtype.bits < quantize_dtype.bits: raise ValueError("Storage unit should be greater or equal to quantized element")' and 'if self.group_size % self.num_elem_per_group != 0: raise ValueError("Group size should be divisible by numbers of elements per storage")'.