Machete weight-only 4-bit GEMM: CUTLASS 3.x tiles plus an offline prepacked int4 B operand, Hopper only
Fields
flops/machete-weight-only-hopper-gemmNotes
The engine-level support matrix is the most decision-relevant fact and is hard-coded in vLLM: get_min_capability() returns 90, and can_implement() returns False with the message 'Machete requires compute capability of 90 (Hopper)' as well as 'Machete only supported on CUDA'. So a Hopper box gets Machete, an Ampere or Ada box falls through to Marlin, a consumer Blackwell (SM120) is not SM90 so also falls through, and ROCm is excluded outright. That is one model, one quantization format, three different kernels depending on the engine and the GPU - the support matrix of the kernel library, not the model, is what determines decode throughput. Also worth noting: because it is CUTLASS-based it inherits the sm_90a arch-accelerated-feature requirement, so building it for a different arch does not work at all.
HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.