inference/research
verifiedhigh · 0.90updated 2026-10-04

CUTLASS / CuTe

template-library

Fields

id
compilers/cutlass-cute
Repo
NVIDIA/cutlass
Language
C++
License
BSD-3-Clause
First release
2017
Category
template-library
Target backends
cuda
Input languages
cutlass-cpp
Output artifacts
cubin
Compilation strategy
template-instantiation
Autotuning
no
Production ready
yes
Production evidence
TensorRT-LLM uses CUTLASS kernels for GEMM and attentionvLLM uses CUTLASS for some fused kernelsPyTorch uses CUTLASS for some GEMM operationsFlashAttention-2/3 use CUTLASS-style abstractions
Strengths
Highest performance: CUTLASS achieves the best known performance for GEMM on NVIDIA GPUsMature: nearly a decade of development, extensive architecture support (Volta through Blackwell)Composable: hierarchical decomposition allows mixing and matching tiling strategiesCuTe DSL (CUTLASS 4.0+): Python interface that provides CUTLASS-level performance with a smoother learning curveExtensive examples: 50+ example kernels covering GEMM, convolution, attention, and more
Weaknesses
Very steep learning curve: C++ template metaprogramming is one of the hardest paradigms in systems programmingLong compile times: template instantiation can take minutes for complex kernelsNVIDIA-only: no AMD, Intel, or other GPU supportCuTe DSL is still in beta (as of CUTLASS 4.8, Sept 2026): not yet production-readyDebugging is extremely difficult: template errors produce thousands-line error messagesNo autotuning: tile sizes and other parameters must be chosen manually
Learning curve
very-high
Compile time (s)
300
notes
CUTLASS is NVIDIA's C++ template library for high-performance GEMM and related computations. It provides abstractions for hierarchical decomposition (thread block, warp, and thread-level tiling) and data movement (async copy, tensor core MMA). CUTLASS has been the foundation for nearly all high-performance NVIDIA GPU kernels since Volta. The learning curve is extremely steep: C++ template metaprogramming requires understanding of partial specialization, SFINAE, and complex type computations. Compile times are significant: a single complex kernel can take minutes to compile due to template instantiation. CUTLASS 4.0 introduced CuTe, a Python DSL that provides the same abstractions with a smoother learning curve and faster compile times. CuTe DSL is in public beta (as of 4.8, Sept 2026) and is expected to graduate by end of summer 2026. For production inference, CUTLASS is used indirectly through TensorRT-LLM and vLLM rather than directly by most teams. The realistic learning curve for CUTLASS C++ is 3-6 months to become productive; for CuTe DSL, 2-4 weeks.

Notes

CUTLASS is NVIDIA's C++ template library for high-performance GEMM and related computations. It provides abstractions for hierarchical decomposition (thread block, warp, and thread-level tiling) and data movement (async copy, tensor core MMA). CUTLASS has been the foundation for nearly all high-performance NVIDIA GPU kernels since Volta. The learning curve is extremely steep: C++ template metaprogramming requires understanding of partial specialization, SFINAE, and complex type computations. Compile times are significant: a single complex kernel can take minutes to compile due to template instantiation. CUTLASS 4.0 introduced CuTe, a Python DSL that provides the same abstractions with a smoother learning curve and faster compile times. CuTe DSL is in public beta (as of 4.8, Sept 2026) and is expected to graduate by end of summer 2026. For production inference, CUTLASS is used indirectly through TensorRT-LLM and vLLM rather than directly by most teams. The realistic learning curve for CUTLASS C++ is 3-6 months to become productive; for CuTe DSL, 2-4 weeks.

Cross-references

This record references no other records — it is a leaf. It is cited by 1.