inference/research
drafthigh · 0.85updated 2026-10-05

ThunderKittens README (HazyResearch) - tile primitives for CUDA kernels

repo
This record is a draft. Not yet verified. Provisional. Treat every number on this page as provisional.

Fields

id
sources/w7k-thunderkittens-readme
Publisher
Hazy Research / Stanford
Kind
repo
Accessed
2026-10-05
notes
A CUDA C++ header/template DSL rather than a separate compiler, which is why it is categorised as a template library and not a compiler-framework. Its own three principles are simplicity, extensibility ('natively embedded into CUDA') and speed. Hardware-model content: tiles no smaller than 16x16 values because a modern GPU 'is a manycore processor where each core can efficiently run ~16x16 matrix multiplies'; first-class support for 'asynchronous WGMMA calls on H100 GPUs and TCGEN05 calls on B200 GPUs', shared-memory bank conflicts, TMA address generation, distributed shared memory, and NVLink/NVSwitch. Dated maintenance facts from the README: ThunderKittens 2.0 (2026-01-11) added Blackwell plus MXFP8 and NVFP4, restructured the repo so kernels compile individually and dropped the Python package, and 'We no longer actively support Ampere GPUs'; Vera Rubin support with new PTX instructions (2026-09-10). Production evidence is the project's own claim, not a third-party measurement: 'many AI companies use it for production-scale training and inference (e.g., Together AI, Jump Trading, and Cursor)' - three named users, no published benchmark.

Notes

A CUDA C++ header/template DSL rather than a separate compiler, which is why it is categorised as a template library and not a compiler-framework. Its own three principles are simplicity, extensibility ('natively embedded into CUDA') and speed. Hardware-model content: tiles no smaller than 16x16 values because a modern GPU 'is a manycore processor where each core can efficiently run ~16x16 matrix multiplies'; first-class support for 'asynchronous WGMMA calls on H100 GPUs and TCGEN05 calls on B200 GPUs', shared-memory bank conflicts, TMA address generation, distributed shared memory, and NVLink/NVSwitch. Dated maintenance facts from the README: ThunderKittens 2.0 (2026-01-11) added Blackwell plus MXFP8 and NVFP4, restructured the repo so kernels compile individually and dropped the Python package, and 'We no longer actively support Ampere GPUs'; Vera Rubin support with new PTX instructions (2026-09-10). Production evidence is the project's own claim, not a third-party measurement: 'many AI companies use it for production-scale training and inference (e.g., Together AI, Jump Trading, and Cursor)' - three named users, no published benchmark.

Cross-references

This record references no other records.