FlashMLA GitHub repository (DeepSeek)

README, MIT. The 2026.09.30 release requires NVIDIA SM100/SM103 and CUDA >= 13.1 and REMOVED Hopper support plus earlier models (V3/V3.2/V4.0), and changed the FP8/FP4 KV cache format - a concrete example of a kernel library whose support window moves and where an older checkout is required. Decode supports only FP8 and FP4 KV cache; unquantized bfloat16 KV is not supported. V4.1 KV cache layout, head_dim 512: 528 bytes/token = 512 bytes of float8_e4m3 covering all 512 dims (the 64 RoPE dims are quantized too, no bf16 part) + 16 bytes of float8_e8m0 scales, one per 32 consecutive e4m3 values; the fp4 variant is 288 bytes/token = 256 bytes of e2m1 (two per byte, even index in the low nibble) + 32 bytes of float8_e4m3 scales, one per 16 e2m1, valid only for extra_k_cache when k_cache is in the V4.1 format. Repo also ships an FP8 sparse-decode deep dive under docs/. Performance numbers (up to 1024 TFlops decode and 1350 TFlops sparse prefill on B200 with CUDA 13.3; 660 TFlops on H800 in the 2025.04 release) are DeepSeek's own.