FlashAttention GitHub repository (FA2/FA3/FA4)

Official repo. README states: FA3 beta is 'optimized for Hopper GPUs (e.g. H100)', requires H100/H800 and CUDA >= 12.3, currently releases FP16/BF16 forward+backward and FP8 forward; FA4 is 'written in CuTeDSL and optimized for Hopper and Blackwell GPUs (e.g. H100, B200)'. Also documents the two ROCm backends (composable_kernel, and Triton via aiter) with their supported GPU lists, that FA-2 added a paged KV cache path (2.5), and that 2.2 optimized for 'iterative decoding when query has very small sequence length' by splitting the KV cache load across thread blocks with a separate kernel to combine results. License BSD-3-Clause.