FlashAttention: Fast and memory-efficient exact attention

Fields

id

sources/src5-flashattention-github

URL

https://github.com/Dao-AILab/flash-attention

Publisher

Dao-AILab (Tri Dao)

Kind

repo

Published

2022-05-01

Accessed

2026-10-04

notes

Official implementation of FlashAttention, FlashAttention-2, FlashAttention-3 (beta, Hopper-optimized), FlashAttention-4 (CuTeDSL, Hopper+Blackwell). 25.1k stars. Triton implementation available. Key for efficient attention in inference. Tested on A100, H100, H800.

Notes

Official implementation of FlashAttention, FlashAttention-2, FlashAttention-3 (beta, Hopper-optimized), FlashAttention-4 (CuTeDSL, Hopper+Blackwell). 25.1k stars. Triton implementation available. Key for efficient attention in inference. Tested on A100, H100, H800.

Cross-references

This record references no other records.