FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Hopper-generation attention: warp-specialization / asynchronous execution of the GEMM and softmax phases on separate warpgroups plus fp8 low-precision attention. It is the primary-source anchor for two mechanisms the workload records name - 'WGMMA/TMA warp specialization' and 'fp8 attention' - and it shows that the mechanism is specifically about keeping tensor cores busy during attention, i.e. about the compute-bound half of prefill.