inference/research
verifiedmedium · 0.80updated 2026-10-17

TT-NN scaled-dot-product attention decode API

repo

Fields

id
sources/ttnn-sdpa-decode-api
Publisher
Tenstorrent
Kind
repo
Accessed
2026-10-17
notes
Generated API page for ttnn.transformer.scaled_dot_product_attention_decode on the 'latest' docs track, read 2026-10-17. A living page with no pinned revision, so the constraints are the state of latest on that date. The page states, in its own words, that the implementation is Flash-Decode and currently only supports MQA while decoding a single token; that the op parallelizes over the batch and the K/V sequence dimension; that a batch position of -1 skips compute for that index; and that the K sequence length must be a multiple of k_chunk_size, which unpaged decode defaults to min(512, the largest power-of-two divisor of the sequence length) while paged decode uses 32 when no program config is given. The same MQA sentence appears on the paged sibling page.

Notes

Generated API page for ttnn.transformer.scaled_dot_product_attention_decode on the 'latest' docs track, read 2026-10-17. A living page with no pinned revision, so the constraints are the state of latest on that date. The page states, in its own words, that the implementation is Flash-Decode and currently only supports MQA while decoding a single token; that the op parallelizes over the batch and the K/V sequence dimension; that a batch position of -1 skips compute for that index; and that the K sequence length must be a multiple of k_chunk_size, which unpaged decode defaults to min(512, the largest power-of-two divisor of the sequence length) while paged decode uses 32 when no program config is given. The same MQA sentence appears on the paged sibling page.

Cross-references

This record references no other records.