AITER-enabled MLA layer inference on MI300X with matrix absorption

Fields

id

sources/amd-aiter-mla-mi300x

URL

https://rocm.blogs.amd.com/software-tools-optimization/aiter-mla/README.html

Publisher

AMD ROCm Blogs

Kind

benchmark

Published

2025-08-25

Accessed

2026-10-04

notes

Measured on MI300X with AITER v0.1.4, ROCm v6.4.0, PyTorch v2.6.0. MLA with matrix absorption delivers up to 2x faster inference vs non-AITER. Speedup increases with context length. Batch sizes 1-512, context 1024-8192.

Notes

Measured on MI300X with AITER v0.1.4, ROCm v6.4.0, PyTorch v2.6.0. MLA with matrix absorption delivers up to 2x faster inference vs non-AITER. Speedup increases with context length. Batch sizes 1-512, context 1024-8192.

Cross-references

This record references no other records.