vLLM#27579: AITER FP8 GEMM performance on MI355

Fields

id

sources/src5-gotcha-vllm-27579-aiter-fp8-gemm

URL

https://github.com/vllm-project/vllm/issues/27579

Publisher

vLLM / AMD

Kind

forum

Published

2025-10-27

Accessed

2026-10-04

notes

PR updates AITER FP8 linear check for different platforms. Removes is_fp8_fnuz check, enabling AITER FP8 GEMM on MI355. Benchmark: DeepSeek-V3 on MI355, AITER FP8 GEMM outperforms Triton GEMM by 10-14% QPS. Accuracy: gsm8k 0.9515. Demonstrates ROCm FP8 kernel performance.

Notes

PR updates AITER FP8 linear check for different platforms. Removes is_fp8_fnuz check, enabling AITER FP8 GEMM on MI355. Benchmark: DeepSeek-V3 on MI355, AITER FP8 GEMM outperforms Triton GEMM by 10-14% QPS. Accuracy: gsm8k 0.9515. Demonstrates ROCm FP8 kernel performance.

Cross-references

This record references no other records.