Dickmann et al. — Evaluating AMD Instinct MI300A APU: Performance Insights on LLM Training via Knowledge Distillation (CUG 2025)

Section 5.2 reports vLLM batch-mode inference on ONE MI300A node (4 APUs, gfx942, 128GB unified HBM per APU, ~5.3 TB/s, ROCm 6.2.2) using 100 diverse prompts. Two rows: KafkaLM-15B-Base (2:4-sparse student distilled from Mistral-Small-24B) and Mistral-Small-24B-Base-2501 teacher. Reports TTFT and aggregate tokens/sec. Paper explicitly notes this is the first published MI300A LLM inference data (MI250X/MI300X exist, MI300A did not).