AMD ROCm blog: 4-bit TurboQuant KV in LMCache on MI355X

Aditi Ghai Rana, Bowen Bao, David Limpus, Inesh Chakrabarti, Spandan Tiwari, Thiago Crepaldi, Ashish Sirasao. ROCm blogs, 2026-08-28. Vendor measurement of an AMD-productionized 4-bit TurboQuant KV path inside LMCache on vLLM V1, ROCm 7.2.3, 2x Instinct MI355X (gfx950, 288 GB HBM each), MiniMax-M2.5 FP8 at TP=2. Workload is a 20-minute replay of 739 anonymized Claude Code sessions, 32 concurrent users, about 100k context, about 12k warm prefix, p95 TTFT SLO of 60 s. Equal-memory config C (HBM prefix cache plus a 128 GB CPU-DRAM L2) reports BF16 vs TurboQuant: 189 vs 385 completed requests, p95 TTFT 140 s vs 53 s, HBM-resident KV 201280 vs 648064 tokens, LMCache hit rate 75.2 percent vs 86.8 percent, bytes copied 3.30 TB vs 1.94 TB. Middle layers are packed uint8 at 134 B/token/head against 512 B/token/head bf16 on the four boundary layers. Quality: 4-bit KV within about 1 point of bf16; DRAM-reloaded RULER within 0.4 points of HBM at 16k/64k/128k. Vendor claim. MiniMax-M2.5 is not a models/ record in this repo.