vLLM#50881: MiniMax-M3 fp8 KV cache garbage under CUDA graphs on SM80

Fields

id

sources/src5-gotcha-vllm-50881-fp8-kv-cache-cudagraph

URL

https://github.com/vllm-project/vllm/issues/50881

Publisher

vLLM

Kind

forum

Published

2026-08-03

Accessed

2026-10-04

notes

MiniMax-M3 with fp8 KV cache on A100/A800 (SM80) generates garbage under CUDA graphs. Root cause: _insert_kv uses plain Python advanced-index scatter, not graph-safe. bf16 path uses fused CUDA op (graph-safe). Fix: decorate with @eager_break_during_capture. Demonstrates KV cache corruption with CUDA graphs.

Notes

MiniMax-M3 with fp8 KV cache on A100/A800 (SM80) generates garbage under CUDA graphs. Root cause: _insert_kv uses plain Python advanced-index scatter, not graph-safe. bf16 path uses fused CUDA op (graph-safe). Fix: decorate with @eager_break_during_capture. Demonstrates KV cache corruption with CUDA graphs.

Cross-references

This record references no other records.