mlx-lm issue #1047: Kimi K2.5 KV cache OOM crash on Mac Studio M3 Ultra 512GB under long prefill

Fields

id

sources/src-github-mlx-lm-issue-1047

URL

https://github.com/ml-explore/mlx-lm/issues/1047

Publisher

ml-explore / community

Kind

repo

Published

2026-03-23

Accessed

2026-10-11

notes

Detailed trace of mlx_lm.server on Mac Studio M3 Ultra (512 GB unified memory) loading 418 GB model and running long prefill (116k tokens) until Metal command buffer crashes with Insufficient Memory (00000008:***).

Notes

Detailed trace of mlx_lm.server on Mac Studio M3 Ultra (512 GB unified memory) loading 418 GB model and running long prefill (116k tokens) until Metal command buffer crashes with Insufficient Memory (00000008:***).

Cross-references

This record references no other records.