inference/research
verifiedmedium · 0.80majorupdated 2026-10-04

Unified memory is not VRAM: capacity, bandwidth, and sharing constraints

[major] hardware

Fields

id
gotchas/local-unified-memory-not-vram
Class
hardware
Severity
major
Concepts
unified-memoryvrammemory-bandwidthapple-siliconmetal
Symptom
A Mac with 128GB unified memory can serve a larger model than a discrete GPU with 24GB VRAM, but the advantage is capacity, not speed. The unified memory bandwidth is shared with the CPU and is not dedicated to the GPU.
Root cause
Apple Silicon uses a unified memory architecture where the CPU, GPU, and Neural Engine share a single memory pool. The GPU can only address part of the advertised capacity (the OS and CPU take their share). The bandwidth figure is the ceiling for the whole system memory subsystem, not a private GPU bus. A discrete GPU's VRAM is private to the GPU with a dedicated high-bandwidth bus (e.g., GDDR7 at 1792 GB/s on RTX 5090 vs LPDDR5X at 614 GB/s on M5 Max).
Workaround
For capacity-bound workloads (large model + long context), unified memory wins. For bandwidth-bound workloads (batch-1 decode), a discrete GPU with HBM/GDDR7 wins. Choose based on whether the workload is capacity-limited or bandwidth-limited.
notes
The honest comparison: a 128GB Mac can hold a 70B model at 4-bit (~35GB weights) with room for KV cache, while a 24GB RTX 4090 cannot. But the Mac's 614GB/s bandwidth yields ~15-25 tokens/sec for a 70B model, while the RTX 4090's 1008GB/s yields ~30-50 tokens/sec for the same model (if it fits). The Mac wins on capacity, the discrete GPU wins on speed. For models that fit in both, the discrete GPU is faster. For models that only fit in the Mac, the Mac is the only option.

Notes

The honest comparison: a 128GB Mac can hold a 70B model at 4-bit (~35GB weights) with room for KV cache, while a 24GB RTX 4090 cannot. But the Mac's 614GB/s bandwidth yields ~15-25 tokens/sec for a 70B model, while the RTX 4090's 1008GB/s yields ~30-50 tokens/sec for the same model (if it fits). The Mac wins on capacity, the discrete GPU wins on speed. For models that fit in both, the discrete GPU is faster. For models that only fit in the Mac, the Mac is the only option.

Cross-references

Affects (10)