inference/research
verifiedmedium · 0.82updated 2026-10-03

GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon (arXiv:2608.28667)

paper

Fields

id
sources/nrg2-arxiv-greenbench-apple-silicon
Publisher
arXiv
Kind
paper
Published
2026-08-24
Accessed
2026-10-03
notes
arXiv preprint, cs.AI, submitted 2026-08-24, v1. The only per-token joule measurements this search found for Apple silicon. Apple M4 Pro (12-core CPU / 16-core Metal 3 GPU / 16-core ANE, 48 GB unified LPDDR5X, 273 GB/s, macOS 15.4, Ollama v0.21.2 wrapping llama.cpp, MacBook Pro Model Mac16,8), five 3-9B models, 1500+ runs, three NLP tasks. Direct macOS powermetrics package power sampled every 2 s, paired with Ollama per-call timing - explicitly NOT a wall meter and NOT TDP. Measured 0.47 W CPU+GPU package power during sustained inference with 8-12 W total system. System-level J/tok: Llama 3.2 3B 0.09, Phi-3 3.8B 0.12, Qwen 2.5 7B 0.20, Mistral 7B 0.22, Gemma 2 9B 0.25. Package-level mJ/tok: 4.4 / 5.4 / 9.4 / 10.4 / 11.6. Compare to their cited Raspberry Pi 4 baseline of 2.61 J/tok for Qwen2.5-0.5B (Husom et al.). Caveat: single user, local deployment, no batching - not comparable to a datacenter serving number.

Notes

arXiv preprint, cs.AI, submitted 2026-08-24, v1. The only per-token joule measurements this search found for Apple silicon. Apple M4 Pro (12-core CPU / 16-core Metal 3 GPU / 16-core ANE, 48 GB unified LPDDR5X, 273 GB/s, macOS 15.4, Ollama v0.21.2 wrapping llama.cpp, MacBook Pro Model Mac16,8), five 3-9B models, 1500+ runs, three NLP tasks. Direct macOS powermetrics package power sampled every 2 s, paired with Ollama per-call timing - explicitly NOT a wall meter and NOT TDP. Measured 0.47 W CPU+GPU package power during sustained inference with 8-12 W total system. System-level J/tok: Llama 3.2 3B 0.09, Phi-3 3.8B 0.12, Qwen 2.5 7B 0.20, Mistral 7B 0.22, Gemma 2 9B 0.25. Package-level mJ/tok: 4.4 / 5.4 / 9.4 / 10.4 / 11.6. Compare to their cited Raspberry Pi 4 baseline of 2.61 J/tok for Qwen2.5-0.5B (Husom et al.). Caveat: single user, local deployment, no batching - not comparable to a datacenter serving number.

Cross-references

This record references no other records.