inference/research
verifiedhigh · 0.85updated 2026-10-03

Where Does the Energy Go? Profiling LLM Agent Inference on Blackwell GPUs (arXiv:2609.29707)

paper

Fields

id
sources/pwr2-arxiv-where-does-energy-go
Publisher
Qi Luo, Kunlin Li, Ziwen Wang, Dongsheng Wang, Yun Chen
Kind
paper
Published
2026-08-31
Accessed
2026-10-03
notes
arXiv preprint, cs.DC, 11 pages. Full-stack energy profiling combining NVML GPU counters, Intel RAPL CPU/DRAM counters and IPMI system-level sensors on 2x NVIDIA RTX PRO 6000 Blackwell with Qwen3.8-27B. Hard numbers that make GPU-only energy accounting wrong: GPU-only telemetry misses 41-45% of system energy across all three workloads. Sequential agent workload consumes 63x more system energy per output token than saturated serving (no batching, context growth across turns, tool-induced idle). Continuous batching improves system-level energy efficiency 3.2x from 1 to 16 requests per second as GPU power plateaus while throughput keeps rising. For the memory-bandwidth-bound reasoning workload throughput is UNCHANGED under per-GPU power caps of 400-600 W and drops sharply at 300 W - i.e. decode is insensitive to a large power-cap range.

Notes

arXiv preprint, cs.DC, 11 pages. Full-stack energy profiling combining NVML GPU counters, Intel RAPL CPU/DRAM counters and IPMI system-level sensors on 2x NVIDIA RTX PRO 6000 Blackwell with Qwen3.8-27B. Hard numbers that make GPU-only energy accounting wrong: GPU-only telemetry misses 41-45% of system energy across all three workloads. Sequential agent workload consumes 63x more system energy per output token than saturated serving (no batching, context growth across turns, tool-induced idle). Continuous batching improves system-level energy efficiency 3.2x from 1 to 16 requests per second as GPU power plateaus while throughput keeps rising. For the memory-bandwidth-bound reasoning workload throughput is UNCHANGED under per-GPU power caps of 400-600 W and drops sharply at 300 W - i.e. decode is insensitive to a large power-cap range.

Cross-references

This record references no other records.