Hardware Corner - Quadro RTX 8000 48GB local LLM test (llama.cpp and ExLlamaV2, June 2025)

Hands-on test of one used Quadro RTX 8000 (TU102, 48 GB GDDR6, 672 GB/s, 260 W, driver 570.144) on Windows 11 with an Intel i9-12900K and 128 GB DDR4-3600. llama.cpp build b5508, CUDA 12.8, Q5_K_M GGUF, 8k context. ExLlamaV2 0.2.9, 5.0 bpw EXL2, 8k context, max_batch_size 1024, cache_8bit True, one GPU, split none. Figures are per-request, not aggregate. The prose comparison with an RTX 3090 gives round numbers without the same methodology detail and is not filed.