inference/research
verifiedlow · 0.60updated 2026-10-16

Hardware Corner — GPU LLM Benchmarks per-GPU pages (llama.cpp, context sweep 4k-256k, March 2026)

benchmark

Fields

id
sources/hardware-corner-gpu-llm-benchmarks-pages-2026-03
Publisher
Hardware Corner
Kind
benchmark
Published
2026-03-01
Accessed
2026-10-16
notes
Per-GPU pages under /gpu-llm-benchmarks/<gpu>/ with Prompt Processing and Token Generation tables by model and context tier (4k/16k/32k/64k/128k/256k). llama.cpp, GGUF Q4_K or MXFP4, single GPU, batch-1. Pages say Updated Benchmarks: March 2026. Some rows annotate a llama.cpp build (8189 on the 5090 Qwen3 8B and 30B-A3B rows, 8233 on the 5080 gpt-oss 20B row) and the 3090 Qwen3 rows annotate CUDA with -fa 1. Third-party in-lab measurement; raw llama-bench logs are not published, so confidence stays at 0.6.

Notes

Per-GPU pages under /gpu-llm-benchmarks/<gpu>/ with Prompt Processing and Token Generation tables by model and context tier (4k/16k/32k/64k/128k/256k). llama.cpp, GGUF Q4_K or MXFP4, single GPU, batch-1. Pages say Updated Benchmarks: March 2026. Some rows annotate a llama.cpp build (8189 on the 5090 Qwen3 8B and 30B-A3B rows, 8233 on the 5080 gpt-oss 20B row) and the 3090 Qwen3 rows annotate CUDA with -fa 1. Third-party in-lab measurement; raw llama-bench logs are not published, so confidence stays at 0.6.

Cross-references

This record references no other records.