Hardware Corner Mar 2026: RTX A6000 Llama 3.3 70B Q4_K decode @ 4k ctx
12.6 tok/s (decode, per-request, batch 1, KV depth 4k)
Fields
id
benchmarks/hc26-a6000-llama33-70b-q4k-decode-4kMetric
decode_tok_s
Value
12.6
Unit
tok/s (decode, per-request, batch 1, KV depth 4k)
Model
Llama 3.3 70B
Methodology
THIRD-PARTY LAB MEASURED (Hardware Corner per-GPU LLM benchmark page, 'Updated Benchmarks: March 2026'). llama.cpp, single GPU, GGUF Q4_K (page labels the quant Q4_K; nearest repo format_id is gguf-q4-k-m — Unsloth Q4_K_XL is what the older Dec-2025 ranking sweep used, and these March page numbers differ from that sweep by <0.1 tok/s on the shared 14B/16k cell, so they are the same harness, not a re-quant). Context length is the KV depth the table column names (4k/16k/64k/128k). Token Generation column = batch-1 per-request decode tok/s, NOT aggregate. Prompt Processing column = prefill tok/s at that context depth. Engine build hash, CUDA version, and exact prompt/output token counts beyond the context-depth bucket are not published on the page.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Single RTX A6000, weights fully in the 48 GB GDDR6. This is the context-depth cell the Dec 2025 ranking page did not publish. format_id gguf-q4-k-m is the nearest standard 4-bit GGUF; the page says Q4_K.
Notes
Single RTX A6000, weights fully in the 48 GB GDDR6. This is the context-depth cell the Dec 2025 ranking page did not publish. format_id gguf-q4-k-m is the nearest standard 4-bit GGUF; the page says Q4_K.