inference/research
verifiedmedium · 0.70updated 2026-10-14

NVIDIA RTX 3090 Ti 24GB llama.cpp cuBLAS: LLaMA-3 8B Q4_K_M pp1024 prefill tok/s

3556.67 tok/s (pp1024 avg, per-request batch-1)

Fields

id
benchmarks/xd-nvidia-rtx-3090-ti-8b-q4-k-m-pp1024
Metric
prefill_tok_s
Value
3,556.67
Unit
tok/s (pp1024 avg, per-request batch-1)
Model
LLaMA-3 8B
Methodology
THIRD-PARTY MEASURED (XiongjieDai GPU-Benchmarks-on-LLM-Inference, RunPod rental). llama.cpp built with cuBLAS (make LLAMA_CUBLAS=1), May-2024 snapshot. NVIDIA RTX 3090 Ti 24GB full offload (-ngl 10000); llama-bench -p 512/1024/4096/8192 -n same. Ubuntu 22.04, CUDA 12.1.1. Value is the 1024-token column. Per-request batch-1, NOT aggregate.
Measured by
third_party
Reproducible
yes

Cross-references

Accelerators (1)

Engine (1)