draftlow · 0.60updated 2026-10-14

llama.cpp CUDA on NVIDIA A10G: Llama-2-7B Q4_0 FA-on pp512 tok/s

2190.43 tok/s (pp512, batch 512)

Fields

benchmarks/llamacpp-cuda-a10g-llama27b-q40-fa-pp512

Metric

prefill_tok_s

Value

2,190.43

Unit

tok/s (pp512, batch 512)

Model

llama-2-7b.Q4_0.gguf (TheBloke, 3.56 GiB, 6.74B params)

Methodology

Same @Hastwell A10G run, pp512.

Measured by

third_party

Reproducible

yes