Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instruct (Kurt, 2026)

Fields

id

sources/bench-meth-kquant

URL

https://arxiv.org/abs/2601.14277

Publisher

arXiv

Kind

paper

Published

2026-01-11

Accessed

2026-10-03

notes

Unified llama.cpp GGUF quantization eval on Llama-3.1-8B-Instruct. Tables 2/3 read directly: WikiText-2 ppl and 5-benchmark scores for 13 quant configs vs an F16 baseline, and pp512/tg128 CPU throughput at fixed settings.

Notes

Unified llama.cpp GGUF quantization eval on Llama-3.1-8B-Instruct. Tables 2/3 read directly: WikiText-2 ppl and 5-benchmark scores for 13 quant configs vs an F16 baseline, and pp512/tg128 CPU throughput at fixed settings.

Cross-references

This record references no other records.