llama.cpp PR #15550: quantize --target-bpw, arguing that same-named quants are not a controlled comparison because bpw differs

Fields

id

sources/qgotcha-llamacpp-15550-target-bpw-bpw-confound

URL

https://github.com/ggml-org/llama.cpp/pull/15550

Publisher

llama.cpp GitHub

Kind

repo

Published

2025-08-24

Accessed

2026-10-03

notes

Author's own argument for bpw-normalized comparison: two models quantized at 'the same Q4_K_M level' can end up at 4.75 and 4.30 bpw, making performance comparisons between models not a controlled experiment. Measurement-methodology evidence, not a bug.

Notes

Author's own argument for bpw-normalized comparison: two models quantized at 'the same Q4_K_M level' can end up at 4.75 and 4.30 bpw, making performance comparisons between models not a controlled experiment. Measurement-methodology evidence, not a bug.

Cross-references

This record references no other records.