A block format's declared block size is not its scale group, and a library default is not a format constant - reading either wrong silently corrupts the memory arithmetic
Fields
gotchas/qzgw-block-size-is-not-the-super-block-and-a-default-is-not-a-constantNotes
The broader lesson is the one this repository has already been burned by twice, in a third disguise. The first burn was quantization records sitting on PROXY ENUM VALUES (nine KV-cache methods filed under weight formats), fixed by reclassifying 45 records. The second was a per-record flag with a correct value but an inverted boolean. This is the same failure shape again: a FIELD THAT IS CORRECT AND INVERTED BY COMPARISON. 256 is a correct number about these formats - it is simply the wrong quantity for this field - and it survives review because anyone checking whether 256 is a real ggml constant says yes. A field that stores a real number about the wrong axis is more dangerous than a field left blank, because it cannot be detected by looking for absence.
THE PROXY-ENUM AUDIT THAT ACCOMPANIED THIS FOUND A SECOND, DISTINCT PROXY CLASS that the 45-record reclassification did not touch. The historical bug was a record filed under the wrong TARGET (a KV method under a weight format). The remaining class is a record filed under the wrong KIND: KERNELS, TOOLKITS and ALGORITHMS sitting under FORMAT enums. Marlin under gptq, Machete under awq, SpQR under gptq, SignRound under gptq, AutoRound under int8, Intel Neural Compressor under int8, ONNX Runtime GenAI model-builder under int8, Olive under int8. None of these misrepresents what the method TARGETS - they are all weight formats, never KV or activation, so the original failure (a query for which formats quantize the KV cache returning nonsense) cannot recur from them. But a query for which formats use GPTQ returns three records that are one kernel and two methods, and Marlin-under-gptq versus Machete-under-awq actively implies two different precisions when both produce identical FP16xFP16 W4A16 output.
ONE RECORD IS ACTIVELY FALSE RATHER THAN MERELY APPROXIMATE: gguf-deprecated-q4-2-q4-3 is filed under gguf-q5-1. It documents two types REMOVED from ggml (type ids 4 and 5, stubs with .type_size = 0 and .is_quantized = false), so anyone selecting gguf-q5-1 from that filter is sent to a record stating that q5_1 is unrelated. Enum values requested rather than added, since shared schema files are parent-owned: exl3, mlx-affine, marlin, machete, spqr, signround, autoround, neural-compressor, olive, ortgenai-model-builder, and a slot for removed GGUF types.
METHOD NOTE FOR THE NEXT AUDIT: the audit that produced this was mechanical rather than interpretive. For each record, check whether the value in weight_group_size is justified by a scale-shared-over-N loop bound in that format's own kernel, and check whether the record's notes contain any explanation for its enum choice. Eleven records had a scheme value their notes never justified. Both halves are scriptable, and the enum-justification half is the one that catches proxies: a record whose notes explain a proxy is fine, and a record whose notes do not is the finding.
Cross-references
Affects (19)
- Quantization formatsGGUF Q2_K / Q2_K_S (2-bit K-quant)
- Quantization formatsGGUF Q3_K_M (3-bit K-quant, medium)contested
- Quantization formatsGGUF Q4_K_M (4-bit K-quant, medium)contested
- Quantization formatsGGUF Q4_K_S (4-bit K-quant, small)contested
- Quantization formatsGGUF Q5_K_M (5-bit K-quant, medium)contested
- Quantization formatsGGUF Q6_K (6-bit K-quant)contested
- Quantization formatsGGUF IQ4_NL (non-linear 4-bit, 32-value blocks, QK4_NL)
- Quantization formatsGGUF IQ2_S (2.5625 bpw, non-linear 2-bit with a 4-bit scale per 64 and sign/high-bit planes)
- Quantization formatsGGUF IQ2_XS (2-bit non-linear, 256 super-block with a 32-element sub-block scale plane)
- Quantization formatsGGUF IQ3_S (3.4375 bpw, non-linear 3-bit with separate sign plane)
- Quantization formatsGGUF IQ1_M (1.75 bpw trellis-flavoured 1-bit with 3-bit sub-block scales and a grid-shift bit)
- Quantization formatsGGUF IQ4_XS (non-linear 4-bit, 256-value super-block with 6-bit sub-block scales)
- Quantization formatsTransformers bitsandbytes integration (load_in_4bit / load_in_8bit via BitsAndBytesConfig)
- Quantization formatsMLX affine quantized linear (mlx-lm's default 4-bit weight format, group_size 64)
- Quantization formatsMXFP4 (OCP microscaling FP4, E8M0 scale)
- Quantization formatsNVFP4 (Blackwell block-scaled E2M1 + E4M3)
- Quantization formatsFP8 E4M3 (per-tensor / per-block scaled)
- Quantization formatsFP8 E5M2
- Quantization formatsINT8 W8A8 rowwise / per-channel