GGUF Q4_1 / Q5_1 (legacy asymmetric 32-value blocks with per-block scale AND min)
Fields
quantization/gguf-legacy-q4-1Notes
CONFIRMED 2026-10-04 against the kernel: QK4_1 = 32 and QK5_1 = 32. Unlike the symmetric q4_0/q5_0, q4_1/q5_1 are ASYMMETRIC - block_q4_1 carries BOTH a `d` and an `m` (min) fp16 per 32 values, so two fp16 per 32 = 5.5 bpw, and dequantize_row_q4_1 computes `d*(q&0xF) - m`. One scale pair per 32 values, no sub-block plane. Weight-only: activations are promoted to q8_K for the dot product and the matmul runs at fp16/bf16, so native_support is deliberately EMPTY and the whole accelerator ecosystem is emulated_support. No accelerator has a GGUF quant path in silicon - this is the single most commonly botched field in this repo. LAYOUT, verified from ggml/src/ggml-common.h: QK4_1 = 32 and block_q4_1 = { GGML_EXTENSION union { struct { ggml_half d; ggml_half m; } ; ggml_half2 dm; } ; uint8_t qs[QK4_1/2]; } = 2*sizeof(ggml_half) + 16 = 36 bytes / 32 values = 4.5 bpw. QK5_1 = 32 and block_q5_1 = same union + uint8_t qh[4] (the 5th bit) + uint8_t qs[QK5_1/2] = 2*2 + 4 + 16 = 40 bytes / 32 = 5.5625 bpw. The critical difference from q4_0/q5_0 is the MIN: x = d*q + m (asymmetric, zero-point-like) rather than x = d*q. This is the same affine form the k-quants use, at 32-value granularity with an fp16 min. STILL IN CIRCULATION and still selectable in the official quantizer: the tool lists Q4_1 at '4.78G, +0.4511 ppl @ Llama-3-8B' and Q5_1 at '5.65G, +0.1062 ppl' (qlab2-quantize-options-table) - both barely better than q4_0's '+0.4685 ppl' despite being wider, which is the practical reason nobody uses them. The one place llama.cpp still assigns them automatically is the imatrix guard in src/llama-quant.cpp: for Q4_0/Q5_0 WITH an imatrix, the first n_layer/8 ffn_down layers are bumped to Q4_1/Q5_1, with the in-source comment 'Guard against craziness in the first few ffn_down layers that can happen even with imatrix for Q4_0/Q5_0 ... Q4_1/Q5_1 do go crazy on ffn_down without an imatrix.' That comment is a direct, quotable statement that the affine min is what saves those tensors - and that it is unstable without calibration. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'gguf-q4_0' (a proxy recorded while the enum lacked a legal value) and is now 'gguf-q4-1'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.