GGUF Q4_0 (legacy block-4 symmetric)
Fields
quantization/gguf-q4-0Notes
CONFIRMED 2026-10-04 against the kernel: QK4_0 = 32, one fp16 d per 32-value block, no zero point. Legacy format: symmetric block-wise 4-bit over blocks of K=32, one FP16 scale per block, no zero-point, codes in [-8,7] packed as nibbles (quant-gguf-kquant-study). It predates k-quants and has no super-block hierarchy. Q4_0 is *worse* on perplexity than both Q4_K_S (7.62) and Q4_K_M (7.56) at comparable size - the direct evidence that format design beats the nominal bit-width label. Weight-only: activations stay in FP16/BF16 at runtime, so there is no '4-bit activations with hardware support' claim to make here. ggml has bespoke dequantize+gemm kernels per type, but no accelerator has a GGUF/k-quant path in silicon; everything is dequantized on the fly. vLLM explicitly labels its GGUF support experimental and under-optimized and positions it as a memory-footprint tool (quant-vllm-gguf-doc). Q4_0 does have good prompt-processing throughput (97.35 pp512 vs 79.57 F16 in that study) but poor decode (4.36 vs 2.83). LEGACY BLOCK LAYOUT, verified from ggml/src/ggml-common.h: QK4_0 = 32 values per block, block_q4_0 = { ggml_half d; uint8_t qs[QK4_0/2]; } = 2 + 16 = 18 bytes / 32 values = 4.5 bpw, symmetric (delta only, no zero point). QK8_0 = 32, block_q8_0 = { ggml_half d; int8_t qs[QK8_0]; } = 2 + 32 = 34 bytes / 32 = 8.5 bpw. Both are 32-value blocks with a single fp16 scale - that is the entire difference from the K-quants, which use 256-value super-blocks with quantized sub-block scales. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: LLAMA.CPP INTEGER STORAGE TYPE, SO STRUCTURALLY ABSENT. Every GGUF type here is a packed integer block format read by a dequant kernel (dequantize_row_*) and then executed as FP16; ggml publishes no int4 or sub-8-bit tensor-core path that consumes these blocks directly. The llama.cpp CUDA and Vulkan backends both run these formats by dequantizing. Consequence for a reader: nothing further to verify - and note the contrast with nvfp4/mxfp4/mxfp8 in this repo, which ARE hardware formats on Blackwell and carry populated native_support lists, so '4-bit' alone does not predict this field. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.