GGUF IQ3_S (3.4375 bpw, non-linear 3-bit with separate sign plane)
Fields
quantization/gguf-iq3-sNotes
CORRECTED 2026-10-04: was 64 (correct) - now with a kernel basis, and the value is the one place in this family that is NOT 32. dequantize_row_iq3_s steps `for (ib32 = 0; ib32 < QK_K/32; ib32 += 2)` and applies TWO 4-bit sub-scales per 64 values (`db1` from the low nibble, `db2` from the high nibble, both off one scales byte). So IQ3_S has a 64-value scale group with 4 sub-scales per super-block, where iq2_s and iq4_xs have 32. IQ3S_N_SCALE = QK_K/64 = 4 confirms this. IQ3_M and IQ3_XS both map to IQ3_S in llama-quant.cpp, so they share this value. SCHEMA MISMATCH: there is no ggml/iq* value in the quantization scheme enum, so this record uses the closest existing value 'gguf-q4_k_m' (or 'gguf-q2_k') and puts the true type in `name` and `bits`. Requested enum additions are listed in the task report. Weight-only by construction: activations are promoted to q8_K (an intermediate dot-product format, not a storage format) and the matmul runs at fp16/bf16, so native_support is deliberately EMPTY and the whole accelerator ecosystem is emulated_support. No accelerator has an i-quant or k-quant path in silicon. COMMON TO ALL IQ-QUANTS, verified from ggml/src/ggml-common.h: they use the QK_K = 256 super-block like the K-quants, but instead of a UNIFORM integer grid they quantize to indices into a per-block CODEBOOK of 256 learned non-uniform 4-bit values (the 'i' in i-quant is for the codebook grid / i-quant from the ik_llama.cpp lineage). The index lookup is table-driven at dequant time, which is why they compress better than K-quants at the same nominal bits and why they dequantize more slowly - the dot product needs a gather from the codebook. This is a real throughput trade, not a free win. BACKEND SITUATION, verified: the CUDA library shipped in llamafile releases is size-optimized and OMITS the IQ-quant kernels (IQ1_*, IQ2_*, IQ3_*, IQ4_*); offloading an IQ-quantized model to an NVIDIA GPU therefore leaves those layers on the CPU with correct output but no GPU acceleration (qlab2-llamafile-support-docs). The Metal and ROCm builds are NOT size-optimized and include full IQ-quant GPU support - so an RX 7900 XTX on ROCm/HIP is in a materially better IQ position than an RTX 4090 on a stock llamafile build. Vulkan has no dedicated IQ-quant path. Upstream llama.cpp has per-type dequant kernels; ik_llama.cpp additionally ships the Trellis family (IQ1_KT/IQ2_KT/IQ3_KT/IQ4_KT, integer-base trellis, CUDA PR 113 / Metal PR 475 / NEON PR 471 / CPU PR 441) and _R4/_R16 row-interleaved variants. IQ3_S is the most interesting layout in the i-quant family and the only one with a SEPARATE SIGN PLANE. ggml-common.h defines IQ3S_N_SCALE = QK_K/64 = 4 and the block as: qs[QK_K/4] = 64 bytes of 2-bit codebook indices (the low part), qh[QK_K/32] = 8 bytes of the 3rd index bit, signs[QK_K/8] = 32 bytes of one-bit-per-element SIGNS, and scales[4] = 4 packed sub-block scales, plus one fp16 d. The static_assert confirms 2 + 13*(QK_K/32) + IQ3S_N_SCALE = 2 + 104 + 4 = 110 bytes, identical in size to block_q3_K, hence the '3.44 bpw' string. So IQ3_S is bit-for-bit the same SIZE as q3_K but replaces the uniform integer grid with a codebook plus an explicit sign bit - the sign plane is what buys the extra effective bits, and it is also why dequantization is a gather plus a sign flip. MIXTURE (llama-quant.cpp): LLAMA_FTYPE_MOSTLY_IQ3_S maps to GGML_TYPE_IQ3_S, and IQ3_S is the PROMOTION TARGET for Q8_EXPERT models when IQ2_S/IQ2_M is selected. The 'IQ3_M' CLI name is NOT its own type: llama_tensor_get_quant_type maps LLAMA_FTYPE_MOSTLY_IQ3_M to GGML_TYPE_IQ3_S, i.e. IQ3_M is a mixture of iq3_s with promoted q4_K/q5_K tensors - the tool's own string calls it '3.66 bpw quantization mix'. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'gguf-q3_k_m' (a proxy recorded while the enum lacked a legal value) and is now 'gguf-iq3-s'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: LLAMA.CPP INTEGER STORAGE TYPE, SO STRUCTURALLY ABSENT. Every GGUF type here is a packed integer block format read by a dequant kernel (dequantize_row_*) and then executed as FP16; ggml publishes no int4 or sub-8-bit tensor-core path that consumes these blocks directly. The llama.cpp CUDA and Vulkan backends both run these formats by dequantizing. Consequence for a reader: nothing further to verify - and note the contrast with nvfp4/mxfp4/mxfp8 in this repo, which ARE hardware formats on Blackwell and carry populated native_support lists, so '4-bit' alone does not predict this field. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.