inference/research
verifiedmedium · 0.80updated 2026-10-04

AQLM (additive / multi-codebook quantization)

2, 3 and 4-bit weights (targeting sub-4-bit); activations remain fp16, so this is w2a16 / w3a16

Fields

id
quantization/aqlm
Scheme
aqlm
Bits
2, 3 and 4-bit weights (targeting sub-4-bit); activations remain fp16, so this is w2a16 / w3a16
Activation scheme
None. AQLM is weight-only. Each weight vector is represented as a sum of codeword lookups from a small learned codebook rather than as a single point on a uniform grid - input-adaptive additive quantization with codebooks jointly optimized across each transformer block.
Kernels
Paper ships fast GPU and CPU implementations for token generation (fast lookup, no per-element dequantization).
Quality delta
From the paper's abstract: AQLM is 'the first scheme that is Pareto optimal in terms of accuracy-vs-model-size when compressing to less than 3 bits per parameter' and 'significantly improves upon all known schemes in the extreme compression (2bit) regime'. No specific perplexity number in the abstract; none recorded.
notes
GROUP SIZE LEFT NULL 2026-10-04: AQLM quantizes each weight to a SUM of codeword indices (additive / multi-codebook), so the representation is a vector over codebooks, not a single grid with a shared scale. There is no group over which one scale is shared, and the whole point of the additive form is that effective bits are non-integer. Any group size here would be invented. Enum mismatch: 'hqq' is the closest existing value because both are codebook-based quantizers solved by iterative optimization rather than a fixed grid. The real distinguishing property is ADDITIVE decomposition - a weight is a sum of codeword indices, so effective bits are non-integer and not comparable to a plain k-bit grid. This is the reference point for the whole sub-4-bit regime: at 2 bits, uniform-grid methods (GPTQ, AWQ, round-to-nearest) collapse, and the labs that succeed do it by changing the representation (additive codebooks, incoherent rotations, dense-and-sparse outlier splits) rather than by tuning the grid. Compare the ternary record: ternary is a 2-level grid (-1,0,+1); AQLM is a learned multi-codebook sum. Both are sub-4-bit but they are not the same thing. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'hqq' (a proxy recorded while the enum lacked a legal value) and is now 'aqlm'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Notes

GROUP SIZE LEFT NULL 2026-10-04: AQLM quantizes each weight to a SUM of codeword indices (additive / multi-codebook), so the representation is a vector over codebooks, not a single grid with a shared scale. There is no group over which one scale is shared, and the whole point of the additive form is that effective bits are non-integer. Any group size here would be invented. Enum mismatch: 'hqq' is the closest existing value because both are codebook-based quantizers solved by iterative optimization rather than a fixed grid. The real distinguishing property is ADDITIVE decomposition - a weight is a sum of codeword indices, so effective bits are non-integer and not comparable to a plain k-bit grid. This is the reference point for the whole sub-4-bit regime: at 2 bits, uniform-grid methods (GPTQ, AWQ, round-to-nearest) collapse, and the labs that succeed do it by changing the representation (additive codebooks, incoherent rotations, dense-and-sparse outlier splits) rather than by tuning the grid. Compare the ternary record: ternary is a 2-level grid (-1,0,+1); AQLM is a learned multi-codebook sum. Both are sub-4-bit but they are not the same thing. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'hqq' (a proxy recorded while the enum lacked a legal value) and is now 'aqlm'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Cross-references

Emulated support (5)