inference/research
verifiedmedium · 0.75updated 2026-10-04

QuIP# (Hadamard incoherence processing + E8 lattice codebooks)

2, 3 and 4-bit weight-only (<=4 bits per weight); activations fp16

Fields

id
quantization/quip-sharp
Scheme
quip
Bits
2, 3 and 4-bit weight-only (<=4 bits per weight); activations fp16
Activation scheme
None. Weight-only PTQ. The mechanism is INCOHERENCE PROCESSING: an orthogonal transform (randomized Hadamard) is applied so weights and activations become approximately incoherent (spherically symmetric, sub-Gaussian with low maximum coherence), which is what makes very low bit grids viable. Then E8-lattice vector-quantization codebooks exploit that ball-shaped sub-Gaussian distribution.
Kernels
quip-sharp reference implementation (Cornell-RelaxML); hardware-efficient E8 codebooks chosen specifically to be decodable in few integer ops.
Quality delta
The abstract claims 'state-of-the-art results in extreme compression regimes (<=4 bits per weight)', 'enables new behaviors in PTQ scaling', and 'supports fast inference', with no numeric perplexity in the abstract. No number recorded.
notes
GROUP SIZE LEFT NULL 2026-10-04: QuIP# is incoherence processing (randomized Hadamard) plus E8 LATTICE CODEBOOKS. A lattice codebook has no shared scale over a group of weights; the incoherence transform is what makes a fixed lattice work. Its precision levels (2/3/4-bit) are not group widths. Enum mismatch: 'hqq' is the closest existing value (both use learned/optimized codebooks). Two things worth carrying forward. First, QuIP# is a successor to QuIP v1 (Chee et al. 2023); the v1 preprint was not retrievable this session and is NOT recorded - do not create a separate quip record claiming details from the v2 paper. Second, the incoherent-rotations idea is the same family as QuaRot and SpinQuant and is the reason W4A4 became plausible at all: rotate, then quantize. Where SpinQuant LEARNS the rotation and QuIP#/QuaRot use a fixed or randomized Hadamard, that difference is the substantive distinction among the three. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'hqq' (a proxy recorded while the enum lacked a legal value) and is now 'quip'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Notes

GROUP SIZE LEFT NULL 2026-10-04: QuIP# is incoherence processing (randomized Hadamard) plus E8 LATTICE CODEBOOKS. A lattice codebook has no shared scale over a group of weights; the incoherence transform is what makes a fixed lattice work. Its precision levels (2/3/4-bit) are not group widths. Enum mismatch: 'hqq' is the closest existing value (both use learned/optimized codebooks). Two things worth carrying forward. First, QuIP# is a successor to QuIP v1 (Chee et al. 2023); the v1 preprint was not retrievable this session and is NOT recorded - do not create a separate quip record claiming details from the v2 paper. Second, the incoherent-rotations idea is the same family as QuaRot and SpinQuant and is the reason W4A4 became plausible at all: rotate, then quantize. Where SpinQuant LEARNS the rotation and QuIP#/QuaRot use a fixed or randomized Hadamard, that difference is the substantive distinction among the three. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'hqq' (a proxy recorded while the enum lacked a legal value) and is now 'quip'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Cross-references

Emulated support (5)