inference/research
verifiedmedium · 0.75updated 2026-10-04

ZeroQuant-V2 (systematic PTQ study + low-rank compensation LoRC)

study spans w4a16 and w4a8; also covers weight-only and activation-only variants across 125M-176B models

Fields

id
quantization/zeroquant-v2
Scheme
zeroquant
Bits
study spans w4a16 and w4a8; also covers weight-only and activation-only variants across 125M-176B models
Activation scheme
None introduced. The paper's own conclusion is that activation quantization is the weak point: 'none of the current methods can achieve the original model quality for quantization with either INT4-weight or INT4-weight-and-INT8-activation'. Its contribution is Low-Rank Compensation (LoRC), which adds low-rank matrices to recover model quality after quantization at minimal size increase.
Kernels
None shipped; inherits GPTQ's approach.
Quality delta
No single number in the abstract. The finding itself is the result: activation quantization is more susceptible to damage than weight quantization, and smaller models often tolerate activation quantization better than larger ones.
notes
GROUP SIZE LEFT NULL 2026-10-04: ZeroQuant-V2 is a systematic PTQ STUDY spanning w4a16 and w4a8 plus a low-rank compensation term, so there is no single group size across the configurations it reports. The low-rank component also means part of the weight is stored outside the grid entirely. Enum mismatch: 'gptq' is the closest enum because ZeroQuant-V2 is a weight-only PTQ method in the GPTQ lineage (it evaluates and extends RTN, GPTQ and ZeroQuant rather than introducing a new grid format). Request the parent add 'zeroquant'. Two things a reader should not get wrong: (1) this is ZeroQuant V2, distinct from the v1 record; v1's headline is INT8 weight+activation with layer-by-layer distillation for BERT and GPT-3 style models, v1 has no separate record here and would need one added; (2) the 'group-wise 3-bit' claim sometimes attributed to ZeroQuant-V2 is NOT supported by the abstract - the abstract mentions no 3-bit configuration at all. Do not record 3-bit for this method without reading the experiments section. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'gptq' (a proxy recorded while the enum lacked a legal value) and is now 'zeroquant'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Notes

GROUP SIZE LEFT NULL 2026-10-04: ZeroQuant-V2 is a systematic PTQ STUDY spanning w4a16 and w4a8 plus a low-rank compensation term, so there is no single group size across the configurations it reports. The low-rank component also means part of the weight is stored outside the grid entirely. Enum mismatch: 'gptq' is the closest enum because ZeroQuant-V2 is a weight-only PTQ method in the GPTQ lineage (it evaluates and extends RTN, GPTQ and ZeroQuant rather than introducing a new grid format). Request the parent add 'zeroquant'. Two things a reader should not get wrong: (1) this is ZeroQuant V2, distinct from the v1 record; v1's headline is INT8 weight+activation with layer-by-layer distillation for BERT and GPT-3 style models, v1 has no separate record here and would need one added; (2) the 'group-wise 3-bit' claim sometimes attributed to ZeroQuant-V2 is NOT supported by the abstract - the abstract mentions no 3-bit configuration at all. Do not record 3-bit for this method without reading the experiments section. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'gptq' (a proxy recorded while the enum lacked a legal value) and is now 'zeroquant'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Cross-references

Emulated support (6)