inference/research
verifiedmedium · 0.75updated 2026-10-04

LQER (low-rank quantization error reconstruction)

w4a8 (4-bit weights, 8-bit activations) - claimed near-lossless

Fields

id
quantization/lqer
Scheme
lqer
Bits
w4a8 (4-bit weights, 8-bit activations) - claimed near-lossless
Activation scheme
8-bit activations. Weights are represented as quantized values PLUS a low-rank correction: an activation-induced scale matrix drives the singular-value distribution of the quantization error towards a favourable shape, letting a truncated SVD absorb the residual. Claimed to work without knowledge distillation, grid search, or gradient-based iterative optimization.
Kernels
Reference implementation open-sourced (ChengZhang-98/lqer). Notable: the computation pattern avoids Scatter/Gather from irregular high-precision weight locations, unlike Atom.
Quality delta
From the abstract: 'near-lossless W4A8 quantization on various LLMs and downstream tasks' without knowledge distillation, grid search, or gradient iterative optimization; near-lossless on six downstream tasks while using 1.36x fewer hardware resources than the leading state-of-the-art method. No per-task numbers in the abstract; none recorded.
notes
GROUP SIZE LEFT NULL 2026-10-04. LEFT NULL 2026-10-04. The paper says only 'quantizing other weights to 4-bit grouped fixed-point' - it states that grouping is used but never publishes the group width in the text available here, and the 1% salient weights go to FP16 rather than to a wider grid. A plausible 128 would be a guess. Enum mismatch: 'gptq' is the closest enum since the base grid is a grouped weight quantizer like GPTQ, but LQER is not a GPTQ variant - it adds a low-rank term to the weight representation, which is a different tradeoff (slightly more memory, much less error) than GPTQ's error-compensation across columns. That 'mixed precision with a structural rather than a sparsity-based split' idea is the same family as Atom and Macaron. Note the activations here are 8-bit, so LQER is W4A8, not W4A4 - it does not need rotations because 8-bit activations have no outliers severe enough to require removing. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'gptq' (a proxy recorded while the enum lacked a legal value) and is now 'lqer'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Notes

GROUP SIZE LEFT NULL 2026-10-04. LEFT NULL 2026-10-04. The paper says only 'quantizing other weights to 4-bit grouped fixed-point' - it states that grouping is used but never publishes the group width in the text available here, and the 1% salient weights go to FP16 rather than to a wider grid. A plausible 128 would be a guess. Enum mismatch: 'gptq' is the closest enum since the base grid is a grouped weight quantizer like GPTQ, but LQER is not a GPTQ variant - it adds a low-rank term to the weight representation, which is a different tradeoff (slightly more memory, much less error) than GPTQ's error-compensation across columns. That 'mixed precision with a structural rather than a sparsity-based split' idea is the same family as Atom and Macaron. Note the activations here are 8-bit, so LQER is W4A8, not W4A4 - it does not need rotations because 8-bit activations have no outliers severe enough to require removing. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'gptq' (a proxy recorded while the enum lacked a legal value) and is now 'lqer'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Cross-references

Emulated support (5)