inference/research
verifiedhigh · 0.90updated 2026-10-03

optimum-quanto GitHub README (design overview)

repo

Fields

id
sources/qlab2-optimum-quanto-readme
Publisher
Hugging Face
Kind
repo
Accessed
2026-10-03
notes
Design details: projection is symmetric per-tensor or per-channel for int8 and float8, and GROUP-WISE AFFINE (with a shift / zero-point) for lower bitwidth. Weights are usually quantized per-channel along the first dimension (output features); matmul weights are always quantized along their first axis because all output features share it. Accelerated matmuls on CUDA: int8-int8, fp16-int4, bf16-int8, bf16-int4. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Notes

Design details: projection is symmetric per-tensor or per-channel for int8 and float8, and GROUP-WISE AFFINE (with a shift / zero-point) for lower bitwidth. Weights are usually quantized per-channel along the first dimension (output features); matmul weights are always quantized along their first axis because all output features share it. Accelerated matmuls on CUDA: int8-int8, fp16-int4, bf16-int8, bf16-int4. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Cross-references

This record references no other records.