inference/research
verifiedhigh · 0.90updated 2026-10-03

ONNX Runtime documentation: Quantize ONNX Models

repo

Fields

id
sources/qlab2-ort-quant-doc
Publisher
ONNX Runtime (Microsoft)
Kind
repo
Accessed
2026-10-03
notes
Canonical ORT quantization doc. QDQ vs QOperator definitions, quantize_dynamic vs quantize_static, the three calibration methods (MinMax/Entropy/Percentile), the U8U8/U8S8/S8S8 support matrix (CPU: U8U8 U8S8 S8S8; GPU: S8S8 only), reduce_range and per_channel guidance, INT4/UInt4 MatMulNBits with HQQ/GPTQ/RTN algorithms and GatherBlockQuantized with RTN only, opset>=10 requirement, and the 2GB model-optimization size limit. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Notes

Canonical ORT quantization doc. QDQ vs QOperator definitions, quantize_dynamic vs quantize_static, the three calibration methods (MinMax/Entropy/Percentile), the U8U8/U8S8/S8S8 support matrix (CPU: U8U8 U8S8 S8S8; GPU: S8S8 only), reduce_range and per_channel guidance, INT4/UInt4 MatMulNBits with HQQ/GPTQ/RTN algorithms and GatherBlockQuantized with RTN only, opset>=10 requirement, and the 2GB model-optimization size limit. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Cross-references

This record references no other records.