inference/research
verifiedhigh · 0.90updated 2026-10-03

MLC LLM python/mlc_llm/quantization/quantization.py QUANTIZATION registry

repo

Fields

id
sources/qlab2-mlc-quantization-py
Publisher
MLC AI
Kind
repo
Accessed
2026-10-03
notes
Authoritative MLC quantization registry: q0f16/q0bf16/q0f32 (no-quant); q3f16_0/1 group_size=40 int3; q4f16_0/1, q4bf16_0/1, q4f32_1 group_size=32 int4, linear_weight_layout KN vs NK, quantize_embedding/final_fc flags; q4f16_2 group_size 32 with embedding and final_fc NOT quantized; q4f16_autoawq group_size=128; q4f16_ft (faster-transformer, int4 into int8 storage); e5m2_e5m2_f16 / e4m3_e4m3_f16 / e4m3_e4m3_f16_max_calibrate per-tensor FP8; fp8_e4m3fn_bf16_block_scale and its _static_activation variant. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Notes

Authoritative MLC quantization registry: q0f16/q0bf16/q0f32 (no-quant); q3f16_0/1 group_size=40 int3; q4f16_0/1, q4bf16_0/1, q4f32_1 group_size=32 int4, linear_weight_layout KN vs NK, quantize_embedding/final_fc flags; q4f16_2 group_size 32 with embedding and final_fc NOT quantized; q4f16_autoawq group_size=128; q4f16_ft (faster-transformer, int4 into int8 storage); e5m2_e5m2_f16 / e4m3_e4m3_f16 / e4m3_e4m3_f16_max_calibrate per-tensor FP8; fp8_e4m3fn_bf16_block_scale and its _static_activation variant. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Cross-references

This record references no other records.