inference/research
verifiedhigh · 0.90updated 2026-10-03

Intel AutoRound step_by_step user guide

repo

Fields

id
sources/qlab2-autoround-step-by-step
Publisher
Intel
Kind
repo
Accessed
2026-10-03
notes
The scheme table: W2A16..W8A16 all default to bits=N, group_size=128, sym=True, act_bits=16. The asymmetric restriction (--asym <=7 bits for auto_round/auto_gptq/auto_awq exports because vLLM serves W8 GPTQ symmetric-only and Marlin supports zero points at 4 bits only; llm_compressor format supports 8-bit asym via Machete SM90+/Conch SM80+). The export-format x scheme matrix (auto_round, llm_compressor, mlx, auto_awq, auto_gptq, gguf). NVFP4 group_size 16, MXFP4 group_size 32. Rounding recipes auto-round-best / auto-round / auto-round-light / auto-round-opt-rtn / auto-round-rtn. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Notes

The scheme table: W2A16..W8A16 all default to bits=N, group_size=128, sym=True, act_bits=16. The asymmetric restriction (--asym <=7 bits for auto_round/auto_gptq/auto_awq exports because vLLM serves W8 GPTQ symmetric-only and Marlin supports zero points at 4 bits only; llm_compressor format supports 8-bit asym via Machete SM90+/Conch SM80+). The export-format x scheme matrix (auto_round, llm_compressor, mlx, auto_awq, auto_gptq, gguf). NVFP4 group_size 16, MXFP4 group_size 32. Rounding recipes auto-round-best / auto-round / auto-round-light / auto-round-opt-rtn / auto-round-rtn. | SOURCE AUDIT 2026-10-03: the 'published' field held a verbatim copy of this notes paragraph instead of a date (field corruption across the qlab2-* batch); published set to null because a GitHub README, doc page or source header has no reliable publication date of its own. Content of the document itself was re-verified live 2026-10-03 and the notes are correct.

Cross-references

This record references no other records.