ExLlamaV3 doc/convert.md (EXL3 trellis group size and codebook options)
repo
Fields
id
sources/qzgw-exl3-convertPublisher
turboderp
Kind
repo
Accessed
2026-10-04
notes
The closest thing EXL3 publishes to a group size, verbatim: 'Each embedding is split into 256-wide groups that are rotated and trellis-coded without calibration, and the following layers are calibrated against the quantized table.' So the trellis group width is 256, and it applies to the embedding explicitly. Also documents -cb/--codebook with 'mul1' (default), 'mcg' or '3inst', and notes 'The mul1 codebook is required by some optimized inference paths'. Other flags: -rcp/--recipe per-tensor bitrate, -hq keep attention in higher precision (final size 'slightly larger than the 4.00 bpw requested', typically +0.05-0.10 bpw for MoE). Accessed 2026-10-04.
Notes
The closest thing EXL3 publishes to a group size, verbatim: 'Each embedding is split into 256-wide groups that are rotated and trellis-coded without calibration, and the following layers are calibrated against the quantized table.' So the trellis group width is 256, and it applies to the embedding explicitly. Also documents -cb/--codebook with 'mul1' (default), 'mcg' or '3inst', and notes 'The mul1 codebook is required by some optimized inference paths'. Other flags: -rcp/--recipe per-tensor bitrate, -hq keep attention in higher precision (final size 'slightly larger than the 4.00 bpw requested', typically +0.05-0.10 bpw for MoE). Accessed 2026-10-04.
Cross-references
This record references no other records.