ExLlamaV3 README (EXL3 quantization, based on QTIP)

EXL3 is 'a streamlined variant of QTIP from Cornell RelaxML', described as making SOTA quantization available on consumer hardware. Conversion 'requires only an input model (in HF format) and a target bitrate', 'By computing Hessians on the fly and thanks to a fused Viterbi kernel, the quantizer can convert a model in a single step'. Also documents '2-8 bit cache quantization' as a separate capability. Acknowledges QTIP (arXiv 2406.11235), QuIP# (2402.04396) and Marlin as the intellectual upstream. This is the successor to the archived ExLlamaV2/EXL2 and has no record in this repo. Accessed 2026-10-04.