ExLlamaV2 EXL2 kernel throughput benchmarks (3090 Ti / 4090)

GROUP SIZE LEFT NULL 2026-10-04 for the same reason as the exl2 record LEFT NULL 2026-10-04 for the same reason as the exl2 record: EXL2's mixed per-tensor bitrate has no single group size. ExLlamaV2 throughput benchmarks from the README (3090 Ti / 4090): Llama GPTQ 7B grpsz 128: 181/205 t/s. Llama GPTQ 13B grpsz 128: 110/114 t/s. Llama GPTQ 33B grpsz 128 act-order: 44/48 t/s. OpenLlama GPTQ 3B grpsz 128 act-order: 259/296 t/s. CodeLlama EXL2 4.0 bpw 34B: 44/50 t/s. Llama2 EXL2 3.0 bpw 7B: 217/257 t/s. Llama2 EXL2 4.0 bpw 7B: 185/211 t/s. Llama2 EXL2 5.0 bpw 7B: 164/179 t/s. Llama2 EXL2 2.5 bpw 70B: 33/38 t/s. TinyLlama EXL2 3.0 bpw 1.1B: 656/770 t/s. TinyLlama EXL2 4.0 bpw 1.1B: 602/700 t/s. KEY OBSERVATION: The throughput scales inversely with bit width - 2.5 bpw is slower than 3.0 bpw which is slower than 4.0 bpw. This is because lower bit widths require more complex dequantization and the memory bandwidth saving is offset by increased compute overhead. The 70B model at 2.5 bpw runs at 33-38 t/s, which is usable but much slower than the 7B model at 4.0 bpw (185-211 t/s). The act-order flag (activation-aware ordering) improves quality but reduces throughput. No quality delta is published in the README - only throughput numbers. The project is archived with development moved to ExLlamaV3. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: WEIGHT-ONLY, SO STRUCTURALLY ABSENT. The bits field keeps activations in FP16, so execution is dequantize-to-FP16 then an FP16 tensor-core GEMM; there is no INT4-x-INT4 (or sub-8-bit x sub-8-bit) silicon instruction for it to target. This is a statement about the arithmetic, not an absence of investigation. Consequence for a reader: nothing further to verify - no silicon path is possible for this format as defined. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.