inference/research
verifiedhigh · 0.95updated 2026-10-04

bitsandbytes csrc/kernels.cu template instantiations (which blocksizes actually have compiled kernels)

repo

Fields

id
sources/qzgw-bnb-kernels-instantiations
Publisher
bitsandbytes
Kind
repo
Accessed
2026-10-04
notes
The set of blocksizes for which a CUDA kernel is actually compiled. kQuantizeBlockwise is instantiated for blocksize 64, 128, 256, 512, 1024, 2048, 4096 for both NF4 and FP4. kQuantizeBlockwiseSmall (the 4-bit small-quantization path) is instantiated ONLY for qblock_size 32 and 64, for both NF4 and FP4. So blocksize 32 and 64 are the only sizes with a fast 4-bit small path; the larger sizes exist but go through the general path. This is the concrete 'format exists but the kernel for it is narrower than the config space' fact. Accessed 2026-10-04.

Notes

The set of blocksizes for which a CUDA kernel is actually compiled. kQuantizeBlockwise is instantiated for blocksize 64, 128, 256, 512, 1024, 2048, 4096 for both NF4 and FP4. kQuantizeBlockwiseSmall (the 4-bit small-quantization path) is instantiated ONLY for qblock_size 32 and 64, for both NF4 and FP4. So blocksize 32 and 64 are the only sizes with a fast 4-bit small path; the larger sizes exist but go through the general path. This is the concrete 'format exists but the kernel for it is narrower than the config space' fact. Accessed 2026-10-04.

Cross-references

This record references no other records.