bitsandbytes csrc/kernels.cu template instantiations (which blocksizes actually have compiled kernels)
repo
Fields
id
sources/qzgw-bnb-kernels-instantiationsPublisher
bitsandbytes
Kind
repo
Accessed
2026-10-04
notes
The set of blocksizes for which a CUDA kernel is actually compiled. kQuantizeBlockwise is instantiated for blocksize 64, 128, 256, 512, 1024, 2048, 4096 for both NF4 and FP4. kQuantizeBlockwiseSmall (the 4-bit small-quantization path) is instantiated ONLY for qblock_size 32 and 64, for both NF4 and FP4. So blocksize 32 and 64 are the only sizes with a fast 4-bit small path; the larger sizes exist but go through the general path. This is the concrete 'format exists but the kernel for it is narrower than the config space' fact. Accessed 2026-10-04.
Notes
The set of blocksizes for which a CUDA kernel is actually compiled. kQuantizeBlockwise is instantiated for blocksize 64, 128, 256, 512, 1024, 2048, 4096 for both NF4 and FP4. kQuantizeBlockwiseSmall (the 4-bit small-quantization path) is instantiated ONLY for qblock_size 32 and 64, for both NF4 and FP4. So blocksize 32 and 64 are the only sizes with a fast 4-bit small path; the larger sizes exist but go through the general path. This is the concrete 'format exists but the kernel for it is narrower than the config space' fact. Accessed 2026-10-04.
Cross-references
This record references no other records.