SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (arXiv:2306.03078)

Fields

id

sources/qhw-src-spqr-paper

URL

https://arxiv.org/abs/2306.03078

Publisher

arXiv

Kind

paper

Published

2023-06-05

Accessed

2026-10-04

notes

SpQR paper by Dettmers et al. Introduces sparse-quantized representation for near-lossless LLM weight compression at 3-4 bits per parameter. Achieves relative accuracy losses of less than 1% in perplexity for LLaMA and Falcon. Enables 33B parameter LLM on single 24 GB consumer GPU with 15% speedup and 4x memory compression.

Notes

SpQR paper by Dettmers et al. Introduces sparse-quantized representation for near-lossless LLM weight compression at 3-4 bits per parameter. Achieves relative accuracy losses of less than 1% in perplexity for LLaMA and Falcon. Enables 33B parameter LLM on single 24 GB consumer GPU with 15% speedup and 4x memory compression.

Cross-references

This record references no other records.