GPTQ: Accurate Post-Training Quantization for GPT models

Fields

id

sources/src5-gptq-github

URL

https://github.com/AutoGPTQ/AutoGPTQ

Publisher

AutoGPTQ

Kind

repo

Published

2023-01-01

Accessed

2026-10-04

notes

GPTQ provides accurate post-training quantization for GPT models. Uses OBQ (Optimal Brain Quantization) for layer-wise quantization. Supports INT4, INT8, FP16. Key for W4A16 quantization. Used by vLLM, llama.cpp for quantized inference.

Notes

GPTQ provides accurate post-training quantization for GPT models. Uses OBQ (Optimal Brain Quantization) for layer-wise quantization. Supports INT4, INT8, FP16. Key for W4A16 quantization. Used by vLLM, llama.cpp for quantized inference.

Cross-references

This record references no other records.