vLLM#47159: Triton W4A16 qzeros assertion fails on GPTQ models

Fields

id

sources/src5-gotcha-vllm-47159-triton-w4a16-qzeros

URL

https://github.com/vllm-project/vllm/issues/47159

Publisher

vLLM

Kind

forum

Published

2026-06-30

Accessed

2026-10-04

notes

GPTQ quantized Qwen 3.6 27B fails with qzeros.shape assertion in triton_w4a16.py. Works on unquantized models. Affects ROCm with TRITON_ATTN backend. Assertion: qzeros.shape == (K // group_size, N // 8). Demonstrates quantization format compatibility issues with Triton kernels.

Notes

GPTQ quantized Qwen 3.6 27B fails with qzeros.shape assertion in triton_w4a16.py. Works on unquantized models. Affects ROCm with TRITON_ATTN backend. Assertion: qzeros.shape == (K // group_size, N // 8). Demonstrates quantization format compatibility issues with Triton kernels.

Cross-references

This record references no other records.