Qwen3-32B Hugging Face config.json (raw)

Read 2026-10-04 as the dense control for the Qwen3-30B-A3B row. hidden_size 5120, 64 layers, 64 attention heads, 8 KV heads (GQA-8), head_dim 128 (n_q*d_h = 8192), intermediate_size 25600, vocab 151936, max_position_embeddings 40960. Same family, same vocab, same context budget as 30B-A3B but dense - so any difference in the attention share is attributable to the MoE, not to the corpus or the tokenizer.