config.json, Qwen/Qwen3-Next-80B-A3B-Instruct

Fields

id

sources/arch-hf-config-qwen3-next

URL

https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-03

notes

48 layers, full_attention_interval 4 (so 12 full-attention layers and 36 gated-delta/linear-attention layers), num_key_value_heads 2, head_dim 256, hidden 2048, num_experts 512, num_experts_per_tok 10, max_position_embeddings 262144. Confirms the hybrid ratio and the very small KV head count.

Notes

48 layers, full_attention_interval 4 (so 12 full-attention layers and 36 gated-delta/linear-attention layers), num_key_value_heads 2, head_dim 256, hidden 2048, num_experts 512, num_experts_per_tok 10, max_position_embeddings 262144. Confirms the hybrid ratio and the very small KV head count.

Cross-references

This record references no other records.