Qwen/Qwen3.8-27B config.json

Raw config.json fetched 2026-10-17. model_type qwen3_5, architectures Qwen3_5ForConditionalGeneration. text_config: hidden_size 5120, num_hidden_layers 64, layer_types is 48 linear_attention + 16 full_attention (full_attention_interval 4), num_attention_heads 24, num_key_value_heads 4, head_dim 256, intermediate_size 17408, hidden_act silu, vocab_size 248320, tie_word_embeddings false, max_position_embeddings 262144, rope_parameters rope_type default with mrope_interleaved true, mrope_section [11, 11, 10], partial_rotary_factor 0.25, rope_theta 10000000. Linear-attention block: linear_num_key_heads 16, linear_num_value_heads 48, linear_key_head_dim 128, linear_value_head_dim 128, linear_conv_kernel_dim 4. No MoE keys. vision_config depth 27, hidden 1152. mtp_num_hidden_layers 1.