inclusionAI/Ling-3.1-flash config.json

Fetched 2026-10-18 from inclusionAI/Ling-3.1-flash config.json (main). model_type bailing_hybrid, architectures BailingMoeV3ForCausalLM. hidden_size 4096, num_hidden_layers 72, layer_group_size 8, first_k_dense_replace 4, num_experts 512, num_experts_per_tok 8, num_shared_experts 1, moe_intermediate_size 1280, moe_shared_expert_intermediate_size 1280. MLA: kv_lora_rank 512, qk_rope_head_dim 64, qk_nope_head_dim 128, qk_head_dim 192, v_head_dim 128, q_lora_rank 1536, num_attention_heads 64, head_dim 128. max_position_embeddings 262144, rope_theta 6000000, rope_scaling null, partial_rotary_factor 0.5. vocab 157184, hidden_act silu, tie_word_embeddings false. quantization_config: fp8 e4m3, activation_scheme dynamic, with lm_head and embeddings excluded. modeling_bailing_moe_v3.py sets attention_layer_type = 'attention' iff (layer_idx+1) % layer_group_size == 0 or layer_idx >= num_hidden_layers // layer_group_size * layer_group_size, else 'linear_attention'.