zai-org/GLM-5.3-Flash config.json

Raw config.json fetched 2026-10-17 (69,416 bytes). model_type glm5_next, architectures Glm5NextForConditionalGeneration. text_config (model_type glm5_next_text): hidden_size 4096, num_hidden_layers 45, layer_types is 34 linear_attention + 11 deepseek_sparse_attention at indices 3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43. MLA: kv_lora_rank 512, qk_rope_head_dim 0, qk_nope_head_dim 256, qk_head_dim 256, v_head_dim 256, q_lora_rank 1536, num_attention_heads 64, num_key_value_heads 64, mla_use_nope true, head_dim 0 (the glm5_next config class forces head_dim to qk_rope_head_dim). DSA indexer: index_n_heads 32, index_head_dim 128, index_topk 2048, index_kpool 4, indexer_types all 45 full. Linear attention (linear_attn_config): num_heads 64, head_dim 128, short_conv_kernel_size 4, 34 kda_layers. MoE: n_routed_experts 288, n_shared_experts 1, num_experts_per_tok 8, moe_intermediate_size 2048, first_k_dense_replace 3 so mlp_layer_types is 3 dense + 42 sparse, intermediate_size 12288, hidden_act silu, swiglu_limit 10. Context: max_position_embeddings 1048576, no rope_scaling and no rope_theta key. vocab_size 154880, num_nextn_predict_layers 1, mhc true, hc_mult 4. vision_config: depth 24, hidden 1024, 16 heads, intermediate 4096, patch 14, image_size 448. quantization_config: fp8 e4m3 dynamic. HF API lastModified 2026-10-15T06:32:52Z; safetensors total 321,323,031,390.