zai-org/GLM-4.5 config.json · inference-research

Fields

id

sources/mbl-cfg-glm-4-5

URL

https://huggingface.co/zai-org/GLM-4.5/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-04

notes

glm4_moe. num_hidden_layers 92, hidden_size 5120, num_attention_heads 96, num_key_value_heads 8, head_dim 128, rope_theta 1000000, rope_scaling null, sliding_window null, max_position_embeddings 131072. NO kv_lora_rank / qk_nope_head_dim / qk_rope_head_dim / v_head_dim keys at all - this is plain GQA, not MLA. MoE: n_routed_experts 160, n_shared_experts present, num_experts_per_tok 8. Read raw 2026-10-04.

Notes

glm4_moe. num_hidden_layers 92, hidden_size 5120, num_attention_heads 96, num_key_value_heads 8, head_dim 128, rope_theta 1000000, rope_scaling null, sliding_window null, max_position_embeddings 131072. NO kv_lora_rank / qk_nope_head_dim / qk_rope_head_dim / v_head_dim keys at all - this is plain GQA, not MLA. MoE: n_routed_experts 160, n_shared_experts present, num_experts_per_tok 8. Read raw 2026-10-04.

Cross-references

This record references no other records.