K2-Horizon-MoVA-36B-A4B config.json

Fields

id

sources/k2-horizon-mova-36b-a4b-config-json

URL

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B/blob/main/config.json

Publisher

IFM

Kind

repo

Published

2026-09-01

Accessed

2026-10-18

notes

config.json fetched 2026-10-18. model_type k2_horizon, custom code via auto_map. hidden 2560, 48 layers, 32 query heads, 8 kv heads, head_dim 128, 100 experts, 8 per token, 1 shared expert, moe_intermediate_size 768, mlp_only_layers [0,1,2], mova_num_experts 64, mova_num_experts_per_tok 4, max_position_embeddings 524288, rope_parameters rope_type default rope_theta 1e7, vocab 250624.

Notes

config.json fetched 2026-10-18. model_type k2_horizon, custom code via auto_map. hidden 2560, 48 layers, 32 query heads, 8 kv heads, head_dim 128, 100 experts, 8 per token, 1 shared expert, moe_intermediate_size 768, mlp_only_layers [0,1,2], mova_num_experts 64, mova_num_experts_per_tok 4, max_position_embeddings 524288, rope_parameters rope_type default rope_theta 1e7, vocab 250624.

Cross-references

This record references no other records.