NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 config.json

Fetched 2026-10-18 from the BF16 repo. Top-level model_type nemotron_h_omni, architectures NemotronH_Omni_Reasoning_V3, vision_config model_type radio (hidden 1280, 32 layers, 16 heads, patch 16). llm_config model_type nemotron_h: hidden_size 4096, 88 layers_block_type (40 mamba, 40 moe, 8 attention at indices 7, 16, 25, 36, 47, 58, 69, 78), num_attention_heads 32, num_key_value_heads 2, head_dim 128, vocab_size 131072, max_position_embeddings 1048576, sliding_window null. MoE scalars, unlike the Bolt child which varies per block: n_routed_experts 512, num_experts_per_tok 22, moe_intermediate_size 2688, n_shared_experts 1, moe_shared_expert_intermediate_size 5376, moe_latent_size 1024. Mamba-2: mamba_num_heads 128, mamba_head_dim 64, ssm_state_size 128, conv_kernel 4. No rope_scaling key. The card and the technical report say 12B active but never state 22 routed experts; this file is the source for experts_per_token.