gemma-4-12B-it config.json (text_config)

gemma4_unified. text_config: num_hidden_layers 48, hidden_size 3840, num_attention_heads 16, num_key_value_heads 8, head_dim 256, global_head_dim 512, num_global_key_value_heads 1, sliding_window 1024, attention_k_eq_v true, num_kv_shared_layers 0. layer_types over 48 layers = 40 sliding_attention + 8 full_attention (full at 0-based indices 5,11,17,23,29,35,41,47, i.e. every 6th). enable_moe_block FALSE, num_experts null -> this is the dense variant. rope_parameters: full_attention rope_type proportional with rope_theta 1e6 and partial_rotary_factor 0.25; sliding_attention rope_type default rope_theta 1e4. max_position_embeddings 262144. Read raw 2026-10-04.