Zyphra/Zamba-7B-v1 config.json

zamba, ZambaForCausalLM. num_hidden_layers 76, hidden_size 3712, num_attention_heads 16, num_key_value_heads 16, sliding_window null, rope_theta 10000, max_position_embeddings 4096. layers_block_type enumerates all 76 blocks: 63 mamba + 13 hybrid (hybrid at 0-based indices 2,7,13,19,25,31,37,43,49,55,61,67,73). attn_layer_offset 4, attn_layer_period 6. mamba: n_mamba_heads 2, mamba_d_state 16, mamba_d_conv 4, mamba_expand 2, mamba_dt_rank 232. head_dim is absent from config; 3712/16 = 232. Read raw 2026-10-04.