inference/research
verifiedhigh · 0.90updated 2026-10-04

falcon-h1-7B-base config.json

repo

Fields

id
sources/mbl-cfg-falcon-h1-7b-base
URL
https://huggingface.co/tiiuae/falcon-h1-7B-base/raw/main/config.json
Publisher
Hugging Face
Kind
repo
Accessed
2026-10-04
notes
falcon_h1, FalconH1ForCausalLM. num_hidden_layers 44, hidden_size 3072, num_attention_heads 12, num_key_value_heads 2, head_dim 128, rope_theta 100000000000, rope_scaling null, max_position_embeddings 262144, attn_layer_indices null (attention runs in parallel with Mamba-2 on every layer). Mamba-2: mamba_d_state 256, mamba_d_head 128, mamba_n_heads 24, mamba_d_conv 4, mamba_chunk_size 128, mamba_use_mlp true. Read raw 2026-10-04.

Notes

falcon_h1, FalconH1ForCausalLM. num_hidden_layers 44, hidden_size 3072, num_attention_heads 12, num_key_value_heads 2, head_dim 128, rope_theta 100000000000, rope_scaling null, max_position_embeddings 262144, attn_layer_indices null (attention runs in parallel with Mamba-2 on every layer). Mamba-2: mamba_d_state 256, mamba_d_head 128, mamba_n_heads 24, mamba_d_conv 4, mamba_chunk_size 128, mamba_use_mlp true. Read raw 2026-10-04.

Cross-references

This record references no other records.