config.json, ibm-granite/granite-4.0-h-tiny

Fields

id

sources/arch-hf-config-granite4

URL

https://huggingface.co/ibm-granite/granite-4.0-h-tiny/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-03

notes

40 layers, layer_types listing exactly 4 'attention' layers among 36 'mamba'; hidden 1536, mamba_expand 2, mamba_d_state 128, mamba_d_head 64, mamba_n_heads 48, mamba_chunk_size 256, mamba_d_conv 4; num_key_value_heads 4, num_experts_per_tok 6, num_local_experts 64, shared_intermediate_size 1024, max_position_embeddings 131072.

Notes

40 layers, layer_types listing exactly 4 'attention' layers among 36 'mamba'; hidden 1536, mamba_expand 2, mamba_d_state 128, mamba_d_head 64, mamba_n_heads 48, mamba_chunk_size 256, mamba_d_conv 4; num_key_value_heads 4, num_experts_per_tok 6, num_local_experts 64, shared_intermediate_size 1024, max_position_embeddings 131072.

Cross-references

This record references no other records.