config.json, Dream-org/Dream-v0-Base-7B

Fields

id

sources/arch-hf-config-dream

URL

https://huggingface.co/Dream-org/Dream-v0-Base-7B/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-03

notes

28 layers, hidden 3584, intermediate 18944, num_attention_heads 28, num_key_value_heads 4 (so head_dim 128), max_position_embeddings 131072, max_window_layers 28, mask_token_id 151666, rope_theta 1e6, use_sliding_window false. GQA head count is retained even though the model is bidirectional and recomputes rather than caching.

Notes

28 layers, hidden 3584, intermediate 18944, num_attention_heads 28, num_key_value_heads 4 (so head_dim 128), max_position_embeddings 131072, max_window_layers 28, mask_token_id 151666, rope_theta 1e6, use_sliding_window false. GQA head count is retained even though the model is bidirectional and recomputes rather than caching.

Cross-references

This record references no other records.