NousResearch/Hermes-4-405B config.json

Fields

id

sources/mbl-cfg-hermes-4-405b

URL

https://huggingface.co/NousResearch/Hermes-4-405B/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-04

notes

num_hidden_layers 126, hidden_size 16384, num_attention_heads 128, num_key_value_heads 8, head_dim 128, rope_theta 500000, max_position_embeddings 131072. rope_scaling {"rope_type":"llama3","factor":8.0,"high_freq_factor":4.0,"low_freq_factor":1.0,"original_max_position_embeddings":8192}. Read raw 2026-10-04.

Notes

num_hidden_layers 126, hidden_size 16384, num_attention_heads 128, num_key_value_heads 8, head_dim 128, rope_theta 500000, max_position_embeddings 131072. rope_scaling {"rope_type":"llama3","factor":8.0,"high_freq_factor":4.0,"low_freq_factor":1.0,"original_max_position_embeddings":8192}. Read raw 2026-10-04.

Cross-references

This record references no other records.