Llama-2-7b-hf config.json (open mirror)

Primary machine-readable geometry for the 7B dense reference model used throughout the workload-profile flop records: hidden_size 4096, intermediate_size 11008, num_hidden_layers 32, num_attention_heads 32, num_key_value_heads 32 (MHA, g=1), head_dim 128 (implicit, = hidden/heads), vocab_size 32000, torch_dtype float16, tie_word_embeddings false. Verbatim. Derived per-layer parameter count from these numbers reproduces the released 6.74B to within rounding, which is the arithmetic check that licenses using these figures in a flop record.