Llama-2-70b-hf config.json (open mirror)

Geometry for the large-MHA/GQA reference: hidden_size 8192, intermediate_size 28672, num_hidden_layers 80, num_attention_heads 64, num_key_value_heads 8 (GQA g=8), head_dim 128, vocab_size 32000, tie_word_embeddings false. NOTE this corrected an error in the Llama-2 paper's own prose: 70B is GQA with 8 KV heads, not MHA with 64. Using n_kv=64 inflates the derived parameter count to 78.4B (+13.6% vs the released 68.98B); using the config's n_kv=8 reproduces 68.98B exactly. This is why flop records must take KV head counts from config.json, not from model-card prose. | SOURCE AUDIT 2026-10-03: the 'publisher' field held a paragraph of prose (authors + affiliations, 61 chars) instead of a publisher name; trimmed to 'Hugging Face' with the authorship detail moved here.