Mistral-7B-v0.1 Hugging Face config.json (raw)

Read 2026-10-04 for the FLOP-crossover record. hidden_size 4096, 32 layers, 32 attention heads, 8 KV heads (GQA-4), head_dim 128, intermediate_size 14336, vocab 32000, sliding_window null. Geometry is IDENTICAL to Llama-3-8B (d=4096, I=14336, n_q*d_h=4096), which is why the two rows of the crossover table coincide - that is a real property of the configs, not a copy error, and it is stated as such in the record.