Mistral AI — 'Cheaper, Better, Faster, Stronger' (Mixtral 8x22B release post, benchmark figures 2 and 4)
blog
Fields
id
sources/mixtral-8x22b-release-blogKind
blog
Accessed
2026-10-19
notes
Read https://mistral.ai/news/mixtral-8x22b for the benchmark figures on models/mixtral-8x22b. As recorded by the reader: The release post has no benchmark prose table; MMLU 77.75% and HumanEval (pass@1) 45.1% were read from the labelled per-model figures embedded in the post (Mixtral 8x22B row, 39B active params). The same figures also give MBPP 71.2%, GSM8K maj@1 78.6%, 'Math maj@4' 41.8% and Arc Challenge 91.3 — none of those names are on the allowed list (Math maj@4 is not MATH-500). The post's only prose numbers are GSM8K maj@8 90.8% and Math maj@4 44.6% for the instructed variant, also not on the allowed list.
Notes
Read https://mistral.ai/news/mixtral-8x22b for the benchmark figures on models/mixtral-8x22b. As recorded by the reader: The release post has no benchmark prose table; MMLU 77.75% and HumanEval (pass@1) 45.1% were read from the labelled per-model figures embedded in the post (Mixtral 8x22B row, 39B active params). The same figures also give MBPP 71.2%, GSM8K maj@1 78.6%, 'Math maj@4' 41.8% and Arc Challenge 91.3 — none of those names are on the allowed list (Math maj@4 is not MATH-500). The post's only prose numbers are GSM8K maj@8 90.8% and Math maj@4 44.6% for the instructed variant, also not on the allowed list.
Cross-references
This record references no other records.