Introducing v0.5 of the AI Safety Benchmark from MLCommons

AI safety benchmarks are needed to assess risks of chat-tuned language models, but existing benchmarks are ad hoc and don't cover the full risk surface.

Introduces a principled approach: use cases, system types, personas, tests, test items. 13 hazard categories, 7 with tests in v0.5. 43,090 test items created with templates. ModelBench platform for evaluation. Explicitly states v0.5 should NOT be used to assess safety of AI systems.

arXiv preprint, no venue claim in comments. Notable for its explicit self-limitation: the authors state v0.5 should not be used to assess safety. This is a rare example of a benchmark authors explicitly warning against premature use. The 43,090 test items and 13 hazard categories are the paper's own numbers. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.