MLPerf Inference benchmark invalidation history
benchmark
Fields
id
sources/meth-mlperf-invalidation-contextPublisher
MLCommons
Kind
benchmark
Published
2024-01-01
Accessed
2026-10-04
notes
MLPerf Inference has had multiple submissions invalidated after publication. Notable cases: Google and NVIDIA submissions in Round 5.0, and AMD/UntetherAI in Round 4.1, were invalidated for violating benchmark rules (e.g., using different hardware than disclosed, or violating latency constraints). This is the canonical example of why benchmark results need independent verification.
Notes
MLPerf Inference has had multiple submissions invalidated after publication. Notable cases: Google and NVIDIA submissions in Round 5.0, and AMD/UntetherAI in Round 4.1, were invalidated for violating benchmark rules (e.g., using different hardware than disclosed, or violating latency constraints). This is the canonical example of why benchmark results need independent verification.
Cross-references
This record references no other records.