inference/research
verifiedhigh · 0.85updated 2026-10-04

MLPerf Inference benchmark invalidation history

benchmark

Fields

id
sources/meth-mlperf-invalidation-context
Publisher
MLCommons
Kind
benchmark
Published
2024-01-01
Accessed
2026-10-04
notes
MLPerf Inference has had multiple submissions invalidated after publication. Notable cases: Google and NVIDIA submissions in Round 5.0, and AMD/UntetherAI in Round 4.1, were invalidated for violating benchmark rules (e.g., using different hardware than disclosed, or violating latency constraints). This is the canonical example of why benchmark results need independent verification.

Notes

MLPerf Inference has had multiple submissions invalidated after publication. Notable cases: Google and NVIDIA submissions in Round 5.0, and AMD/UntetherAI in Round 4.1, were invalidated for violating benchmark rules (e.g., using different hardware than disclosed, or violating latency constraints). This is the canonical example of why benchmark results need independent verification.

Cross-references

This record references no other records.