MLPerf Results Change Log - invalidated and modified results
Fields
sources/ev-mlperf-results-change-logNotes
CRITICAL FOR MLPerf AUDITING. The official MLPerf change log documents that published results are sometimes modified or invalidated. Key findings: (1) Round 5.0 (Sept 2025): Google and NVIDIA results invalidated - 'Submitted as preview, but validation results were not submitted to available in the subsequent round.' (2) Round 4.0 (Aug 2024): Dell, HPE, Lenovo, Qualcomm results invalidated for same reason. (3) Round 4.1 (Apr 2025): AMD and UntetherAI results invalidated. (4) Round 4.0 Training: Multiple submitters (ASUSTeK, Dell, GigaComputing, NVIDIA, Quanta, Supermicro) had benchmark names changed from 'gnn' to 'RGAT' - indicating the original benchmark name was misleading. This log demonstrates that: (a) Preview results may never become available - they are not validated for production use. (b) Even published results can be invalidated post-hoc. (c) Benchmark names can be adjusted after publication, affecting comparability. This is direct evidence that MLPerf results require careful auditing before use in vendor comparisons.
Cross-references
This record references no other records.