MLPerf v5.1 Supermicro AS-4126GS-NBR-LCC 8x B200-SXM-180GB llama3.1-8b Interactive run_1 harness summary log
Fields
sources/bench-mlperf-v5-1-supermicro-b200-llama3-8b-interactive-logNotes
THE UNITS PROOF. The v5.1 summary_results.json publishes this run as llama3.1-8b / Interactive / 122269.0 / Performance_Units='Queries/s'. The raw harness log for the identical run prints 'Completed samples per second : 947.90' and 'Completed tokens per second: 122268.73'. 122269 divided by 947.90 is 129.0 - the token count per sample - so the figure is tokens per second and the published unit label 'Queries/s' is simply wrong. Every one of the eight v5.1 llama3.1-8b Interactive rows carrying 'Queries/s' has this defect. This is the cleanest available demonstration that a number copied out of an MLPerf leaderboard can carry a unit that changes its meaning by a factor of ~129, with no error message anywhere. | DEAD-LINK VERIFIED 2026-10-04. THREE independent GET probes returned HTTP 404 with body '404: Not Found' on raw.githubusercontent.com. The path cites MLPerf v6.1 ('inference_results_v6.1') while the record is titled v5.1 - the directory name is AS-4126GS-NBR-LCC_B200-SXM-180GBx8_TRT, which no longer exists in that repo. GitHub's contents API for closed/Supermicro/results now lists only 1-node-2S-GNR_128C, 1-node-2S-GNR_86C, 1-node-4S-GNR_86C, 1-node-4x-BMG-B70, 8xMI355X_2xEPYC-9575F, AS-8126GS-NB3RT_B300-SXM-288GB_TRT, SYS-822GS-NB3RT - Supermicro's B200 directory is gone from v6.1. THE FIGURE STANDS: the units proof (122269.0 Queries/s = 947.90 samples/s x 129 tokens/sample = 122268.73 tokens/s) was read from this log when it existed and is a mathematical identity, not a reading that can rot. Do not delete. | 2026-10-05 REPAIR: REPAIRED - THIS WAS NOT A DEAD LINK, IT WAS A MIS-TYPED REPO, AND THE UNITS PROOF IS RE-READ FROM A LIVE FILE. The earlier pass concluded the log was absent from MLCommons and that Supermicro's B200 directory was gone from v6.1. Both halves of that were right about v6.1 and wrong about the record: the record is TITLED v5.1, the directory AS-4126GS-NBR-LCC_B200-SXM-180GBx8_TRT does not exist in inference_results_v6.1 (its results listing is 1-node-2S-GNR_128C, 1-node-2S-GNR_86C, 1-node-4S-GNR_86C, 1-node-4x-BMG-B70, 8xMI355X_2xEPYC-9575F, AS-8126GS-NB3RT_B300-SXM-288GB_TRT, SYS-822GS-NB3RT - and an authenticated code search for AS-4126GS in v6.1 returns 0 results), but it EXISTS in inference_results_v5.1, where the Supermicro results directory still lists AS-4126GS-NBR-LCC_B200-SXM-180GBx8_TRT among five B200 systems. The URL had simply been written against the v6.1 repo. Corrected URL returns HTTP 200, and the units proof was RE-READ FROM THAT LIVE FILE, not trusted from the old note: line 7 'Completed samples per second : 947.90', line 8 'Completed tokens per second: 122268.73', line 33 the same token figure in the summary block. 122269.0 / 947.90 = 129.0 tokens per sample, so the v5.1 summary_results.json unit label 'Queries/s' on this run is wrong by a factor of 129 and the raw log is the ground truth that shows it. This is now a first-party-verified live citation for the repo's cleanest unit-mislabelling proof. METHOD LESSON, recorded because it cost a false death: a repo-swap (v5.1 vs v6.1) is indistinguishable from a deleted file if you only check the ONE repo the URL names. Checking the sibling results repo - which is what the record's own title pointed at - turned a dead link into a live one. I am the third instrument artefact this repo has now caught in its own audit (after rent-cerebras and the five mlcommons repo roots); the pattern is always the same, a single probe of a single hypothesis.
Cross-references
This record references no other records.