MLPerf Inference: Datacenter benchmark suite description
Fields
sources/ev-mlperf-inference-datacenter-pageNotes
Official MLCommons page describing the Inference Datacenter benchmark. Key structural facts: (1) Four scenarios defined: Single stream, Multiple stream, Server, Offline - each with different latency constraints and throughput metrics. (2) Two divisions: Closed (same model as reference, 'apples-to-apples' hardware comparison) and Open (different model or retraining allowed, 'fosters innovation'). (3) Three availability categories: Available (purchasable/rentable), Preview (submittable next round), RDI (experimental/internal). (4) 'Published results are sometimes modified or invalidated for various reasons' - acknowledges results can change post-publication. (5) The page links to the rules document as 'the official source of truth' for benchmark definitions. This page confirms that cross-vendor comparison requires checking that submissions are in the same division AND same scenario.
Cross-references
This record references no other records.