Contested paper audit: cross-index verification resolves venue disputes but leaves preprints and schema gaps contested
[minor] measurement
Fields
id
gotchas/w11-contested-paper-audit-resolutions-and-unresolved-gapsClass
measurement
Severity
minor
Concepts
papervenueprovenanceauditcontestedpmlrarxiviclrcrossref
Symptom
30 papers were marked status=contested after wave-9/10 venue and hardware sweeps. An audit across arXiv API metadata, PMLR, Crossref, and conference indexes resolved only two records to verified (deepspeed-moe at ICML 2022, whisper at ICML 2023), while 24 records remain contested because their cited preprint sources lack or contradict the published venue attribution, or their true venues are absent from the schema enum.
Root cause
The repo's provenance rule requires that status may change away from contested only when the finding names one venue that the paper's own sources agree on AND that venue is in schemas/paper.schema.json's venue enum. Most papers in data/papers/ cite arXiv preprint source records (pap-arch-*, pap-eff-*) whose abstracts and metadata describe preprints rather than conference proceedings, causing sources to disagree with conference index findings. Furthermore, venues like ACM FAccT, EACL, and demo tracks represent schema gaps absent from the venue enum.
Workaround
Record dated findings citing specific proceedings URLs and quotes in the paper's notes field while preserving both values. Resolve status to verified only where all declared sources and proceedings deposits agree (such as PMLR proceedings deposits matching the author comment). Report enum gaps (FAccT, EACL) for parent schema extension rather than forcing status changes.
notes
Audited 30 contested papers: 4 excluded as parent-fixed hardware citations (eagle-2, leviathan, s1, prm800k), 1 skipped as not-recorded in scratch scripts (speechgpt), 2 resolved to verified (deepspeed-moe, whisper), and 23 verified in notes but retained as contested due to source disagreements or schema enum gaps.
Notes
Audited 30 contested papers: 4 excluded as parent-fixed hardware citations (eagle-2, leviathan, s1, prm800k), 1 skipped as not-recorded in scratch scripts (speechgpt), 2 resolved to verified (deepspeed-moe, whisper), and 23 verified in notes but retained as contested due to source disagreements or schema enum gaps.
Cross-references
Affects (25)
- PapersBlock Diffusion: Interpolating Between Autoregressive and Diffusion Language Models (BD3-LM)contested
- PapersCareless Whisper: Speech-to-Text Hallucination Harmscontested
- PapersDeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- PapersDeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
- PapersDINOv2: Learning Robust Visual Features without Supervisioncontested
- PapersEAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- PapersEAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycontested
- PapersHigh Fidelity Neural Audio Compression (EnCodec)contested
- PapersLinear Transformers Are Secretly Fast Weight Programmerscontested
- PapersLarge Language Diffusion Models (LLaDA)contested
- PapersThe LLM Surgeoncontested
- PapersBreak the Sequential Dependency of LLM Inference Using Lookahead Decodingcontested
- PapersMagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decodingcontested
- PapersMedusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscontested
- PapersMegaBlocks: Efficient Sparse Training with Mixture-of-Expertscontested
- PapersFrom Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecontested
- PapersDetecting Pretraining Data from Large Language Modelscontested
- PapersMTEB: Massive Text Embedding Benchmarkcontested
- PapersNative Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
- PapersA Length-Extrapolatable Transformercontested
- PapersOLMoE: Open Mixture-of-Experts Language Modelscontested
- PapersSALMONN: Towards Generic Hearing Abilities for Large Language Modelscontested
- PapersStructured Denoising Diffusion Models in Discrete State-Spaces (SEDD)contested
- PapersTurning Whisper into a Real-Time Transcription System (Whisper-Streaming)contested
- PapersWhisper: Robust Speech Recognition via Large-Scale Weak Supervision