inference/research
verifiedhigh · 0.92majorupdated 2026-10-10

Sixteen of the eighteen venue='other' papers name a real venue with a publisher deposit - the gap was a missing enum member, not missing information

[major] toolchain

Fields

id
gotchas/w10-b-venue-other-is-a-provenance-claim-not-a-vocabulary-claim
Class
toolchain
Severity
major
Concepts
venueschema-gapprovenancecrossrefpmlropenreviewenumjmlrppoppsigircikmenlspsc22
Symptom
Auditing all 18 papers with venue='other' in this repo, on the expectation that most were genuinely unverifiable, produced the opposite result: every one of them names a real venue, and 12 of the 15 in-scope records have a PUBLISHER deposit in hand that was already sitting in data/sources/. Three had been written off as unverifiable when the deposit existed all along. marlin was recorded as PPoPP on Semantic Scholar alone, with the note flagging that ACM DL returns 403 and the attribution 'was NOT independently confirmed' - Crossref has the deposit: DOI 10.1145/3710848.3710871, 'Proceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming', event "PPoPP '25", pages 239-251, issued 2025-02-28. plaid was recorded as CIKM 2022 with the note 'crossref did not surface a proceedings entry, so other is used rather than guessing at kdd' - Crossref has it: DOI 10.1145/3511808.3557325, CIKM '22, pages 1747-1756. carbon-intensity was recorded as FAccT on 'the arXiv COMMENT FIELD alone' - Crossref has it: DOI 10.1145/3531146.3533234, ACM FAccT '22, pages 1877-1894. mx-ftptq was ENLSP 'from Semantic Scholar' - PMLR v262 has it: sharify24a, 'Proceedings of The 4th NeurIPS Efficient Natural Language and Speech Processing Workshop', PMLR 262:232-240.
Root cause
Two separate causes, and they produce the same 'other'. The first is a real information deficit, which is genuinely rare: 3 of 18 (careless-whisper, mteb, whisper-streaming) are status='contested' and belong to a sibling crawler. The second - the dominant one - is that a venue search which does not find a deposit in ONE index is being read as a fact about the PAPER rather than a fact about the INDEX. This is the exact failure w6p-acl-or-crossref-index-misses-venue already recorded for MoE-Lightning, where four ML-conference indexes were all negative and the paper was in fact at ASPLOS. The same pattern recurs here with a new mechanism: the query was field-scoped. ColBERT's Crossref deposit is 10.1145/3397271.3401075 but the record carries 10.1145/3397271.3401555, a note-level DOI typo; anyone querying by the quoted DOI gets a 404 and concludes no deposit exists, when the deposit is sitting under a DOI one four-digit suffix away (see w10-a-source-doi-resolves-to-a-different-paper). The perverse result is that this repo distrusts aggregator venue strings and then treats a single missing aggregator result as more authoritative than the publisher's own deposit.
Workaround
Before recording venue='other' for a paper with a named venue, run a title-first bibliographic query against Crossref (query.bibliographic=<record name>, filter type=proceedings-article or journal-article) and read the container-title off the top hit; that single query covers ACM, IEEE, USENIX and ACL deposits and is what surfaced MIS, PPoPP, CIKM and FAccT here. For PMLR-hosted papers the index is proceedings.mlr.press, not Crossref - ENLSP/MARLIN-class workshops are PMLR volumes and will not appear in a Crossref title search at all. For TMLR papers the first-party index is OpenReview (api2.openreview.net/notes/search), where content.venue='Accepted by TMLR' with venueid='TMLR' is the authoritative signal. Record 'other' only after all three are negative, and when you do, say which three you checked, because a bare 'other' is indistinguishable from an unexamined one. This repo already has the rule in its own schema notes - the 2026-10-05 survey entry says venue='other' 'still holds 18 papers spanning at least nine DISTINCT venues' and that the problem is provenance, not vocabulary; this wave measures 18 records across 12 venues and confirms it.
notes
Severity major because every unclassified record is a paper a reader filtering on venue cannot find, and because 3 of the 15 in-scope records carried notes asserting a verification status that was simply false - the workaround had been done, badly, and then recorded as done. FULL CENSUS of the 18, measured 2026-10-10 and classified by whether the real venue exists in schemas/paper.schema.json's enum (29 members + 'other'): RE-CLASSIFIABLE TO AN ENUM MEMBER: 0. There are NONE - every one of the 18 real venues is absent from the enum, so no venue field could be changed without inventing a value that would fail validation. GENUINELY AMBIGUOUS: 3 (careless-whisper, mteb, whisper-streaming), all status='contested', all correctly left alone - careless-whisper is a real four-index conflict (Crossref says ACM FAccT 2024 via 10.1145/3630106.3658996, arXiv v2 says IEEE ICASSP 2025) and all three are outside this crawler's write scope by instruction. SCHEMA GAPS: 15, which split cleanly into two kinds that need a policy decision before any member is added. (A) CONFERENCES missing from the enum: sigir (colbert SIGIR '20, DOI 10.1145/3397271.3401075 pp.39-48; splade SIGIR '21, DOI 10.1145/3404835.3463098 pp.2288-2292), cikm (plaid, DOI 10.1145/3511808.3557325 pp.1747-1756), ppopp (marlin, DOI 10.1145/3710848.3710871 pp.239-251), sc22/HPC (deepspeed-inference, DOI 10.1109/SC41404.2022.00051 pp.1-15), facct (carbon-intensity, DOI 10.1145/3531146.3533234 pp.1877-1894), enlsp (mx-ftptq, PMLR v262 sharify24a pp.232-240). (B) JOURNALS missing from the enum: tmlr (frugalgpt, OpenReview forum cSimKw5p6R venueid=TMLR; contriever, forum jKN1pXi7b0 venueid=TMLR; nomic-embed, forum IPmzyQSiQE venueid=TMLR), taslp (hubert, DOI 10.1109/TASLP.2021.3122291 pp.3451-3460; soundstream, DOI 10.1109/TASLP.2021.3129994 pp.495-507), jmlr (switch-transformers, JMLR vol 23 2022, papers 21-0998, pp.1-39 on jmlr.org), acm-tos (mooncake-kimi, DOI 10.1145/3773772), nature (paper-deepseek-r1, arXiv journal_ref 'Nature volume 645, pages 633-638 (2025)' independently confirmed by Crossref DOI 10.1038/s41586-025-09422-z). The journal half is a DIFFERENT kind of schema change from the conference half: every existing enum member is a conference, so adding journals is a decision about whether the field encodes 'venue' or 'publication venue', and it should not be made by a crawler scoped to data edits. That is why this crawler classified rather than edited, and why it reclassified NOTHING. Verified independently by this crawler rather than trusted from the notes: Crossref by exact-title bibliographic query for colbert, splade, plaid, marlin (exact-DOI lookup), carbon-intensity, deepspeed, hubert, soundstream, mooncake, deepseek-r1; PMLR v262 volume page for mx-ftptq; OpenReview notes/search for frugalgpt, nomic-embed, contriever; jmlr.org/papers/v23 for switch-transformers. No schema file was edited and no venue field was changed. RESOLVED 2026-10-10 (crawler/w11-venue-enum): Added the 11 missing venues to schemas/paper.schema.json venue enum (conferences: sigir, cikm, ppopp, sc22, facct, enlsp; journals: tmlr, taslp, jmlr, acm-tos, nature), added optional venue_kind enum (conference|journal), and reclassified the 12 records whose notes and data/sources/ deposit records agreed (colbert, splade -> sigir; plaid -> cikm; marlin -> ppopp; deepspeed-inference -> sc22; frugalgpt, contriever, nomic-embed -> tmlr; hubert, soundstream -> taslp; switch-transformers -> jmlr; mooncake-kimi -> acm-tos). The remaining 3 papers (carbon-intensity-ai-cloud-instances, mx-ftptq, paper-deepseek-r1) were retained as venue='other' with an explanatory note because their data/sources/ records do not contain a publisher deposit confirming the candidate venue. RESOLVED 2026-10-10 (crawler/w12-venue-rest): Verified publisher deposits for the remaining 3 papers (carbon-intensity-ai-cloud-instances -> facct via Crossref DOI 10.1145/3531146.3533234; mx-ftptq -> enlsp via PMLR v262 sharify24a; paper-deepseek-r1 -> nature via Crossref DOI 10.1038/s41586-025-09422-z / Nature article page). Appended confirming sentences to their cited data/sources/ records and reclassified all 3 from venue='other' to their respective venues with venue_kind ('conference' for facct and enlsp, 'journal' for nature). All 15 in-scope papers are now reclassified, leaving only the 3 contested papers at venue='other'.

Notes

Severity major because every unclassified record is a paper a reader filtering on venue cannot find, and because 3 of the 15 in-scope records carried notes asserting a verification status that was simply false - the workaround had been done, badly, and then recorded as done. FULL CENSUS of the 18, measured 2026-10-10 and classified by whether the real venue exists in schemas/paper.schema.json's enum (29 members + 'other'): RE-CLASSIFIABLE TO AN ENUM MEMBER: 0. There are NONE - every one of the 18 real venues is absent from the enum, so no venue field could be changed without inventing a value that would fail validation. GENUINELY AMBIGUOUS: 3 (careless-whisper, mteb, whisper-streaming), all status='contested', all correctly left alone - careless-whisper is a real four-index conflict (Crossref says ACM FAccT 2024 via 10.1145/3630106.3658996, arXiv v2 says IEEE ICASSP 2025) and all three are outside this crawler's write scope by instruction. SCHEMA GAPS: 15, which split cleanly into two kinds that need a policy decision before any member is added. (A) CONFERENCES missing from the enum: sigir (colbert SIGIR '20, DOI 10.1145/3397271.3401075 pp.39-48; splade SIGIR '21, DOI 10.1145/3404835.3463098 pp.2288-2292), cikm (plaid, DOI 10.1145/3511808.3557325 pp.1747-1756), ppopp (marlin, DOI 10.1145/3710848.3710871 pp.239-251), sc22/HPC (deepspeed-inference, DOI 10.1109/SC41404.2022.00051 pp.1-15), facct (carbon-intensity, DOI 10.1145/3531146.3533234 pp.1877-1894), enlsp (mx-ftptq, PMLR v262 sharify24a pp.232-240). (B) JOURNALS missing from the enum: tmlr (frugalgpt, OpenReview forum cSimKw5p6R venueid=TMLR; contriever, forum jKN1pXi7b0 venueid=TMLR; nomic-embed, forum IPmzyQSiQE venueid=TMLR), taslp (hubert, DOI 10.1109/TASLP.2021.3122291 pp.3451-3460; soundstream, DOI 10.1109/TASLP.2021.3129994 pp.495-507), jmlr (switch-transformers, JMLR vol 23 2022, papers 21-0998, pp.1-39 on jmlr.org), acm-tos (mooncake-kimi, DOI 10.1145/3773772), nature (paper-deepseek-r1, arXiv journal_ref 'Nature volume 645, pages 633-638 (2025)' independently confirmed by Crossref DOI 10.1038/s41586-025-09422-z). The journal half is a DIFFERENT kind of schema change from the conference half: every existing enum member is a conference, so adding journals is a decision about whether the field encodes 'venue' or 'publication venue', and it should not be made by a crawler scoped to data edits. That is why this crawler classified rather than edited, and why it reclassified NOTHING. Verified independently by this crawler rather than trusted from the notes: Crossref by exact-title bibliographic query for colbert, splade, plaid, marlin (exact-DOI lookup), carbon-intensity, deepspeed, hubert, soundstream, mooncake, deepseek-r1; PMLR v262 volume page for mx-ftptq; OpenReview notes/search for frugalgpt, nomic-embed, contriever; jmlr.org/papers/v23 for switch-transformers. No schema file was edited and no venue field was changed.

RESOLVED 2026-10-10 (crawler/w11-venue-enum): Added the 11 missing venues to schemas/paper.schema.json venue enum (conferences: sigir, cikm, ppopp, sc22, facct, enlsp; journals: tmlr, taslp, jmlr, acm-tos, nature), added optional venue_kind enum (conference|journal), and reclassified the 12 records whose notes and data/sources/ deposit records agreed (colbert, splade -> sigir; plaid -> cikm; marlin -> ppopp; deepspeed-inference -> sc22; frugalgpt, contriever, nomic-embed -> tmlr; hubert, soundstream -> taslp; switch-transformers -> jmlr; mooncake-kimi -> acm-tos). The remaining 3 papers (carbon-intensity-ai-cloud-instances, mx-ftptq, paper-deepseek-r1) were retained as venue='other' with an explanatory note because their data/sources/ records do not contain a publisher deposit confirming the candidate venue.

RESOLVED 2026-10-10 (crawler/w12-venue-rest): Verified publisher deposits for the remaining 3 papers (carbon-intensity-ai-cloud-instances -> facct via Crossref DOI 10.1145/3531146.3533234; mx-ftptq -> enlsp via PMLR v262 sharify24a; paper-deepseek-r1 -> nature via Crossref DOI 10.1038/s41586-025-09422-z / Nature article page). Appended confirming sentences to their cited data/sources/ records and reclassified all 3 from venue='other' to their respective venues with venue_kind ('conference' for facct and enlsp, 'journal' for nature). All 15 in-scope papers are now reclassified, leaving only the 3 contested papers at venue='other'.

Cross-references