Turning Whisper into a Real-Time Transcription System (Whisper-Streaming)

Whisper is a non-streaming 30s-window model, so naive streaming to it either blocks for 30s or transcribes unstable prefixes.

Local agreement policy: run the Whisper decoder on every incoming buffer and emit only the longest prefix on which the current hypothesis and the previous hypothesis agree, with a self-adaptive latency that delays the rolling buffer until agreement stabilises. Buffers are re-encoded on every step, so the encoder runs repeatedly per utterance rather than once.

Reports 3.3 s latency on an unsegmented long-form speech test set, and demonstrates production use as a live transcription component at a multilingual conference. The number is a latency figure on the authors' own test set with no GPU specified, so treat it as a demo-scale result.

Published as an IJCNLP-AACL 2023 system demonstration (arXiv comment). The mechanism matters for anyone serving Whisper: local agreement re-runs the encoder-decoder over the buffer every step, so the encoder cost scales with (steps x buffer length) rather than with audio length. That is the price of retrofitting streaming onto a fixed-window model, and it is why production streaming ASR uses Conformer/Zipformer transducers instead. Recorded here as the in-repo representative of the 'Whisper VAD / segmentation / chunking' family. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass): venue_track set to 'workshop' from the ACL Anthology bib index (anthology.bib, 131650 entries) by normalized title match. Evidence: anthology id 2023.ijcnlp-demo.3, booktitle 'Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations'. The Anthology id is the authoritative signal here and it is unambiguous, so this is an additive clarification of a track, not a change of venue - `venue` is untouched and no status change is implied. The reason this matters for inference readers: a Findings paper and a main-track paper at the same venue are not the same kind of evidence, and until this field existed the repo could not say which was which. VENUE CORRECTION + VENUE_TRACK 2026-10-04 (paper-fields pass, agent w3p): venue 'other' -> 'other', venue_track 'workshop'. EVIDENCE: the ACL Anthology bib index (anthology.bib, 131650 entries, fetched 2026-10-04) contains this exact title as anthology id 2023.ijcnlp-demo.3 with booktitle 'Proceedings of the 13th International Joint Conference on NLP and the 3rd Conference of the Asia-Pacific Chapter of the ACL: System Demonstrations'. That is a proceedings record, so 'no venue found' was an under-claim, not a fact. BOTH VALUES RETAINED HERE PER THE REPO CONTRACT: the prior value was 'other'. status set to contested because this changes a peer-reviewed attribution. arxiv_id unchanged (2307.14743). 2026-10-10: Crossref bibliographic query (crbatch.py: https://api.crossref.org/works?query.bibliographic=Turning%20Whisper%20into%20a%20Real-Time%20Transcription%20System) confirms container "Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (System Demonstrations)" (IJCNLP-AACL 2023 demo). Venue is IJCNLP-AACL 2023 demo (schema enum gap: demo track missing from schemas/paper.schema.json). Venue remains other; status remains contested.