inference/research
verifiedhigh · 0.90updated 2026-10-05

Audio input token count explodes relative to text output

memory bound

Fields

id
flops/aud-flop-audio-token-explosion
Class
custom_kernel
Bound by
memory
Scales with
audio_durationsample_ratemel_binsencoder_layersencoder_hidden_size
Hardware parameters
parametermemory_bandwidtheffect2026-10-04: affected_by_hardware previously held ['gpu-vram', 'memory-bandwidth'] - hardware quantities, not products.basisunverified
Workarounds
Use 30-second windowing (Whisper's max_source_positions=1500 at 50Hz mel frame rate)Use VAD to skip non-speech segmentsUse streaming ASR for long audio (with quality tradeoff)
notes
HARDWARE PARAMETERS, NOT RECORD IDS. 2026-10-04: affected_by_hardware previously held ['gpu-vram', 'memory-bandwidth'] - hardware quantities, not products. They could never resolve, so the repo was carrying permanent false debt against a budget that exists to catch NEW bad references. Moved into this note; the field is now EMPTY and that is the correct value, not a gap. Why: These were hardware PARAMETERS ('gpu-vram', 'memory-bandwidth'), not products. But the record is ALSO part-independent - it is derived from Whisper model configs across five model sizes, so naming any GPU would INVENT a hardware dependency it does not claim. HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.

Notes

HARDWARE PARAMETERS, NOT RECORD IDS. 2026-10-04: affected_by_hardware previously held ['gpu-vram', 'memory-bandwidth'] - hardware quantities, not products. They could never resolve, so the repo was carrying permanent false debt against a budget that exists to catch NEW bad references. Moved into this note; the field is now EMPTY and that is the correct value, not a gap. Why: These were hardware PARAMETERS ('gpu-vram', 'memory-bandwidth'), not products. But the record is ALSO part-independent - it is derived from Whisper model configs across five model sizes, so naming any GPU would INVENT a hardware dependency it does not claim.

HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.

Cross-references

This record references no other records — it is a leaf. It is cited by 0.