FlashInfer issue #4110: '[Compilation] GDN decode cold kernel compilation time is too long on SM90/100/120/121' - 30-50 min per test on every arch
forum
Fields
id
sources/ev-flashinfer-issue-4110-gdn-cold-compilePublisher
FlashInfer
Kind
forum
Accessed
2026-10-04
notes
Filed 2026-07-23, OPEN as of 2026-10-04. Short and quantitative: 'tests/gdn/test_decode_delta_rule.py runs very long (30-50 mins on every arch) and has been straining our CI'; 'Most time is spent on cold kernel compilation. On a B200 I locally see ~800 cases that collapses to ~200 cute.compile calls based on the cache key logic'; 'For some kernels, B/tile_v/T/cache flags/pool-strides each key a new compile.' That is the structural statement of WHEN a JIT build step must be repeated: any change to batch size, tile_v, dtype/T, cache flag or pool stride produces a new kernel specialization and a fresh compile. Read with [[ev-flashinfer-issue-4317-cute-dsl-no-cache]] for the measured per-kernel seconds.
Notes
Filed 2026-07-23, OPEN as of 2026-10-04. Short and quantitative: 'tests/gdn/test_decode_delta_rule.py runs very long (30-50 mins on every arch) and has been straining our CI'; 'Most time is spent on cold kernel compilation. On a B200 I locally see ~800 cases that collapses to ~200 cute.compile calls based on the cache key logic'; 'For some kernels, B/tile_v/T/cache flags/pool-strides each key a new compile.' That is the structural statement of WHEN a JIT build step must be repeated: any change to batch size, tile_v, dtype/T, cache flag or pool stride produces a new kernel specialization and a fresh compile. Read with [[ev-flashinfer-issue-4317-cute-dsl-no-cache]] for the measured per-kernel seconds.
Cross-references
This record references no other records.