TensorRT 11.3 Optimization guide: builder tactic profiling, layer fusion, engine build time
Fields
sources/w7k-tensorrt-optimization-11-3Notes
The builder's autotuning mechanism, stated by NVIDIA: the builder 'profiles each layer's available tactics, picks the best one, and emits a serialized engine', and build time 'can range from seconds for small networks to tens of minutes (or longer) for large models with complex topology' - a cost that 'shows up wherever builds happen often, such as in CI pipelines'. Also documents the contention trap: concurrent streams share SMs, register files, L2 and DRAM bandwidth, so 'kernels TensorRT picked while profiling against a fully available GPU may no longer be optimal at runtime', and the mitigation is to profile against a deliberately reduced share of the GPU. Layer fusion and pointwise fusion are covered with their supported patterns. Cited for autotuning=true, the aot strategy, and the build-time figure; the seconds-to-tens-of-minutes range is NVIDIA's own and is recorded as a range, not a point.
Cross-references
This record references no other records.