Getting Started with CUDA Graphs · inference-research

Fields

id

sources/flop-nvidia-cuda-graphs

URL

https://developer.nvidia.com/blog/cuda-graphs/

Publisher

NVIDIA Technical Blog (Alan Gray)

Kind

blog

Published

2019-09-05

Accessed

2026-10-03

notes

Documents CUDA Graphs: capture a launch sequence once and replay it, removing per-kernel launch overhead. This is the mechanism engines reach for at batch 1, where per-op launch latency is comparable to the work itself. Also the reason batching engines want static shapes.

Notes

Documents CUDA Graphs: capture a launch sequence once and replay it, removing per-kernel launch overhead. This is the mechanism engines reach for at batch 1, where per-op launch latency is comparable to the work itself. Also the reason batching engines want static shapes.

Cross-references

This record references no other records.