Serverless GPU & LLM Inference Benchmark - container cold start comparison (Sep 2026)

Updated September 2026, carries its own disclosure: 'Disclosure: some links on this page are affiliate links (marked as such). If you sign up through them we may earn a commission, at no extra cost to you. This never influences our testing methodology or rankings.' CONTAINS A KNOWN WRONG NUMBER, recorded here so it is not reused: it lists 'Replicate A100 (80GB) median cold start 6.5 s', which contradicts Replicate's own per-second pricing page already in this corpus (ppl-cloud-replicate-pricing, accessed 2026-10-04) - per-second billing of a private model implies cold start is billed, and 6.5 s for a 70B-class weights pull from remote storage is not credible on its own terms. Their other figures: Modal A100 40GB ~1.8 s ('Modal uses custom containerization primitives allowing image memory snapshots to restore in under 2 seconds'), RunPod A100 80GB ~4.2 s, Together AI 'Instant (Pooled)', Lambda Labs 'On-Demand VM, no scale-to-zero (billed per hour)'. Their own summary of the shape of the problem, useful: the ~1.8 s figures are IMAGE-RESTORE times, not model-serving cold starts - no model weight load is included. Use this source for the shape (serverless cold start is dominated by image restore plus weight load, not by container start) not for the numbers. 2026-10-11: Re-read page. Confirms affiliate comparison table of container restore times. Retained at confidence 0.6 (contested) due to known flawed cold-start numbers.