inference/research
verifiedhigh · 0.95majorupdated 2026-10-03

FP8 on RDNA3 blows the 600 s engine-ready timeout on first start even though weights load in seconds

[major] config

Fields

id
gotchas/vllm-rocm-fp8-cold-start-timeout
Class
config
Severity
major
Concepts
rocm
Symptom
vLLM serve of an FP8 checkpoint on a gfx1100 Radeon aborts during startup with an engine-ready timeout after waiting VLLM_ENGINE_READY_TIMEOUT_S (default 600 s); measured first init was 1474.26 s. The message blames 'slow weight loading' while the weights actually loaded in 16.33 s, and the GPU sits idle.
Root cause
The wheel ships zero tuned W8A8 block-FP8 configs for device_name=AMD_Radeon_Graphics (218 device-keyed configs total: 76 AMD Instinct, 142 NVIDIA). With no config hit, warmup JIT-compiles the Triton GEMM kernels shape by shape on the CPU: W8A8 tuning alone searches 5 shapes for ~23.5 min, and first inference triggers a further ~43 min of Triton JIT staging that /health does not reflect.
Workaround
Set VLLM_ENGINE_READY_TIMEOUT_S=1800 for the first cold start. Once ~/.triton/cache is warm, engine init drops to ~49 s and later starts take ~2 min; the cost is one-time per host until that cache is cleared. Optionally drop tuned configs under both RDNA3 device-name strings (AMD_Radeon_RX_7900_XTX and AMD_Radeon_Graphics) as the issue reporter did.
notes
FP8 does work on gfx1100 - the blocker is cold-start cost, not an architecture refusal, despite community trackers listing FP8 as 'does not run on RDNA3'. Note the same thread reports Triton on RDNA3 hanging indefinitely at num_stages=5 and with BLOCK_SIZE_M=128/BLOCK_SIZE_N=256/num_warps=8.

Notes

FP8 does work on gfx1100 - the blocker is cold-start cost, not an architecture refusal, despite community trackers listing FP8 as 'does not run on RDNA3'. Note the same thread reports Triton on RDNA3 hanging indefinitely at num_stages=5 and with BLOCK_SIZE_M=128/BLOCK_SIZE_N=256/num_warps=8.

Cross-references