vLLM documentation - Engine arguments, LoadConfig (load-format and safetensors-load-strategy)

Fetched 2026-10-04 from /configuration/engine_args.html, section LoadConfig. --load-format default 'auto' ('auto' will try to load the weights in the safetensors format and fall back to the pytorch bin format if safetensors format is not available). Alternatives verbatim: 'instanttensor' (load safetensors on CUDA devices using InstantTensor, 'enables distributed loading with pipelined prefetching and fast direct I/O'); 'ipc_cache' ('map post-quantized weights from a local weight cache daemon via CUDA IPC for fast engine restarts. Launch the daemon with `vllm preload` first'); 'npcache' (numpy cache to speed up loading); 'dummy' (random values, mainly for profiling); 'tensorizer' (CoreWeave); 'runai_streamer' and 'runai_streamer_sharded'; 'sharded_state' ('efficient loading of tensor-parallel models'); 'mistral'; 'modelexpress'; 'Other custom values can be supported via plugins.' --safetensors-load-strategy choices eager / lazy / prefetch / torchao / None, with the doc's own guidance quoted verbatim and this is the load-vs-eager answer: 'None (default): Uses memory-mapped (lazy) loading. When an NFS filesystem is detected and the total checkpoint size fits within 90% of available RAM, prefetching is enabled automatically.'; '"lazy": Weights are memory-mapped from the file. This enables on-demand loading and is highly efficient for models on local storage. Unlike the default (None), auto-prefetch on NFS is not performed.'; '"eager": The entire file is read into CPU memory upfront before loading. This is recommended for models on network filesystems (e.g., Lustre, NFS) as it avoids inefficient random reads, significantly speeding up model initialization. However, it uses more CPU RAM.'; '"prefetch": Checkpoint files are read into the OS page cache before workers load them, speeding up the model loading phase. Useful on network or high-latency storage.'; '"torchao": ... Needs torchao >= 0.14.0.' Also --download-dir ('Directory to download and load the weights, default to the default cache directory of Hugging Face') and --safetensors-prefetch-num-threads. This is the configuration-choice record for lazy vs eager: the doc's own recommendation inverts by storage type, and lazy is explicitly only 'highly efficient for models on local storage'.