inference/research
verifiedhigh · 0.85updated 2026-10-03

vLLM docs: NixlConnector Usage Guide

repo

Fields

id
sources/pd-vllm-nixl-connector-docs
Publisher
vLLM project
Kind
repo
Accessed
2026-10-03
notes
NixlConnector 'provides fully asynchronous send/receive operations using the NIXL library'. Transport: UCX is the primary default backend, configured with UCX-specific env vars (UCX_TLS, UCX_NET_DEVICES) and explicitly NOT NCCL variables ('NCCL environment variables (like NCCL_IB_HCA, NCCL_SOCKET_IFNAME) are not applicable to NixlConnector'). Alternative backends selectable via kv_connector_extra_config.backends, with a LIBFABRIC example. Operationally important: kv_connector_extra_config kv_lease_duration sets the prefiller's lease, and a high vllm:nixl_num_kv_expired_reqs metric means the lease is too short for the network or workload - i.e. the producer's buffer lifetime is a function of transfer bandwidth.

Notes

NixlConnector 'provides fully asynchronous send/receive operations using the NIXL library'. Transport: UCX is the primary default backend, configured with UCX-specific env vars (UCX_TLS, UCX_NET_DEVICES) and explicitly NOT NCCL variables ('NCCL environment variables (like NCCL_IB_HCA, NCCL_SOCKET_IFNAME) are not applicable to NixlConnector'). Alternative backends selectable via kv_connector_extra_config.backends, with a LIBFABRIC example. Operationally important: kv_connector_extra_config kv_lease_duration sets the prefiller's lease, and a high vllm:nixl_num_kv_expired_reqs metric means the lease is too short for the network or workload - i.e. the producer's buffer lifetime is a function of transfer bandwidth.

Cross-references

This record references no other records.