inference/research
verifiedmedium · 0.70updated 2026-10-10

vLLM issue #38967: vLLM >= 0.18.0 segfaults in NCCL cuMemCreate with TP>1 on RTX 4090 (SM 89)

forum

Fields

id
sources/comm-github-vllm-38967-4090-tp-cumem-segfault
Publisher
vLLM GitHub issues
Kind
forum
Accessed
2026-10-04
notes
Reporter zhouliang5266, 2x RTX 4090 48GB (SM 89), vLLM 0.18.0, NCCL 2.27.5, openEuler 22.03 SP4. Only ONE independent reporter; the reporter later conceded he had not isolated the cause and suggested closing. Workarounds tried and reported as failing: NCCL_CUMEM=0 + NCCL_P2P_DISABLE=1, mp executor, larger --shm-size, VLLM_USE_V1=0 (unavailable, V0 removed in 0.18.0). SWAP TEST: same v0.18.0 + TP=2 works on two other identical machines, segfaults only on the third (cross-NUMA SYS topology) - so NOT a universal 4090 behaviour. Single report, unresolved. Verified 2026-10-10: Read issue '[Bug] vLLM >= 0.18.0 NCCL segfault (cuMemCreate) with TP>1 on RTX 4090 (SM 89)' (https://github.com/vllm-project/vllm/issues/38967); confirms bug report, environment details, workarounds tried, swap test results, and unresolved status.

Notes

Reporter zhouliang5266, 2x RTX 4090 48GB (SM 89), vLLM 0.18.0, NCCL 2.27.5, openEuler 22.03 SP4. Only ONE independent reporter; the reporter later conceded he had not isolated the cause and suggested closing. Workarounds tried and reported as failing: NCCL_CUMEM=0 + NCCL_P2P_DISABLE=1, mp executor, larger --shm-size, VLLM_USE_V1=0 (unavailable, V0 removed in 0.18.0). SWAP TEST: same v0.18.0 + TP=2 works on two other identical machines, segfaults only on the third (cross-NUMA SYS topology) - so NOT a universal 4090 behaviour. Single report, unresolved. Verified 2026-10-10: Read issue '[Bug] vLLM >= 0.18.0 NCCL segfault (cuMemCreate) with TP>1 on RTX 4090 (SM 89)' (https://github.com/vllm-project/vllm/issues/38967); confirms bug report, environment details, workarounds tried, swap test results, and unresolved status.

Cross-references

This record references no other records.