llama.cpp issue #21006: PR #20908 broke multi-host rpc-server tensor-split balancing - the client loads the model only on the RPC workers and nothing on the local machine
forum
Fields
id
sources/comm-github-llamacpp-21006-rpc-tensor-split-regressionPublisher
llama.cpp GitHub issues
Kind
forum
Accessed
2026-10-04
notes
Reporter stew675, build b8487, llama.cpp releases b8492 onward to filing date, Linux Fedora 43, AMD Ryzen AI Max+ 395 (so AMD hardware - cited here only for the RPC mechanism, not for NVIDIA performance). Qwen3.5-397B-A17B IQ4_NL, --rpc to two machines, --tensor-split 1,1,1. Symptom: 'Remote RPC server crashed or returned malformed response' even though the workers did not crash. 11 comments.
Notes
Reporter stew675, build b8487, llama.cpp releases b8492 onward to filing date, Linux Fedora 43, AMD Ryzen AI Max+ 395 (so AMD hardware - cited here only for the RPC mechanism, not for NVIDIA performance). Qwen3.5-397B-A17B IQ4_NL, --rpc to two machines, --tensor-split 1,1,1. Symptom: 'Remote RPC server crashed or returned malformed response' even though the workers did not crash. 11 comments.
Cross-references
This record references no other records.