inference/research
verifiedmedium · 0.75updated 2026-10-04

llama.cpp issue #21006: PR #20908 broke multi-host rpc-server tensor-split balancing - the client loads the model only on the RPC workers and nothing on the local machine

forum

Fields

id
sources/comm-github-llamacpp-21006-rpc-tensor-split-regression
Publisher
llama.cpp GitHub issues
Kind
forum
Accessed
2026-10-04
notes
Reporter stew675, build b8487, llama.cpp releases b8492 onward to filing date, Linux Fedora 43, AMD Ryzen AI Max+ 395 (so AMD hardware - cited here only for the RPC mechanism, not for NVIDIA performance). Qwen3.5-397B-A17B IQ4_NL, --rpc to two machines, --tensor-split 1,1,1. Symptom: 'Remote RPC server crashed or returned malformed response' even though the workers did not crash. 11 comments.

Notes

Reporter stew675, build b8487, llama.cpp releases b8492 onward to filing date, Linux Fedora 43, AMD Ryzen AI Max+ 395 (so AMD hardware - cited here only for the RPC mechanism, not for NVIDIA performance). Qwen3.5-397B-A17B IQ4_NL, --rpc to two machines, --tensor-split 1,1,1. Symptom: 'Remote RPC server crashed or returned malformed response' even though the workers did not crash. 11 comments.

Cross-references

This record references no other records.