Sharding model weights across cards instead of keeping them host-resident more than doubles both prefill and decode on 4x MI50, at ~7 GB extra VRAM per card

llama-bench on 4x MI50 32GB, ROCm 7.2.1, -sm tensor -ngl 99 -nr 1, -p 512 and -p 4096, -n 64 and -n 512, -b 512 -ub 512 -r 2, -lm dio, prompt cache off. Env: GGML_ENABLE_CUSTOM_AR=1 HSA_FORCE_FINE_GRAIN_PCIE=1 GPU_MAX_HW_QUEUES=8 LLAMA_ENABLE_MTP_OPT=1 HIP_VISIBLE_DEVICES=0,1,2,3 LLAMA_PLE_SHARD=0 vs =1. Paired results, same session: pp4096 227.5 +/- 13.1 host-resident -> 501.2 +/- 1.7 sharded (+120%); tg512 23.5 +/- 0.3 -> 37.1 +/- 0.9 (+58%); tg64 20.4 +/- 2.5 -> 31.8 +/- 1.8 (+56%). VRAM per card ~21.7 GB -> 28.6 GB (+6.9 GB). Perplexity 2.3108 +/- 0.059 -> 2.3052 +/- 0.059.

COMMUNITY-REPORTED MEASUREMENT with a rare and valuable property: the reporter included a PERPLEXITY CHECK alongside the throughput, so this is not just a speed win that quietly broke the model - accuracy is unchanged within the reported error bars. That makes this the best-evidenced community performance claim in this slice. The mechanism is pipeline-parallel-expert (PLE) shard mode, an out-of-tree feature of the gfx906 llama.cpp fork, so it is NOT available in stock llama.cpp on any hardware. Clock and power state not stated. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-radeon-instinct-mi50x). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface. FORMAT_ID BACKFILL 2026-10-08 (w10-bench-formatid): format_id was null and is now 'gguf-q4-k-m'. EVIDENCE, quoted from this record's own text: model: "Qwen3.8-Flash-Next UD-Q4_K_XL". NO NUMBER CHANGED - value, metric and unit are untouched; this adds metadata, it does not re-measure. The value was NOT taken from a model name, from a sibling record, or from outside knowledge. Left null everywhere the record's own unit/methodology does not name a format for this run: 63 of the 150 records in data/benchmarks/ are still null and each one has its reason, because a null with a reason is correct and a guessed value is not. PROXY DECLARED: the measured load is Q4_K_XL, a Q4_K-family mixture; the repo has no gguf-q4-k-xl record, so gguf-q4-k-m is the existing family id - read it as Q4_K-family, not as the Q4_K_M mixture specifically