llama.cpp under ROCm on 2x RX 7900 XTX (gfx1100): Qwen3.8-27B Q4_K_XL batch-1 tensor-split decode throughput

PER-REQUEST decode throughput, batch 1, single stream - explicitly framed as the llama-server serving case for one user at a time on this hardware class. Tensor-split across 2x RX 7900 XTX under ROCm on gfx1100. Model Qwen3.8-27B at Q4_K_XL, 17.545 GB, which streams once per token at batch 1. Accompanying measured arithmetic, which is the reason this is worth a record: the pair moves 825.7 GB/s = 43.0% of their combined 1920 GB/s DRAM peak, leaving 57% of aggregate bandwidth idle. Single-card baseline is 632.5 of 960 GB/s = 65.9% UMC on the faster card, 623.5 on the other. So adding a second card delivers only 1.31x throughput despite neither card being bandwidth-saturated. The issue decomposes this into a 1.518x solo-inefficiency factor and a 1.532x tensor-split penalty, 2.326x combined. OMITTED BY SOURCE: exact llama.cpp commit and ROCm version, prompt length, generation length, KV-cache size, thread count, which ROCm backend path, and clock state. Projections in the issue (72.1 tok/s at per-card-equal efficiency, 109.4 tok/s at 100% UMC) are explicitly labelled projections from measured bandwidths and are NOT recorded as measurements.

Valuable precisely because it quantifies a failure mode that generic tok/s figures hide: consumer-RDNA multi-card tensor parallelism underperforms badly on consumer cards, where the only link between GPUs is PCIe rather than a fabric. 1.31x for 2x the hardware is a poor result, and the per-card DRAM utilisation drops from 65.9% to 43.0% when the split is introduced - the split actively makes each card less efficient. Cross-checked against a second model in the same issue: a 7B at Q8_0 keeps 63.9% of its solo per-card bandwidth when split, so the penalty is a roughly constant fraction that multiplies whatever the batch-1 kernel already achieves. PRACTICAL TAKEAWAY: on RDNA, do the arithmetic before assuming a second card buys you a second card's worth of tokens per second. ACCELERATOR ID CAVEAT: no accelerators/amd-radeon-rx-7900-xtx record exists in this repo; planned id, validator will warn. measured_by third_party and status draft: community research issue, no build or version pinned. The underlying measurement (47.06 t/s, 825.7 GB/s, 65.9%/43.0%) is concrete and cross-checked within the issue; the derived percentages are the author's own arithmetic on those measurements. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-radeon-rx-7900-xtx). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface. FORMAT_ID BACKFILL 2026-10-08 (w10-bench-formatid): format_id was null and is now 'gguf-q4-k-m'. EVIDENCE, quoted from this record's own text: methodology: "Model Qwen3.8-27B at Q4_K_XL, 17.545 GB, which streams once per token at batch 1". NO NUMBER CHANGED - value, metric and unit are untouched; this adds metadata, it does not re-measure. The value was NOT taken from a model name, from a sibling record, or from outside knowledge. Left null everywhere the record's own unit/methodology does not name a format for this run: 63 of the 150 records in data/benchmarks/ are still null and each one has its reason, because a null with a reason is correct and a guessed value is not. PROXY DECLARED: the measured load is Q4_K_XL, a Q4_K-family mixture; the repo has no gguf-q4-k-xl record, so gguf-q4-k-m is the existing family id - read it as Q4_K-family, not as the Q4_K_M mixture specifically