REPORTED (unverified, Reddit): 16x MI50 32GB serving DeepSeek-V3.2 AWQ 4-bit at 10 tok/s decode and 2000 tok/s prefill at 69k context

AS REPORTED, NOT DOCUMENTED: 'Deepseek 3.2 AWQ 4bit @ 10 tok/s (output) // 2000 tok/s (input of 23k tok) on vllm-gfx906-deepseek with 69000 context length.' Power draw 550 W idle / 2400 W peak inference. 16 x MI50 32GB. No batch size, no TP/PP split, no concurrency, no quant group size, no ROCm or vLLM version, no clock state, no prompt/output length beyond '23k tok' input. Setup guide referenced at github.com/ai-infos/guidances-setup-16-mi50-deepseek-v32.

COMMUNITY-REPORTED AND NOT REPRODUCIBLE - recorded for the shape of the result, not the digits. The author states outright: 'it might be a good alternative to CPU hardwares as RAM price increases and the prompt processing speed will be much better with 16 TB/s bandwidth + tensor parallelism', and describes himself as 'a random guy with average software dev background using LLMs to make it run'. What IS worth taking from this: it demonstrates that a ~700B-class MoE served in AWQ 4-bit is achievable on 512 GB of used-market HBM2, and the idle-versus-peak power spread (550 W to 2400 W) is a useful reminder that a 16-card node is not idle-draw. Do NOT compare the 10 tok/s against any measured vLLM figure in this repo - the methodology fields that would make it comparable (parallelism configuration, concurrency, context depth, precision policy) are all absent, and the same reporter's own Discord posts on the same system describe wildly different figures depending on precision settings, including that fp32 compute was added specifically to avoid garbage output. Related Discord figures from the same author on related setups, all equally unmethodologised: 4.3 tok/s at TP2/PP16, and 'around 14 tok/s TG 135 tok/s PP (16k tok prompt / 140k ctx max on 16 MI50s)' for a different model and quant. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-radeon-instinct-mi50x). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface. CONVENTION AUDIT 2026-10-05 (agent w6-benchmarks): VALUE UNCHANGED. The record states no batch size, no TP/PP split and no concurrency - its own methodology says so - and therefore it does not state whether 10.0 tok/s is per-request or aggregate either. It is filed metric=decode_tok_s, which reads per-request, but the source text is a Discord-style line ("Deepseek 3.2 AWQ 4bit @ 10 tok/s (output)") with no qualification, and the author describes himself as "a random guy with average software dev background". This is the corpus's clearest instance of the trap in a record that is not honest enough to resolve it: at 10 tok/s aggregate over 16 MI50s the per-request figure would be a fraction of a token per second, which is implausible, so per-request is almost certainly right - "almost certainly" is not a basis for a capacity plan. FLAGGED, NOT REPAIRED: inventing "per-request" would be filling in an absent condition, which the repo's own rule forbids because a plausible value is worse than an absent one. The existing status draft and the notes' "Do NOT compare the 10 tok/s against any measured vLLM figure in this repo" are already the correct treatment; this entry records that the sweep reached the same verdict independently and adds the roofline arithmetic, which is worth recording because it is the one thing derivable here: DeepSeek-V3.2 MoE is 685.4e9 total / 37e9 active, so at AWQ 4-bit the active stream is 685.4e9/2 x (37/685.4) = 18.5 GB read per token, and 16 MI50s at 1024 GB/s give a roof of 16 x 1024e9 / 18.5e9 = 885.6 tok/s. At 10 tok/s this run sits at 1.1% of roof, which is the finding, not a roofline violation: a ~700B MoE on 16 PCIe cards with no GPU-to-GPU fabric is bandwidth- and allreduce-bound at a small fraction of its ceiling, and the author's own note that the same setup reports "around 14 tok/s" on a related configuration is a 1.4x methodological spread worth respecting. NOTE THE MoE CORRECTION IS LOAD-BEARING HERE: dividing by total weights instead of active gives a 47.8 tok/s roof and would have reported this compliant record at 20.9% of roof by a route that looks like a finding but is an artefact of the divisor. FORMAT_ID BACKFILL 2026-10-08 (w10-bench-formatid): format_id was null and is now 'awq'. EVIDENCE, quoted from this record's own text: methodology, quoting the reporter verbatim: "Deepseek 3.2 AWQ 4bit @ 10 tok/s (output) // 2000 tok/s (input of 23k tok)" - the format is part of the run description, not a checkpoint name pulled off a listing. NO NUMBER CHANGED - value, metric and unit are untouched; this adds metadata, it does not re-measure. The value was NOT taken from a model name, from a sibling record, or from outside knowledge. Left null everywhere the record's own unit/methodology does not name a format for this run: 60 of the 150 records in data/benchmarks/ are still null and each one has its reason, because a null with a reason is correct and a guessed value is not.