W8A8 int8 quantization on MI50 is 4-6x SLOWER than fp16/GPTQ, not faster, because it falls to a TritonScaledMMLinearKernel path and never engages the hardware INT8 path

vLLM benchmark_serving.py, --dataset-name sharegpt, --num-prompt 32, --sonnet-input-len 512 --sonnet-output-len 128, --max-concurrency 1. vLLM 0.8.5 with V1 disabled ('rocm is experimental on VLLM_USE_V1=1. Falling back to V0 Engine'), PyTorch 2.7.0+rocm6.3, HIP 6.3.42131, MIOpen 3.3.0, gfx906:sramecc+:xnack-, Ubuntu 24.04.2, python 3.12.3, kernel 6.11. 32/32 requests succeeded. Output token throughput 5.26 tok/s; total token throughput 10.81 tok/s; mean TTFT 1727.72 ms; mean TPOT 183.26 ms; mean ITL 183.39 ms. Served tp1 with --max_model_len 8192 --gpu_memory_utilization 0.95, quantization=compressed-tensors, scheme CompressedTensorsW8A8Int8, kernel TritonScaledMMLinearKernel, attention via Triton FA in ROCmBackend, KV cache fp8 with Q and prob scale 1.0. Launch: vllm serve <model> --max_model_len 8192 -tp 1 --gpu_memory_utilization 0.95 --disable-log-requests --port 8040.

COMMUNITY-REPORTED MEASUREMENT, and the strongest-sourced benchmark in this slice because it is a public GitHub issue carrying the full benchmark_serving.py output, the full startup log, and the exact environment. Two things make it more useful than the raw 10.81 tok/s: (1) the reporter's diagnostic observation that 'during the benchmark the GPU hasn't ever pulled more than 110W, and has mostly hanged around 80-100W, while with GPTQ models it easily pulls up to 190-200W' - a power-draw signature that distinguishes a path that is not engaging the hardware from one that is; (2) the explicit contrast in the issue title, that W8A8 'should in theory make use of the mi50's support for INT8 and thus actually improve pp and tg'. It does not, because compressed-tensors W8A8 routes to TritonScaledMMLinearKernel and gfx906 has no MFMA. The comparison point is 3.6-3.7 tok/s decode on 1x MI50 for Llama-2-7B w8a8 from the same fork's Discord benchmark tables (christopherlin, 2025-09-15, vllm 0.10.1_update1, --max-model-len 4096, output length 128) - the same order of magnitude, so this is a persistent property of the format on this hardware, not one bad run. Issue is locked and still open. LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: NOT STATED by the source - absent, not inferred. A throughput figure with no scenario cannot be placed against an MLPerf Offline or Server number (they are different measurements; on identical hardware the Offline-minus-Server difference is negative in 5 of 9 systems measured). ABSENT CONDITIONS (1 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - warmup: unrecoverable from comm-nlzy-vllm-gfx906-issue-4-w8a8, comm-gfx906-discord-server-log - would need the raw harness log's warmup phase or the benchmark command line (loadgen min_duration/min_query_count, or trtllm-bench --warm_up N / --num_runs N) None of these were filled in. A plausible value is worse than an absent one: it would be trusted. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-radeon-instinct-mi50x). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface. FORMAT_ID BACKFILL 2026-10-08 (w10-bench-formatid): format_id was null and is now 'int8-w8a8'. EVIDENCE, quoted from this record's own text: methodology: "quantization=compressed-tensors, scheme CompressedTensorsW8A8Int8". NO NUMBER CHANGED - value, metric and unit are untouched; this adds metadata, it does not re-measure. The value was NOT taken from a model name, from a sibling record, or from outside knowledge. Left null everywhere the record's own unit/methodology does not name a format for this run: 63 of the 150 records in data/benchmarks/ are still null and each one has its reason, because a null with a reason is correct and a guessed value is not.