SGLang DSPARK speculative decoding A/B on DeepSeek-V4.1-Flash, 8x RTX PRO 6000: 76.5-78.9 vs 36.4-36.5 tok/s on prose at block_size=2

PER-REQUEST single-stream decode, and this is an A/B rather than a single number: same server, same 8x RTX PRO 6000 Blackwell Max-Q 96 GB (SM120, PCIe, no NVLink), same TP=8/EP=8/DP=1 production configuration, only the speculative path toggled, and 'clean legs, each verified via server_info + acceptance length'. DSPARK ON, block_size 2: prose 76.5-78.9 tok/s, structured 120.3-120.5 tok/s. DSPARK OFF: 36.4-36.5 tok/s prose, 36.5 tok/s structured. Acceptance length 3.0 tokens/step with speculation vs 1.0 without. The decisive observation is that WITH SPECULATION OFF the model flattens to ~36.5 tok/s 'regardless of prompt class', i.e. the 2.1x on prose and 3.3x on structured are not prompt-dependent luck - speculation is the whole difference. The reporter's own summary: 'speculation is the difference between usable and unusable latency on this hardware'.

This is the best-controlled causal measurement in this whole addition because it is a within-engine A/B with the configuration verified server-side rather than assumed - the author notes they 'recorded a full sweep of false data' earlier by trusting container env vars, which is precisely the mistake this repo's methodology rule exists to prevent. TWO CAVEATS BOTH STATED BY THE AUTHOR. (1) The headline uses block_size=2 but PRODUCTION SHIPS block_size=6, at which the reporter measures ~237-252 tok/s structured (vs 120 at block 2) - so the production number is roughly double the number recorded here, and the reason block_size=2 is the one recorded is that it gives the cleanest ON/OFF legs. (2) 'Not established - No 24 h soak ... block sizes 4/8/12 not re-measured with the fixed sweep tool, and >32K prefill/long-context degradation not characterised.' Config traps the same report documents, because they change what any of these numbers mean: --enable-expert-parallel is a vLLM flag and is rejected by SGLang, which takes ep_size; docker run must pass TP_SIZE/DP_SIZE/EP_SIZE explicitly or it 'silently serves TP8 defaults'; ep_size must be <= tp_size or the MoE rank computation divides by zero; DP attention is unreachable on this model ('V4.1 vision currently supports TP/EP without DP, CP, PP or MoE A2A'). LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: MLPerf-style server scenario. ABSENT CONDITIONS (3 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - input length: unrecoverable from ev-sglang-issue-40877-production-deepseek-v41-flash - would need the run's ISL or the prepared dataset's input-length profile - output length: unrecoverable from ev-sglang-issue-40877-production-deepseek-v41-flash - would need the run's OSL or the harness's configured max_tokens - warmup: unrecoverable from ev-sglang-issue-40877-production-deepseek-v41-flash - would need the raw harness log's warmup phase or the benchmark command line (loadgen min_duration/min_query_count, or trtllm-bench --warm_up N / --num_runs N) None of these were filled in. A plausible value is worse than an absent one: it would be trusted. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (nvidia-rtx-pro-6000-blackwell). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface.