vLLM 0.26.0 prefill throughput for Command A W4A4 at 512-128K context on 2xH100

Hardware: 2x NVIDIA H100 80GB SXM, TP2. Engine: vLLM 0.26.0. Model: Command A W4A4 (218B total / 25B active MoE, NVFP4 experts). Measurement: saturated prefill (max-batch concurrent requests, output_len=1). Context: 128K tokens. Batch size: 3 (max that fits in KV cache at 128K). Cache state: cold (random prompts, unique seeds). Sibling prefill throughput points: 512 = 11587 tok/s (B=85), 2K = 11357 tok/s (B=30), 8K = 10843 tok/s (B=10), 32K = 10084 tok/s (B=4), 64K = 9274 tok/s (B=2), 128K = 8080 tok/s (B=3). Per-token cost: below ~8K it falls as fixed per-step overhead amortizes; beyond 32K the O(L^2) full-attention layers take over, and by 128K per-token cost is ~35% above its 8K best.

Published by SamMausberg (capp-vllm-bench) in August 2026. Shows that prefill throughput degrades only ~25% from 8K (10843 tok/s) to 128K (8080 tok/s) despite the 16x increase in context length. This is because 24 of 32 layers use sliding-window attention capped at 4096 tokens, so only 8 layers pay the O(L^2) cost. The 11.5K tok/s at 2xH100 sustains roughly 580 TFLOPS (~29% of aggregate dense BF16 peak), which is credible for a token-choice MoE routing through Marlin dequant on every expert GEMM. LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: NOT STATED by the source - absent, not inferred. A throughput figure with no scenario cannot be placed against an MLPerf Offline or Server number (they are different measurements; on identical hardware the Offline-minus-Server difference is negative in 5 of 9 systems measured). ABSENT CONDITIONS (3 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - engine version: unrecoverable from bench-capp-vllm-longctx-ttft - engine_id=vllm is named but no build/commit/container digest is published - output length: unrecoverable from bench-capp-vllm-longctx-ttft - would need the run's OSL or the harness's configured max_tokens - warmup: unrecoverable from bench-capp-vllm-longctx-ttft - would need the raw harness log's warmup phase or the benchmark command line (loadgen min_duration/min_query_count, or trtllm-bench --warm_up N / --num_runs N) None of these were filled in. A plausible value is worse than an absent one: it would be trusted. FORMAT_ID BACKFILL 2026-10-08 (w10-bench-formatid): format_id was null and is now 'nvfp4'. EVIDENCE, quoted from this record's own text: methodology: "Model: Command A W4A4 (218B total / 25B active MoE, NVFP4 experts)". NO NUMBER CHANGED - value, metric and unit are untouched; this adds metadata, it does not re-measure. The value was NOT taken from a model name, from a sibling record, or from outside knowledge. Left null everywhere the record's own unit/methodology does not name a format for this run: 63 of the 150 records in data/benchmarks/ are still null and each one has its reason, because a null with a reason is correct and a guessed value is not.