vLLM 0.26.0 TTFT for Command A W4A4 (218B MoE) at 512-128K context on 2xH100

Hardware: 2x NVIDIA H100 80GB SXM, NVLink NV18, TP2. Engine: vLLM 0.26.0. Model: Command A W4A4 (218B total / 25B active MoE, NVFP4 experts with BF16 attention/KV). Measurement: single-request TTFT (max_concurrency=1, output_len=1), the latency floor for prefill. Context: 128K tokens. Cache state: cold (random prompts with unique seeds per run, prefix caching on but cannot contaminate). All 91 runs completed with zero preemptions. Sibling TTFT points: 512 = 78 ms, 2K = 216 ms, 8K = 785 ms, 32K = 3390 ms, 64K = 7290 ms, 128K = 16800 ms. Sliding-window architecture (24 of 32 layers cap at 4096-token window) makes long context unusually cheap to hold.

Published by SamMausberg (capp-vllm-bench) in August 2026. The most detailed public TTFT breakdown at long context. Key findings: (1) TTFT scales super-linearly with context: 78ms at 512, 785ms at 8K, 3.39s at 32K, 7.29s at 64K, 16.8s at 128K. (2) The sliding-window architecture means only 8 of 32 layers pay for long context, so 128K costs 1.9x a 64K sequence (not 2x). (3) Saturated prefill throughput at 128K is 8080 tok/s (B=3), only 30% below the 8K peak of 10843 tok/s. (4) Chunked prefill budget (8192 default) is the right trade-off; dropping to 2048 costs 8-18% prefill throughput but halves TTFT under load. (5) CUDA graphs are essential: 5.8x on single-stream decode. LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: single-request latency - no load scenario applies. ABSENT CONDITIONS (2 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - engine version: unrecoverable from bench-capp-vllm-longctx-ttft - engine_id=vllm is named but no build/commit/container digest is published - output length: unrecoverable from bench-capp-vllm-longctx-ttft - would need the run's OSL or the harness's configured max_tokens None of these were filled in. A plausible value is worse than an absent one: it would be trusted. FORMAT_ID BACKFILL 2026-10-08 (w10-bench-formatid): format_id was null and is now 'nvfp4'. EVIDENCE, quoted from this record's own text: methodology: "Model: Command A W4A4 (218B total / 25B active MoE, NVFP4 experts with BF16 attention/KV)". NO NUMBER CHANGED - value, metric and unit are untouched; this adds metadata, it does not re-measure. The value was NOT taken from a model name, from a sibling record, or from outside knowledge. Left null everywhere the record's own unit/methodology does not name a format for this run: 63 of the 150 records in data/benchmarks/ are still null and each one has its reason, because a null with a reason is correct and a guessed value is not.