Command A W4A4 on vLLM: a performance breakdown on 2xH100 (capp-vllm-bench)

Comprehensive vLLM 0.26.0 benchmark of CohereLabs/command-a-plus-05-2026-w4a4 (218B total / 25B active MoE, NVFP4 experts) on 2xH100 80GB with TP2. Measures TTFT (single-request, max_concurrency=1, output_len=1), saturated prefill (max-batch concurrent, output_len=1), and decode (256 output tokens) at context lengths 512, 2K, 8K, 32K, 64K, 128K. All 91 runs completed with zero preemptions. Prefix caching on but cannot contaminate (random prompts with unique seeds per run). Sliding-window architecture (24 of 32 layers cap at 4096-token window) makes long context unusually cheap. Also measures chunked-prefill budget effects, CUDA graphs, KV cache memory, and decode interference.