vLLM 0.20 TTFT for Qwen3.5-9B at 32K-256K context (dense FA2, chunk 4K)

Serving: vLLM 0.20, chunk 4K, dense baseline FA2. Context: 32K tokens. Hardware: GPU 80G (specific GPU model not stated). Concurrency: not explicitly stated (likely single-request or low concurrency given TTFT measurement). Cache state: cold (no prefix caching mentioned). Output: first token only. Value 2944 ms at 32K. Sibling points from same leaderboard: 64K = 6863 ms, 100K = 11780 ms, 128K = 17652 ms, 256K = 50915 ms.

Published by SOTA2 Research (formerly WizWand) on 2026-06-05. Part of a TTFT leaderboard spanning 32K-256K context. The 32K TTFT of 2.944s for a 9B model on an 80G GPU is notably high, suggesting either a very long prompt processing time or significant scheduling overhead. The scaling from 32K to 256K is roughly linear in context length (2.9s to 50.9s, ~17x for 8x context), which is consistent with O(L^2) attention dominating at long context. Sparse attention variant (distilled indexer) achieves 1.07x-1.93x speedup over dense. PROVENANCE RAISED 2026-10-08 (w8-bench-src): the leaderboard was Tier D because www.sota2.com is an aggregator/researcher-org host with no vendor identity in the classifier's registry. The UNDERLYING DOCUMENT is now identified and cited: the sota2 paper page for this work is 'How Much Dense Attention is Necessary? Oracle-Guided Sparse Prefill for Full/GQA Layers in Hybrid Long-Context Models', arXiv:2606.07703 (26pp, 11 tables, Hongxing Wang, Harenome Razanajato, Zhen Zhang, Yujie Yuan, Hongsheng Liu), fetched and read 2026-10-08 as [[w8-src-arxiv-2606-07703-dense-attention-oracle]]. The leaderboard's own 'Related benchmarks' table links to that paper and the abstract carries the TTFT claims in the same words: 'Preliminary single-card TTFT measurements show distilled-indexer sparse serving speedups of 1.71x for Qwen3.5-0.8B on NPU and 1.93x for Qwen3.5-9B on GPU against its dense FlashAttention-2 baseline.' So the aggregator page is now a Tier-C-author-preprint measurement. VALUE UNCHANGED (2944 ms) AND THE HONEST LIMIT IS THE POINT: the paper itself labels these measurements PRELIMINARY, it does not name the GPU model ('GPU 80G' is a leaderboard label), and neither the paper page nor the leaderboard states concurrency or cache state - so this record's 'reproducible: false' and its note about the unstated hardware stay exactly as they were. Raising the tier did not manufacture a missing condition.