A40 vLLM: Qwen/Qwen2.5-7B-Instruct median TTFT, 100 concurrent
1093.55 ms (median time to first token, 100 concurrent)
Fields
id
benchmarks/dm-a40-vllm-qwen25-7b-instruct-c100-ttftMetric
ttft_ms
Value
1,093.55
Unit
ms (median time to first token, 100 concurrent)
Model
qwen2.5-7b-instruct
Methodology
Database Mart vLLM bench on one NVIDIA A40 48 GB GDDR6. Model Qwen/Qwen2.5-7B-Instruct, quantization column '16' (bf16/fp16 not distinguished; format_id null, no fp16 record exists). vLLM commit not printed. Single GPU, tensor parallel 1. 100 requests, input length 100 tokens, output length 600 tokens, benchmark duration 42.94 s, total input tokens 10000, total generated tokens 48267. AGGREGATE across the concurrent requests, not a per-request decode rate.
Measured by
third_party
Reproducible
yes
notes
Median TTFT. The page reports the card reaching 80 C+ under these loads; no watt figure is printed.
Notes
Median TTFT. The page reports the card reaching 80 C+ under these loads; no watt figure is printed.