Database Mart vLLM benchmark, NVIDIA A40, 50 and 100 concurrent
Fields
sources/databasemart-vllm-a40Notes
Database Mart hosted-GPU report, page footer dated 10/07/2026. Engine is vLLM built from github.com/vllm-project/vllm with no release tag or commit printed, so the engine version is unpinned. Weights are the Hugging Face checkpoint at the precision the table labels 'Quantization 16' (bf16/fp16 not distinguished); format_id is null because data/quantization/ has no fp16/bf16 record. Single GPU, tensor parallel 1. Input length fixed at 100 tokens, output length fixed at 600 tokens. The 'Output (tokens/s)' column is aggregate generated tokens across the concurrent requests divided by benchmark duration, NOT a per-request decode rate. Median TTFT is the median time to first token under that same concurrency. Third-party measurement by the GPU host; confidence capped because the harness and the vLLM commit are unpublished. Card: NVIDIA A40, Ampere, 48 GB GDDR6, 10,752 CUDA cores. Both the 50-concurrent and the 100-concurrent scenarios are internally consistent (request count matches total input tokens / 100) and both are recorded. The page notes 14B models needed --max-model-len 4096 to load, and that the card reached 80 C+ under load.
Cross-references
This record references no other records.