inference/research
verifiedlow · 0.60updated 2026-10-17

Database Mart vLLM benchmark, RTX PRO 5000 Blackwell (48 GB)

benchmark

Fields

id
sources/databasemart-vllm-rtx-pro-5000-blackwell
Publisher
Database Mart
Kind
benchmark
Published
2026-10-07
Accessed
2026-10-17
notes
Database Mart hosted-GPU report, page footer dated 10/07/2026. Engine is vLLM built from github.com/vllm-project/vllm with no release tag or commit printed, so the engine version is unpinned. Weights are the Hugging Face checkpoint at the precision the table labels 'Quantization 16' (bf16/fp16 not distinguished); format_id is null because data/quantization/ has no fp16/bf16 record. Single GPU, tensor parallel 1. Input length fixed at 100 tokens, output length fixed at 600 tokens. The 'Output (tokens/s)' column is aggregate generated tokens across the concurrent requests divided by benchmark duration, NOT a per-request decode rate. Median TTFT is the median time to first token under that same concurrency. Third-party measurement by the GPU host; confidence capped because the harness and the vLLM commit are unpublished. Card: NVIDIA RTX Pro 5000 Blackwell Server Edition, 48 GB GDDR7 with ECC, 1,344 GB/s, 300 W, PCIe 5.0 x16. This is the 48 GB SKU, not the 72 GB workstation SKU. Request count 1000. The page headline quotes about 2,879 output tok/s for Llama-3.1-8B; the table prints 2,878.77 and that is the recorded value.

Notes

Database Mart hosted-GPU report, page footer dated 10/07/2026. Engine is vLLM built from github.com/vllm-project/vllm with no release tag or commit printed, so the engine version is unpinned. Weights are the Hugging Face checkpoint at the precision the table labels 'Quantization 16' (bf16/fp16 not distinguished); format_id is null because data/quantization/ has no fp16/bf16 record. Single GPU, tensor parallel 1. Input length fixed at 100 tokens, output length fixed at 600 tokens. The 'Output (tokens/s)' column is aggregate generated tokens across the concurrent requests divided by benchmark duration, NOT a per-request decode rate. Median TTFT is the median time to first token under that same concurrency. Third-party measurement by the GPU host; confidence capped because the harness and the vLLM commit are unpublished. Card: NVIDIA RTX Pro 5000 Blackwell Server Edition, 48 GB GDDR7 with ECC, 1,344 GB/s, 300 W, PCIe 5.0 x16. This is the 48 GB SKU, not the 72 GB workstation SKU. Request count 1000. The page headline quotes about 2,879 output tok/s for Llama-3.1-8B; the table prints 2,878.77 and that is the recorded value.

Cross-references

This record references no other records.