Markaicode: vLLM vs Ollama benchmark — Red Hat GuideLLM study on A100-PCIE-40GB

Fields

id

sources/markaicode-vllm-vs-ollama-guidellm-a100-pcie

URL

https://markaicode.com/benchmarks/cuda-qwen-benchmark/

Publisher

Markaicode (citing Red Hat Developer GuideLLM study, Aug 2025)

Kind

benchmark

Accessed

2026-10-15

notes

Republishes Red Hat's GuideLLM v0.2.1 study with full test environment: single NVIDIA A100-PCIE-40GB, driver 550.144.03, CUDA 12.4, OpenShift 4.17.15, vLLM 0.9.1 vs Ollama 0.9.2, Llama-3.1-8B-Instruct fp16, concurrency 1..256, 300 s per level. Peak: vLLM 793 output tok/s vs Ollama 41 tok/s (OLLAMA_NUM_PARALLEL=32 tuned); P99 TTFT 80 vs 673 ms.

Notes

Republishes Red Hat's GuideLLM v0.2.1 study with full test environment: single NVIDIA A100-PCIE-40GB, driver 550.144.03, CUDA 12.4, OpenShift 4.17.15, vLLM 0.9.1 vs Ollama 0.9.2, Llama-3.1-8B-Instruct fp16, concurrency 1..256, 300 s per level. Peak: vLLM 793 output tok/s vs Ollama 41 tok/s (OLLAMA_NUM_PARALLEL=32 tuned); P99 TTFT 80 vs 673 ms.

Cross-references

This record references no other records.