arXiv:2507.00418 — Serving LLMs in HPC Clusters: QAic 100 Ultra vs A100/V100/A6000/H200/MI300A (vLLM)

Fields

id

sources/arxiv-2507-00418-qaic-vs-gpu-llm-serving

URL

https://arxiv.org/abs/2507.00418

Publisher

arXiv / UCSD (PEARC'25)

Kind

paper

Published

2025-07-01

Accessed

2026-10-14

notes

PEARC'25 paper. vLLM serving benchmark on NRP. Table 2 = peak aggregate tok/s per accelerator per model; concurrency swept 32-256 concurrent requests, best card-count config chosen per cell. QAic = Cloud AI 100 Ultra (maps to accelerator qualcomm-ai200, 128GB, 4 SoC/card). AGGREGATE multi-request throughput, NOT per-request decode. MI300A used where a model fit (128GB APU).

Notes

PEARC'25 paper. vLLM serving benchmark on NRP. Table 2 = peak aggregate tok/s per accelerator per model; concurrency swept 32-256 concurrent requests, best card-count config chosen per cell. QAic = Cloud AI 100 Ultra (maps to accelerator qualcomm-ai200, 128GB, 4 SoC/card). AGGREGATE multi-request throughput, NOT per-request decode. MI300A used where a model fit (128GB APU).

Cross-references

This record references no other records.