inference/research
verifiedlow · 0.35updated 2026-10-15

Kuhaku413/rocm-inference-bench — AMD MI250X / MI300X vs CUDA LLM inference table

repo

Fields

id
sources/kuhaku413-rocm-inference-bench
Publisher
Kuhaku413 (GitHub)
Kind
repo
Accessed
2026-10-15
notes
Single-commit benchmark-suite README with a results table: Llama-3-8B BF16 on MI250X = 42.3 tok/s (16.2GB), MI300X FP8 = 68.7 tok/s (12.1GB), Mistral-7B BF16 on MI250X = 38.9 tok/s (14.8GB). No methodology detail beyond PyTorch+ROCm6.x, no batch/prompt-length specified — treat as low-confidence community numbers.

Notes

Single-commit benchmark-suite README with a results table: Llama-3-8B BF16 on MI250X = 42.3 tok/s (16.2GB), MI300X FP8 = 68.7 tok/s (12.1GB), Mistral-7B BF16 on MI250X = 38.9 tok/s (14.8GB). No methodology detail beyond PyTorch+ROCm6.x, no batch/prompt-length specified — treat as low-confidence community numbers.

Cross-references

This record references no other records.