Intel Gaudi 3 AI Accelerator — Inference Model Performance Data (software 1.24, Optimum-Habana)

Fields

id

sources/intel-gaudi3-model-performance-1-24

URL

https://www.intel.com/content/www/us/en/developer/platform/gaudi/model-performance.html

Publisher

Intel

Kind

benchmark

Accessed

2026-10-14

notes

Vendor-published max-throughput table for Gaudi 3 (Optimum-Habana, Gaudi sw 1.24, FP8). Verbatim: model/precision/input_len/output_len/#HPU/batch_size/throughput. Numbers are AGGREGATE tokens/sec across the batch (huge batch sizes), not per-request decode.

Notes

Vendor-published max-throughput table for Gaudi 3 (Optimum-Habana, Gaudi sw 1.24, FP8). Verbatim: model/precision/input_len/output_len/#HPU/batch_size/throughput. Numbers are AGGREGATE tokens/sec across the batch (huge batch sizes), not per-request decode.

Cross-references

This record references no other records.