AWS Neuron docs — Trn1/Trn1n Inference Performance tables

Fields

id

sources/aws-neuron-trn1-inference-benchmarks

URL

https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/benchmarks/trn1/trn1-inference-performance.html

Publisher

AWS

Kind

benchmark

Accessed

2026-10-15

notes

Vendor reference inference numbers for Trn1/Trn1n (Trainium1). NeuronPerf-driven. Encoder models (BERT/RoBERTa/etc) reported as inference/sec; encoder-decoder (flan-t5-xl, t5-3b) reported as tokens/sec counting BOTH input and output tokens, NeuronX Distributed TP=8 on trn1.32xlarge. Last update 2026-02-26, SDK versions 2.17-2.28. Numbers are aggregate throughput across the data-parallel run mode, not per-request decode.

Notes

Vendor reference inference numbers for Trn1/Trn1n (Trainium1). NeuronPerf-driven. Encoder models (BERT/RoBERTa/etc) reported as inference/sec; encoder-decoder (flan-t5-xl, t5-3b) reported as tokens/sec counting BOTH input and output tokens, NeuronX Distributed TP=8 on trn1.32xlarge. Last update 2026-02-26, SDK versions 2.17-2.28. Numbers are aggregate throughput across the data-parallel run mode, not per-request decode.

Cross-references

This record references no other records.