AWS Neuron: trn1.32xlarge (Trainium1), NeuronX Distributed, t5-3b, TP=8, batch 1, throughput-opt
116.29 tok/s (counts input+output tokens; TP=8 across 16 Trainium chips)
Fields
id
benchmarks/aws-trn1-neuronxd-t53b-fp32-tp8-b1-tpsMetric
tps_aggregate
Value
116.29
Unit
tok/s (counts input+output tokens; TP=8 across 16 Trainium chips)
Model
t5-3b
Methodology
VENDOR-MEASURED (AWS Neuron docs, NeuronPerf). NeuronX Distributed on trn1.32xlarge (16x Trainium1), Tensor Parallel degree 8, DP 1, batch 1, seq_len 128, in_len 128, out_len 84, FP32 weights / Matmult-BF16 autocast, Neuron 2.17.0. Throughput tokens/sec counts BOTH input and output tokens — NOT pure decode.
Measured by
vendor
Reproducible
yes
notes
Encoder-decoder text-generation row. Engine = AWS NeuronX Distributed (Neuron SDK PyTorch tensor-parallel runtime); engine_id uses the repo's alt-neuron-compiler engine tag.
Notes
Encoder-decoder text-generation row. Engine = AWS NeuronX Distributed (Neuron SDK PyTorch tensor-parallel runtime); engine_id uses the repo's alt-neuron-compiler engine tag.