AWS Neuron docs: bert-base-uncased on trn1.2xlarge, real-time, 2085 inf/s (p50 0.94ms)
2085.45 inferences/s (encoder fwd pass, batch-1, latency-optimized)
Fields
id
benchmarks/aws-neuron-trn1-bert-base-rt-inf-sMetric
tps_aggregate
Value
2,085.45
Unit
inferences/s (encoder fwd pass, batch-1, latency-optimized)
Model
bert-base-uncased
Methodology
VENDOR-MEASURED (AWS Neuron docs, NeuronPerf). trn1.2xlarge, PyTorch 2.8, Neuron 2.26, Data Parallel run-mode, batch 1, seq_len 128, FP32 w/ Matmult-BF16 autocast. Encoder model — unit is inferences/sec, NOT tokens/sec.
Measured by
vendor
Reproducible
yes
notes
Encoder (BERT) row — counts whole-sequence inferences, not generative tokens. Included to record the vendor-published latency-optimized figure; not comparable to decode tok/s.
Notes
Encoder (BERT) row — counts whole-sequence inferences, not generative tokens. Included to record the vendor-published latency-optimized figure; not comparable to decode tok/s.