AWS Neuron docs: ResNet-50 on inf1.xlarge batch, 2207 img/s (p50 18ms)
2207.0 images/s (batch 10, fp16)
Fields
id
benchmarks/aws-neuron-inf1-resnet50-batch-inf-sMetric
tps_aggregate
Value
2,207
Unit
images/s (batch 10, fp16)
Model
ResNet-50
Methodology
VENDOR-MEASURED (AWS Neuron docs, Inf1 Inference Performance, NeuronPerf, last update 2024-09-16, Neuron 2.20). inf1.xlarge = 1 Inferentia1 chip (4 NeuronCores). Encoder/CNN models only — NO generative LLM rows exist for Inf1. Units are inferences/sec (BERT, seq_len 128) or images/sec (ResNet/YOLO), NOT tokens/sec. TensorFlow 1.15, batch 10, Data Parallel, fp16.
Measured by
vendor
Reproducible
yes
notes
CNN row — highest-throughput published Inf1 figure; images/sec, not tokens.
Notes
CNN row — highest-throughput published Inf1 figure; images/sec, not tokens.