inference/research
verifiedhigh · 0.85updated 2026-10-04

AWS Trn2.48xlarge: 20.8 FP8 PFLOPS (16-chip instance)

20.8 PFLOPS FP8

Fields

id
benchmarks/alt-aws-trn2-fp8-tps
Metric
tps_aggregate
Value
20.8
Unit
PFLOPS FP8
Model
N/A (peak compute specification)
Methodology
Vendor-published instance specification. 16 Trainium2 chips in a 4x4 2D torus with NeuronLink-v3 at 1024 GB/s/chip. FP8 peak compute. Also: 10.7 PFLOPS FP16/BF16/TF32, 41 PFLOPS sparse, 2.9 PFLOPS FP32.
Measured by
vendor
Reproducible
no
notes
This is a vendor-published peak compute specification, not a measured inference throughput. The 20.8 PFLOPS is the theoretical FP8 peak for the 16-chip instance. No inference throughput (tokens/sec) is published by AWS for Trainium2. The per-chip FP8 figure of 1300 TFLOPS is derived: 20.8 PFLOPS / 16 chips. LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: NOT STATED by the source - absent, not inferred. A throughput figure with no scenario cannot be placed against an MLPerf Offline or Server number (they are different measurements; on identical hardware the Offline-minus-Server difference is negative in 5 of 9 systems measured). ABSENT CONDITIONS (5 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - engine version: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - engine_id=alt-neuron-compiler is named but no build/commit/container digest is published - batch/concurrency: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the achieved in-flight batch or the --max-concurrency value - input length: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the run's ISL or the prepared dataset's input-length profile - output length: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the run's OSL or the harness's configured max_tokens - warmup: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the raw harness log's warmup phase or the benchmark command line (loadgen min_duration/min_query_count, or trtllm-bench --warm_up N / --num_runs N) None of these were filled in. A plausible value is worse than an absent one: it would be trusted.

Notes

This is a vendor-published peak compute specification, not a measured inference throughput. The 20.8 PFLOPS is the theoretical FP8 peak for the 16-chip instance. No inference throughput (tokens/sec) is published by AWS for Trainium2. The per-chip FP8 figure of 1300 TFLOPS is derived: 20.8 PFLOPS / 16 chips.

LOAD-CONDITION AUDIT (bq- 2026-10-04). SCENARIO: NOT STATED by the source - absent, not inferred. A throughput figure with no scenario cannot be placed against an MLPerf Offline or Server number (they are different measurements; on identical hardware the Offline-minus-Server difference is negative in 5 of 9 systems measured). ABSENT CONDITIONS (5 of 7 checked: model, hardware, batch-or-concurrency, input length, output length, engine version, warmup): - engine version: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - engine_id=alt-neuron-compiler is named but no build/commit/container digest is published - batch/concurrency: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the achieved in-flight batch or the --max-concurrency value - input length: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the run's ISL or the prepared dataset's input-length profile - output length: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the run's OSL or the harness's configured max_tokens - warmup: unrecoverable from alt-trn2-arch, asic-aws-trn2-instances - would need the raw harness log's warmup phase or the benchmark command line (loadgen min_duration/min_query_count, or trtllm-bench --warm_up N / --num_runs N) None of these were filled in. A plausible value is worse than an absent one: it would be trusted.

Cross-references