AWS Neuron documentation - Inferentia2 Architecture page (per-chip compute, memory, DMA, NeuronLink)

FIRST-PARTY HARDWARE DOCUMENTATION. Read in full 2026-10-05. VERBATIM: 'At the heart of each Inf2 instance are up to twelve Inferentia2 chips (each with two NeuronCore-v2 cores).' Component table, VERBATIM: Compute 'Two NeuronCore-v2 cores, delivering 380 INT8 TOPS, 190 FP16/BF16/cFP8/TF32 TFLOPS, and 47.5 FP32 TFLOPS.'; Device Memory '32GiB of high-bandwidth device memor (HBM) (for storing model state), with 820 GiB/sec of bandwidth.'; Data Movement '1 TB/sec of DMA bandwidth, with inline memory compression/decompression.'; NeuronLink 'NeuronLink-v2 for chip-to-chip interconnect enables high-performance collective compute for co-optimization of latency and throughput.'; Programmability 'Inferentia2 supports dynamic shapes and control flow, via ISA extensions of NeuronCore-v2 and custom-operators via the deeply embedded GPSIMD engines.' WHY THIS PAGE WAS FETCHED IN THIS PASS, and what it settles. The repo's aws-inferentia2 record carried an fp8 row at 380 TFLOPS whose own notes said '380 TFLOPS FP8 is inferred as 2x the vendor's 190 TFLOPS FP16 per-chip figure. AWS does not publish a per-chip FP8 figure; this is a direct inference' - and it was filed with basis 'vendor-spec-sheet', which misrepresented an inference as a transcription. This page shows where 380 actually comes from and it is NOT an FP8 number: the 380 is INT8 TOPS. 380 INT8 TOPS is a real first-party per-chip figure, and filing it as 380 FP8 TFLOPS was a precision error that happened to preserve the number. Two more rows this page makes possible that the record did not have: 47.5 FP32 TFLOPS, and the 1 TB/s DMA bandwidth as a figure distinct from the 820 GiB/s device-memory bandwidth. TWO WORDING TRAPS IN THIS TABLE, both recorded so a future pass does not fall into them: (1) the device-memory bandwidth is printed as '820 GiB/sec', not GB/s - the repo's existing 820 sits in a field named memory_bandwidth_gbps, so 820 GiB/s is actually about 881 GB/s and the stored value is the vendor's NUMBER with the wrong unit label attached. That is a units discrepancy, not a rounding difference, and it is recorded rather than silently converted. (2) 'with inline memory compression/decompression' appears in the Data Movement row, not the Device Memory row, so it must not be read as a property of the HBM tier.