NVFP4: Block-scaled FP4 for Blackwell (arXiv:2404.14469 - related work)

Fields

id

sources/qhw-src-nvfp4-paper

URL

https://arxiv.org/abs/2404.14469

Publisher

arXiv

Kind

paper

Published

2024-04-22

Accessed

2026-10-04

notes

Related work on FP4 quantization. The NVFP4 format is documented in NVIDIA's TensorRT-LLM and Transformer Engine documentation. Key facts: E2M1 element format, E4M3 per-16-element block scale, FP32 per-tensor global scale. Native only on Blackwell (SM100/SM103/SM120). The format is W4A4 (both weights and activations quantized to FP4). This is distinct from MXFP4 which uses E8M0 power-of-two scales per 32 elements.

Notes

Related work on FP4 quantization. The NVFP4 format is documented in NVIDIA's TensorRT-LLM and Transformer Engine documentation. Key facts: E2M1 element format, E4M3 per-16-element block scale, FP32 per-tensor global scale. Native only on Blackwell (SM100/SM103/SM120). The format is W4A4 (both weights and activations quantized to FP4). This is distinct from MXFP4 which uses E8M0 power-of-two scales per 32 elements.

Cross-references

This record references no other records.