FP8 KV cache quantization (E4M3 keys and values)

GROUP SIZE NOT APPLICABLE BY CONSTRUCTION: this scheme quantizes the KV cache, not the weights, so there are no weight groups to size. A number in this field would be a category error. (Where the underlying method has a real grouping - e.g. KIVI per-channel keys, KVQuant per-channel - that lives in the method's own record, not in weight_group_size.) FP8 KV cache quantization reduces KV cache memory by 2x compared to FP16. The key question is accuracy: FP8 E4M3 has ~2 decimal digits of precision, which is marginal for attention keys that feed into softmax. The repo already documents that sliding-window attention plus FP8 KV plus speculative decoding causes silent corruption (see existing records). The attention kernel interaction is critical: the KV cache is quantized after the KV projection, so the attention kernel must dequantize before computing attention scores. This dequantization adds latency to the decode phase. The vLLM docs mention FP8 KV cache as a feature but the SGLang docs do not list it as a separate quantization method - it may be handled internally. HARDWARE SUPPORT: FP8 is natively supported on Hopper (SM90) and Ada (SM89) with FP8 tensor cores. On Ampere (SM80), FP8 is emulated via FP16 with reduced precision. The accuracy delta is expected to be larger on Ampere where the FP8 path is emulated. This is a priority for further research: finding a source that reports measured accuracy delta on a named benchmark.