DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

DSA = lightning indexer + fine-grained token selection. Index score is a ReLU-based dot product of q and w (ReLU 'for throughput consideration'); the indexer 'has a small number of heads and can be implemented in FP8'. Only the top-2048 key-value entries are attended per query. DSA is instantiated on the MQA mode of MLA because 'each key-value entry must be shared across multiple queries for computational efficiency'. The indexer is trained separately with a KL loss, detached from the main graph.