DeepSeek-V3 Technical Report (arXiv:2412.19437)

Fields

id

sources/model-cn-deepseek-v3-report

URL

https://arxiv.org/abs/2412.19437

Publisher

arXiv / DeepSeek-AI

Kind

whitepaper

Published

2024-12-27

Accessed

2026-10-03

notes

Primary source for DeepSeek-V3's vendor-stated figures: 671B total / 37B activated per token, MLA with kv_lora_rank 512 and q_lora_rank 1536, 256 routed + 1 shared expert with 8 active per token, first 3 layers dense, YaRN context extension. Also carries the V2/V2.5/V3 comparison table (236B/21B, 72B, 405B) used to date the MLA lineage.

Notes

Primary source for DeepSeek-V3's vendor-stated figures: 671B total / 37B activated per token, MLA with kv_lora_rank 512 and q_lora_rank 1536, 256 routed + 1 shared expert with 8 active per token, first 3 layers dense, YaRN context extension. Also carries the V2/V2.5/V3 comparison table (236B/21B, 72B, 405B) used to date the MLA lineage.

Cross-references

This record references no other records.