inclusionAI/LLaDA2.2-mini model card and config.json
Fields
sources/hf-llada2-2-miniNotes
Primary source for the LLaDA2.2-mini record. HF API createdAt 2026-09-05T05:16:19Z, lastModified 2026-10-14T06:40:22Z (card text refresh inside the watcher window; the weights are the September 5 checkpoint). Model card states a MoE diffusion language model, 16B total excluding embeddings, 1.4B activated, 20 layers, 16 attention heads, 4 KV heads, 256 experts with 8 activated per token, context 128K, vocab 157184, Apache-2.0. config.json (resolve/main, fetched 2026-10-17) confirms model_type llada2_moe, architectures LLaDA2MoeModelLM, hidden_size 2048, num_hidden_layers 20, num_attention_heads 16, num_key_value_heads 4, head_dim 128, intermediate_size 5120, moe_intermediate_size 512, num_experts 256, num_experts_per_tok 8, num_shared_experts 1, first_k_dense_replace 1, max_position_embeddings 131072, vocab_size 157184, rope_theta 3000000, rope_scaling null, partial_rotary_factor 0.5, use_cache false, tie_word_embeddings false, hidden_act silu. Safetensors parameter total reported by the HF API is 16255643392 BF16. The card's technical report link is https://github.com/inclusionAI/LLaDA2.X/blob/main/LLaDA2_2_tech_report.pdf (a GitHub PDF, not an arXiv id). Benchmark numbers on the card (BFCL v4 47.68, AIME 2026 35.05, GPQA-Diamond 44.41, LongBench v2 34.99, agentic average 59.00) are vendor claims. The card publishes no absolute tok/s for the mini; the sibling flash card's TPS table is not used here. Serving recipe on the card is SGLang from main with dllm-algorithm JointThresholdInDel and, in the example, tensor parallel size 8.
Cross-references
This record references no other records.