LLaDA-8B-Instruct

Diffusion language model, not an autoregressive one. config.json: model_type 'llada', architectures ['LLaDAModelLM'], d_model 4096, n_layers 32, n_heads 32, n_kv_heads 32, max_sequence_length 4096, block_type 'llama'. Crucially use_cache is false and there is no KV-cache machinery at all - the config has no sliding_window, no num_key_value_heads distinction worth noting (32/32), and the README states generation works by iteratively unmasking tokens. kv_cache_bytes_per_token = 0, but NOT because memory is flat: there is no autoregressive KV cache because the model never generates token-by-token. Each denoising step runs a full bidirectional forward pass over the entire sequence, so the cost per generated token is (number of denoising steps) x (one full-sequence forward pass), and compute scales with sequence length x steps rather than with a growing cache. This makes the workload compute-bound in a way autoregressive serving is not, and it is why llama.cpp/vLLM KV-cache management has nothing to contribute here. gqa_ratio = 1 (32 heads, 32 KV heads). head_dim derived as 4096 / 32 = 128. active_params_b equals params_b because every weight participates in every denoising step. Practical caveat: custom modeling code (modeling_llada.py) is required, and vLLM support is via a separate diffusion-LM path rather than the normal causal-LM path. context_scaling is null: context length is native to the pretrained config, no rope_scaling. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=diffusion. REQUIRED KERNEL CLASS: no KV cache and no decode loop at all - this is the one family in the repo where an autoregressive serving engine is the wrong shape, not merely the wrong kernel. flop/diffusion-lm-denoising-steps: each denoising step is a FULL forward pass over the current sequence, cost per emitted token = S * 2 * N / Lg for S steps and generation length Lg, so the per-token cost is a function of two user-facing hyperparameters rather than a constant, the attention is bidirectional so there is no causal prefix to reuse, and the model is dense so there is no MoE sparsity to amortise. Arithmetic intensity per step is prefill-like, so part selection inverts (compute and tensor cores, not memory bandwidth), TTFT is excellent and inter-token latency is terrible, and the weight stream is paid S times. ENGINE JOIN (cited): vLLM has exactly ONE diffusion row in its entire architecture registry - DiffusionGemmaForBlockDiffusion - and NO LLaDA row and NO Dream row [[mdl2-vllm-supported-models-2026-10]] [[pap-arch-vllm-supported-models]]. SGLang by contrast gives diffusion its own top-level docs section separate from 'Large Language Models', with its own launch flags (--dllm-algorithm LowConfidence or JointThreshold, --dllm-algorithm-config, --no-dllm-fdfo) and its own scheduling primitive, First-Done-First-Out: 'each request leaves the batch as soon as its block is resolved, instead of advancing in lockstep' - a scheduling model that does not exist for the AR path [[mdl2-sglang-diffusion-language-models]]. llama.cpp carries LLADA, LLADA_MOE, DREAM and RND1 in llm_arch_is_diffusion and adds two diffusion-only tensors, LLM_TENSOR_MASKED_EMBD_CENTROIDS and LLM_TENSOR_MASKED_EMBD_ORDERING, so its diffusion path needs a different state structure and not just a different sampler [[mdl2-llamacpp-arch-class-registry]]. ik-llama-cpp lists DFlash among the features that landed there first [[ik-llama-cpp]]. HONEST COVERAGE GAP: SGLang's supported diffusion set is LLaDA2.0 (mini, flash), SDAR/JetLM (8B-Chat dense, 30B-A3B-Chat MoE) and DiffusionGemma - this checkpoint is NOT among them, and Dream 7B is not either [[mdl2-sglang-diffusion-language-models]]. So for this exact checkpoint: vLLM = no native row, SGLang = no table row, and no engine record in this repo states otherwise. Custom modeling code (modeling_llada.py) is required, and use_cache is false in config.json, which is the machine-readable form of the same statement. CONTEXT_SCALING_METHOD (none-verified): GSAI-MML/LLaDA-8B-Instruct config has max_sequence_length 4096 and no rope_scaling, but it is a diffusion LM with its own position convention; the config declares nothing to classify as rope context extension.