Seed-OSS-36B-Instruct

ByteDance's open dense model, and a useful data point: it is DENSE, with no MoE and no MLA, at 36B. kv_cache_bytes_per_token = 64 layers x 8 kv_heads x head_dim 128 x 2 x 2 bytes = 64 x 8 x 128 x 2 x 2 = 262,144 B/token (256 KiB), the same as DeepSeek-R1-Distill-Qwen-32B and 3.7x DeepSeek-V3. head_dim 128 is explicit. gqa_ratio 10 (80 heads / 8 kv heads) - a wider GQA than typical. hidden 5120, intermediate 27648, vocab 201088, tie_word_embeddings false. Our config arithmetic gives 36.6B against a 36B name, consistent. Context: max_position_embeddings 524288 with rope_theta 10000000 and rope_scaling rope_type 'default' with no other parameters - i.e. natively long, not extended. That is a 512k native window on a dense model, which is unusual and is the main serving claim. Confidence 0.85: config, safetensors total and the Apache-2.0 licence are all confirmed from the card frontmatter (license: apache-2.0) and README badge. The card states no explicit active-parameter figure, but this is a dense model so active = total by construction. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. CONTEXT_SCALING_METHOD (native): SCHEME RECLASSIFIED (2026-10-04): was null. rope_scaling {rope_type: 'default'} with rope_theta 1e7 over max_position_embeddings 524288 - a default (identity) rope declaration, i.e. native.