Grok 1

PARAMS: 314B is xAI's own stated figure. ACTIVE IS NULL AND DELIBERATELY SO: neither the model card nor run.py states an active-parameter count. 314B total with 8 experts and 2 selected per token implies roughly 314/8 plus attention and embeddings, but deriving an active count that way assumes perfectly uniform experts and that the attention tower is 1/8 of the model, and xAI does not publish it. Leaving it null rather than guessing. KV: 64 * 8 * 128 * 2 * 2 bytes = 262,144 bytes/token = 256 KiB/token, derived from run.py's num_layers=64, num_kv_heads=8 and key_size=128 (emb_size 48*128 = 6144). CONTEXT IS 8192, which is the headline serving fact: at 128k-era context lengths this model is not deployable at all. 8k on 256 KiB/token is 2 GiB of KV per sequence. WEIGHTS ARE NOT SAFETENSORS. xai-org/grok-1 ships int8 quantized custom ckpt-* shards and has NO config.json; every architecture field here comes from the JAX reference implementation in run.py plus the model card's specification list, cross-checked against each other. The reference MoE layer is explicitly described by xAI as not efficient (chosen to avoid custom kernels). Grok 2 and later, and the Grok Diffusion diffusion LM, are NOT recorded here: their weights were not retrievable in this session. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. CONTEXT_SCALING_METHOD (none-verified): xai-org/grok-1 ships weights with no config.json in the repo (HTTP 404), so no rope declaration exists to classify.