Nous-Hermes-2-Mixtral-8x7B-DPO

CONTEXT SCALING: null is a VERIFIED value, checked against config rather than left blank. config.json declares rope_scaling null, rope_theta 1000000, max_position_embeddings 32768. MoE record: params_b 46.70 is the safetensors total, active_params_b 12.9 is Mixtral's published activated-parameter figure for this architecture (2 of 8 experts per token plus attention, router and embeddings), NOT a value read from config.json - config.json only states num_local_experts 8 and num_experts_per_tok 2. Do not conflate the two: all 46.70B of weights must be resident, while per-token compute is ~12.9B. fine-tuned from mistralai/Mixtral-8x7B-v0.1 via OpenHermes-2.5 then DPO; _name_or_path names NousResearch/OpenHermes-2.5-Mixtral-8x7B-epoch4. hidden 4096, 32 layers, 32 query heads / 8 KV heads, rope_theta 1000000, 32768 native context, vocab 32002. head_dim derived as 4096 / 32 = 128; gqa_ratio = 4. KV bytes/token at bf16 (2 bytes/element): 32 layers x 8 KV heads x 128 head_dim x 2 (K and V) x 2 bytes = 131072 B/token = 128.0 KiB/token. KV is per-token modest (1.75 GiB at its native 32k context), but 46.7B weights at bf16 (93.4 GB) is the binding constraint - a 2x80GB SXM minimum before any KV at all. context_scaling is null: context length is native to the pretrained config, no rope_scaling. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card.