Jamba-1.5-Large

GEOMETRY FILLED 2026-10-17 from the public mradermacher Q2_K GGUF header [[gguf-jamba-1-5-large-q2k]], because the official config.json is still gated (HTTP 401). Parsed, not inferred: block_count 72, embedding_length 8192, feed_forward_length 24576, attention.head_count 64, head_count_kv = 8 at layers 4,12,20,28,36,44,52,60,68 and 0 on the other 63 layers, ssm.state_size 16, ssm.inner_size 16384, ssm.conv_kernel 4, ssm.time_step_rank 512, context_length 262144, expert_count 16, expert_used_count 2. head_dim is not a header field; 8192/64 = 128. gqa_ratio = 64/8 = 8. KV at bf16 counts only the 9 attention layers: 9 x 8 kv heads x 128 head_dim x 2 (K+V) x 2 bytes = 36,864 B/token = 36 KiB/token. The 63 Mamba layers keep a fixed state (state 16, inner 16384) that does not grow with context and is not in this number. This is a third-party conversion of ai21labs/AI21-Jamba-Large-1.5, not AI21 own config.json. DEPLOYMENT: llama.cpp runs it — mradermacher ships Q2_K through Q8_0 and IQ4_XS GGUF (Q2_K is 143.8 GB). vLLM serves the family (JambaForCausalLM, fused MoE, ExpertsInt8) per the Jamba-1.5 paper. No ollama tag was found. The 94B active figure and the 8x-smaller-KV claim stay the paper numbers [[arch-jamba-15]]. ai21labs/AI21-Jamba-Large-1.5 returned HTTP 401 on a direct config read this session, so the attention geometry of the Jamba-1.5-Large could not be machine-verified. Left null. The sibling LFM2 and Falcon-H1 hybrids in this set show that the pattern (num_key_value_heads 2-8, head_dim 64-128 on a minority of layers) is knowable only from the config, not inferred from the family, so no value is assumed here. 94B active parameters per the Jamba-1.5 paper (arXiv:2408.12570). The original Jamba (arXiv:2403.19887) states the design goal: 'Trading off attention layers for Mamba layers reduces the total size of the KV cache. Our architecture aims to provide not only a small number of active parameters but also an 8x smaller KV cache compared to a vanilla Transformer of comparable size.' The 52B Jamba-v0.1 configuration uses roughly a 1:3 Mamba:Transformer block ratio. Jamba-1.5 adds ExpertsInt8 quantization. 256K effective context. The exact layer counts and KV head geometry for Jamba-1.5-Large could not be verified from the config.json (HuggingFace returns 404 for the raw config), so kv_cache_bytes_per_token is null. The 8x KV reduction claim is relative to a vanilla Transformer of comparable size, not an absolute number. CONTEXT_SCALING_METHOD (none-verified): ai2/jamba-1.5-large config.json is gated (HTTP 401) to an unauthenticated fetch; context is 256k per the record but the rope declaration could not be read.