Jamba-1.5: Hybrid Transformer-Mamba Models at Scale

Scaling a hybrid Transformer-Mamba MoE to larger sizes exposed that naive hybrid and MoE choices do not survive at scale, and the models needed a serving-friendly expert quantisation to be affordable.

Instruction-tuned successors of Jamba at two sizes: Jamba-1.5-Large with 94B active parameters and Jamba-1.5-Mini with 12B active, both with an effective context of 256K tokens, on the same interleaved Transformer + Mamba MoE backbone as Jamba. The serving-relevant new mechanism is ExpertsInt8, a quantisation scheme for the MoE expert weights only, which is what makes the 94B-active model affordable to serve: quantising experts rather than attention lets the quantisation error stay confined to the FFN blocks while keeping the attention path at higher precision. Layer structure is inherited from Jamba, so KV cache is again proportional only to the attention layers.

arXiv preprint; no journal-ref on the abs page. 256K effective context is described as the largest among open-weight models at the time of release, which is a context-length claim and not a throughput claim. speedup_reported is null because the paper reports quality benchmarks plus the ExpertsInt8 compression ratio and does not state a tok/s speedup with baseline and settings. Engine support inherited and verified: vLLM native JambaForCausalLM lists AI21-Jamba-1.5-Large and AI21-Jamba-1.5-Mini; llama.cpp has LLM_ARCH_JAMBA in llm_arch_is_hybrid(). The expert-only int8 idea is a recurring pattern worth cross-referencing against other MoE quantisation records. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.