nomic-ai/nomic-embed-text-v2-moe config.json

NomicBertModel with MoE. n_embd 768, n_layer 12, n_head 12, n_positions 2048, max_trained_positions 2048, causal FALSE. MoE: num_experts 8, moe_top_k 2, moe_every_n_layers 2 (so 6 of 12 layers carry MoE FFNs), num_shared_experts 0, moe_impl megablocks, ffn_div 1. num_key_value_heads and head_dim absent (bidirectional encoder, KV cache does not apply); 768/12 = 64 if a head dim is needed. Read raw 2026-10-04.