Jina Reranker v2 Base Multilingual

CROSS-ENCODER RERANKER: kv_cache_bytes_per_token = 0, KV formula does not apply. As with every reranker here, N passages means N independent full forward passes over query+passage pairs with no KV reuse across pairs — prefill-like and compute-bound, cost linear in passages x pair length. Unlike a decoder, the query prefix gains nothing from being the same across the 100 pairs. The BASE (not the 0.6B multilingual variant) at 278437633 params: XLM-RoBERTa-sized, hidden 768, 12 layers, intermediate 3072, vocab 250002 — half the depth and 55% the params of bge-reranker-v2-m3, so it is roughly 2x cheaper per pair, which is the tradeoff this record exists to make explicit. 1026 max_position_embeddings (1024 usable) — over BERT's 512, well under bge-reranker-v2-m3's 8194. Multilingual. LICENSE: CC-BY-NC-4.0, non-commercial, same constraint as jina-embeddings-v3 — the Jina family is the strongest multilingual multifunctionality available in this slice and the least commercially usable. Check the license before the benchmark. 986k downloads. GGUF ports DO exist (gpustack, cstr, and community Q4_K_M / Q8_0 builds) — unusually good llama.cpp coverage for a cross-encoder; no first-party ONNX port found. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=cross-encoder. REQUIRED KERNEL CLASS: N independent full forward passes over (query + passage) with no cache reuse and no early exit - flop/cross-encoder-reranking derives AI = t at batch 1 (544 flop/byte for BERT-base geometry at t=544, i.e. 1.84x ABOVE H100's 295.4 ridge) and AI = N*t when batched. It is compute-bound at batch 1, which makes buying FLOPS rather than bandwidth the correct procurement decision, and it is the one common serving workload where the usual quantization advice inverts: quantization helps only because it doubles peak FLOPS, not because it cuts bytes. Cost scales as candidates x sequence length. SUPPORTED BY (cited): vLLM pooling's cross-encoder row list is explicit - BertForSequenceClassification, GemmaForSequenceClassification (BAAI/bge-reranker-v2-gemma), GteNewForSequenceClassification, LlamaBidirectionalForSequenceClassification, ModernBertForSequenceClassification, Qwen2ForSequenceClassification (mixedbread-ai/mxbai-rerank-base-v2), Qwen3ForSequenceClassification (Qwen/Qwen3-Reranker-0.6B), RobertaForSequenceClassification, XLMRobertaForSequenceClassification (BAAI/bge-reranker-v2-m3) - with per-model score templates shipped for the rerankers that need a specific prompt format [[vllm-pooling-models]]. TEI has reranking first-class since v0.4.0 for CamemBERT, RoBERTa, XLM-RoBERTa and GTE sequence-classification cross-encoders at /rerank [[hf-text-embeddings-inference]]. SGLang splits the class in two and says so: 'Cross-encoder rerank models: run with --is-embedding (embedding runner)' vs 'Decoder-only rerank models: run WITHOUT --is-embedding and use next-token logprob scoring (yes/no)' - and its BGE-reranker row carries a hard kernel constraint, 'Currently only support attention-backend triton and torch_native', with the example launch passing --attention-backend triton, --disable-radix-cache and --chunked-prefill-size -1 [[mdl2-sglang-embedding-rerank-models]]. THE CLASS BOUNDARY THAT DECIDES ENGINE CHOICE: a cross-encoder is NOT late interaction. ColBERT-style multi-vector scoring is a different kernel class - a MaxSim over token-level embedding matrices, bandwidth- and capacity-bound on the index rather than compute-bound in the encoder, scaling with corpus size rather than batch size [[colbert-reference]]. No mainstream generation engine here serves it except vLLM, whose token_embed pooling type plus late-interaction MaxSim scoring is the only first-class ColBERT endpoint in the repo [[vllm-pooling-models]]. TEI cannot: its pooling options are cls, mean, splade and last-token, with no token-wise output mode, so 'a ColBERT model cannot be served correctly here' [[hf-text-embeddings-inference]]. llama.cpp-embedding-server cannot either: --pooling none gives raw per-token output, 'which is a building block for ColBERT but not a served ColBERT' [[llama-cpp-embedding-server]]. BATCHING TRAP, cited: reranking through Triton's sequence batcher requires modelling the query-document pair as ONE stateful unit rather than two independent sequences [[triton-inference-server]] - get that wrong and the pair is scored separately.