inference/research
verifiedhigh · 0.85updated 2026-10-04

Jina Embeddings v4

dense 3.24B

Fields

id
models/jina-embeddings-v4
Family
jina
Vendor
Jina AI
Release year
2025
Architecture
dense
Params (B)
3.24
Active params (B)
3.24
Hidden size
2,048
Layers
36
KV heads
2
Head dim
128
GQA ratio
8
Max context
128,000
Context scaling method
mrope
Activation
silu
Attention variant
causal decoder (Qwen2.5-VL text backbone) used as a bi-encoder via mean pooling, with multi-vector projector
KV cache (bytes/token)
36,864
FLOPs per active token
6,480,000,000
Open weights
yes
License
CC-BY-NC-4.0
embedding dims
2,048
notes
THE MULTIMODAL EMBEDDER: Qwen2.5-VL text backbone (hidden 2048, 36 layers, 16 heads, 2 KV heads, vocab 151936, 128000 max_position_embeddings, sliding_window 32768) with a multi-vector projector (dim 128) for ColBERT-style late interaction. Matryoshka dims [128,256,512,1024,2048]. Task names: retrieval, text-matching, code. KV cache applies (36 * 2 * 128 * 2 * 2 = 18432 bytes/token at bf16) but is irrelevant for embedding (single forward pass, mean pooling). 3.24B params, 6.48e9 flops/token — GPU-class work. Supports both single-vector and multi-vector output. CC-BY-NC-4.0 (non-commercial). GGUF and ONNX ports exist. KV BYTES CORRECTED (2026-10-04): stored 18432 was 0.500x the value its own stated derivation produces. 36*2*128*2*2 = 36864, not 18432; the stored figure was exactly half. Arithmetic re-done from the same config fields; no source disagreement, so the correction stands without contest. CONTEXT_SCALING_METHOD (mrope): rope_scaling carries mrope_section [16,24,24]; Qwen2.5-VL text backbone.
task
embedding

Notes

THE MULTIMODAL EMBEDDER: Qwen2.5-VL text backbone (hidden 2048, 36 layers, 16 heads, 2 KV heads, vocab 151936, 128000 max_position_embeddings, sliding_window 32768) with a multi-vector projector (dim 128) for ColBERT-style late interaction. Matryoshka dims [128,256,512,1024,2048]. Task names: retrieval, text-matching, code. KV cache applies (36 * 2 * 128 * 2 * 2 = 18432 bytes/token at bf16) but is irrelevant for embedding (single forward pass, mean pooling). 3.24B params, 6.48e9 flops/token — GPU-class work. Supports both single-vector and multi-vector output. CC-BY-NC-4.0 (non-commercial). GGUF and ONNX ports exist.

KV BYTES CORRECTED (2026-10-04): stored 18432 was 0.500x the value its own stated derivation produces. 36*2*128*2*2 = 36864, not 18432; the stored figure was exactly half. Arithmetic re-done from the same config fields; no source disagreement, so the correction stands without contest.

CONTEXT_SCALING_METHOD (mrope): rope_scaling carries mrope_section [16,24,24]; Qwen2.5-VL text backbone.

Cross-references

This record references no other records — it is a leaf. It is cited by 0.