Jina Embeddings v3
dense 0.572B
Fields
id
models/jina-embeddings-v3Family
jina
Vendor
Jina AI
Release year
2024
Architecture
dense
Params (B)
0.572
Active params (B)
0.572
Hidden size
1,024
Layers
24
Max context
8,194
Context scaling method
native
Activation
gelu
Attention variant
bidirectional encoder (XLM-RoBERTa with LoRA adapters), full attention, no causal mask
KV cache (bytes/token)
0
FLOPs per active token
1,140,000,000
Open weights
yes
License
CC-BY-NC-4.0
embedding dims
1,024
notes
Encoder: kv_cache_bytes_per_token = 0, KV formula does not apply. XLM-RoBERTa-large backbone (hidden 1024, 24 layers, 16 heads, vocab 250002, 8194 max_position_embeddings) with LoRA adapters for 5 tasks: retrieval.query, retrieval.passage, separation, classification, text-matching. LoRA rank 4, alpha 1. Matryoshka dimensions [32,64,128,256,512,768,1024] — can truncate to any of these with minimal quality loss. Rotary embeddings (base 20000). Task-specific prefixes required: 'Represent the query for retrieving evidence documents: ' for queries, 'Represent the document for retrieval: ' for passages. CC-BY-NC-4.0 (non-commercial). GGUF and ONNX ports exist.
task
embedding
Notes
Encoder: kv_cache_bytes_per_token = 0, KV formula does not apply. XLM-RoBERTa-large backbone (hidden 1024, 24 layers, 16 heads, vocab 250002, 8194 max_position_embeddings) with LoRA adapters for 5 tasks: retrieval.query, retrieval.passage, separation, classification, text-matching. LoRA rank 4, alpha 1. Matryoshka dimensions [32,64,128,256,512,768,1024] — can truncate to any of these with minimal quality loss. Rotary embeddings (base 20000). Task-specific prefixes required: 'Represent the query for retrieving evidence documents: ' for queries, 'Represent the document for retrieval: ' for passages. CC-BY-NC-4.0 (non-commercial). GGUF and ONNX ports exist.
Cross-references
This record references no other records — it is a leaf. It is cited by 0.