GTE Base English v1.5 · inference-research

Fields

id

models/gte-base

Family

gte

Vendor

Alibaba (Alibaba-NLP)

Release year

2023

Architecture

dense

Params (B)

0.109

Active params (B)

0.109

Hidden size

768

Layers

12

Max context

8,192

Context scaling

NTK-aware RoPE (factor 2.0, theta 500000)

Context scaling method

ntk-aware

Activation

gelu

Attention variant

bidirectional encoder (NewModel), full attention, no causal mask

KV cache (bytes/token)

0

FLOPs per active token

219,000,000

Open weights

yes

License

MIT

embedding dims

768

notes

Encoder: kv_cache_bytes_per_token = 0, KV formula does not apply. Same parameter count as bge-base-en-v1.5 (109M) — both are BERT-base geometry (hidden 768, 12 layers, intermediate 3072). Key difference: max_position_embeddings 8192 vs BGE's 512, using RoPE with NTK scaling. This is the model that showed Alibaba's GTE family could match BGE on retrieval while supporting longer context. Vocab 30528 (6 more than BERT's 30522). pack_qkv true. MIT license. GGUF and ONNX ports exist. CONTEXT_SCALING_METHOD (ntk-aware): rope_scaling {type: 'ntk', factor 2.0} at rope_theta 500000 - NTK-aware base adjustment, a 2x extension from 4096.

task

embedding

Notes

Encoder: kv_cache_bytes_per_token = 0, KV formula does not apply. Same parameter count as bge-base-en-v1.5 (109M) — both are BERT-base geometry (hidden 768, 12 layers, intermediate 3072). Key difference: max_position_embeddings 8192 vs BGE's 512, using RoPE with NTK scaling. This is the model that showed Alibaba's GTE family could match BGE on retrieval while supporting longer context. Vocab 30528 (6 more than BERT's 30522). pack_qkv true. MIT license. GGUF and ONNX ports exist.

CONTEXT_SCALING_METHOD (ntk-aware): rope_scaling {type: 'ntk', factor 2.0} at rope_theta 500000 - NTK-aware base adjustment, a 2x extension from 4096.

Cross-references

This record references no other records — it is a leaf. It is cited by 0.