Text Embeddings Inference GitHub repository

Fields

id

sources/niche-tei-repo

URL

https://github.com/huggingface/text-embeddings-inference

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-03

notes

TEI README, v1.9 line. Embeddings/reranking/classifier server in Rust with a Python router; claims no model graph compilation step, small images for serverless, token-based dynamic batching, safetensors AND onnx weight loading, and OpenTelemetry + Prometheus. Documented hardware matrix: CPU x86_64 and aarch64 images; Volta explicitly NOT SUPPORTED; Turing image experimental with Flash Attention OFF by default due to precision issues; Ampere 8.0/8.6, Ada, Hopper images; Blackwell 100/120/121 experimental. States GPUs with CUDA compute capability < 7.5 (V100, Titan V, GTX 1000 series) are unsupported and CUDA 12.2+ drivers required. Separate ROCm images for MI200/MI300. Metal for local Mac use. API metadata 2026-10-03: created 2023-10-13, last push 2026-09-29, v1.9.4 (2026-09-15), Apache-2.0, 457 open issues, 5069 stars.

Notes

TEI README, v1.9 line. Embeddings/reranking/classifier server in Rust with a Python router; claims no model graph compilation step, small images for serverless, token-based dynamic batching, safetensors AND onnx weight loading, and OpenTelemetry + Prometheus. Documented hardware matrix: CPU x86_64 and aarch64 images; Volta explicitly NOT SUPPORTED; Turing image experimental with Flash Attention OFF by default due to precision issues; Ampere 8.0/8.6, Ada, Hopper images; Blackwell 100/120/121 experimental. States GPUs with CUDA compute capability < 7.5 (V100, Titan V, GTX 1000 series) are unsupported and CUDA 12.2+ drivers required. Separate ROCm images for MI200/MI300. Metal for local Mac use. API metadata 2026-10-03: created 2023-10-13, last push 2026-09-29, v1.9.4 (2026-09-15), Apache-2.0, 457 open issues, 5069 stars.

Cross-references

This record references no other records.