Infinity embedding/reranking server README

Fields

id

sources/embed-infinity-embeddings-github

URL

https://github.com/michaelfeil/infinity

Publisher

michaelfeil

Kind

repo

Accessed

2026-10-03

notes

Infinity (michaelfeil/infinity) README. MIT. Backends: PyTorch, optimum (ONNX/TensorRT) and CTranslate2, with FlashAttention on NVIDIA CUDA, AMD ROCm, CPU, AWS INF2 and Apple MPS. Dynamic batching, tokenization in worker threads, FastAPI, OpenAI-aligned OpenAPI spec. Note the separate infiniflow/infinity repo is the Milvus-ecosystem vector DATABASE, a different project with a confusingly similar name.

Notes

Infinity (michaelfeil/infinity) README. MIT. Backends: PyTorch, optimum (ONNX/TensorRT) and CTranslate2, with FlashAttention on NVIDIA CUDA, AMD ROCm, CPU, AWS INF2 and Apple MPS. Dynamic batching, tokenization in worker threads, FastAPI, OpenAI-aligned OpenAPI spec. Note the separate infiniflow/infinity repo is the Milvus-ecosystem vector DATABASE, a different project with a confusingly similar name.

Cross-references

This record references no other records.