inference/research
verifiedhigh · 0.90updated 2026-10-04

vLLM TPU inference plugin (tpu-inference) - GitHub README

repo

Fields

id
sources/alt-tpu-vllm-inference-readme
URL
https://github.com/vllm-project/tpu-inference
Publisher
vllm-project
Kind
repo
Published
2025-05-06
Accessed
2026-10-04
notes
vLLM's TPU hardware plugin. Key facts read from the README: (1) tpu-inference is a single hardware plugin inside the vLLM project that unifies JAX and PyTorch under ONE lowering path, so PyTorch model definitions run on TPU with no code changes; announced 2025/10. (2) Compatible TPU generations: recommended v7x (Ironwood), v5e, v6e (Trillium); experimental v3, v4, v5p. (3) Ships a public support-matrix dashboard with per-model Unit Test / Correctness Test / Performance Test status, generated from support_matrices/ in-repo, last updated 2026-08-27 - a rarer and more honest release than vendors usually publish. (4) Explicitly states that out-of-the-box high performance is NOT yet universal: core components are still being implemented, so Recommended Models and Recommended Features pages list what is actually validated. (5) Recipes exist for v7x and v6e under AI-Hypercomputer/tpu-recipes. (6) Repo metadata via GitHub API 2026-10-04: created 2025-05-06, pushed 2026-10-04, 453 stars, 329 forks, 531 open issues, Apache-2.0.

Notes

vLLM's TPU hardware plugin. Key facts read from the README: (1) tpu-inference is a single hardware plugin inside the vLLM project that unifies JAX and PyTorch under ONE lowering path, so PyTorch model definitions run on TPU with no code changes; announced 2025/10. (2) Compatible TPU generations: recommended v7x (Ironwood), v5e, v6e (Trillium); experimental v3, v4, v5p. (3) Ships a public support-matrix dashboard with per-model Unit Test / Correctness Test / Performance Test status, generated from support_matrices/ in-repo, last updated 2026-08-27 - a rarer and more honest release than vendors usually publish. (4) Explicitly states that out-of-the-box high performance is NOT yet universal: core components are still being implemented, so Recommended Models and Recommended Features pages list what is actually validated. (5) Recipes exist for v7x and v6e under AI-Hypercomputer/tpu-recipes. (6) Repo metadata via GitHub API 2026-10-04: created 2025-05-06, pushed 2026-10-04, 453 stars, 329 forks, 531 open issues, Apache-2.0.

Cross-references

This record references no other records.