inference/research
verifiedhigh · 0.95updated 2026-10-04

JetStream README (archived notice)

repo

Fields

id
sources/alt-jetstream-readme
Publisher
Google AI Hypercomputer
Kind
repo
Published
2024-03-01
Accessed
2026-10-04
notes
JetStream is Google's own throughput- and memory-optimized LLM inference engine for XLA devices, starting with TPUs. Its own repo description: 'a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome)'. The README carries an explicit archival notice: core functionality was migrated into the vLLM tpu-inference plugin and JetStream was to be archived (read-only) on 2026-02-01; archival is explicitly NOT deletion. Two reference engine implementations are named: JAX (google/maxtext) and PyTorch (google/jetstream-pytorch). Published as PyPI package google-jetstream. Docs cover GKE deployment, Prometheus observability and JAX profiler integration. Repo metadata via GitHub API 2026-10-04: created 2024-03-01, last push 2026-01-05 (consistent with the archival), 464 stars, 26 open issues, Apache-2.0, not flagged archived by the API.

Notes

JetStream is Google's own throughput- and memory-optimized LLM inference engine for XLA devices, starting with TPUs. Its own repo description: 'a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome)'. The README carries an explicit archival notice: core functionality was migrated into the vLLM tpu-inference plugin and JetStream was to be archived (read-only) on 2026-02-01; archival is explicitly NOT deletion. Two reference engine implementations are named: JAX (google/maxtext) and PyTorch (google/jetstream-pytorch). Published as PyPI package google-jetstream. Docs cover GKE deployment, Prometheus observability and JAX profiler integration. Repo metadata via GitHub API 2026-10-04: created 2024-03-01, last push 2026-01-05 (consistent with the archival), 464 stars, 26 open issues, Apache-2.0, not flagged archived by the API.

Cross-references

This record references no other records.