inference/research
verifiedmedium · 0.80updated 2026-10-04

LiteRT (TFLite successor)

cpu, nnapi, gpu-delegate, coreml, hexagon, xnnpack

Fields

id
engines/local-litert
Repo
tensorflow/tensorflow
Languages
C++JavaKotlinSwiftPython
License
Apache-2.0
First release
2025
Design goal
edge/mobileminimal-depslow-latencyembedded
Scheduling
none
Backends
cpunnapigpu-delegatecoremlhexagonxnnpack
Formats
tflite
Notable features
LiteRT is the successor to TensorFlow Lite, announced 2025Maintains .tflite model format - existing TFLite models are compatibleSupports Android (NNAPI, GPU delegate, Hexagon), iOS (CoreML, Metal), and embedded LinuxSmaller binary footprint than TFLite - stripped unused ops and delegatesSupports quantization: int8, int4, float16Part of the TensorFlow ecosystem - seamless transition from TF training to mobile deployment
Hardware caveats
.tflite format is TensorFlow-specific - no GGUF, ONNX, or safetensors support without conversionLLM support is limited - TensorFlow Lite is primarily for vision and small modelsNo continuous batching or paged KV cache - single-stream inference onlyGPU delegate performance varies significantly by device and driverSmaller LLM ecosystem than llama.cpp or MLX
Best for
TensorFlow users who want to deploy models on-device without leaving the TF ecosystemAndroid apps using NNAPI or GPU delegate for hardware accelerationiOS apps using CoreML for Neural Engine accelerationVision and small models on mobile - TFLite's traditional strength
Avoid for
Large language models - TFLite/LiteRT is not optimized for LLM inferenceNon-TensorFlow models - conversion to TensorFlow is requiredMaximum performance on Apple Silicon - MLX is faster on MetalHigh-concurrency serving - no batching or scheduling support
Docs
https://www.tensorflow.org/lite
notes
LiteRT is the rebranded TensorFlow Lite, announced 2025. It maintains compatibility with existing .tflite models while reducing binary footprint. For local LLM inference, LiteRT is not the primary choice - it is optimized for vision and small models. However, it is the right choice for TensorFlow-centric workflows and for models that fit within its optimization envelope. For LLM-specific work, llama.cpp, MLX, or ONNX Runtime Mobile are better choices. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release v2.21.0 published 2026-03-06, 5353 contributors (distinct committers including anonymous), open_issues_count 3275 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 200700 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set. ||| DEPLOYMENT: NO EVIDENCE OF PRODUCTION DEPLOYMENT FOUND as of 2026-10-04 - repositories, READMEs, docs and issue trackers were searched. See [[ev-production-evidence-is-scarce-and-nearly-always-single-site]] for the grading scheme; note that for an artifact-evaluation or paper-code release, non-deployment is documented intent rather than a negative finding.

Notes

LiteRT is the rebranded TensorFlow Lite, announced 2025. It maintains compatibility with existing .tflite models while reducing binary footprint. For local LLM inference, LiteRT is not the primary choice - it is optimized for vision and small models. However, it is the right choice for TensorFlow-centric workflows and for models that fit within its optimization envelope. For LLM-specific work, llama.cpp, MLX, or ONNX Runtime Mobile are better choices. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release v2.21.0 published 2026-03-06, 5353 contributors (distinct committers including anonymous), open_issues_count 3275 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 200700 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set. ||| DEPLOYMENT: NO EVIDENCE OF PRODUCTION DEPLOYMENT FOUND as of 2026-10-04 - repositories, READMEs, docs and issue trackers were searched. See [[ev-production-evidence-is-scarce-and-nearly-always-single-site]] for the grading scheme; note that for an artifact-evaluation or paper-code release, non-deployment is documented intent rather than a negative finding.

Cross-references

This record references no other records — it is a leaf. It is cited by 7.