id
engines/local-executorch
Design goal
edge/mobileminimal-depslow-latencyembedded
Backends
cpucoremlnnapiqnnvulkanxnnpack
Notable features
ExecuTorch is PyTorch's official on-device inference engine, announced 2024Uses .pt2 export format via torch.export - models are exported from PyTorch, not converted from other formatsSupports Android (NNAPI, QNN), iOS (CoreML), and Linux embedded targetsQuantization: supports int8, int4, and uint8 quantization via PyTorch quantization toolsDelegation to hardware backends: CoreML (Apple Neural Engine), NNAPI (Android), QNN (Qualcomm), Vulkan (GPU)Part of the PyTorch ecosystem - seamless transition from training to on-device deployment
Hardware caveats
.pt2 format is PyTorch-specific - no GGUF, ONNX, or safetensors support without conversionLLM support is early - fewer optimized kernels than llama.cpp for transformer architecturesNo continuous batching or paged KV cache - single-stream inference onlyDelegation coverage varies by platform - not all ops are delegated to hardware backendsSmaller community and ecosystem than llama.cpp for local LLM inference
Best for
PyTorch users who want to deploy models on-device without leaving the PyTorch ecosystemAndroid apps using NNAPI or QNN for hardware accelerationiOS apps using CoreML for Neural Engine accelerationModels already in PyTorch format that need mobile deployment
Avoid for
Non-PyTorch models - conversion to PyTorch is requiredMaximum performance on Apple Silicon - MLX is faster on MetalHigh-concurrency serving - no batching or scheduling supportModels only available in GGUF format
notes
ExecuTorch is the newest entry in the on-device inference space, backed by PyTorch/Meta. Its main advantage is seamless PyTorch integration - models can be exported and deployed without format conversion. However, the LLM ecosystem is less mature than llama.cpp, and the .pt2 format is PyTorch-specific. For local LLM inference, MLX (Apple) or llama.cpp (cross-platform) are currently better choices, but ExecuTorch is the right choice for PyTorch-centric workflows. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release v1.5.1 published 2026-09-23, 626 contributors (distinct committers including anonymous), open_issues_count 1309 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 5078 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.