Design goal
edge/mobileminimal-depslow-latencyembedded
Notable features
ncnn is a high-performance neural network inference framework optimized for mobile platformsUses its own model format converted from ONNX/Caffe via ncnn2onnx and onnx2ncnn toolsVulkan GPU acceleration on Android - the primary GPU backendARM NEON CPU optimization for mobile processorsNo dependencies beyond Vulkan - extremely small binary footprintBattle-tested by Tencent in consumer products (WeChat, QQ, etc.)
Hardware caveats
Closed model format - .ncnn is proprietary, converted from ONNX/CaffeNo GGUF, safetensors, or direct PyTorch support - conversion requiredVulkan-only GPU backend - no CUDA, Metal, or OpenCL on iOSNo continuous batching or paged KV cache - single-stream inference onlyLLM support is limited - primarily designed for vision and small modelsNo iOS GPU acceleration - Vulkan is Android-only
Best for
Android apps requiring Vulkan GPU acceleration with minimal dependenciesMobile vision models (detection, segmentation, face recognition)Embedded Linux devices with Vulkan supportApplications where binary footprint is critical
Avoid for
iOS deployment - no Metal backend, Vulkan is Android-onlyLarge language models - not optimized for LLM inferenceCUDA or ROCm hardware - no desktop GPU supportHigh-concurrency serving - no batching or scheduling support
notes
ncnn is Tencent's mobile inference framework, optimized for Android with Vulkan GPU acceleration. It is distinct from MNN (Alibaba) in that it uses Vulkan as its primary GPU backend rather than OpenCL. For local LLM inference, ncnn is not the primary choice - it is optimized for vision and small models. However, it is the right choice for Android-centric mobile deployment with minimal dependencies. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release 20260526 published 2026-05-26, 386 contributors (distinct committers including anonymous), open_issues_count 1223 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 23906 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.