License
AGPL-3.0 for KoboldCpp and KoboldAI Lite; bundled GGML, llama.cpp and stable-diffusion.cpp remain MIT
Design goal
ease-of-useminimal-depssingle-file-distributionmulti-modalmulti-model
Backends
cudarocmhipvulkanmetalcpu-avx512cpu-avx2arm64-neon
Formats
ggufggml-legacy-bin
Notable features
One executable, zero install — a GUI launcher opens on double-click and the web UI is served at localhost:5001.Far more than text: the same process serves Stable Diffusion/SDXL/SD3/Flux image generation and editing, video generation (WAN 2.2, LTX2.3), Whisper speech-to-text and multimodal audio, TTS (Qwen3TTS, Kokoro, OuteTTS, Parler, Dia), and music (ACE Step).Bundled KoboldAI Lite WebUI with memory, world info, author notes, character cards, and the classic llama.cpp WebUI.Speaks many API dialects at once: KoboldAI, OpenAI, Ollama, A1111/Forge, ComfyUI, Whisper, XTTS and OpenAI speech clients.An integrated agent (launchable from the Admin tab or with `--agent`) plus MCP server support and tool calling.Can connect the UI to external providers instead of a local model (AI Horde, OpenAI-compatible, Anthropic, OpenRouter, Gemini, Grok, Mistral) via `--nomodel`.
Hardware caveats
Release binaries are backend-specific and this is the single most practical constraint: `koboldcpp.exe` / `koboldcpp-linux-x64` are CUDA builds, `koboldcpp-nocuda.exe` / `koboldcpp-linux-x64-nocuda` are the smaller CPU+Vulkan builds, and `koboldcpp-oldpc.exe` / `-oldpc` exist for older CPUs and GPUs.A dedicated GPU is optional. Model size and context length determine memory need, not GPU presence.AMD: there is no ROCm build in the main release. The README points AMD users at Vulkan in the mainline release as a starting point and lists a third-party ROCm fork (YellowRoseCx/koboldcpp-rocm) whose maintenance status must be checked yourself. Arch users have separate `koboldcpp-cuda` and `koboldcpp-hipblas` AUR packages.macOS binaries are Apple Silicon ARM64 only (`koboldcpp-mac-arm64`); Intel Macs require a source build.For ARM/Termux builds, LLAMA_PORTABLE=1 disables native ARM instruction optimizations — a real speed cost, and the reason it exists is to keep the build working at all.Docker is explicitly discouraged for local use: it targets expert cloud-GPU use on x86-64 Ubuntu, may perform poorly on Windows/macOS, ARM systems may fail, and CPU feature detection can select slower fallback binaries.Backward compatibility with legacy GGML `.bin` models is retained, but some newer features are unavailable for those files.
Best for
Roleplay, creative writing, and story workflows — the bundled KoboldAI Lite UI is built for it and nothing else is this complete.A single-user home box that should do text, images, speech, and music without five separate installs.AMD owners who have no working ROCm path and want a working local GPU experience today via Vulkan.People who want a GUI instead of flags: the launcher exposes every backend and tensor-split setting.Offline creative tooling with zero dependency chain.
Avoid for
Serving many concurrent users — no scheduler, no continuous batching, single model per process.Maximum tok/s on a CUDA box: it is a llama.cpp fork and the tuning surface is not where upstream effort goes.Licenses that forbid AGPL — KoboldCpp itself is AGPL-3.0 even though its bundled llama.cpp parts are MIT.Anything needing a production-grade API surface; it is a UI-first personal tool.
notes
As of access the default branch is `concedo`, not master. The project maintains a `koboldcpp.com` phishing warning, which is a reasonable signal about ecosystem maturity. Treated here as a distribution/UX engine over llama.cpp kernels; the ggml backend matrix and quantization-type coverage belong to the llama.cpp record. I did not find a primary-source statement of how much slower the Vulkan path is than native CUDA/HIP for this project, so no number is claimed. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-02 (0.1 months), latest release v1.122.1 published 2026-09-26, 2112 contributors (distinct committers INCLUDING ANONYMOUS - a heavily anonymous-contributed project, so read as 'active committers', not as a team of 2112 people), open_issues_count 529 (issues PLUS pull requests), NOT archived, 11943 stars. DEPLOYMENT: NO EVIDENCE OF PRODUCTION DEPLOYMENT FOUND as of 2026-10-04 - see [[ev-production-evidence-is-scarce-and-nearly-always-single-site]]. Note that this is a weaker negative than for a small library: the README publishes an offline batch-inference throughput TUNING GUIDE with a worked example (a log line showing 'Decode batch. #running-req: 233, #token: 370959, token usage: 0.82, cuda graph: True, gen throughput (token/s): 4594.01, #queue-req: 317'), which is evidence that operators tune this engine seriously but is explicitly a configuration guide, not a report of anyone's deployment. FEATURE AXES: all seven UNSTATED. Worth noting for this record's unusual breadth (text, image, video, ASR, TTS and music from one process): 'chunked prefill' and 'disaggregated prefill/decode' are questions that may not have a meaningful answer for a single-user local GUI-launched process, which is itself worth recording rather than leaving blank.