KServe documentation - LLMInferenceService overview (GenAI-first CRD built on llm-d)

Fetched 2026-10-04, version 0.20 docs. 'LLMInferenceService is a Kubernetes Custom Resource Definition (CRD) introduced in KServe as part of its strategic shift towards GenAI-first architecture. Built on the foundation of llm-d ... Features like KV-cache aware scheduling, disaggregated prefill-decode serving, and distributed inference'. DUAL-TRACK STRATEGY, verbatim: 'InferenceService: Optimized for Predictive AI workloads'; 'LLMInferenceService: Purpose-built for Generative AI workloads (Large Language Models)'; 'Can you use InferenceService for LLMs? Yes, but only for basic single-node deployments. Advanced features like prefill-decode separation, multi-node orchestration, and intelligent scheduling are not available.' Composable configuration via LLMInferenceServiceConfig resources for model config, workload templates, ROUTER SETTINGS and scheduler policies. Four named deployment patterns: Single-Node ('less than 7B parameters'), Multi-Node ('Distributed inference with LeaderWorkerSet for medium-large models'), Prefill-Decode ('Disaggregated serving for cost optimization'), DP+EP ('Data and Expert Parallelism for MoE models (Mixtral, DeepSeek-R1)'). Confirms that router behaviour is a first-class configuration surface in the deployment topology, not a deployment detail.