KServe LLMInferenceService documentation

Fields

id

sources/serve-kserve-docs-llmisvc

URL

https://kserve.github.io/website/docs/getting-started/genai-first-llmisvc

Publisher

KServe

Kind

whitepaper

Accessed

2026-10-04

notes

LLMInferenceService is a Kubernetes CRD designed specifically for LLMs with advanced scheduling and routing capabilities including prefix-aware routing, disaggregated serving, and fine-grained GPU scheduling. Creates InferencePool, InferenceModel, and EPP (Endpoint Picker Pod) resources. Uses Envoy Gateway for traffic routing.

Notes

LLMInferenceService is a Kubernetes CRD designed specifically for LLMs with advanced scheduling and routing capabilities including prefix-aware routing, disaggregated serving, and fine-grained GPU scheduling. Creates InferencePool, InferenceModel, and EPP (Endpoint Picker Pod) resources. Uses Envoy Gateway for traffic routing.

Cross-references

This record references no other records.