inference/research
verifiedhigh · 0.93updated 2026-10-04

LeaderWorkerSet (LWS) and DisaggregatedSet - Kubernetes SIGs README

repo

Fields

id
sources/dep-k8s-lws-readme
Publisher
Kubernetes SIGs (lws), co-designed with llm-d (CNCF sandbox)
Kind
repo
Accessed
2026-10-04
notes
Fetched 2026-10-04. LWS is 'An API for deploying a group of pods as a unit of replication. It aims to address common deployment patterns of AI/ML inference workloads, especially multi-host inference workloads where the LLM will be sharded and run across multiple devices on multiple nodes.' Features verbatim: 'Group of Pods as a unit' - each replica is a 'super pod'; 'Unique pod identity: Each pod in the group has a unique index from 0 to n-1'; 'Parallel creation: Pods in the group will have the same lifecycle and be created in parallel'; 'Gang Scheduling: Each replica with a group of pods can be scheduled in an all-or-nothing manner (Alpha level, API may change in the future)'; dual templates, one for the leader and one for workers. DisaggregatedSet 'builds on it to add advanced disaggregated workload deployment with support for autoscaling, rollouts and failure handling'. Gang scheduling being ALPHA is the material maturity fact: a multi-node LLM replica is not yet scheduled atomically by upstream Kubernetes, which means partial placement is a state you can observe. KServe's LLMInferenceService exposes exactly this as its multi-node deployment pattern, and vLLM production-stack PR #1007 adds multi-node multiprocessing deployment via LeaderWorkerSet.

Notes

Fetched 2026-10-04. LWS is 'An API for deploying a group of pods as a unit of replication. It aims to address common deployment patterns of AI/ML inference workloads, especially multi-host inference workloads where the LLM will be sharded and run across multiple devices on multiple nodes.' Features verbatim: 'Group of Pods as a unit' - each replica is a 'super pod'; 'Unique pod identity: Each pod in the group has a unique index from 0 to n-1'; 'Parallel creation: Pods in the group will have the same lifecycle and be created in parallel'; 'Gang Scheduling: Each replica with a group of pods can be scheduled in an all-or-nothing manner (Alpha level, API may change in the future)'; dual templates, one for the leader and one for workers. DisaggregatedSet 'builds on it to add advanced disaggregated workload deployment with support for autoscaling, rollouts and failure handling'. Gang scheduling being ALPHA is the material maturity fact: a multi-node LLM replica is not yet scheduled atomically by upstream Kubernetes, which means partial placement is a state you can observe. KServe's LLMInferenceService exposes exactly this as its multi-node deployment pattern, and vLLM production-stack PR #1007 adds multi-node multiprocessing deployment via LeaderWorkerSet.

Cross-references

This record references no other records.