Portkey AI Gateway - Load balancing (weighted, sticky) documentation

Fetched 2026-10-04. Available on all Portkey plans. Strategy shapes: {"strategy":{"mode":"loadbalance"},"targets":[...]}, targets carry provider or override_params (a specific model) plus a numeric weight. Five documented patterns: Between Providers (model from request), Between Models, Multiple API Keys (same provider, to spread across per-account rate limits), Cost Optimization (cheap vs premium), Gradual Migration (old to new model) - so the gateway's routing decision is expressed as a weighted distribution over heterogeneous targets that may be different vendors or different models. Mechanism: 'Weight normalization: Portkey normalizes weights to sum to 100%. Example: weights 5, 3, 1 become 55%, 33%, 11%.' Defaults weight 1, minimum 0 ('stops traffic without removing from config'). Sticky load balancing: {"sticky":{"enabled":true,"hash_fields":["metadata.user_id"],"ttl":3600}} with default TTL 3600 s; mechanism is hash the listed fields, look up the previous target from cache, and 'Sticky sessions use a two-tier cache system (in-memory + Redis) for fast lookups and persistence across gateway instances in distributed deployments.' Cautions the doc itself gives: 'Each LLM has different latency and pricing' and to monitor usage per LLM because weight distribution affects spend. '@provider-slug/model-name' format routes automatically. This is the purest statement of the gateway layer's added value: weighted routing plus stickiness plus per-target accounting, none of which the engine can do.