inference/research
verifiedhigh · 0.90updated 2026-10-04

sgl-model-gateway WASM rate-limit example README and its documented limitations

repo

Fields

id
sources/ops-sglang-wasm-ratelimit-example
Publisher
SGLang project
Kind
repo
Published
main branch as of 2026-10-04
Accessed
2026-10-04
notes
Defaults: 60 requests per minute per identifier; identifier priority API Key then IP Address then Request ID; returns 429 Too Many Requests when the limit is exceeded. The Limitations section is the operator-relevant part, verbatim: 'Per-instance state (not shared across workers)', 'No cross-process state sharing', 'Memory growth with unique identifiers', 'State lost on instance restart'. The Overview adds: 'This is a simplified demonstration. Since WASM components are stateless, each worker thread maintains its own counter. For production, implement rate limiting at the router/host level with shared state.' The companion wasm-guest-auth example returns 401 for missing or invalid keys on /api and /v1, accepts Authorization Bearer, Authorization ApiKey and x-api-key, and hardcodes a single EXPECTED_API_KEY compiled into the guest module.

Notes

Defaults: 60 requests per minute per identifier; identifier priority API Key then IP Address then Request ID; returns 429 Too Many Requests when the limit is exceeded. The Limitations section is the operator-relevant part, verbatim: 'Per-instance state (not shared across workers)', 'No cross-process state sharing', 'Memory growth with unique identifiers', 'State lost on instance restart'. The Overview adds: 'This is a simplified demonstration. Since WASM components are stateless, each worker thread maintains its own counter. For production, implement rate limiting at the router/host level with shared state.' The companion wasm-guest-auth example returns 401 for missing or invalid keys on /api and /v1, accepts Authorization Bearer, Authorization ApiKey and x-api-key, and hardcodes a single EXPECTED_API_KEY compiled into the guest module.

Cross-references

This record references no other records.