Orca: A Distributed Serving System for Transformer-Based Generative Models

Introduces iteration-level scheduling (continuous batching) so a new request joins the running batch as soon as one sequence finishes, rather than waiting for a whole batch. The scheduling half of the prefill/decode asymmetry problem. The site blocks plain curl (403) but the page was retrieved successfully by the extraction tool.