Orca: A Distributed Serving System for Transformer-Based Generative Models (OSDI 2022)

Fields

id

sources/sched-orca-osdi22

URL

https://www.usenix.org/conference/osdi22/presentation/yu

Publisher

USENIX Association

Kind

paper

Published

2022-07

Accessed

2026-10-04

Archived URL

https://www.usenix.org/system/files/osdi22-yu.pdf

notes

The paper that introduced iteration-level scheduling. Key result: 36.9x throughput improvement over NVIDIA FasterTransformer at the same latency for GPT-3 175B. Also introduced selective batching (batching only a subset of operations). The OSDI '22 presentation page was fetched and verified.

Notes

The paper that introduced iteration-level scheduling. Key result: 36.9x throughput improvement over NVIDIA FasterTransformer at the same latency for GPT-3 175B. Also introduced selective batching (batching only a subset of operations). The OSDI '22 presentation page was fetched and verified.

Cross-references

This record references no other records.