Worth Running

Models ranked by capability against what they actually need. Two numbers that are usually conflated, kept apart here. Resident is the whole checkpoint at 4-bit — what has to fit somewhere in VRAM plus system RAM. Read / token is only the weights the decode step touches, which is what speed scales with and is far smaller for a mixture-of-experts model: a 600B MoE reads 13.5 GB per token and still needs a 279 GB file. Need adds the KV cache forone user or for 16 concurrent users at an 8k context, the part that grows with users. Offload marks models that stay fast when the routed experts sit in system RAM.

Capability is relative, not a raw average. Each benchmark is scored against the best result recorded for that same benchmark in the catalog, so 100 means "best known on this test" and a model tested on AIME and MMLU-Pro can be compared with one tested on SWE-bench and Terminal-Bench. Averaging raw percentages would instead rank a model by which tests it happened to be run on. The line under each name lists the tests and their relative scores; thin marks a ranking carried by only one or two benchmarks, and vendor marks scores the lab published about itself.

Scores are the model at its default serving configuration, not its ceiling. This matters where a lab publishes several reasoning levels: gpt-oss sets reasoning_effort to mediumwhen you do not pass one, and its published high column is a test-time-compute ceiling — ranking on that put gpt-oss-20b above models it does not actually beat. Where a source documents the setting it is recorded on the entry; where it publishes only a ceiling, the record says so.