inference/research
contestedlow · 0.55updated 2026-10-05

Humanity's Last Exam: Cerebras GPT-5.6 Sol Ultrafast vs Claude Fable 5

7.0 x speedup in WALL-CLOCK TIME to complete the same 2500 Humanity's Last Exam questions. BASELINE: Claude Fable 5 with Claude Code at 78h27m; Cerebras GPT-5.6 Sol Ultrafast with Codex at 11h11m. This is a time-to-completion ratio between two DIFFERENT models on DIFFERENT hardware, which is why the record is status: contested and confidence 0.55 - a speedup_ratio is the honest key but it does not make the comparison sound.

Fields

id
benchmarks/alt-cerebras-hle-benchmark
Metric
speedup_ratio
Value
7
Unit
x speedup in WALL-CLOCK TIME to complete the same 2500 Humanity's Last Exam questions. BASELINE: Claude Fable 5 with Claude Code at 78h27m; Cerebras GPT-5.6 Sol Ultrafast with Codex at 11h11m. This is a time-to-completion ratio between two DIFFERENT models on DIFFERENT hardware, which is why the record is status: contested and confidence 0.55 - a speedup_ratio is the honest key but it does not make the comparison sound.
Model
GPT-5.6 Sol Ultrafast vs Claude Fable 5
Methodology
Cerebras ran GPT-5.6 Sol Ultrafast with Codex on xhigh reasoning (July 10) and Claude Fable 5 with Claude Code on xhigh reasoning (July 13-15). Both completed all 2500 HLE questions. Cerebras: 11h11m. Claude: 78h27m. Comparable accuracy.
Measured by
vendor
Reproducible
no
notes
Vendor-commissioned benchmark. The 7x speedup is wall-clock time to complete the same 2500-question benchmark with comparable accuracy. This is a quality + speed benchmark, not a pure throughput measurement. The comparison is between Cerebras hardware running GPT-5.6 Sol and unspecified hardware running Claude Fable 5. METRIC CORRECTION 2026-10-05 (schema authority pass, agent w5-schema). Value UNCHANGED; `metric` moved quality -> speedup_ratio. Both members were live in this record's case and the enum forced a choice between two wrong answers: `quality` implies a score and this is a ratio, and there was no dimensionless member at all. This record is one of the five named in gotcha bench-metric-enum-hides-a-dimensionless-ratio, whose root_cause records that the `quality` catch-all was carrying accuracy deltas, PUE ratios and power ratios at once.

Notes

Vendor-commissioned benchmark. The 7x speedup is wall-clock time to complete the same 2500-question benchmark with comparable accuracy. This is a quality + speed benchmark, not a pure throughput measurement. The comparison is between Cerebras hardware running GPT-5.6 Sol and unspecified hardware running Claude Fable 5.

METRIC CORRECTION 2026-10-05 (schema authority pass, agent w5-schema). Value UNCHANGED; `metric` moved quality -> speedup_ratio. Both members were live in this record's case and the enum forced a choice between two wrong answers: `quality` implies a score and this is a ratio, and there was no dimensionless member at all. This record is one of the five named in gotcha bench-metric-enum-hides-a-dimensionless-ratio, whose root_cause records that the `quality` catch-all was carrying accuracy deltas, PUE ratios and power ratios at once.

Cross-references

Accelerators (1)

Engine (1)

Interconnect (1)