exo GitHub repository · inference-research

Fields

id

sources/niche-exo-repo

URL

https://github.com/exo-explore/exo

Publisher

exo labs

Kind

repo

Accessed

2026-10-03

notes

exo README: turns a set of consumer devices into one AI cluster, with automatic device discovery, day-0 RDMA over Thunderbolt 5 (claimed 99% latency reduction between devices), topology-aware automatic model parallelisation, and tensor parallelism. Vendor speedups to treat as claims: up to 1.8x on 2 devices and 3.2x on 4. Screenshots show 4x M3 Ultra Mac Studio running Qwen3-235B, DeepSeek v3.1 671B and Kimi K2 Thinking at 8-bit/native 4-bit. macOS/Apple Silicon oriented (MLX, Metal ToolChain from Xcode, macmon for monitoring; the README notes macmon 0.6.1 crashes on Apple M5). API metadata 2026-10-03: created 2024-06-24, last push 2026-10-03, v1.0.71 (2026-04-23), Apache-2.0, 949 open issues, 47731 stars.

Notes

exo README: turns a set of consumer devices into one AI cluster, with automatic device discovery, day-0 RDMA over Thunderbolt 5 (claimed 99% latency reduction between devices), topology-aware automatic model parallelisation, and tensor parallelism. Vendor speedups to treat as claims: up to 1.8x on 2 devices and 3.2x on 4. Screenshots show 4x M3 Ultra Mac Studio running Qwen3-235B, DeepSeek v3.1 671B and Kimi K2 Thinking at 8-bit/native 4-bit. macOS/Apple Silicon oriented (MLX, Metal ToolChain from Xcode, macmon for monitoring; the README notes macmon 0.6.1 crashes on Apple M5). API metadata 2026-10-03: created 2024-06-24, last push 2026-10-03, v1.0.71 (2026-04-23), Apache-2.0, 949 open issues, 47731 stars.

Cross-references

This record references no other records.