AcruxCore docs blog - How much overhead does an LLM gateway add? (five-path controlled measurement)

Measured 2026-07-31, vendor's own gateway, published with methodology - treat as vendor_claim but the method is sound and reusable. Five paths all hit the SAME upstream (OpenAI gpt-4o-mini) with the same account, key, prompt and parameters (max_tokens 5, temperature 0), so model time is a shared constant and the residual is the path. Medians of 60 INTERLEAVED rounds with rotating order and a discarded warm-up, pooled connections, production build not dev server, reporting medians with 95% CI plus p95/p99. Result: BYO no-trace 753 ms, OpenAI direct 757 ms, BYO tracing on 766 ms, LOCAL GATEWAY 825 ms, HOSTED GATEWAY 1320 ms. Gateway software cost, computed per-round so shared blips cancel: 'Local gateway - OpenAI direct = +42 ms (95% CI +17 ms to +81 ms)' - described as '~40 milliseconds for routing, resolving your model name, applying your key, recording the call, and accounting the cost. On a request that already takes ~750 ms of model time, the gateway is roughly 5% of what you were already going to wait.' BYO SDK vs direct: -8.5 ms (CI -41.6 to +14.2), 'the interval straddles zero ... the SDK adds nothing measurable to a direct provider call.'