Hidden inference costs: egress, persistent storage, and cold-storage retrieval are not in any headline GPU or token rate
Fields
supply/rent-hidden-costs-egress-storage-coldload-2026-10Notes
The reason this record exists is that none of the headline rates in this slice include any of these, and each one can exceed the compute. THREE ARITHMETIC RESULTS A READER SHOULD BE ABLE TO REPRODUCE. (1) THE MODEL-LOAD COST. Loading one 70B checkpoint at BF16 is roughly 140 GB, at FP8 roughly 70 GB. From Google Coldline at $0.000005479/GiB-hour, 70 GB costs about $0.0038 per HOUR to sit in cold storage - so you can leave a model in Coldline essentially free and still pay real money to pull it: what you cannot avoid is the RESTORE, not the resting, and the cost lands as restore-request charges or as egress, not as the storage line. Put differently, storing weights is nearly free and MOVING them is not, which is why 'we keep our weights in cheap cold storage' is a networking-cost claim wearing a storage-cost disguise. (2) THE STORAGE-FEE-THAT-BITES list price. Azure's Ultra disk at $0.000164/GiB-hour means a 28,000 GiB temp store - the ND H100 v5 node in this repo - costs 28000 x 0.000164 = about $4.59 per HOUR while attached, which is 37% of the $12.29 on-demand H100-hour it serves. Attach a Premium SSD v2 at 28000 GiB and it is $3.08/hour, still 25% of the GPU. That ratio is the answer to 'why is my inference node more expensive than its GPU rate': local NVMe inside the instance is cheap, durable MANAGED disk is not, and an inference deployment that does not need to survive a reboot is being charged for persistence it never uses. (3) EGRESS DOMINATES ONCE THERE ARE MANY USERS. At an AWS-style $0.09/GB-class outbound rate, moving 140 GB of weights once costs about $12.60 - more than two H100-hours on Lambda - so a model-reload-per-tenant or per-cold-start design pays a networking bill that no GPU-hour estimate predicts. Treat egress as part of the architecture decision, not as an afterthought. Caveat on this record's own numbers: only the GCS and Azure figures above were transcribed verbatim from their pages on 2026-10-04. The S3 and AWS egress numbers come from worked examples and general statements on those pages, and the AWS tiered outbound table itself is JS-rendered and was NOT captured - see the pre-existing ppl-sup-aws-egress-negative record for that limitation. price_usd therefore carries the GCS Standard per-GiB-hour rate as the single most defensible number on the page.
EXPORT CONTROL (2026-10-04): NOT APPLICABLE IN THE HARDWARE-SHIPMENT SENSE - this record prices a token-based API, so no accelerator is transferred across a border and the ECCN 3A090 hardware licence regime does not attach to the transaction. Recorded null rather than false precisely because a token API is where export control instead becomes an END-USER/TERMS question: the provider applies geographic and end-user restrictions to service access (the AI Diffusion Rule briefly put a tiered country structure on exactly this kind of service before being rescinded on 2025-05-13), and a customer can still be refused under sanctions or end-use policy regardless of where the compute sits. Anyone reading this record for a restricted destination should look at the provider's terms, not at the EAR. Sources for the regime: spl-bis-2025-05-ai-diffusion-rescission, spl-bis-2026-05-d5-guidance.
EXPORT CONTROL (2026-10-04): NOT APPLICABLE IN THE HARDWARE-SHIPMENT SENSE - this record prices a token-based API, so no accelerator crosses a border and the ECCN 3A090 hardware licence regime does not attach to the transaction. Recorded NULL rather than false precisely because a token API is where export control instead becomes an END-USER and TERMS question: the provider applies geographic and end-user restrictions to service access (the AI Diffusion Rule briefly put a tiered country structure on exactly this kind of service before being rescinded and unenforced from 2025-05-13), and a customer can still be refused under sanctions or end-use policy regardless of where the compute sits. For a restricted destination, read the provider's terms, not the EAR. Sources: spl-bis-2025-05-ai-diffusion-rescission, spl-bis-2026-05-d5-guidance.
LEAD TIME: null - correct and not a gap. A token-priced serverless API is provisioned on request, so there is no lead time to publish; the binding constraint when a model is at capacity is provider-side queuing and any per-account concurrency limit, which this repo records in notes where a vendor publishes one (see the DeepSeek concurrency figures already here). Deliberately NOT recorded: any 'model capacity lead time' estimate, because vendors publish none and the SEO 'GPU lead time' numbers that circulate do not apply to a rented endpoint. Probe log in source spl-negative-no-oem-publishes-lead-time.
Cross-references
This record references no other records — it is a leaf. It is cited by 0.