inference/research
verifiedmedium · 0.80updated 2026-10-04

Hidden inference costs: egress, persistent storage, and cold-storage retrieval are not in any headline GPU or token rate

multi-cloud (AWS us-east-1, Google Cloud us-west4, Azure) - the rates are per-cloud and per-region

Fields

id
supply/rent-hidden-costs-egress-storage-coldload-2026-10
Kind
cloud
Region
multi-cloud (AWS us-east-1, Google Cloud us-west4, Azure) - the rates are per-cloud and per-region
Channels
hidden-costegressstorage
Price (USD)
0
Price basis
MIXED UNITS - the price_usd field carries ONE number, the Google Cloud Storage Standard rate, and the rest of the record carries the others with their units stated. (a) Google Cloud Storage per-GiB-per-HOUR, us-west4, observed 2026-10-04: Standard $0.000027397, Nearline $0.000013699, Coldline $0.000005479, Archive $0.000001644, Rapid Cache $0.0001233. Storage is 'prorated to the sub-second for each object'. (b) Azure Managed Disk per-GiB-per-HOUR: Premium SSD v2 $0.00011, Ultra $0.000164; the same page's monthly table gives Premium SSD LRS from $0.01825/GiB-month and Standard SSD LRS from $0.01533/GiB-month, and snapshot storage on Standard at $0.05/GB-month, on Premium SSD at $0.12/GB-month. (c) Amazon S3 per GB-MONTH: the page's worked examples use $0.06/GB-month for Standard storage and $0.20/GB for a full upload; GET $0.0004 per 1,000 requests and PUT $0.005 per 1,000 requests on S3 Tables, $0.10/GB/day file-search storage is on the OpenAI page not here. (d) AWS Data Transfer Out to the internet: 100 GB free per month aggregated across all AWS services and regions (except China and GovCloud), then rate-tiered PERISHABLE-FIGURE STAMP (added 2026-10-04, uniform across data/supply/ so it is greppable). PRICE CLASS: PUBLISHED LIST PRICE - an asking rate a new customer is quoted, NOT a realised, discounted or negotiated transaction rate. A real bill is at or below this and typically well below it once commitments, credits and committed-use discounts apply. OBSERVED: 2026-10-04 RE-VERIFY BY: 2027-01-04 - the repo's quarterly re-verification bar. After that date this figure is presumed drifted and must be re-checked against its source before it is quoted again. A price without its observation date is noise; this stamp exists so a reader can tell a fresh figure from a stale one without reading prose. UNIT: MIXED UNITS READING RULE: MAJOR UNIT WARNING: this record carries MULTIPLE UNITS at once. price_usd is the single number named first in price_basis and every other figure carries its own unit. Do not mix a per-GiB-per-hour storage rate with a per-GB-month rate, or a per-1M-token rate with a per-GPU-hour rate.
Availability
unknown
Lead time source
structural-na
Lead time basis
lead_time_weeks is null because NO NUMBER APPLIES, and the reason is specific rather than generic: this record prices a metered service term (tokens, egress, storage) rather than hardware capacity, so there is no order-to-delivery interval to report at all. Calling this 'no published lead time' would be wrong in the other direction - nothing was withheld.
Export control regime
no EAR citation: no controlled item is transferred by this transaction; not an EAR citation: provider service terms and end-use policy (the AI Diffusion Rule briefly tiered service access before its 2025-05-13 rescission)
Export control note
MECHANISM: a token-priced API, so no accelerator crosses a border and the 3A090 hardware licence regime does not attach to the transaction. Export control becomes an END-USER AND TERMS question instead: the provider applies geographic and end-use restrictions to service access, and a customer can still be refused under sanctions or end-use policy wherever the compute sits. A reader targeting a restricted destination should read the provider's terms, not the EAR.
notes
The reason this record exists is that none of the headline rates in this slice include any of these, and each one can exceed the compute. THREE ARITHMETIC RESULTS A READER SHOULD BE ABLE TO REPRODUCE. (1) THE MODEL-LOAD COST. Loading one 70B checkpoint at BF16 is roughly 140 GB, at FP8 roughly 70 GB. From Google Coldline at $0.000005479/GiB-hour, 70 GB costs about $0.0038 per HOUR to sit in cold storage - so you can leave a model in Coldline essentially free and still pay real money to pull it: what you cannot avoid is the RESTORE, not the resting, and the cost lands as restore-request charges or as egress, not as the storage line. Put differently, storing weights is nearly free and MOVING them is not, which is why 'we keep our weights in cheap cold storage' is a networking-cost claim wearing a storage-cost disguise. (2) THE STORAGE-FEE-THAT-BITES list price. Azure's Ultra disk at $0.000164/GiB-hour means a 28,000 GiB temp store - the ND H100 v5 node in this repo - costs 28000 x 0.000164 = about $4.59 per HOUR while attached, which is 37% of the $12.29 on-demand H100-hour it serves. Attach a Premium SSD v2 at 28000 GiB and it is $3.08/hour, still 25% of the GPU. That ratio is the answer to 'why is my inference node more expensive than its GPU rate': local NVMe inside the instance is cheap, durable MANAGED disk is not, and an inference deployment that does not need to survive a reboot is being charged for persistence it never uses. (3) EGRESS DOMINATES ONCE THERE ARE MANY USERS. At an AWS-style $0.09/GB-class outbound rate, moving 140 GB of weights once costs about $12.60 - more than two H100-hours on Lambda - so a model-reload-per-tenant or per-cold-start design pays a networking bill that no GPU-hour estimate predicts. Treat egress as part of the architecture decision, not as an afterthought. Caveat on this record's own numbers: only the GCS and Azure figures above were transcribed verbatim from their pages on 2026-10-04. The S3 and AWS egress numbers come from worked examples and general statements on those pages, and the AWS tiered outbound table itself is JS-rendered and was NOT captured - see the pre-existing ppl-sup-aws-egress-negative record for that limitation. price_usd therefore carries the GCS Standard per-GiB-hour rate as the single most defensible number on the page. EXPORT CONTROL (2026-10-04): NOT APPLICABLE IN THE HARDWARE-SHIPMENT SENSE - this record prices a token-based API, so no accelerator is transferred across a border and the ECCN 3A090 hardware licence regime does not attach to the transaction. Recorded null rather than false precisely because a token API is where export control instead becomes an END-USER/TERMS question: the provider applies geographic and end-user restrictions to service access (the AI Diffusion Rule briefly put a tiered country structure on exactly this kind of service before being rescinded on 2025-05-13), and a customer can still be refused under sanctions or end-use policy regardless of where the compute sits. Anyone reading this record for a restricted destination should look at the provider's terms, not at the EAR. Sources for the regime: spl-bis-2025-05-ai-diffusion-rescission, spl-bis-2026-05-d5-guidance. EXPORT CONTROL (2026-10-04): NOT APPLICABLE IN THE HARDWARE-SHIPMENT SENSE - this record prices a token-based API, so no accelerator crosses a border and the ECCN 3A090 hardware licence regime does not attach to the transaction. Recorded NULL rather than false precisely because a token API is where export control instead becomes an END-USER and TERMS question: the provider applies geographic and end-user restrictions to service access (the AI Diffusion Rule briefly put a tiered country structure on exactly this kind of service before being rescinded and unenforced from 2025-05-13), and a customer can still be refused under sanctions or end-use policy regardless of where the compute sits. For a restricted destination, read the provider's terms, not the EAR. Sources: spl-bis-2025-05-ai-diffusion-rescission, spl-bis-2026-05-d5-guidance. LEAD TIME: null - correct and not a gap. A token-priced serverless API is provisioned on request, so there is no lead time to publish; the binding constraint when a model is at capacity is provider-side queuing and any per-account concurrency limit, which this repo records in notes where a vendor publishes one (see the DeepSeek concurrency figures already here). Deliberately NOT recorded: any 'model capacity lead time' estimate, because vendors publish none and the SEO 'GPU lead time' numbers that circulate do not apply to a rented endpoint. Probe log in source spl-negative-no-oem-publishes-lead-time.

Notes

The reason this record exists is that none of the headline rates in this slice include any of these, and each one can exceed the compute. THREE ARITHMETIC RESULTS A READER SHOULD BE ABLE TO REPRODUCE. (1) THE MODEL-LOAD COST. Loading one 70B checkpoint at BF16 is roughly 140 GB, at FP8 roughly 70 GB. From Google Coldline at $0.000005479/GiB-hour, 70 GB costs about $0.0038 per HOUR to sit in cold storage - so you can leave a model in Coldline essentially free and still pay real money to pull it: what you cannot avoid is the RESTORE, not the resting, and the cost lands as restore-request charges or as egress, not as the storage line. Put differently, storing weights is nearly free and MOVING them is not, which is why 'we keep our weights in cheap cold storage' is a networking-cost claim wearing a storage-cost disguise. (2) THE STORAGE-FEE-THAT-BITES list price. Azure's Ultra disk at $0.000164/GiB-hour means a 28,000 GiB temp store - the ND H100 v5 node in this repo - costs 28000 x 0.000164 = about $4.59 per HOUR while attached, which is 37% of the $12.29 on-demand H100-hour it serves. Attach a Premium SSD v2 at 28000 GiB and it is $3.08/hour, still 25% of the GPU. That ratio is the answer to 'why is my inference node more expensive than its GPU rate': local NVMe inside the instance is cheap, durable MANAGED disk is not, and an inference deployment that does not need to survive a reboot is being charged for persistence it never uses. (3) EGRESS DOMINATES ONCE THERE ARE MANY USERS. At an AWS-style $0.09/GB-class outbound rate, moving 140 GB of weights once costs about $12.60 - more than two H100-hours on Lambda - so a model-reload-per-tenant or per-cold-start design pays a networking bill that no GPU-hour estimate predicts. Treat egress as part of the architecture decision, not as an afterthought. Caveat on this record's own numbers: only the GCS and Azure figures above were transcribed verbatim from their pages on 2026-10-04. The S3 and AWS egress numbers come from worked examples and general statements on those pages, and the AWS tiered outbound table itself is JS-rendered and was NOT captured - see the pre-existing ppl-sup-aws-egress-negative record for that limitation. price_usd therefore carries the GCS Standard per-GiB-hour rate as the single most defensible number on the page.

EXPORT CONTROL (2026-10-04): NOT APPLICABLE IN THE HARDWARE-SHIPMENT SENSE - this record prices a token-based API, so no accelerator is transferred across a border and the ECCN 3A090 hardware licence regime does not attach to the transaction. Recorded null rather than false precisely because a token API is where export control instead becomes an END-USER/TERMS question: the provider applies geographic and end-user restrictions to service access (the AI Diffusion Rule briefly put a tiered country structure on exactly this kind of service before being rescinded on 2025-05-13), and a customer can still be refused under sanctions or end-use policy regardless of where the compute sits. Anyone reading this record for a restricted destination should look at the provider's terms, not at the EAR. Sources for the regime: spl-bis-2025-05-ai-diffusion-rescission, spl-bis-2026-05-d5-guidance.

EXPORT CONTROL (2026-10-04): NOT APPLICABLE IN THE HARDWARE-SHIPMENT SENSE - this record prices a token-based API, so no accelerator crosses a border and the ECCN 3A090 hardware licence regime does not attach to the transaction. Recorded NULL rather than false precisely because a token API is where export control instead becomes an END-USER and TERMS question: the provider applies geographic and end-user restrictions to service access (the AI Diffusion Rule briefly put a tiered country structure on exactly this kind of service before being rescinded and unenforced from 2025-05-13), and a customer can still be refused under sanctions or end-use policy regardless of where the compute sits. For a restricted destination, read the provider's terms, not the EAR. Sources: spl-bis-2025-05-ai-diffusion-rescission, spl-bis-2026-05-d5-guidance.

LEAD TIME: null - correct and not a gap. A token-priced serverless API is provisioned on request, so there is no lead time to publish; the binding constraint when a model is at capacity is provider-side queuing and any per-account concurrency limit, which this repo records in notes where a vendor publishes one (see the DeepSeek concurrency figures already here). Deliberately NOT recorded: any 'model capacity lead time' estimate, because vendors publish none and the SEO 'GPU lead time' numbers that circulate do not apply to a rented endpoint. Probe log in source spl-negative-no-oem-publishes-lead-time.

Cross-references

This record references no other records — it is a leaf. It is cited by 0.