deploybase

Google Cloud vs AWS vs Azure GPU Pricing Comparison

Deploybase · April 17, 2025 · GPU Pricing

Contents

Google Cloud vs AWS GPU Pricing

Google cloud vs aws gpu pricing matters. Azure too. Costs differ by 20-30% depending on instance type, region, and commitment. Pick wrong and developers waste $60k+ annually. This guide shows the math so developers don't.

Instance Pricing Comparison

AWS P5 (8x H100, 640GB): ~$55.04/hour on-demand = ~$6.88/hour per GPU. NVLink 4.0 bandwidth is excellent; includes 400Gbps EFA networking.

Azure ND H100 v5 (8x H100): $88.49/hour = $11.06/hour per GPU.

Google Cloud A3 (8x H100): ~$88.49/hour = ~$11.06/hour per GPU.

Ordering by on-demand cost: AWS cheapest, GCP and Azure similarly priced. AWS offers the most competitive H100 pricing among hyperscalers.

The Math

100 hours/month (8x H100 node, on-demand):

  • AWS: $5,504
  • GCP: $8,849
  • Azure: $8,849

AWS wins by a significant margin. GCP and Azure are similarly priced.

1,000 hours/month:

  • AWS: $55,040
  • GCP: $88,490
  • Azure: $88,490

AWS saves $33,000-$33,500 monthly vs GCP and Azure.

24/7 (730 hours):

  • AWS: $40,179
  • GCP: $64,598
  • Azure: $64,598

AWS is roughly $24,000/month cheaper than GCP and Azure for H100.

Reserved Instances and Commitments

AWS RIs (p5.48xlarge, 8x H100) 1-year: ~$38.53/hour (~30% off on-demand), 3-year: estimated ~$33/hour (~40% off)

Azure Reservations (ND H100 v5, 8x H100) 1-year: ~$62/hour (~30% off), 3-year: ~$44/hour (~50% off)

GCP Committed Use Discounts (a3-highgpu-8g, 8x H100) 1-year: ~$61.94/hour (30% off), 3-year: ~$53.10/hour (40% off)

AWS remains cheapest even with commitments. GCP and Azure offer similar committed rates.

Spot/Preemptible Pricing

AWS Spot (p5.48xlarge): ~$16.51/hour (~70% off on-demand $55.04/hr). Can get interrupted.

Azure Spot (ND H100 v5): ~$26.55/hour (~70% off $88.49/hr). Same terms.

GCP Preemptible (a3-highgpu-8g): Not available on A3 instances.

Note: GCP does not offer preemptible/spot pricing on A3 H100 instances. AWS and Azure spot pricing provides significant savings for fault-tolerant workloads.

Data Transfer and Egress Costs

Egress: All charge $0.02/GB (GCP sometimes $0.03).

10TB/day example: 300TB/month = $6,000/month. Not small.

Intra-region: Free on all three.

Rule: Same region data and compute always. Cross-region kills budgets.

VPN: $0.05/hour or $30-40/month on all three.

Direct connections (Direct Connect, ExpressRoute, Interconnect): $0.30/hour+ depending on bandwidth. Pricier but lower latency if mission-critical.

Total Cost of Ownership Scenarios

Development (100 GPU hours/month, 8x H100 node + 10GB transfer):

  • AWS on-demand: $5,504 + $2 egress ≈ $5,506
  • Azure on-demand: $8,849 + $2 egress ≈ $8,851
  • GCP on-demand: $8,849 + $1 egress ≈ $8,850

AWS wins by a large margin. GCP and Azure are similar.

Production inference (3,000 hours/month + 500GB transfer, 1-year commitment):

  • AWS 1-year reserved: ~$38.53 × 3,000 + $10 = $115,600
  • GCP 1-year CUD: ~$61.94 × 3,000 + $15 = $185,835
  • Azure 1-year reserved: ~$62 × 3,000 + $10 = $186,010

AWS saves $70,000+ monthly vs GCP and Azure with commitments.

Large fine-tuning (10,000 hours + 500GB egress, 1-year commitment):

  • AWS 1-year reserved: ~$38.53 × 10,000 + $10 = $385,310
  • GCP 1-year CUD: ~$61.94 × 10,000 + $15 = $619,415
  • Azure 1-year reserved: ~$62 × 10,000 + $10 = $620,010

AWS saves $235,000+ over 10,000 hours vs hyperscaler alternatives. For sustained H100 workloads, AWS is the cheapest hyperscaler.

Regional Pricing Variations

Same rate across zones in a region usually, but some zones cost more.

AWS: US East baseline, West 5-10% more, EU 10-15% more, Asia 20-30% more.

Azure: US baseline, EU 10-15% more, Asia 20-30% more.

Deploy in cheap regions when latency allows. Latency usually matters less than cost.

Provider Selection Framework

Pick AWS if: locked into AWS already, need multi-region, need SageMaker, willing to pay more for ecosystem.

Pick Azure if: already invested in Microsoft ecosystem, need AD integration, have Azure ML already, or compliance requirements favor Microsoft.

Pick GCP if: using GCP ecosystem tools (BigQuery, Vertex AI, TPUs), prefer Google ML tools, or need A100 single-GPU flexibility (a2-highgpu-1g at $3.67/hr).

Pick multi-cloud if: can't afford single-provider outages, workloads are portable, operational complexity OK.

FAQ

Which provider is cheapest for long-running LLM inference? AWS P5 instances are cheapest among hyperscalers for H100 on-demand ($55.04/hr for 8 GPUs). For specialized providers, RunPod (~$2.69/hr per H100) and Vast.ai are significantly cheaper than any hyperscaler. Check current pricing before committing to long-term contracts.

Can we move between providers easily? Yes, models and code are portable. Infrastructure changes take 2-4 weeks. Data transfer costs are 1-2% of total migration cost. Switching is feasible but not trivial.

Do committed use discounts lock us in? Yes. Reservations and CUDs are non-refundable in most cases. Break-even is usually 3-6 months. Only commit if confident in 1-3 year usage requirements.

What's the sweet spot for GPU cluster size? 8-GPU clusters (single node) simplify operations. 16-64 GPU clusters gain efficiency but require distributed training complexity. Stay with 8-GPU nodes until scaling pain is obvious.

Sources

  • AWS P5 instance pricing (March 2026)
  • Azure ND H100 instance pricing (March 2026)
  • Google Cloud A3 instance pricing (March 2026)
  • Reserved instance discount structures
  • Spot and preemptible pricing analysis
  • 2026 cloud GPU cost optimization guides