Deploybase CLIGPU + LLM pricing in your terminal.
dbdeploybase

GPU PRICING

Google Cloud vs AWS vs Azure GPU Pricing Comparison

Compare GPU costs across Google Cloud, AWS, and Azure for LLM inference and training. Pricing data and recommendations.

By Deploybase · April 17, 2025

Contents

Google Cloud vs AWS GPU Pricing

Google cloud vs aws gpu pricing matters. Azure too. Costs differ by 20-30% depending on instance type, region, and commitment. Pick wrong and teams waste $60k+ annually. This guide shows the math to help avoid that.

Instance Pricing Comparison

AWS P5 (8x H100, 640GB): ~$55.04/hour on-demand = ~$6.88/hour per GPU. NVLink 4.0 bandwidth is excellent; includes 400Gbps EFA networking.

Azure ND H100 v5 (8x H100): $88.49/hour = $11.06/hour per GPU.

Google Cloud A3 (8x H100): ~$88.49/hour = ~$11.06/hour per GPU.

Ordering by on-demand cost: AWS cheapest, GCP and Azure similarly priced. AWS offers the most competitive H100 pricing among hyperscalers.

The Math

100 hours/month (8x H100 node, on-demand):

  • AWS: $5,504
  • GCP: $8,849
  • Azure: $8,849

AWS wins by a significant margin. GCP and Azure are similarly priced.

1,000 hours/month:

  • AWS: $55,040
  • GCP: $88,490
  • Azure: $88,490

AWS saves $33,000-$33,500 monthly vs GCP and Azure.

24/7 (730 hours):

  • AWS: $40,179
  • GCP: $64,598
  • Azure: $64,598

AWS is roughly $24,000/month cheaper than GCP and Azure for H100.

Reserved Instances and Commitments

AWS RIs (p5.48xlarge, 8x H100) 1-year: ~$38.53/hour (~30% off on-demand), 3-year: estimated ~$33/hour (~40% off)

Azure Reservations (ND H100 v5, 8x H100) 1-year: ~$62/hour (~30% off), 3-year: ~$44/hour (~50% off)

GCP Committed Use Discounts (a3-highgpu-8g, 8x H100) 1-year: ~$61.94/hour (30% off), 3-year: ~$53.10/hour (40% off)

AWS remains cheapest even with commitments. GCP and Azure offer similar committed rates.

Spot/Preemptible Pricing

AWS Spot (p5.48xlarge): ~$16.51/hour (~70% off on-demand $55.04/hr). Can get interrupted.

Azure Spot (ND H100 v5): ~$26.55/hour (~70% off $88.49/hr). Same terms.

GCP Preemptible (a3-highgpu-8g): Not available on A3 instances.

Note: GCP does not offer preemptible/spot pricing on A3 H100 instances. AWS and Azure spot pricing provides significant savings for fault-tolerant workloads.

Data Transfer and Egress Costs

Egress: All charge $0.02/GB (GCP sometimes $0.03).

10TB/day example: 300TB/month = $6,000/month. Not small.

Intra-region: Free on all three.

Rule: Same region data and compute always. Cross-region kills budgets.

VPN: $0.05/hour or $30-40/month on all three.

Direct connections (Direct Connect, ExpressRoute, Interconnect): $0.30/hour+ depending on bandwidth. Pricier but lower latency if mission-critical.

Total Cost of Ownership Scenarios

Development (100 GPU hours/month, 8x H100 node + 10GB transfer):

  • AWS on-demand: $5,504 + $2 egress ≈ $5,506
  • Azure on-demand: $8,849 + $2 egress ≈ $8,851
  • GCP on-demand: $8,849 + $1 egress ≈ $8,850

AWS wins by a large margin. GCP and Azure are similar.

Production inference (3,000 hours/month + 500GB transfer, 1-year commitment):

  • AWS 1-year reserved: ~$38.53 × 3,000 + $10 = $115,600
  • GCP 1-year CUD: ~$61.94 × 3,000 + $15 = $185,835
  • Azure 1-year reserved: ~$62 × 3,000 + $10 = $186,010

AWS saves $70,000+ monthly vs GCP and Azure with commitments.

Large fine-tuning (10,000 hours + 500GB egress, 1-year commitment):

  • AWS 1-year reserved: ~$38.53 × 10,000 + $10 = $385,310
  • GCP 1-year CUD: ~$61.94 × 10,000 + $15 = $619,415
  • Azure 1-year reserved: ~$62 × 10,000 + $10 = $620,010

AWS saves $235,000+ over 10,000 hours vs hyperscaler alternatives. For sustained H100 workloads, AWS is the cheapest hyperscaler.

Regional Pricing Variations

Same rate across zones in a region usually, but some zones cost more.

AWS: US East baseline, West 5-10% more, EU 10-15% more, Asia 20-30% more.

Azure: US baseline, EU 10-15% more, Asia 20-30% more.

Deploy in cheap regions when latency allows. Latency usually matters less than cost.

Provider Selection Framework

Pick AWS if: locked into AWS already, need multi-region, need SageMaker, willing to pay more for ecosystem.

Pick Azure if: already invested in Microsoft ecosystem, need AD integration, have Azure ML already, or compliance requirements favor Microsoft.

Pick GCP if: using GCP ecosystem tools (BigQuery, Vertex AI, TPUs), prefer Google ML tools, or need A100 single-GPU flexibility (a2-highgpu-1g at $3.67/hr).

Pick multi-cloud if: can't afford single-provider outages, workloads are portable, operational complexity OK.

FAQ

Which provider is cheapest for long-running LLM inference? AWS P5 instances are cheapest among hyperscalers for H100 on-demand ($55.04/hr for 8 GPUs). For specialized providers, RunPod (~$2.69/hr per H100) and Vast.ai are significantly cheaper than any hyperscaler. Check current pricing before committing to long-term contracts.

Can we move between providers easily? Yes, models and code are portable. Infrastructure changes take 2-4 weeks. Data transfer costs are 1-2% of total migration cost. Switching is feasible but not trivial.

Do committed use discounts lock us in? Yes. Reservations and CUDs are non-refundable in most cases. Break-even is usually 3-6 months. Only commit if confident in 1-3 year usage requirements.

What's the sweet spot for GPU cluster size? 8-GPU clusters (single node) simplify operations. 16-64 GPU clusters gain efficiency but require distributed training complexity. Stay with 8-GPU nodes until scaling pain is obvious.

Sources

  • AWS P5 instance pricing (March 2026)
  • Azure ND H100 instance pricing (March 2026)
  • Google Cloud A3 instance pricing (March 2026)
  • Reserved instance discount structures
  • Spot and preemptible pricing analysis
  • 2026 cloud GPU cost optimization guides