deploybase

Azure GPU Cloud Pricing: Complete Guide for Every GPU (March 2026)

Deploybase · June 25, 2025 · GPU Pricing

Contents

Azure GPU Pricing Overview

Azure offers GPU compute through multiple instance families. Pricing sits between AWS and specialized providers like Vast.AI.

The tradeoff: Azure costs more per GPU-hour than Vast.AI or RunPod. But developers get Microsoft service integration, compliance certifications, and global availability. Worth it for production workloads tied to the Azure ecosystem.

Here's what matters: instance families, pricing structure, optimization tactics.

Azure Instance Families for ML

ND series (ND A100 v4, ND H100 v5): NVIDIA A100 and H100 SXM GPUs (training focus) NC series (NC A100 v4, NC H100 v5): NVIDIA A100 and H100 GPUs (general ML) NV series: NVIDIA A10, L40S GPUs (visualization and inference focus) NDm series: Multi-node configurations with H100/A100

ND series handles modern ML training. NC is legacy but still available. NV targets inference. NDm supports distributed training.

Pricing Structure

Azure bills hourly for VMs. Commitment discounts available for 1 or 3 years. Additional costs include storage, data transfer, and managed services.

Base VM charges don't include:

  • Storage: $0.10-0.25/GB/month depending on type
  • Data transfer: $0.02/GB outbound
  • IP addresses: $2.88/month if static

Total costs exceed GPU hourly rates. Plan architecture carefully.

Azure Instance Pricing by Type

ND H100 v5 Instances (Training)

Standard_ND96isr_H100_v5:

  • 8x H100 SXM (80GB): ~$88.49/hr
  • Per H100: ~$11.06/hr
  • Memory: 1900GB RAM
  • CPUs: 96 vCPUs
  • Storage: Premium SSD capable

8-GPU instance costs roughly $88.49/hr. Single H100 NVL availability: $6.98/hr (Standard_NC40ads_H100_v5). Most large-scale work uses full 8-GPU ND instances or multi-node clusters.

Cost comparison:

  • Azure ND H100 v5: $11.06/hr per GPU (8-pack)
  • Azure NC H100 NVL: $6.98/hr (single GPU)
  • AWS p5 H100: $6.88/hr per GPU (p5.48xlarge)
  • RunPod H100: $2.69/hr
  • Vast.AI H100: $2.00-3.50/hr

Azure ND H100 is significantly more expensive than AWS per GPU. Azure NC H100 NVL single GPU ($6.98) is comparable to AWS p5 per-GPU cost ($6.88).

ND A100 v4 Instances (Training)

Standard_ND96asr_v4:

  • 8x A100 SXM (80GB): ~$28.50/hr
  • Per A100: ~$3.56/hr
  • Memory: 900GB RAM
  • CPUs: 96 vCPUs

A100 instances are generally cheaper than H100 but slower for large-scale training.

Cost comparison:

  • Azure ND A100 v4: $3.56/hr per GPU
  • AWS p4d.24xlarge A100: $3.83/hr per GPU
  • RunPod A100: $1.39/hr
  • Vast.AI A100: $0.80-1.50/hr

Azure is slightly cheaper than AWS for A100 but more expensive than specialized providers.

NV A10 Instances (Inference)

Standard_NV36ads_A10_v5:

  • 1x NVIDIA A10 (24GB): ~$2.88/hr
  • Memory: 110GB RAM
  • CPUs: 36 vCPUs

A10 for inference workloads. 24GB VRAM handles most inference models. Pricing is reasonable for production inference.

Larger instances:

  • Standard_NV72ads_A10_v5 (2x A10): ~$5.76/hr

GPU scaling is linear. Adding GPUs roughly doubles cost.

Legacy NC V100 Instances

Standard NC24s_v3:

  • 4x V100 (16GB): ~$3.95/hr per GPU
  • Memory: 448GB RAM
  • CPUs: 24 vCPUs

V100 is older. Prices reflect legacy hardware. Not recommended for new work unless locked into existing infrastructure.

Regional Pricing Variations

Azure pricing varies significantly by region:

East US: ~$88.49/hr for 8x H100 West US 2: ~$94.68/hr for 8x H100 (+7%) Europe West: ~$105.30/hr for 8x H100 (+19%) Southeast Asia: ~$115.04/hr for 8x H100 (+30%) Australia East: ~$117.69/hr for 8x H100 (+33%)

US regions are cheapest. European and Asia-Pacific regions cost 20-33% more. For non-latency-critical work, US regions always win.

Commitment Discounts

Azure offers reserved instances for cost reduction:

1-Year Commitment

ND H100 (8x):

  • On-demand: $88.49/hr
  • Reserved: ~$57.52/hr (-35%)

Annual cost: $57.52 × 8,760 = $503,875

Significant savings for committed workloads.

3-Year Commitment

ND H100 (8x):

  • On-demand: $88.49/hr
  • Reserved: ~$42.48/hr (-52%)

3-year cost: $42.48 × 8,760 × 3 = $1,116,518

Better per-hour rate than 1-year but requires long-term commitment.

Spot Pricing

Azure Spot VMs provide 70-80% discounts:

ND H100 Spot:

  • On-demand: $88.49/hr
  • Spot: ~$17.70/hr (80% discount)

Spot is excellent for interruptible workloads. Research, testing, and checkpointed training work well. Production serving doesn't.

Compute Savings Plan

Azure's flexible commitment model:

  • Commitment to compute dollars for 1-3 years
  • 17% discount (1-year)
  • 28% discount (3-year)

More flexible than reserved instances. Switch between regions and instance types.

Cost Optimization Strategies

1. Use Spot for non-critical work

80% discount on Spot is dramatic. Development, testing, non-critical training: use it.

2. Right-size instances

Don't rent 8x GPU instances if developers only need 1 GPU. Rent single instances (if available) or NV series for inference.

3. Commit if workload is predictable

Continuous training? Reserve instances. One-off projects? Pay hourly.

4. Minimize data transfer

Keep data in Azure Storage. Use same region for compute and storage. Avoid downloading to local machines.

5. Stop instances when idle

Stopped instances don't charge for compute. Storage still charges. Stop when not training.

6. Use appropriate instance family

ND for training. NV for inference. Don't use expensive ND for inference work.

7. Monitor and audit costs

Azure Cost Management tools track spending. Set up alerts. Monitor for unused resources.

Total Cost of Ownership

Example: ND H100 v5 (8x H100 SXM) reserved for 1 year

  • GPU: $57.52/hr × 8,760 = $503,875
  • Storage (1TB): $0.10 × 1000 × 12 = $1,200
  • Data transfer (100GB/month): 0.02 × 100 × 12 = $24
  • IP address (static): $2.88 × 12 = $35
  • Total: ~$505,134/year

Compare to Vast.AI on-demand:

  • 8x H100 on Vast.AI: ~$2.50 × 8 = $20/hr = ~$175,200/year

Vast.AI is significantly cheaper than Azure reserved pricing for full utilization. Azure advantage is integration, compliance, and SLAs; Vast.AI advantage is dramatically lower cost with no upfront commitment.

For single GPU workloads, Vast.AI is 3-5x cheaper than Azure on-demand.

Azure vs Competitors

H100 GPU pricing (March 2026):

  • Azure ND H100 v5: $11.06/hr per GPU (8-pack, $88.49/hr total)
  • Azure NC H100 NVL: $6.98/hr (single GPU)
  • AWS p5: $6.88/hr per GPU ($55.04/hr total for 8x)
  • RunPod: $2.69/hr
  • Vast.AI: $2.00-3.50/hr

A100 GPU pricing:

  • Azure ND A100 v4: $3.56/hr per GPU
  • AWS p4d.24xlarge: $3.83/hr per GPU
  • RunPod: $1.39/hr
  • Vast.AI: $0.80-1.50/hr

A10/A10G pricing:

  • Azure NV A10 v5: $2.88/hr per GPU
  • AWS g5 (A10G): $1.87/hr per GPU
  • RunPod A10: ~$0.74/hr
  • Vast.AI: $0.50-0.90/hr

Azure is mid-range. Cheaper than AWS, more expensive than specialized providers. Production integration is the main advantage.

Data Transfer Costs

Azure Data Transfer pricing:

  • Outbound to internet: $0.09/GB
  • Outbound to other regions: $0.02/GB
  • Inbound: Free
  • Between services in same region: Free

Keep data regional. Downloading 100GB of model weights to local machine costs $9. Unnecessary transfers add up.

Storage Options

Azure Storage pricing varies by type:

Hot storage: $0.0184/GB/month ($18/TB/month) Cool storage: $0.0073/GB/month ($7/TB/month) Archive storage: $0.0036/GB/month (~$3.60/TB/month)

Use hot storage for active datasets. Archive for backups. Select based on access patterns.

Integration with Azure Services

Azure's advantage over specialized providers is integration:

  • Azure ML: Built-in MLOps, automated training, deployment
  • Azure Synapse: Data warehouse + ML pipeline integration
  • Azure Cognitive Services: Pre-built ML models
  • Azure DevOps: CI/CD integration for ML

If the workflow uses these services, Azure is efficient. If pure ML compute, specialized providers are cheaper.

FAQ

Is Azure cheaper than AWS for GPU?

Roughly similar pricing. Azure slightly cheaper on ND series. AWS cheaper on smaller g instances. Production considerations push toward both.

Can I use Azure for casual ML projects?

Yes, but it's expensive. Specialized providers like Vast.AI are 50-70% cheaper for single-GPU work.

What's the best way to start with Azure GPU?

Use Spot instances for testing. Reserved instances for committed workloads. Compute Savings Plans for flexibility.

How do I minimize Azure GPU costs?

Use Spot. Commit for predictable workloads. Right-size instances. Stop when idle. Keep data regional.

Can I mix instance types in Azure?

Distributed training usually requires identical hardware. Mixing complicates orchestration. Keep consistent.

Should I use Azure or Vast.AI?

Azure if integrating with broader Azure services or needing compliance certifications. Vast.AI if pure ML compute on a budget.

How do I estimate Azure GPU costs?

(GPU hours × hourly rate) + storage + data transfer. Budget conservatively. Use Azure's pricing calculator.

Sources

  • Azure VM Pricing (as of March 2026)
  • Azure Instance Type Specifications
  • Azure Commitment Discounts Documentation
  • Spot Pricing Data (March 2026)

Last updated: March 2026. Pricing reflects market rates as of March 22, 2026.