deploybase

H100 on Azure: Pricing, Specs & How to Rent

Deploybase · May 12, 2025 · GPU Pricing

Contents

H100 Azure Pricing

H100 on Azure runs $3.00-$4.50/hr (single GPU) as of March 2026. ND96asr_v4 gives developers 8x H100 PCIe. Spot instances cut costs 50-70% but terminate without warning. Availability varies by region.

H100 Specs

  • 80GB HBM3 memory
  • 3.35 TB/s memory bandwidth
  • 900GB/s NVLink (8x per node)
  • FP32: 67 TFLOPS
  • Tensor: 989 TFLOPS

Enough for 70B+ parameter training. Compare to A100 for smaller models or H200 for higher bandwidth.

How to Rent

Pick one of three options:

On-Demand: Hourly billing. No commitment.

Reserved: 1-year or 3-year. Saves 40-60%.

Spot: Spare capacity. 50-70% cheaper. Interrupts without warning.

Deployment: Log into Azure portal, select ND96asr_v4, pick the region (US East/West or EU), deploy. 5-10 minutes to run. Virtual Machine Scale Sets handle auto-scaling for batch jobs.

Provider Comparison

ProviderCost/GPUBest For
RunPod$2.69/hrCheapest, no frills
Lambda$3.78/hr SXMMulti-GPU NVLink
Azure$3.50-4.50/hrproduction compliance
CoreWeave$6.16/hrKubernetes clusters
Vast.AI$2.00-3.50/hrCheap, interruptions ok

Azure wins if developers need HIPAA/FedRAMP and Azure DevOps integration. Otherwise RunPod or Lambda are cheaper.

When to Use Azure H100

Healthcare/Government: Need HIPAA or FedRAMP? Azure is certified. RunPod isn't.

Multi-region failover: Azure's global infrastructure helps. Others can't.

production DevOps: Using Azure DevOps, SQL Server, Power BI? Integration pays for itself.

Otherwise pick RunPod ($2.69/hr) and save 25%.

FAQ

What is the cheapest way to run H100 on Azure? Spot instances offer the lowest rates at 50-70% discounts, but face interruption. Reserved instances with one-year terms reduce standard rates by 40%. For non-critical workloads tolerating interruptions, spot pricing ranges $0.90-$1.50 per hour.

Can I mix H100 and H200 GPUs in one Azure deployment? Azure instance families segregate GPU types. Single deployments cannot mix H100 and H200. Distributed training frameworks like Horovod can federate jobs across separate H100 and H200 clusters if needed.

How long does it take to provision H100 on Azure? Initial deployment takes 5-10 minutes after resource approval. Capacity constraints may cause delays. Reserving capacity in advance guarantees faster provisioning for time-sensitive projects.

Is Azure H100 cheaper than RunPod or Lambda? No. RunPod charges $2.69/hr (SXM) and Lambda $3.78/hr (SXM) versus Azure's $3.50-$4.50/hr. RunPod is the cheapest option; Azure's deep integrations with organizational tools and compliance certifications justify higher costs for regulated workloads.

What happens if Azure doesn't have H100 capacity? Availability varies by region and time. Teams should check capacity before committing budgets. Alternative regions or spot instance queues provide fallback options.

Sources