deploybase

A100 on AWS: Pricing, Specs & How to Rent

Deploybase · January 27, 2025 · GPU Pricing

Contents

A100 Hardware Specifications

The NVIDIA A100 is the most widely deployed data center GPU. It comes in 40GB and 80GB HBM2/HBM2e variants delivering 312 TFLOPS BF16 tensor core performance (19.5 TFLOPS FP32). Two form factors exist: PCIe (standard interconnect) and SXM (specialized modules with higher bandwidth).

The A100 has 6,912 CUDA cores and 432 tensor cores. Memory bandwidth reaches 2 TB/s on the SXM variant, ideal for training transformers and fine-tuning large language models. Supports INT8, INT4, and TF32 precision for faster training (FP8 is not natively supported on A100).

A100 on AWS Pricing

AWS offers A100 GPU instances through EC2, with pricing structured around instance family and availability zone. As of March 2026, AWS charges approximately $2.745 per hour per A100 GPU in on-demand configurations (p4d.24xlarge at $21.96/hr for 8 GPUs). Reserved instances can lower costs to around $1.37 per GPU per hour with a one-year commitment.

AWS bundles A100 GPUs in the p4d instance family. The p4d.24xlarge contains eight A100 GPUs alongside 768GB of CPU RAM and 8TB of NVMe SSD storage. This configuration costs $21.96 per hour on-demand. Multi-GPU setups benefit from AWS's NVMe storage and high-bandwidth interconnect, though the bundled compute often exceeds individual requirements.

Pay-as-developers-go works for short experiments. Batch jobs spanning days or weeks benefit from spot discounts (50-70% savings). Same hardware, interruption risk.

Comparing AWS with Other Providers

Lambda Labs charges $1.48 per hour for a single A100 PCIe GPU, significantly underpricing AWS's on-demand rates. RunPod offers A100 PCIe at $1.19 per hour and A100 SXM at $1.39 per hour, making it among the most economical options. CoreWeave bundles eight A100 GPUs in their offerings at $21.60 per hour, translating to $2.70 per GPU.

These alternatives matter when budgets tighten. Savings of 50% on compute represent material differences in project viability. However, AWS provides value through its ecosystem integration, Kubernetes support via EKS, and predictable SLA commitments. Teams already committed to AWS infrastructure may prefer the convenience despite higher per-GPU costs.

For batch processing, AWS GPU pricing becomes competitive when reserved instances apply. For interactive workloads demanding immediate availability, specialist GPU cloud providers win on hourly rates.

Getting Started on AWS

Accessing A100 instances on AWS requires an active account with appropriate permissions. Handle to the EC2 console and search for p4d instance types. Request a quota increase if needed. AWS processes most quota requests within hours.

Create a new instance using the AWS Management Console. Select Ubuntu 22.04 LTS or the deep learning AMI image. Configure security groups to permit SSH access from trusted IPs. Attach an IAM role for programmatic access if deploying via infrastructure-as-code.

Once launched, install NVIDIA CUDA drivers and cuDNN libraries. Most deep learning AMIs come pre-configured. Verify GPU availability with the nvidia-smi command. Install PyTorch or TensorFlow depending on framework preference.

For production workloads, use AWS CloudFormation or Terraform to automate provisioning. Define instance configurations as code for reproducibility. Set up CloudWatch alarms for cost monitoring and unexpected spend.

FAQ

Q: What is the hourly rate for A100 on AWS? A: As of March 2026, the p4d.24xlarge instance containing eight A100s costs $21.96 per hour on-demand through AWS EC2, which works out to approximately $2.745 per GPU.

Q: Can I use spot instances to save money? A: Yes. AWS spot instances offer 50-70% discounts but carry interruption risk. Spot instances work well for fault-tolerant batch jobs and distributed training workloads.

Q: How does AWS A100 pricing compare to RunPod and Lambda? A: Both cost significantly less. RunPod charges $1.19 per hour for A100 PCIe and $1.39 per hour for A100 SXM. Lambda Labs charges $1.48 per hour for A100. AWS's on-demand per-GPU rate of $2.745 is roughly 2.3x higher than RunPod.

Q: Does AWS offer discounted yearly commitments? A: Yes. Reserved instances reduce costs to approximately $1.65 per hour for one-year commitments. Three-year commitments offer steeper discounts but less flexibility.

Q: What software do I need to run on A100 instances? A: Install NVIDIA CUDA 12.x, cuDNN, and a deep learning framework like PyTorch or TensorFlow. Most AWS deep learning AMIs include these pre-installed.

Sources