NVIDIA A40
Ampere Architecture
Active
Launched October 2020
Core Specifications
VendorNVIDIA
ArchitectureAmpere
Form FactorPCIe
VRAM48 GB
Memory Bandwidth696 GB/s
TDP300 W
Compute Performance
| Precision | TFLOPs |
|---|---|
| FP32 | 37.4 |
| FP16 | 150 |
| BF16 | 150 |
Performance Benchmarks
image gen
| Configuration | Precision | Performance | Source |
|---|---|---|---|
| Stable Diffusion XL, 1024x1024 | FP16 | 1.5 images_per_second | View |
llm inference
| Configuration | Precision | Performance | Source |
|---|---|---|---|
| LLaMA 13B, batch inference | FP16 | 4,200 tokens_per_second | View |
llm train
| Configuration | Precision | Performance | Source |
|---|---|---|---|
| LLaMA 13B, workstation training | BF16 | 7,500 tokens_per_second | View |
Quick Stats
Supported Platform
This NVIDIA accelerator can be orchestrated and managed with hosted·ai
Learn more at hosted·ai