[WIP] GPU Selection Guide

Major 4x GPU Types

• Data Center, e.g., T4, A2, L4

• Visual Computing, e.g., A2000, A4000, 4000ADA, A5000, A6000, 6000ADA

• Gaming, e.g., 3050, 3060, 4070, 4080

• Jetson, e.g.,. Xavier NX, AGX Xavier, Orin NX, AGX Orin

4x GPU Types Comparison

	Latest GPU Architecture	Performance per Watt	Performance per $	Product Life Cycle	Accelerated INT8 Inference
Data Center (e.g., L4)	Ada Lovelace	✭✭✭✭✭	✭✭	✭✭✭	✓
Visual Computing (e.g., 6000ADA)	Ada Lovelace	✭✭✭	✭✭✭	✭✭	N/A
Gaming (e.g., RTX4080)	Ada Lovelace	✭✭	✭✭✭✭✭	✭	N/A
Jetson (e.g., AGX Orin)	Ampere	✭✭✭	✭✭	✭✭✭✭✭	✓

CUDA Cores to Single-Precision(FP32) TFLOPs

In addition to Tensor Cores and DLA (Deep Learning Accelerator), CUDA cores contibute the major FP32 peformacne.

For example,

The NVIDIA GTX 3060 has 3,584 CUDA cores and a clock speed of 1.78GHz. To calculate the TFLOPS of the GTX 3060:

TFLOPS = (3,584 CUDA cores x 1.78GHz x 2) / 1,000,000,000 TFLOPS = 12.4 TFLOPS

GPU Cards

Name	MSRP (USD$)	Gen	TFLOPS (FP32)	TFLOPS (Tensor)	TOPS (INT8)	Dimension (mm)	Power Consumption (W)	Memo
4000ADA	1250	Ada Lovelace	19.2	306.8 (FP8 + Sparsity)	N/A	69 x 168 x Dual Slot	70	Next gen of A2000