Compute

The computational resources—processing power, memory, and infrastructure—required to train and run AI models.

Compute refers to the processing power and infrastructure needed to train and run AI models—a critical resource that shapes what’s possible in AI.

Hardware

• GPUs: NVIDIA H100, A100 (dominant for AI)
• TPUs: Google’s custom AI accelerators
• Custom chips: Amazon Trainium, Microsoft Maia
• CPUs: Supporting roles, some inference

Scale

Training frontier models requires enormous compute:

• GPT-4: Estimated $100M+ in compute
• Training runs: Weeks to months on thousands of GPUs
• Inference: Millions of queries per day

Access Models

• Cloud: AWS, Azure, GCP rental
• On-premise: Building own data centers
• Startups: Compute credits, partnerships
• Research: Academic clusters, grants

Constraints

• Supply: GPU shortages, NVIDIA dominance
• Cost: Training runs cost millions
• Energy: Massive power requirements
• Geopolitics: Export controls on chips

Trends

Compute requirements double roughly every 6-10 months for frontier models, creating concentration toward well-resourced players.