The Economics of AI

The Economics of AI

AI is expensive to build and, if you're not careful, expensive to run. Understanding the economics — training costs, inference costs, and how the market is evolving — helps you make smarter decisions about what to build and how.

Training Costs

Training a frontier model is one of the most capital-intensive activities in modern technology. These costs are driven by GPU time — training runs use thousands of GPUs for weeks or months, and the largest runs now use far more than that.

For most developers and companies: you will never train a frontier model. You will use an existing model (via API or open weights) and adapt it via fine-tuning, prompting, or RAG. The economics firmly support this approach — a serious custom fine-tune on an open-weight base model costs a tiny fraction of a frontier training run.

As of October 2026: OpenAI's CEO has said GPT-4 training cost over $100 million; frontier training runs today cost hundreds of millions of dollars or more. A serious custom fine-tune on an open-weight base model typically costs thousands to tens of thousands of dollars by comparison.

Inference Costs

Inference (generating responses) is much cheaper than training, but adds up at scale.

A rule of thumb: 1,000,000 tokens is roughly 750,000 words or a few hundred typical chat interactions.

For high-volume consumer applications, inference cost is a real consideration — and the model tier you choose can change the bill by an order of magnitude or more for the same traffic.

As of October 2026: frontier pricing is roughly $2–10 per million input tokens and $10–50 per million output tokens. Budget/small-model tiers run around $0.10 input / $0.50 output per million tokens — 20–100x cheaper than frontier pricing. Local models remain essentially free to run once the hardware is paid for. Worked example: a product with 100,000 daily active users, each sending 10 messages averaging 500 tokens (split evenly between input and output), processes 500M tokens/day — 250M input, 250M output. At mid-range frontier pricing ($5 in / $25 out per million) that's roughly $7,500/day; at budget-tier pricing ($0.10 in / $0.50 out per million) the same traffic costs roughly $150/day — about 50x cheaper for tasks that don't need frontier-level quality.

The Cost Curve is Falling

AI inference costs have dropped dramatically and continue to fall:

• Each new generation of frontier model tends to deliver more capability per dollar than the one before it
• Open-weight models running on commodity hardware are closing the gap with proprietary APIs, putting further downward pressure on API pricing
• Hardware improvements (H100 → Blackwell-generation chips like the B200/GB200 → whatever comes next) continue to increase throughput per dollar
• Techniques like quantisation, speculative decoding, and batching keep improving efficiency

This isn't just a trend to expect — it already happened once: earlier editions of this course predicted that a model costing $10/million tokens would cost roughly $1/million within two years, and pricing broadly followed that curve. The same logic applies going forward: whatever a frontier model costs today, expect a fraction of that within a couple of years.

Practical guidance: start with the cheapest model that meets your quality bar. Use a budget-tier model for simple tasks, and reserve frontier models for tasks where the quality difference actually justifies the cost. Structure your product so you can swap models as prices evolve.