Mental model
Optimize cost per acceptable outcome, not cost per token. A cheaper request that fails more often or consumes more human review can make the product more expensive.
Theory
Choose a meaningful unit such as resolved case, approved document, qualified lead, or successful tool transaction. Measure variable model input, cached input, output, embeddings, reranking, tools, storage, and data transfer. Add allocated infrastructure, evaluation, observability, moderation, support, and human review. Divide by accepted outcomes rather than attempts.
Segment economics by task type, model route, customer tier, and failure class. The distribution matters: a small group of long contexts or retry loops can dominate spend. Connect unit cost to latency and quality gates so optimization does not shift cost into user time, incidents, or downstream rework.
Alternatives and trade-offs
Smaller models reduce unit price but may increase retries or review. Caching saves repeated computation but introduces freshness risk. Shorter outputs reduce cost and latency when the task permits. Batch processing improves utilization but delays results. Revenue-based metrics are useful only after product value is demonstrated.
Failure modes and misconceptions
Provider invoice totals omit engineering and review costs. Token averages hide heavy tails. Counting every generated answer as successful inflates margins. Fallback chains can multiply spend during incidents. Premature cost optimization can lock in a system that does not solve the user problem.
Knowledge check
How does the cost per request differ from the cost per successful, policy-compliant task?
Decision scenario
A document reviewer tracks cost per accepted review. A smaller model is cheaper per call but produces more escalations; routing only simple documents to it improves total economics without weakening the quality gate.
Learning outcomes
- Explain AI Unit Economics as a system mechanism rather than a slogan.
- Compare its alternatives, trade-offs, and production failure modes.
- Apply the concept to a decision and identify evidence that would validate it.
Trade-offs
Using AI Unit Economics can improve capability or control, but it also introduces cost, latency, complexity, and failure modes that must be measured against an explicit objective.