Cheaper Tokens, Bigger Bills: The Jevons Paradox of AI
AI token prices collapsed roughly 300x since 2023, yet compute spend keeps setting records. Why cheaper intelligence means bigger bills, not smaller ones.
Note: Statistics and figures reflect data available as of August 2026. Verify for latest figures.
Two numbers, three years apart.
When GPT-4 launched in 2023, OpenAI charged $30 per million input tokens and $60 per million output tokens. Today you can buy comfortably better-than-GPT-4 quality for cents per million tokens, and the cheapest capable models sit near $0.10 per million input. Stanford HAI's AI Index has tracked the same collapse using a fixed capability benchmark: the cost of querying a model scoring GPT-3.5-class accuracy on MMLU fell roughly 280x in about two years, from around $20 per million tokens to $0.07.
Now the other number. Microsoft, Alphabet, Amazon and Meta are collectively guiding to something in the region of $725 billion of capital expenditure in calendar 2026, up about 77% from roughly $410 billion the prior year, per the spend trackers compiled from their earnings guidance and reported by CNBC in February. Alphabet raised its ceiling again at Q2 2026 earnings. Meta has raised twice.
So the price of a unit of intelligence fell by two orders of magnitude, and the total bill for intelligence went up by three quarters in a single year.
If that feels like a contradiction, it isn't. It's the oldest result in resource economics, and it has a name.
Jevons, in one paragraph
In 1865, William Stanley Jevons noticed something awkward about the steam engine. Every improvement in coal efficiency was supposed to reduce Britain's coal consumption. Instead, consumption kept climbing. Cheaper steam power made steam power worth using in places it had never made sense before, and the new uses swamped the savings from the old ones. Efficiency didn't shrink demand. It unlocked it.
Tokens are the new coal. Every price cut moves a whole tier of previously uneconomic ideas into the "obviously worth doing" column, and the aggregate bill goes up even as every individual call gets cheaper.
I want to be precise about the claim, because Jevons is thrown around loosely. The paradox holds when demand is highly elastic and far from saturation. That describes AI inference in 2026 almost perfectly. It will not describe it forever. More on that later.
Why prices fell
Four things happened at once, and none of them was magic.
Distillation and model compression. Once you have an expensive frontier model, you can use it to train much smaller models that retain most of the useful behaviour on most tasks. The frontier keeps moving up, and last year's frontier gets squeezed into something you can serve at a tenth of the cost.
Open-weight models became genuinely good. This is the one that changes market structure rather than just cost curves. Vercel's AI Gateway production index for July 2026 reported open-weight models handling 29% of all gateway tokens in June, up from 11% in April, with DeepSeek alone at 22.6% of token volume. The presence of a credible free-to-self-host alternative disciplines every closed provider's price list, whether or not you actually deploy it. I dug into how deep that adoption runs in the open-source AI piece.
Inference hardware got competitive. NVIDIA still owns the training market, but inference is where custom silicon has traction. Google's Ironwood TPU generation was positioned explicitly for large-scale inference. Amazon's Trainium and Microsoft's Maia are in production fleets. Analyst estimates put custom ASIC shipment growth at roughly 44.6% year-on-year in 2026 against about 16.1% for merchant GPUs. Treat vendor price-performance claims with suspicion, but the direction is not in doubt.
Serving got better. Continuous batching, paged attention, speculative decoding, KV-cache reuse, quantised weights. Unglamorous engineering that quietly multiplied tokens-per-GPU-hour several times over on identical hardware.
Why volume exploded anyway
Here's the part the "AI is getting cheap" headlines miss. Google's Sundar Pichai said at I/O in May 2026 that Google processes 3.2 quadrillion tokens per month across its surfaces, up roughly sevenfold from 480 trillion a year earlier. Vercel's gateway saw token volume grow 29% month-on-month in June alone.
Three forces are doing that work.
Agents are absurdly token-hungry. A chat turn is one request and one response. An agentic task is a loop, and every iteration re-sends the accumulated context: the full history, all prior tool outputs, every tool definition. Published estimates put typical agentic tasks at 5 to 30 times the tokens of a comparable chat interaction, and a 2026 arXiv study of agentic coding tasks found consumption orders of magnitude above simple code-chat, with input tokens rather than output tokens dominating the bill. A 20-step agent does not cost 20 chat turns. It costs far more, because context compounds.
This matters more now that the agents actually work. The 2026 AI Index reported agent success on real-world tasks improving from 20% to 77.3%. Nobody runs expensive loops that fail. Everybody runs expensive loops that succeed.
Every SaaS product now has an AI feature. When inference cost 3 cents a call, "should we add AI here?" was a business case with a spreadsheet. At current prices it's a default. Summarise this. Draft that. Explain this chart. Each individual feature is trivially cheap. There are now tens of thousands of them, running on every page view.
Reasoning models trade tokens for quality. Extended thinking is, economically, a decision to burn a large number of invisible output tokens to get a better answer. We have effectively invented a dial that converts money into accuracy, and product teams have discovered they like turning it.
Who captures the value
Follow the money and it doesn't stop at the model labs.
NVIDIA reported record revenue of $81.6 billion in Q1 FY2027 (the quarter ended April 2026), of which $75.2 billion was data centre, up 85% year on year. That is more than 90% of the company from a single segment that barely existed at this scale four years ago.
The hyperscalers are converting the same demand into cloud growth. AWS did about $42.2 billion in Q2 2026 at roughly 37% growth, its fastest since 2021, with its AI business and its own chips each above $25 billion annualised. Azure and other cloud services grew 43%, crossing $100 billion over the trailing twelve months. Google Cloud reported $24.8 billion for the quarter and a backlog above $500 billion.
Read that backlog number again. It is contracted future compute consumption, and it exists because customers now believe their token usage is going up, not down.
The pattern is the picks-and-shovels one, with a wrinkle. The hyperscalers are not just reselling NVIDIA. They are progressively substituting their own silicon into the inference tier, which is exactly where volume is growing fastest. That is a deliberate attempt to keep the margin rather than pass it to Santa Clara.
Who gets squeezed
Frontier labs are in a genuinely hard position. Training costs scale with ambition, inference margins are set by whoever prices most aggressively, and open weights put a ceiling on what you can charge for commodity capability. The reported numbers show how differently this can go. OpenAI has been reported as projecting roughly a $14 billion loss for 2026 on gross margins in the high 30s, while Anthropic is reported to have pushed gross margin to around 60% and to be approaching operating profitability, largely on inference efficiency and an enterprise-weighted revenue mix rather than higher prices. Both sets of figures come from press reporting and leaked projections rather than audited statements, so hold them loosely. The structural point stands: selling tokens is a harder business than selling the machines that make them.
Wrapper startups get caught in the middle. If your gross margin depends on a spread between what the model costs you and what you charge, falling model prices sound like good news. They usually aren't, because your customers read the same headlines and expect the cut passed through, while your own token consumption per user rises as you add agentic features. Cheaper inputs plus higher volume plus price-aware customers is a margin squeeze, not a windfall.
Enterprises without cost governance. This is the one I see most often, and the most fixable. Teams budget AI as a per-seat licence and then ship an agent that burns a million tokens per task. The unit price fell, so nobody put a meter on it. I wrote a full framework for this in the AI spend governance piece, and the short version is that the cheapness of tokens is precisely why the volume needs watching.
The counterargument I owe you
The honest case against everything above is that Jevons effects end. They always do, eventually.
The mechanism is saturation. Demand is elastic only while there are unserved use cases. Once every document is summarised and every ticket triaged, further price cuts stop generating new volume and simply reduce the bill. Add on-device inference, where small models running on a phone or laptop cost the provider nothing at all, and a genuine plateau in cloud inference demand becomes conceivable. The Gemma, Phi and MiniCPM families are real evidence that small models can carry a surprising amount of routine work.
There's also an efficiency ceiling argument running the other way. A July 2026 arXiv scenario analysis of inference economics argued that some of the big serving levers are approaching hard limits, with KV-cache quantisation within sight of information-theoretic bounds and mixture-of-experts sparsification hitting quality floors. If per-token efficiency gains slow while demand keeps compounding, the bill grows faster, not slower.
I find the saturation case plausible and the current evidence against it. The most interesting datapoint is from Vercel's July index: blended price per token stayed flat in June after rising almost 20% in May, because cheap open-weight volume was offset by a roughly 12% rise in closed frontier prices. Prices are no longer falling in a straight line, and volume grew 29% that month anyway. That's the opposite of saturation.
Watch for the turn. It will show up as token volume growth decelerating while prices are still falling. Nothing in the current data looks like that.
What to actually do about it
If you're building. Design for token volume from day one, not price. Instrument tokens per task, not tokens per call, because the agent loop is where budgets die. Cache aggressively, prune context deliberately, and route by task: a cheap open-weight model for classification and extraction, the expensive one only where reasoning earns its keep. Assume your per-user consumption will rise faster than unit prices fall.
If you're buying. Negotiate on committed volume rather than list price, because list price is going to keep moving and your consumption is the thing with leverage. Avoid multi-year lock-in at today's rates. Insist on per-team and per-workload token attribution before you scale any agent programme, and treat "the API is cheap now" as a reason for governance rather than an excuse to skip it.
If you're investing. I'm not a licensed financial adviser and nothing here is investment advice. But the analytical frame I'd apply is this: in a Jevons regime, the durable economics sit closer to the constrained resource than to the abundant one. Compute, power and the silicon that turns one into the other are constrained. Tokens are abundant and getting more so. That doesn't make infrastructure a good trade at any price, and it says nothing about valuations, which are a separate question entirely. Ask what happens to that thesis if demand growth decelerates, because that is the risk the whole structure rests on.
The cheapest thing in AI right now is the intelligence. Everything around it, the chips, the racks, the transformers, the electricity, is getting more expensive and more contested. Power in particular has quietly become the binding constraint, which I've written about separately in the grid piece. That is not an anomaly to be corrected. It is what a Jevons paradox looks like from the inside.
Related Reading
Enjoying this article?
Get posts like this in your inbox. No spam, unsubscribe anytime.
Related Articles
Open Source AI in 2026: The 89% Adoption Rate Nobody Talks About
Open-Source vs Paid AI Tools: Which Is Actually More Cost-Effective in 2026?
