In March 2023, OpenAI's GPT-4 API launched at $30 per million input tokens. By July 2026, budget-tier model endpoints from all major providers sit at $0.10 per million input tokens or below. That is a three-hundred-fold reduction in three years, and the natural reading of it is: build more. The natural reading is the wrong one.
The argument for adding a language model to a product has never been cheaper. For anything except genuinely extreme volume, inference cost will not register in the unit economics. What teams have treated as a green light to ship AI features is actually the removal of the one question that was reliably getting answered before the budget was approved.
#02What the gate was doing
When inference cost $30 per million tokens, the question was unavoidable: will this feature justify its running cost? The question was imperfect as a filter. It favoured high-frequency, high-volume features regardless of whether the underlying task actually benefited from a language model. It was blind to low-frequency, high-value cases. But it was a gate, and gates do something. They make the team think about whether the feature earns its keep before the feature ships.
At $0.10 per million tokens, that calculation disappears. The cost projection does not appear in the planning document because there is nothing to project. A feature that processes ten thousand requests a day costs roughly one pound in inference at budget-tier pricing. The gate is gone. Nothing has replaced it.
#03The test
The correct test was never the cost test. It is this: does this workflow actually get better when you replace deterministic logic with a language model? The class of tasks where the answer is yes is specific. Natural language or unstructured content that requires judgement rather than lookup. A range of acceptable outputs wide enough that the model's probabilistic character is a feature rather than a defect. A cost of error that is low, or visible enough that the user can catch it. Document classification at volume. Intent routing across bounded options. First-draft generation that a skilled person will edit.
Most AI features that shipped in the past two years are not in that class. They are in a different class: the chat interface added because it looks like what an AI product looks like. The generated summary of content the user already read. The ask-a-question button that wraps a query the product team could have surfaced as a filter. The model was cheap. The question of whether the model was the right tool was not asked.
#04The gains that are real
Cheap inference has made real things possible that were not viable before. Automated document processing at the scale a legal or insurance team actually works at was not economically viable at 2023 prices; it is now. Real-time translation of support conversations at volume. Anomaly detection on event streams that would have required prohibitive compute three years ago. These gains belong to teams that started with the workflow problem and found that a language model was the correct primitive to solve it. The pricing collapse did not make bad ideas into good ones. It made it cheaper to find out whether a good idea would work.
Writer's 2026 enterprise AI adoption survey found fewer than one in four organisations reaching measurable return on investment from their AI initiatives, with 79 per cent reporting challenges despite sustained investment. The survey did not find that inference was too expensive. The pattern it describes is of products that shipped because the cost barrier had dropped, not because the use case had been validated.
Cheap inference changed the budget conversation. It has not changed what a product needs to do to earn a renewal. Enterprise customers audit their software estate. They remove or do not renew products whose AI features do not demonstrably save time or reduce the cost of a task the team cares about. The feature that costs nothing to run and does nothing the user values is a credibility cost, not a neutral addition. At $30 per million tokens, there was an economic prompt to ask whether the feature earned its keep. At $0.10, that prompt is gone. The discipline it imposed needs to be supplied deliberately.
“The feature that costs nothing to run and does nothing the user values is a credibility cost, not a neutral addition.”