We had the meeting in April, and a version of it has happened three times since. The client — a professional services firm running a document review tool that had been in production for fourteen months — was in a routine performance review when the operations director asked something that was not on the agenda. She had attended a conference the previous week, watched a demonstration of one of the newer foundation models on legal reasoning tasks, and wanted to know why the system her firm was paying to maintain did not seem to be following. The system worked. The recall scores were solid. The team used it every day. But the general models she had seen were visibly sharper on the tasks her system performed, and hers was not keeping pace.
The honest answer was that hers would not keep pace — not automatically, and not without sustained engineering effort that had not been scoped. A fine-tuned model does not benefit from improvements to the foundation model it was built on. When a new model version arrives with better instruction-following or stronger reasoning on the tasks the fine-tune was trained to perform, the fine-tuned variant does not receive those capabilities. It remains on the base it diverged from at training time. The gap between the fine-tuned model and the current frontier is a function of how much the foundation has improved since training, minus however much a retraining programme has recovered. In fourteen months, with two significant foundation model releases in the period, that gap had grown in ways we had not made explicit when the programme was designed.
We have now reviewed or directly managed eleven production AI systems that used fine-tuning as their primary domain adaptation strategy. Three are performing well against their original intent, still delivering a measurable advantage over the current foundation model on their target tasks. The other eight are in retraining cycles they did not budget for, carrying performance advantages that have narrowed to single-digit percentage points, or being converted to retrieval-augmented architectures by the teams now managing them. The fine-tuning recommendation those eight received in 2024 was made in good faith, by people working with the information available at the time. The failure was not in the recommendation. It was in what the recommendation did not say.
#02Why the recommendation looked reasonable in 2024
Foundation models in 2024 had a genuine performance gap on domain-specific tasks. A model evaluating clinical discharge summaries against a hospital's notation conventions, or classifying legal clauses against a firm's internal taxonomy, or extracting structured fields from invoices in a non-standard format, would make systematic errors on a well-labelled evaluation set that fine-tuning would substantially reduce. The gap was measurable. The improvement from fine-tuning was measurable. The recommendation was grounded in evidence.
There was also a coherent theory of why the advantage would persist. Domain vocabulary, document structure conventions, institutional classification taxonomies — foundation models learn these imperfectly from general pre-training, and the assumption was that the fine-tuned model would maintain its advantage in proportion to the domain's distinctiveness. The expectation was that the investment in data curation and training would compound over time as the labelled dataset grew. That turned out to be the load-bearing assumption nobody was examining closely.
What happened, beginning in approximately mid-2025 and visible across most major model families by early 2026, was that the foundation models improved faster than the fine-tuning programmes could follow. In four of the eleven programmes I am describing, the fine-tuned model's advantage over the current foundation model on its target task had declined from 14 to 22 percentage points at programme launch to 3 to 7 percentage points twelve months later. That narrowing was attributable almost entirely to foundation model improvement rather than to any degradation in the fine-tune. The fine-tune had not worsened. The world it was trained in had moved.
#03The cost structure the business case did not include
The time cost in most fine-tuning business cases is the initial data assembly and training run: a labelling programme, a training job on a managed cloud service, an evaluation cycle, and a deployment. That is the number in the budget. The operational cost of maintaining a fine-tuned model in a production environment — over a deployment lifetime measured in years, against a foundation model ecosystem improving on a six-month cadence — rarely appears in the original programme.
In the legal review system from the opening of this piece, the fine-tune had been retrained twice since launch. Each cycle required approximately 80 hours of legal associate time for labelling updates, two weeks of ML engineer time for the training and evaluation pipeline, and four to six weeks of regression testing to confirm the retrained model had not degraded on tasks the prior version handled correctly. Total retraining cost across the fourteen-month deployment, at standard internal billing rates, was approximately £38,000. The original programme budget contained a retraining line of £8,000 — a figure set without reference to how often the foundation model would update in ways that required following.
Across the eleven programmes in this review, the mean time between retraining cycles was 4.6 months. Three of the eight struggling programmes had no retraining provision at all — designed with a one-time fine-tune and no explicit plan for ongoing maintenance. Those three are the ones where the performance gap has widened most visibly, and where the conversation about rebuilding to a retrieval architecture is now most advanced. In each case, the cost of the retraining programme required to bring them current has exceeded the estimated cost of the rebuild.
The volume of labelling required per retraining cycle also did not remain constant. In the legal classification programme, it grew by 34 per cent between the first and second cycles. The model's failure modes had shifted as the foundation improved in areas adjacent to the target task — the gaps the fine-tune was filling had moved, and the labelled examples covering the previous gaps were no longer the right examples for the current ones. Data curation for fine-tuning is not a one-time investment that depreciates slowly. It is an ongoing programme whose scope is set by a model ecosystem that has not asked for your input.
“Three percentage points of task accuracy is not nothing. But it needs to be weighed against the ongoing retraining cost of the system that produces it — a cost that was absent from every business case in this group.”
#04Where fine-tuning genuinely earned its cost
The three programmes where fine-tuning has held its value share a structural feature: they were solving a constraint, not a performance gap. Each had something — a deployment constraint, a scale economics argument, or a proprietary data advantage — that made fine-tuning the better choice independently of how the foundation model was performing on the same task. That independence is what has preserved the value.
The first was a medical imaging report processing system running inside a private network where API calls to external inference providers were not permitted by the client's information security policy. The fine-tuned model ran on-premises, on hardware the client controlled. There was no viable alternative for a system requiring model inference at that security boundary. Fine-tuning was the mechanism by which foundation model capability was adapted to the task and to the deployment constraint simultaneously. The deployment constraint has not changed.
The second was a high-volume invoice processing pipeline running approximately 37,000 invoices per day. At API pricing against a frontier model, retrieval-augmented extraction would have cost more than the fine-tuned model running on dedicated hardware at that throughput. The economic case for fine-tuning was not about task accuracy — the accuracy profiles were comparable — but about unit economics at a volume where a 4p difference per call compounds into over £500,000 per year. That arithmetic has also not changed.
The third had a data advantage that made fine-tuning the obvious choice. The client had assembled 58,000 labelled examples of a genuinely novel annotation task involving domain vocabulary that does not appear in any foundation model's training data and does not yield to retrieval augmentation because the knowledge is not in documents — it is in the labelled training set itself. That dataset represents years of specialist effort and constitutes real competitive advantage. Fine-tuning was the mechanism by which it was operationalised.
In none of these three was fine-tuning chosen to outperform a well-prompted current foundation model on a benchmark it might later close. Each solved a constraint the foundation model alone could not. That distinction — constraint versus performance gap — is the one that separates durable fine-tuning investment from advantage with an expiry date.
#05What the rebuild programmes found
In five of the eight programmes where teams have rebuilt or are actively rebuilding to retrieval-augmented architectures, the evaluation story is instructive. In three of the five, the rebuilt retrieval system — using a current foundation model with a curated knowledge base, a pgvector index, and an explicit structured prompt — matched or exceeded the fine-tuned model's performance on the original target task within the first month of operation. In two of those three, a domain expert panel reviewing both systems could not identify a consistent quality difference at the task level. The fine-tuning had been covering for prompting and retrieval design that had not been optimised, not for a capability the foundation model lacked.
What made retrieval augmentation competitive for these tasks where it had not been in 2024 is not the size of the context window, which is frequently cited. It is the quality of instruction-following in current frontier models. Claude 3.7 Sonnet and GPT-4o are materially better than their 2024 predecessors at maintaining extraction schema consistency across variable document structures, following complex multi-step formatting instructions, and flagging uncertainty rather than confabulating. Those improvements removed most of the cases where fine-tuning was bridging a genuine capability gap. A well-structured prompt with representative few-shot examples now covers the majority of domain adaptation requirements that a fine-tune was solving twelve months ago.
The single most reliable predictor we have found of whether a fine-tuning advantage will survive foundation model improvement is this: does the gap between the fine-tuned model and the prompted baseline close when the prompt is properly optimised against the same evaluation set? In seven of the eleven programmes, that comparison was run against a prompt set up once and never iterated. In the three rebuild cases where the gap closed on prompt optimisation, fine-tuning had been covering a prompting deficiency rather than a model one. It is the comparison that would have changed the recommendation. It is not a difficult comparison to run.
#06Three questions before any fine-tuning programme now
We have reduced the gate for approving fine-tuning to three questions, each requiring a specific rather than arguable answer. Each is designed to surface the reasoning failure that produced the eight struggling programmes — which in all cases was not bad engineering but an incomplete framing of the original decision.
First: is the performance gap attributable to a genuine capability the model lacks, or to a prompting and context strategy that has not been properly optimised? The question requires a demonstration. The fine-tuned model must be compared against the best available prompted baseline — one iterated against the evaluation set, not written once. If the gap closes on prompt optimisation, fine-tuning is solving a prompting problem and will erode as foundation models improve. If it persists through careful prompting, there may be a genuine case.
Second: what is the retraining plan, and who owns it? The answer must name a person, a cadence, a budget, and a trigger condition for when a retraining cycle begins. An answer of 'we will revisit that when needed' is not a plan — it is a statement that nobody has priced the ongoing cost. In the eight programmes where this question was not answered specifically, the retraining bill arrived on a different team's budget and at a different time than anyone expected.
Third: is there a deployment constraint, a scale economics argument, or a proprietary data advantage that makes fine-tuning the right choice independently of benchmark performance? If yes, the programme is on firm ground regardless of what the foundation model does next. If no — if fine-tuning is being proposed because it is expected to outperform a well-prompted current model — then the question is whether that advantage will still exist in twelve months. The evidence across these eleven programmes is that in most cases it will not, and that the foundation model's improvement trajectory is not something any individual fine-tuning programme has a mechanism to follow without sustained engineering investment.
These were askable questions in 2024. The evidence to answer the third one confidently was thinner then. It is not thin now. The organisations starting fine-tuning programmes in mid-2026 are doing so with eighteen months of production data on how the economics play out — data that was not available when most of the programmes in this review were commissioned. Making the same framing error in those conditions is a choice rather than an oversight.