A machine learning model stops learning the instant training ends. During training it studies examples and adjusts its internal settings to find patterns; once that finishes, those settings are frozen. From then on the model only applies what it learned to new inputs, which is called inference. It does not absorb anything from the live data it processes. So as the real world shifts away from the data it was trained on, the model slowly goes stale. That gradual decay is called drift, and it is the single most common reason a model that launched brilliantly is quietly mediocre a year later.
If that surprises you, you are in good company. A lot of sensible business owners assume a model keeps getting smarter the more it is used, the way a new employee does. It does not. Understanding why is the difference between treating AI as a one-off purchase and treating it as a system that needs looking after.
Training and inference are two different jobs
It helps to separate the two phases clearly, because people use "the model" to mean both and that is where the confusion starts.
Training is the learning phase. You show the model a large set of historical examples, each with a known answer, and it repeatedly tweaks millions of internal numbers (the "weights") until its predictions match the answers as closely as possible. This is the expensive, compute-heavy part. It happens once, in a controlled environment, before anything goes live.
Inference is the working phase. You hand the finished model a new input it has never seen and it produces an answer, fast and cheaply. This is what happens every time a customer gets a recommendation, a document gets classified, or a forecast gets generated. The weights do not change. The model is, in effect, a very sophisticated frozen function: same input, same output, every single time.
A useful mental picture: training is studying for an exam, inference is sitting the exam. Once you walk into the exam hall, you stop learning. You answer questions with what is already in your head. The model is permanently in the exam hall.
This is also why large language models behave the way they do. When you chat with one it can feel like it is learning from your conversation, but it is not retaining anything between sessions unless a developer deliberately builds that in. If you are curious about what is actually happening under the hood, we wrote a plain-English explainer on how LLMs generate answers, and a related one on why AI sometimes makes things up.
So does machine learning ever keep learning?
Almost never automatically, and that is by design.
It is technically possible to build systems that update continuously as new data arrives. It is also a fast way to break things. If a model retrained itself on every incoming example with no checks, a burst of unusual data, a spam attack, or a logging bug could quietly corrupt it overnight, and you would have no fixed version to roll back to. So in practice, responsible teams keep training and serving separate. The live model stays frozen and predictable. Learning happens deliberately, in a controlled retraining run, with the new model tested before it ever replaces the old one.
That control is a feature, not a limitation. It means you always know exactly which version is making decisions, you can audit it, and you can reproduce its behaviour. For anyone operating under GDPR or the EU AI Act, that predictability is not optional. A model that silently rewrites itself is a model you cannot explain to a regulator, an auditor, or a customer who asks why they were turned down.
Why models go stale: the four flavours of drift
Here is the uncomfortable truth. Your model can be flawless on launch day and still get worse without anyone touching a line of code. The world it learned does not stand still. This decay is usually broken into a few types.
- Data drift. The inputs change. A fraud model trained on last year's transaction patterns meets this year's payment habits, new merchants, new devices. The data looks different from what it studied, so its judgement gets shakier.
- Concept drift. The relationship itself changes. The thing you are predicting starts behaving by new rules. "What makes a customer likely to churn" in a calm market is not the same as during a price war or a recession. The model's understanding of cause and effect is now out of date.
- Label drift. What counts as the right answer shifts. A support-ticket classifier trained when "billing" meant one thing struggles after you launch a new product line and "billing" now covers situations it never saw.
- Upstream drift. Nothing about the world changed, but your data did. A supplier reformats a feed, a sensor is recalibrated, a column gets renamed, a currency field switches from euros to cents. The model receives subtly broken inputs and confidently produces nonsense.
That last one is sneaky and common. The model is fine; the pipeline feeding it quietly changed. It is one reason solid data engineering matters as much as the model itself. A clever model on a fragile data feed is a liability waiting to surface. If you want to gauge whether your own data is in shape for this kind of work, our piece on whether your data is AI-ready is a good starting point.
How do you know a model has drifted?
You do not feel drift. There is no error message, no crash, no red light. The model keeps answering with the same confidence it always had — it is just wrong more often. That is what makes it dangerous. A broken model fails loudly; a drifted one fails politely.
The only way to catch it is to watch for it. In practice that means a few habits:
- Track predictions against reality. Where you eventually learn the true outcome — did the customer actually churn, was the invoice actually fraudulent — compare it to what the model said and watch the accuracy trend over weeks and months.
- Monitor the inputs, not just the outputs. If the shape of the incoming data starts shifting away from the training data, that is an early warning, often visible before accuracy visibly drops.
- Set thresholds and alerts. Decide in advance what level of slippage is unacceptable and get told automatically when you cross it, rather than discovering it in a quarterly review.
This monitoring layer is part of what separates a tidy demo from a system you can actually run a business on. We go deeper on that gap in machine learning in production and in our overview of the production AI stack.
What retraining actually involves
Retraining is the cure for drift, and it is simply running the training phase again — this time with fresh, recent data added to the mix — to produce a new frozen model that reflects the world as it is now. The old model is retired, the new one takes over, and the clock starts ticking again.
How often should you do it? There is no universal answer, and anyone who gives you one without asking about your data is guessing. It depends entirely on how fast your world moves. A model predicting something stable — say, which machines on a factory line need maintenance — might stay healthy for many months. A model reacting to fast-moving consumer behaviour or market prices might need refreshing every few weeks. The right cadence is the one your monitoring tells you to use, not a number plucked from the air.
A few things make retraining far less painful when the time comes:
- You kept good, labelled data flowing. Retraining needs recent examples with known outcomes. If you are not capturing those along the way, you have nothing to retrain on. This is, again, mostly a data engineering question rather than a modelling one.
- The process is repeatable. Retraining should be a button you press, not a research project you restart from scratch. The first build is where you invest in making the second, third, and tenth builds cheap.
- New models are tested before they go live. A retrained model is a new model. It should be checked against the current one before replacing it — occasionally a refresh makes things worse, and you want to find that out in testing, not in production.
If you are weighing AI for the first time, this is exactly the kind of total cost that a one-off "build the model" quote tends to hide. We break down the real numbers in what AI implementation costs and how to keep a first project sensibly scoped in our guide to scoping a proof of concept.
What this means for your business
None of this should put you off machine learning. It should change how you budget for it. A model is not a painting you hang on the wall; it is more like a garden. The planting is the visible effort, but the upkeep is what keeps it alive. Plan for three things from the start:
- Monitoring, so drift is caught early instead of discovered in lost revenue.
- A retraining path, so refreshing the model is routine rather than a crisis.
- Reliable data plumbing, so the model is always fed clean, consistent inputs.
Get those right and a machine learning system can serve you well for years, quietly adapting as you feed it. Skip them and even a brilliant launch turns into a slow, invisible decline. The teams who win with AI are rarely the ones with the cleverest model — they are the ones who treat it as something to be maintained, not just bought.
Where Crux Digits fits
We build machine learning systems that are meant to live in production, not just impress in a demo — which means the monitoring, the retraining path, and the data engineering are part of the plan from day one, not bolted on after something goes stale. We work in fixed-scope projects with clear prices, human oversight built in, and a strict EU and GDPR-first stance, so you always know what a model is doing and why.
If you are wondering whether your data and use case are ready for this — or whether they are not yet — that is a conversation worth having early. Have a look at our AI consulting in the Netherlands, our transparent pricing, or just get in touch and tell us what you are trying to do. No hype, no pressure — an honest read on whether it is worth doing.
Frequently asked questions
Does a machine learning model keep learning after it is deployed?
No. A model stops learning the moment training ends. Once deployed it only performs inference — applying its frozen settings to new inputs to produce answers. It does not absorb or remember the live data it processes. Any further learning requires a deliberate retraining run, done in a controlled environment, not automatic updates during normal use.
What is the difference between training and inference?
Training is the learning phase, where the model studies historical examples and adjusts its internal weights to find patterns — it is compute-heavy and happens once before launch. Inference is the working phase, where the finished, frozen model takes a new input and produces an answer quickly. The weights do not change during inference. A useful analogy: training is studying for an exam, inference is sitting it.
What is model drift and why does it happen?
Model drift is the gradual decline in a model's accuracy over time, even when nothing about the model changes. It happens because the model is frozen on the data it was trained on, while the real world keeps moving. Inputs change (data drift), the underlying relationships change (concept drift), the definition of a correct answer changes (label drift), or the data feed itself changes upstream. The model keeps answering confidently while quietly becoming more wrong.
How often should a machine learning model be retrained?
There is no universal schedule — it depends entirely on how fast your domain changes. A model predicting something stable, like machine maintenance, may stay healthy for many months. A model tracking fast-moving consumer behaviour or market prices may need refreshing every few weeks. The right cadence comes from monitoring the model's accuracy and inputs, not from a fixed number. Retrain when the data tells you to.
How can I tell if my model has drifted if there are no error messages?
Drift is silent — the model keeps answering with full confidence while being wrong more often, so you have to actively watch for it. Track the model's predictions against real outcomes once you know them, monitor whether incoming data is shifting away from the training data, and set thresholds that alert you automatically when accuracy slips past an acceptable level. Without this monitoring, drift usually surfaces as lost revenue rather than a warning.