Model drift is a change in a model's behavior over time without any change to the model itself as deployed. In classical machine learning it refers to the world moving away from the training distribution. With hosted foundation models it has a second and more abrupt cause: the provider changed the model underneath you.
Two kinds, with different remedies
Data drift is the world changing. Inputs shift away from what the model was trained on — new products, new customer language, a changed economic environment — and accuracy degrades gradually. It is detected by monitoring input distributions and outcomes, and corrected by retraining.
Provider drift is the vendor changing the artifact. A hosted model is updated, deprecated or silently re-pointed, and behavior changes between one call and the next with nothing on your side having moved. It is not gradual, it cannot be corrected by retraining, and the only defenses are pinning versions where the provider supports it and detecting the change quickly.
The second is the one that surprises regulated institutions, because a validated model can stop being the model that was validated without any event in the institution's own change control.
Why it is a governance problem, not only a quality one
A model that quietly gets worse is a product issue. A model that quietly changes after it was reviewed is an evidentiary one.
The question an examiner asks is which model was in use at a given time and under what policy. Answering it requires the version identifier to be captured on every call, not recorded once at deployment. If the only record is a deployment ticket from March, there is no way to establish what was actually serving traffic in July — which is the same argument that makes policy versioning and tamper-evident records load-bearing rather than tidy.
Drift also has to be distinguished from an adversary. Data poisoning looks like drift from the outside — behavior changed, no deployment — and retraining fixes the first while reproducing the second.
Detecting it
Three practical controls, none of which require a model-monitoring platform to start.
Capture the model identifier on every request, including what the gateway actually routed to after any rewrite. A control plane on the request path does this by construction, which is also what makes a model allowlist enforceable.
Run a fixed probe set on a schedule — a small, stable battery of prompts whose expected behavior is known, executed regularly so a change in the provider's artifact shows up as a change in the answers.
Alert on version transitions rather than reading changelogs. The record showing which identifier served traffic is the earliest signal available.
US supervisors excluded generative and agentic AI from model risk management scope in SR 26-2, so the ongoing-monitoring expectation that once covered this is no longer a framework an institution can point at. See runtime governance and examination readiness.
Related terms
Data poisoning · SR 11-7 and SR 26-2 · AI bill of materials · Tamper-evident audit log · Full glossary
Put this in the path of your own agents.
Policy enforced inline between your agents and every model and tool they reach, with a record bound to the human who owns it.
Request a Demo Read the docs