Talking AI — Model drift: the AI you install is not a finished AI

A predictive AI learns yesterday's world: the moment things shift — new behaviors, new prices, a new season — its forecasts quietly degrade, with no error message. That is model drift: accuracy slips while the dashboard stays green. The right question to ask a vendor is not "how accurate is it on demo day?" but "how do you monitor drift in production, how often is the model retrained, and who alerts me when its accuracy drops?" The trap is assuming a model that was delivered and validated stays good on its own. Any small or midsize company should write ongoing accuracy monitoring and a retraining schedule into the contract as deliverables, on par with go-live.

Source: U.S. Food and Drug Administration, "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations" (draft guidance), January 2025.