Your model passed testing, then met different data in production
A churn model can pass every test and still send retention discounts to customers who were never leaving. When I worked on building the data layer for churn models at a subscription business, nothing compared the data a model received in production with the data it had been trained on. Nothing looked wrong, because nothing was looking. Then an automated check was added to make that comparison, and it kept flagging. The issues had been there all along, and they were resolved once the time zones, handled differently in the feature calculations for training and for production, were fixed.
Every wrong prediction has a price per customer: someone wrongly flagged as leaving gets a retention call or a discount they did not need; someone who is actually leaving gets no offer at all. The offline score that justified the model says nothing about either.
A model that disappoints in production is often not predicting worse. It is being fed different data. Yet when results dip, a common reflex is to retrain or tune the model. When the gap is in the calculation, retraining gives it fresh data calculated the training way, while production keeps calculating the same inputs differently, so the gap survives every new version.
The same feature, calculated twice
A feature is one of the inputs a model learns from, such as a customer's usage in the last 30 days. It is calculated once for training, from history in the data warehouse, and again in production, from live data at the moment of each prediction. Two calculations, often written by different people in different code, drift apart in small ways. For instance, one cuts the 30 days at midnight UTC and the other at local midnight. Both match the same line in the design document, and every customer near the boundary gets a slightly different number, one the model was never tested on.
The year the model has not seen
The second gap is coverage. A model trained on a few months of history has not met the seasonal patterns a full year brings, such as a holiday period. In production it meets them for the first time and reads them as a change in customer behaviour. Here the data is calculated correctly; it is simply new to the model, and no amount of tuning teaches it a season it has never seen.
The check that belongs to nobody
Whoever builds a feature is right about its definition. Whoever uses it in the model is right about the model. Checking that the model receives in production the same data it was tested on belongs to neither by default. In my case, the check arrived when the people building the features started working closely with the people using them, and it stopped being nobody's job. A scaleup launching its first model rarely has that check, because nothing has failed loudly yet.
The leadership mandate
The tempting fix is a tool. A feature store, a system that calculates features once and serves them to both training and production, earns its place when several models share features. It does not decide which definition of "usage in the last 30 days" is right, or whether a few months of history is enough. Those decisions, and the check that proves they hold, need an owner before launch rather than after the first disappointing quarter. It is the data contract question again, with the model as the data consumer that cannot object.
The path forward
Before a model goes live, make sure of three things:
- Each feature has one owner for its definition, in training and in production.
- What the model receives in production is compared automatically with what it was trained on, and repeated flags are treated as a data problem to trace, not as noise.
- The training data covers at least one full business cycle, or someone has written down which seasons it has not seen yet.
The same gap is arriving with AI features built on language models, where the test questions are curated and the production questions are not. The question to ask before launch does not change: who checks that what the model meets in production is what it was tested on?