Data Drift Detection
Data drift detection monitors whether the statistical distribution of a model's incoming input data is shifting away from the distribution it was trained or validated on — a leading indicator that model performance may be about to degrade, even before that degradation shows up in accuracy metrics.
Drift detection matters because it catches a problem before it becomes visible in outcome metrics — by the time accuracy has visibly dropped, the underlying data has often already shifted significantly, and the lag between cause and visible effect can be weeks. Watching the input distribution directly gives an earlier warning: new categories of input appearing, a shift in typical input length or format, or a change in the frequency of certain topics can all signal the model is now facing data unlike what it was validated against.
Common statistical techniques compare a rolling window of recent inputs against a reference distribution — population stability index, KL divergence, or simpler summary-statistic comparisons depending on the data type — and flag when the divergence crosses a threshold. The practical difficulty is distinguishing meaningful drift from normal seasonal or short-term variation: a threshold set too sensitive fires constantly on noise, while one set too loose misses real, gradual shifts until they've already caused damage.
