A model without monitoring is an incident that has not happened yet

🏢 The situation

The churn model (modules 5–10) still lives as "a notebook Igor runs on Mondays". Lena:

"Embarrassing to admit, but that's the truth of half the industry. Time to grow up: the model must become a service — with an API, versioning, monitoring and a plan for 'everything is broken'. Wrap it in FastAPI + Docker, set up drift monitoring. This is the last skill separating you from a middle: a model that can't be operated reliably doesn't exist."

🎯 Your task

1. Wrap the model in a REST API (FastAPI) and a container (Docker).
2. Understand model versioning and reproducibility.
3. Set up monitoring: service metrics, data drift, quality.

---

📚 Theory

Maturity levels of ML systems

0. A notebook + manual runs ("us, today").
1. Model as a service: API, container, CI/CD, versions.
2. An automated pipeline: retraining on schedule/trigger, validation gates.
3. Full MLOps: feature store, model registry, automatic rollback, model A/B.

Most teams only need levels 1–2. Don't build level 3 for the badge.

The model as an artifact

• Serialization: joblib/pickle (⚠️ library versions must match!), ONNX — a portable format.
• Versioning: the model + code + data + metrics are pinned together (MLflow Model Registry, W&B). The question "which model is in production and what was it trained on?" must be answerable in 10 seconds.
• Reproducibility: pinned random_state, package versions (a lock file), code in git.

The three layers of ML service monitoring

1. Service (like any backend): p50/p99 latency, RPS, 5xx errors.
2. Data drift: the input feature distribution has drifted from training (a new customer segment, a broken field in a source). Metrics: PSI, KS test per feature. Prediction drift — a shift in the score distribution — is the cheapest early signal.
3. Quality (concept drift): true labels arrive late (we learn about churn 30 days later) → compute delayed quality, compare with the offline estimate.

The rule: a model without monitoring is an incident that hasn't happened yet. Data always changes.

Model rollout strategies

• Shadow mode: the new model predicts in parallel; answers are logged but not used. A free check on real traffic.
• Canary: 5% of traffic to the new model; metrics fine → roll out.
• Rollback: the previous version always on standby; switching by config, not by deployment.

---