🏢 The situation
Datacore's logistics team is complaining: the customer's delivery cost is computed as the "average city tariff", and the company loses money on long-distance and heavy orders.
Lena:
"Your first model! We have history: 80,000 deliveries with the actual cost. Features — weight, dimensions, distance, city, urgency. Start with linear regression. Yes, everyone wants neural nets right away, but the team rule is: simple baseline first. If a linear model gets MAE below $0.80 — we already save millions. And be ready to explain to Max where the number comes from: finance won't sign off on a 'black box'."
🎯 Your task
- Understand how linear regression and gradient descent work.
- Train the model in scikit-learn, evaluate it with MAE/RMSE.
- Interpret the coefficients for the business.
📚 Theory
Linear regression
The model predicts a number as a weighted sum of features:
Training means finding the weights w that minimize the error on historical data. The classic loss function is MSE (mean squared error).
Gradient descent — how the model "learns"
This is the core of almost all ML, including neural networks (Andrew Ng devotes the first week of his course to it):
- Start with random weights.
- Compute the error on the data.
- Compute the gradient — the direction of the steepest error increase.
- Take a step against the gradient:
w = w − α·∇L, whereαis the learning rate. - Repeat until convergence.
- α too large → we jump over the minimum; the error oscillates or grows.
- α too small → training takes forever.
Feature scaling
Parcel weight: 0.1–50 kg. Distance: 1–9000 km. With such different scales gradient descent converges poorly and the coefficients are incomparable. The fix is standardization: x' = (x − mean) / std (StandardScaler).
Regression metrics
| Metric | Formula (essence) | Property |
|---|---|---|
| MAE | mean |y − ŷ| | same units ($), robust to outliers, business-friendly |
| RMSE | root of mean (y − ŷ)² | punishes large errors harder |
| R² | share of explained variance | 1 — ideal, 0 — no better than the mean |
If RMSE ≫ MAE — there are individual big misses worth investigating.
Train/test split — the sacred rule
Quality is measured on data the model did not see during training. Otherwise you're measuring memory, not generalization.