The model learned that phones go with cases, knowing nothing about products

🏢 The situation

The big quarterly project. Victor (CTO) at kickoff:

"The 'You may also like' block on the home page drives 4% of revenue, and right now it's just the bestseller list, identical for everyone. Competitors do personalization and grow conversion by 15–20%. We're building our own recommender. Dataset: 2M users, 300K products, view and purchase history."

Lena assigns the roles: you own the candidate model and the offline metrics.

🎯 Your task

1. Understand the approaches: popularity, collaborative filtering, content-based.
2. Master matrix factorization (ALS) and implicit feedback.
3. Measure ranking quality: precision@k, recall@k, NDCG.

---

📚 Theory

The three families of recommenders

1. Non-personal: bestsellers, trends. The baseline that must be in every comparison. Often the only option for new users.
2. Collaborative filtering (CF): "similar users buy similar things". Works purely on behavior; needs no product descriptions. Weakness — the cold start (a new product/user without history).
3. Content-based: recommend similar by attributes (category, brand, description text, image). Solves the product cold start.

Production systems are hybrids with a two-stage architecture: first a light model selects hundreds of candidates (retrieval), then a heavy model ranks them precisely (ranking). That's how YouTube, Amazon and every large marketplace work.

Matrix factorization

The "users × products" interaction matrix is gigantic and 99.99% empty. The idea: decompose it into the product of two narrow matrices — user embeddings U (n×k) and product embeddings V (m×k), k ≈ 32–256. A product's score for a user = the dot product of their vectors. Similar tastes → nearby vectors.

Explicit vs implicit feedback

• Explicit: 1–5 ratings. Rare in e-commerce.
• Implicit: views, cart, purchases. There are no "negatives": the absence of a purchase ≠ "dislike" — maybe the user just never saw it. Implicit data needs special methods (ALS with confidence weights, BPR).

Ranking metrics

The user sees the top-K (say, 10 slots) — what matters is the top of the list:

• precision@k — the share of relevant items in the top-k;
• recall@k — the share of all relevant items found;
• NDCG@k — accounts for position: a relevant item in 1st place is worth more than in 10th;
• coverage/novelty — are we recommending the same things to everyone?

Validation — by time: train on history up to date T, evaluate on purchases after T.

---