Recommendation Systems Interview Questions

Sample Answer

You should use the weighted matrix factorization framework from Hu, Koren, and Volinsky (2008), where you define a binary preference variable $p_{ui} = 1$ if user $u$ interacted with item $i$ and a confidence weight $c_{ui} = 1 + \alpha \cdot f(r_{ui})$ that scales with interaction frequency. The loss function becomes $$\min_{U,V} \sum_{u,i} c_{ui}(p_{ui} - u_u^T v_i)^2 + \lambda(\|U\|^2 + \|V\|^2)$$ where the sum is over all user-item pairs, not just observed ones. This is critical because with implicit feedback, the absence of interaction is ambiguous (the user may not have seen the item), so you treat all zeros as negative signals but with low confidence. ALS is preferred here over SGD because the confidence-weighted formulation allows a closed-form per-row update that efficiently handles the dense summation.

Sample Answer

You could do user-based CF, which finds similar users and recommends what they liked, or item-based CF, which finds similar items to what a user already interacted with. Item-based wins here because with 300 million users, computing and storing user-user similarities is far more expensive and unstable, since user profiles change frequently as new interactions arrive. Item-item similarity is more stable over time (a product's co-purchase pattern shifts slowly), the item catalog is 30x smaller than the user base making precomputation feasible, and Amazon's original published research confirmed this exact reasoning. You should also note that item-based CF provides more interpretable recommendations ("because you bought X"), which directly improves click-through rates.

Sample Answer

You could use a pure content-based fallback that switches on when interaction count drops below a threshold, or a hybrid model that learns to weight content and collaborative signals jointly. The hybrid wins here because a hard switch creates a jarring quality discontinuity at the threshold boundary, while a hybrid model like a two-tower architecture with both content features (video embeddings, audio, hashtags) and collaborative features (user/item interaction embeddings) can learn to smoothly downweight the collaborative signal when it is unreliable. In practice, you can implement this by feeding user interaction count as an input feature so the model learns the reliability of collaborative signals as a function of data density. This also avoids maintaining two separate systems in production, reducing serving complexity.

Sample Answer

First, you need to think about what content signals are available for a podcast episode: you have the audio itself, the transcript (via ASR), episode title and description text, show-level metadata like genre and creator, and episode duration. You would extract text embeddings from transcripts and descriptions using a sentence transformer, audio embeddings from a model like Audio Spectrogram Transformer for tone and style, and categorical features like genre encoded via learned embeddings. Next, you combine these into a unified episode embedding, potentially through a fusion network that learns cross-modal interactions. For the hybrid piece, you can use a weighted combination in a ranking model where collaborative features (e.g., user-podcast interaction matrix factorization embeddings) are concatenated with content features, and the model learns feature importance. The key insight is to use the content tower as a prior that regularizes the collaborative embeddings for sparse items, for example by adding a loss term $\lambda \|e_{\text{collab}} - e_{\text{content}}\|^2$ that pulls collaborative embeddings toward content embeddings when interaction data is limited.

Sample Answer

First, you need to ask what the MF baseline is actually failing at. If the interaction matrix is sparse and the catalog is relatively static, MF with well-tuned regularization is surprisingly hard to beat, and a neural model may just overfit. Next, consider whether you have rich side features like user demographics, item metadata, or contextual signals, because that is where neural collaborative filtering shines: it can learn nonlinear feature interactions through its hidden layers, something MF with linear embeddings cannot capture via $\hat{y}_{ui} = f(\mathbf{p}_u, \mathbf{q}_i; \theta)$ where $f$ is a learned MLP. Then you should run an A/B test, not just compare offline metrics, because improvements in NDCG do not always translate to engagement gains. If the neural model does not meaningfully move online metrics after a fair comparison, the engineering cost of training, serving, and maintaining it is not justified.

Sample Answer

This question is checking whether you can navigate the tension between capturing long-term user behavior and the quadratic self-attention cost $O(n^2 d)$ that makes naive transformer application impractical. You should truncate to the most recent $k$ interactions (typically 50 to 200) as your primary context window, since recency dominates relevance in most engagement prediction tasks. For longer-range signals, you can compress older history into a fixed-size summary embedding using a separate aggregation module or hierarchical attention, then concatenate it with the recent-window transformer output. At inference time, you cache the key-value pairs from previous forward passes so you only compute attention for the new interaction incrementally, similar to how autoregressive language models serve tokens. Techniques like sparse attention patterns or linear attention approximations are also worth mentioning, but in practice at Meta-scale systems, the truncation plus cached KV approach is the most deployed solution.

Sample Answer

This question is checking whether you can move beyond a single pointwise relevance score and reason about multi-objective optimization in a production setting. You would take the top-$k$ candidates from your ranking model and apply a re-ranking function that combines the relevance score with diversity penalties (e.g., Maximal Marginal Relevance or a determinantal point process) and business boost terms as a weighted sum or constrained optimization. You can formulate this as selecting an ordered slate $S$ that maximizes $\sum_{i \in S} \alpha \cdot \text{rel}(i) + \beta \cdot \text{diversity}(i, S) + \gamma \cdot \text{boost}(i)$ where the weights are tuned via online A/B tests. The key design choice is keeping this layer lightweight, typically greedy sequential selection, so it adds minimal latency on top of the ranking stage.

Sample Answer

The standard move is to precompute any feature that is expensive to calculate or depends on large aggregations, such as a user's 30-day purchase history or an item's average rating. But here, feature freshness matters because some signals like "items added to cart in the last 5 minutes" lose value if stale, so those need to be computed on the fly from a streaming pipeline (e.g., Kafka into a low-latency key-value store). A good rule of thumb: if the feature changes slower than your serving SLA (say, once per hour vs. a 20ms budget), precompute it; if it changes within the session, compute it at request time or via near-real-time streaming. You should also consider that user-side features are fetched once per request while item-side features are fetched per candidate, so item features benefit more from precomputation to avoid a latency blowup proportional to the candidate set size.

Sample Answer

Get this wrong in production and new content never surfaces, creating a stale ecosystem that hurts both creators and users. The right call is to ensure your item tower relies heavily on content features (image embeddings from a vision model, text from pin descriptions, board metadata) rather than purely on engagement features, so new pins still produce meaningful embeddings at index time. You can also use a "warm-up" exploration policy that gives new pins a small guaranteed impression budget, then blend the resulting engagement signals back into the embedding via periodic re-indexing. For the user tower, you should include the user's recent interaction sequence so that even if the item is new, the dot product $\langle u, v \rangle$ in the shared embedding space can still capture semantic relevance based on content similarity to what the user has engaged with before.

Sample Answer

The standard move is to audit your training data and loss function for implicit popularity amplification, since models trained on implicit feedback naturally learn that popular items get more positives. But here, the feedback loop matters because popular items get more exposure, which generates more clicks, which reinforces the model's preference in a rich-get-richer cycle. You should apply inverse propensity scoring (IPS) where each training example is weighted by $1 / p(\text{exposure})$ to down-weight items that were shown disproportionately, or use a causal debiasing approach that separates the popularity effect from genuine relevance. On the serving side, add a diversity-aware re-ranker or reserve a fraction of slots for exploration using Thompson Sampling over new podcast embeddings, so fresh content gets enough impressions to learn a reliable quality estimate.

Sample Answer

Get this wrong in production and your model learns to replicate the existing ranking rather than true relevance, creating a self-reinforcing loop where position dictates clicks and clicks dictate position. The right call is to train a position-bias model: factor the click probability as $$P(\text{click} | \text{item}, \text{pos}) = P(\text{examine} | \text{pos}) \cdot P(\text{click} | \text{examine}, \text{item})$$ where the examination probability depends only on position and the relevance probability depends only on item features. You feed position as a separate input during training so the model learns to disentangle the two, then at serving time you set position to a constant (or remove it) so the model scores purely on relevance. You can also run occasional randomization experiments where a small percentage of traffic gets shuffled rankings to collect unbiased labels, which serves as ground truth for calibrating your propensity estimates.

Sample Answer

Get this wrong in production and you introduce massive variance and inconsistent user experiences. The right call is user-level randomization in almost all recommendation A/B tests. Session-level randomization means the same user could see the old algorithm in one session and the new one in the next, which contaminates any measurement of long-term engagement or retention and inflates variance due to within-user variation across groups. You should use user-level assignment with a stable hash on user ID, which gives you clean causal estimates and a consistent experience. The only scenario where session-level might be acceptable is if you are explicitly studying session-context effects and your metric is purely within-session, like session duration.

Sample Answer

$\text{MAP@10}$ sounds reasonable but breaks under graded relevance because it treats relevance as binary, so you would have to threshold your 0-to-4 labels and lose information. Using raw precision-based metrics also does not reward putting a 4-star item above a 1-star item. That leaves $\text{NDCG@10}$, which is designed for graded relevance: it uses a gain function like $2^{rel_i} - 1$ that naturally distinguishes between relevance levels and applies a logarithmic position discount via $$DCG@K = \sum_{i=1}^{K} \frac{2^{rel_i} - 1}{\log_2(i+1)}$$. You normalize by the ideal DCG so the metric is comparable across queries with different numbers of relevant items.