Machine Learning Interview Questions

Sample Answer

Define the target as return within a fixed window after delivery, and train with a time based cutoff so you only use examples whose label has had time to resolve. Delayed labels mean you either wait for maturity, or use techniques like censoring aware labeling, but you must not treat unknowns as negatives. Use a cost sensitive objective, for example weighted log loss where false positives and false negatives reflect downstream cost, or optimize expected cost: $$\mathbb{E}[c_{FP}\,\mathbf{1}[\hat y=1,y=0] + c_{FN}\,\mathbf{1}[\hat y=0,y=1]].$$ Calibrate probabilities so thresholding can be tuned to changing business tradeoffs.

Sample Answer

You could do classification on $y \in \{0,1\}$ for watch within 24 hours, or regression on watch time, for example minutes watched, conditioned on exposure. Classification wins here because the product question is binary and exposure creates a clear prediction timestamp, while regression mixes intent with duration noise. A strong baseline is a per user and per title popularity model, for example $\hat p = \sigma(b_u + b_i)$, which often beats naive global averages. If your model cannot beat user and item bias, your features or labeling are likely wrong.

Sample Answer

First, you note logistic regression assumes log odds is approximately linear in features, so you choose features that make effects close to additive, like log transforms for spend, bucketed counts, and explicit interaction terms for known non linearities. Next, independence is not about features being independent, it is about examples, so you avoid leakage across repeated impressions by grouping by user or session in splits, and you use robust validation that respects time and user identity. Then you check for correlated rows, like many impressions from one user, and mitigate with downsampling, per user weighting, or clustered standard errors if doing inference. Finally, you tie this back to data quality, if your labels are noisy due to delayed attribution, linear models will look stable but actually learn bias, so you validate attribution windows and missingness explicitly.

Sample Answer

You could treat the offline metric as wrong and rerun the A/B test, or you could treat the online result as real and audit the offline setup. The audit path wins here because offline improvements often come from label or exposure shifts, leakage, or optimizing a surrogate that is misaligned with engagement. Check that offline evaluation replays the same candidate generation, filtering, and position bias as production, and segment by traffic slices where the A/B regressed. Then align metrics, for example use counterfactual metrics like IPS or optimize for calibrated expected value, and only rerun online once you have a falsifiable hypothesis.

Sample Answer

First, define the prediction time $t$ and the label window, for example churn in $(t, t+1]$, then only use features available at or before $t$. Next, split by time, not random, so training is earlier weeks, validation is a later block, and test is the most recent block, which matches deployment and prevents future info leaking backward. If users appear multiple times, group by user within each time block or use a rolling origin evaluation so the model never trains on a later snapshot of the same user than it is evaluated on. Finally, confirm leakage by checking suspiciously high feature importance for post-outcome events like cancellation confirmations or retention offers sent after churn.

Sample Answer

This question is checking whether you can separate model quality from measurement quality under production constraints. You should first verify evaluation parity: same dataset version, same label definition, same joins, and identical filtering, because small pipeline shifts often change the evaluation population. Then run a backtest with the old model scored through the new pipeline and the new model scored through the old pipeline to isolate whether features or scoring logic changed. If the drop only appears on one split, check for temporal drift or a label delay issue, and use bootstrap CIs to see if 0.04 is statistically meaningful.

Sample Answer

First, you check whether the offline split matched production, if you used random split, you redo it as a time split to reproduce the drop. Next, you inspect feature drift, especially token frequency and new token rate, and quantify it with PSI or KL on hashed buckets. Then you add stronger $L_1$ or elastic net to prune brittle token weights, cap vocab by minimum document frequency, and consider feature hashing with a stable hash to reduce churn. Finally, you validate with rolling window backtests, ablate token features versus meta features, and monitor calibration and precision at fixed recall to ensure the fix addresses the real failure mode.

Sample Answer

This question is checking whether you can tie an overfitting symptom to targeted interventions and validate with clean ablations. You start by fixing the evaluation, use a strict time split per store and a metric aligned to the business, then apply early stopping as the baseline since it directly matches the observed divergence. Next, you run controlled ablations, early stopping only, then add weight decay, then dropout, then adjust lag features, keeping everything else fixed and logging validation by horizon. You pick the winner based on improved validation across horizons and stability across folds, not just a single best epoch.

Sample Answer

This question is checking whether you can control both variance and compute in an ensemble without guessing. You reduce per tree cost by limiting `max_depth`, using larger `min_samples_leaf`, and subsampling features via `max_features`, which also decorrelates trees and can improve validation. You tune `n_estimators` until OOB or validation stabilizes, since more trees mostly reduce variance but increase latency and memory. If latency is the bottleneck, you can distill the forest or export a smaller forest, accepting a small AUC drop for predictable inference time.

Sample Answer

The standard move is to use native categorical handling if your GBDT implementation supports it, or use target encoding with strong regularization and out of fold computation. But here, leakage matters because target encoding can memorize rare categories, so you must compute encodings on training folds only, add smoothing toward the global mean, and optionally add noise. For missingness, trees can learn missing as its own branch if supported, otherwise you add missing indicators and impute with a constant so the model can split on "is missing". If the feature is time varying, you also ensure encodings and imputations are computed with only past data to avoid temporal leakage.

Sample Answer

Get this wrong in production and your offline lift turns into online churn because the model chases noise and degrades on fresh traffic. Boosting adds trees sequentially to fit residuals, so after the signal is captured, later trees fit idiosyncrasies in the training sample and validation loss rises. The right call is early stopping on a clean validation set, plus smaller `learning_rate` with an appropriate `n_estimators`, and stronger regularization like shallower trees, larger `min_data_in_leaf`, subsampling rows and columns, and $L_1/L_2$ penalties. You also watch for dataset shift, because apparent overfit can be a validation split that does not match serving.

Sample Answer

The standard move is to subsample, run UMAP or t-SNE, and eyeball clusters. But here, distortion matters because these methods can invent apparent separation by optimizing local neighborhoods, not global geometry. You should tune and report sensitivity to key hyperparameters, t-SNE perplexity, UMAP $n\_neighbors$ and $min\_dist$, and rerun with multiple seeds to check stability. Quantitatively, you can evaluate neighborhood preservation, for example trustworthiness, and verify any claimed clusters also separate in the original space via kNN agreement or centroid distances.

Sample Answer

Get this wrong in production and you amplify popularity, drown out niche content, and make cold-start items invisible. The right call is to learn sequence-based embeddings, for example skip-gram style on sessions or a two-tower setup with self-supervised objectives, and regularize against popularity via sampling or debiasing. For cold start, you should blend metadata or content features into the embedding space, then calibrate with a gating model. You evaluate with offline ranking metrics like NDCG or recall at $k$ on time-split data, plus embedding quality checks like nearest-neighbor coherence and stability across time.