Case In Point

Downloadable Code Files

Getting Started with Code Files

Getting Started

Introduction

Experiment Goals

Key Members

Product KPI

The North Star Metric

Driver Metric

Guardrail Metrics

Secondary Metrics

Segmentation Metrics

Launch Criteria

Experiment Data

Amazon AB Testing Case - Question

Amazon AB Testing Case - Solution

Code Walkthrough

Understand the Business Problem

State the Experiment Hypothesis

Hypothesis Statement

Significance Level

Statistical Power

Minimum Detectable Effect (MDE)

Design the Experiment

Randomization Unit

Sample Size Calculation

Experiment Duration

Run the Experiment

Experiment Platform

Peeking at P-Values

Assess Validity Threats

Stable Unit Treatment Value Assumption

Survivorship Bias

Sample Ratio Mismatch

Primacy Effect

Novelty Effect

Holiday Effect

Other Effects to Consider

AA Test

Conduct Statistical Inference

Chi-Squared Test of Independence

Z-Test for Proportions

T-Test for Means

Decide Whether to Launch

Interpret Test Results (Relative vs Absolute Difference)

Launch Decisions

Decision Rule

Decision Rule with Confidence Interval

Holdout Set

[Google] An experiment is statistically significant, but the sample is low. Should you change?

[Amazon] An AB test on "frequently bought together" showed improve in test, but not after launch. What happened?

[Amazon] New UI and algorithm are tested on product search. How would you design an experiment?

[Facebook] An experiment on News Feed showed increase in CTR, but the p-value is 0.06. Should you launch the change?

[Instacart] What are important statistical parameters in AB testing?

Interview Cases

[Amazon] Testing Recommender Systems

[Uber] Improving Driver Enrollment Rate

Mock Interviews

Develop a comprehensive understanding of how to approach data science case interviews by learning key frameworks and practicing with real-world examples from leading companies like Lyft, Meta, Amazon, and LinkedIn. This course covers a wide array of data science case types, including product metrics, experimentation, and machine learning, providing you with the essential tools and knowledge to excel in case-based interviews. Each module is designed to offer a deep dive into practical case frameworks and step-by-step solutions, giving you the ability to effectively communicate your thought process during interviews. Whether you're new to case studies or looking to refine your skills, this course offers 70+ case examples that reflect real challenges faced by data scientists in top tech companies. This course is an invaluable resource for anyone preparing for data science interviews that emphasize problem-solving, technical case responses, and data-driven decision making.

Case Assessment

Effective Case Response

Lyft Example

Preparation Strategy

What Is a Data Science Case?

[Applied Statistics] Survey Weighting

[Machine Learning] Regression Modeling Framework

[Marketing] Attribution Modeling

[Marketing] Marketing Mix Modeling - Part 1

[Marketing] Marketing Mix Modeling - Part 2

[Marketing] Measuring Marketing Performance

Technical Case Frameworks

[Amazon] Evaluate Variance

[Meta] Distribution of Comments

[Meta] Distribution of False Positives

[Meta] Parents on Facebook

[Meta] Revenue Analytics on Facebook Games

Applied Statistics

[Google] Search Metric Shifts

[Instacart] Measure Active Users

[Instagram] Measure Success on Stories

[LinkedIn] Engagement Increased, but Revenue Decreased

[LinkedIn] Estimating Business Travelers

[Lyft] Measuring Driver Satisfaction

[Meta] Decreasing User Retention After Sign-Ups

[Meta] Facebook Groups

[Meta] Measuring Performance of News Feed

[Meta] Measuring Spam on Facebook

[Meta] P-Value > Significance Level (Alpha)

[Netflix] Improve Conversions

[Netflix] Long-Term Metric

[Uber] Rideshare Engagement

Product Metrics

[Amazon] AB Testing Multiple Changes

[Amazon] Change Declined After Launch

[Google] How would you improve Google Maps?

[Google] Peeking at the P-Value

[Hotel Tonight] Metric Change

[Instacart] Statistical Parameters in AB Testing

[LinkedIn] Premium Subscription Funnel

[LinkedIn] Should LinkedIn Have Videos?

[Meta] How would you improve Facebook Messenger?

[Netflix] Video Preview

[Uber Eats] Measure Long-Term Effects

[Uber] Incentivizing Inactive Users on Uber Eats

Experimentation

[Amazon] Detect Fraudulent Users

[Amazon] Estimate Customer Lifetime Value

[Amazon] Sales Forecasting

[Apple] Meaningful Customer Segmentation

[Google] Forecast Airline Ticket Prices

[LinkedIn] Optimize Email Campaigns

[LinkedIn] Salary Estimation

[Meta] Fraud Features

[Meta] Optimize Cold Sales Calls

[Redfin] Pricing Optimization

Machine Learning

[Instacart] Free Delivery Opportunity Sizing

[Meta] Portal Launch Opportunity Sizing

Opportunity Sizing

[Apple] Marketing Mix Modeling on iPhone Sales

[CVS] Marketing Outreach Experiment

[LinkedIn] Email Conversions

[Uber] Measure the Impact of Super Bowl Ads

Marketing

[AWS] Propensity Modeling

[BCG Gamma] Customer Satisfaction Survey

[McKinsey] Employee Attrition

Consulting Cases

Case-In-Point