Introduction

Downloadable Code Files

Getting Started with Code Files

Understand Business Problem

Experiment Goals

Key Members

Product KPI

The North Star Metric

Driver Metric

Guardrail Metrics

Secondary Metrics

Segmentation Metrics

Launch Criteria

Experiment Data

Amazon AB Testing: Question

Amazon AB Testing: Solution

Code Walkthrough

State the Experiment Hypothesis

Hypothesis Statement

Hypothesis Statement: Question 1

Chi-Squared Test of Independence: Question

Hypothesis Statement: Solution 1

Hypothesis Statement: Question 2

Hypothesis Statement: Solution 2

Significance Level

Significance Level: Question

Significance Level: Solution

Statistical Power

Statistical Power: Question 1

Statistical Power: Solution 1

Statistical Power: Question 2

Statistical Power: Solution 2

Minimum Detectable Effect (MDE)

Minimum Detectable Effect: Question

Minimum Detectable Effect: Solution

Design the Experiment

Randomization Unit

Sample Size Calculation

Sample Size Calculation: Question

Sample Size Calculation: Solution

Experiment Duration

Experiment Duration: Question

Experiment Duration: Solution

Run the Experiment

Experiment Platform

Peeking at P-Values

Assess Validity Threats

Stable Unit Treatment Value Assumption

Survivorship Bias

Sample Ratio Mismatch

Primacy Effect

Novelty Effect

Holiday Effect

Other Effects to Consider

AA Test

Conduct Statistical Inference

Chi-Squared Test of Independence

Chi-Squared Test of Independence: Solution

Z-Test for Proportions

Z-Test for Proportions: Question

Z-Test for Proportions: Solution

T-Test for Means

T-Test for Means: Question

T-Test for Means: Solution

Launch Decision

Decide Whether to Launch

Interpret Test: Relative Difference

Launch Decisions

Decision Rule

Confidence Interval

Holdout Set

Practice Cases

Peeking at the P-Value

Failed to Generalize

Multiple Testing

No Statistical Signifiance

Statistical Parameters

Mock Interviews

Amazon Testing Recommender System

Improve Driver Enrollments on Uber

Ab Testing

Last Updated:&nbsp;01/01/23

<h2 class="wp-block-heading">Problem</h2>

Duration: 10 Minutes Difficulty: Medium

An interviewer at&nbsp;Amazon&nbsp;asked:

Amazon tested a new feature that displayed “frequently bought together” on the product page. For instance, when a user is on the product page of a lamp, the feature displays LED lightbulbs that other users have also bought. The experiment shows an improvement.&nbsp;

Last Updated : &nbsp; 01/01/23 Problem Duration : 10 Minutes Difficulty : Medium An interviewer at&nbsp; Amazon &nbsp;asked: Amazon tested a new feature…

DataInterview

Failed to Generalize

Problem#

Unlock the full lesson

Problem