Last Updated: August 2021

Problem

Duration: 5 Minutes
Difficulty: Easy

An interviewer at Amazon asked:

Given that you have sales data with revenue, users, and country, what would be the difference if you were to compare the following variances?

The variance of the revenue across users
The variance of the revenue means across countries

Assume that there is a many-to-one relationship between users and their respective countries.

Tips & Hints

This section provides tips and hints to help you practice this case problem, effectively. The practice tip section highlights how to practice this question. The solution hint section provides clues on how to solve this problem if you get stuck. Skip the solution hint if you want to try this problem on your own.

Practice Tip

To practice this problem effectively, set a timer for 5 minutes and explain your solution out loud to yourself or a practice buddy.
Make sure to use a whiteboard, sheet of paper, or word document to jot down your ideas. Jotting down your ideas will help structure your response.
Share your solution either on the course discussion or Slack channel to receive feedback from peers and instructors.
As you read the solution dialogue, sometimes you will see “*Address this question” after an interviewer asks a follow-up to the candidate. Pause the reading for a minute to respond to this follow-up as though you are the candidate.

Solution Hint

Actively recall the variance formula as this will help you solve the problem:

If you get stuck, create a simple dataset with four to five observations. Calculate the variances and evaluate the outcomes.

Solution

[Interviewer] Given that you have sales data with revenue, users, and country, what would be the difference if you were to compare the following variances?

The variance of the revenue across users
The variance of the revenue means across countries

Assume that there is a many-to-one relationship between users and their respective countries.

[Candidate] I would like to ask clarifying questions if that is okay with you. The first variance seems straightforward to calculate. However, calculating the second variance is unclear to me. Do I need to calculate the revenue means across users within each country, then calculate the variance on those means at the country level?

[Interviewer] Good question. Yes, that is correct.

[Candidate] Thank you for the clarification. To compare the two variances, I would like to recall the variance formula, test it on a sample dataset and provide a comparison.

[Interviewer] Sounds good to me.

[Candidate] First, I would like to jot down the variance formula.

[Commentary] Notice that the candidate actively recalled the variance formula. This shows to the interviewer that the candidate readily knows the fundamental concept in statistics. In addition, the candidate can use this formula for the next step of the solution, which involves testing the two variances on a sample dataset.

[Interviewer] Okay, sounds good. What’s the next step?

[Candidate] Now, I would like to create a sample dataset to test the two types of variances.

User	Country	Revenue
1	A	1
2	A	2
3	A	3
4	B	4
5	B	4

The global average, x.bar, would be (1 + 2 + 3 + 4 + 4) / 5 = 14 / 5 = 2.8. I’m going to round the average to 3 to make the calculation easy to calculate.

So the variance of the user level becomes:

Variance (User-Level) = [ (1 – 3)^2 + (2 – 3)^2 + (3 – 3)^2 + (4 – 3)^2 + (4 – 3)^2 ] / 5

= [ (-2)^2 + (-1)^2 + (0)^2 + (1)^2 + (1)^2 ] / 5

= [ 4 + 1 + 0 + 1 + 1 ] / 5

= 7 / 5

= 1.4

On the other hand, the variance at the country level becomes:

x.bar.countryA = (1 + 2 + 3) / 3 = 2

x.bar.countryB = (4 + 4) / 2 = 4

x.bar.countryMeans = (2 + 4) / 2 = 3

Variance (Country-Level) = [ (2 – 3)^2 + (4 – 3)^2 ] / 2

= [ (-1)^2 + (1)^2 ] / 2

= [ 1 + 1 ] / 2

= 2 / 2

= 1

[Commentary] Note the way that the candidate is grounded with numbers. This not only helps the candidate make sense of the two variances, but also demonstrates to the interviewer that she’s comfortable in manipulating numbers, which is a core competency as a data scientist.

Also, note that the candidate rounded the global average from 2.8 to 3.0. She applied this to estimate the variance easily without the hassle of subtracting from decimal and applying the square root of a float value, which is difficult to perform with mental math. She simplifies the problem to her advantage so she can focus on the big picture of the problem – comparing the properties of the two variances.

[Interviewer] Now, what can you tell me about the difference between the two variances? *Address this question.

[Candidate] I can spot three differences between the two.

The variance at the user level will often be smaller than at the country level. Though the test dataset shows otherwise as the variance of the user level is 1.4 while the country level is 1.0, this is an exception. The reason that the variance at the user level will generally be smaller is because of the larger sample size in the denominator. I would imagine real-life data would have billions or millions of user observations and 150 to 200 countries. The denominator of variance of the user level would be much larger, so the variance will be smaller.
Users in larger countries would have more influence over the overall variance in the first variance. If country A contains 300 million users and country B contains 1 million users. The variance at the user level will be skewed more towards variability from the 300 million users in country A. On the other hand, the variance at the country level is agnostic to the user size per country as their revenues averaged first then the variance is computed.
The country-level variance is more prone to outliers. Simply speaking, the country-level variance has less sample size, so if one country is an influential outlier, the variance will bias towards the outlier.

[Interviewer] Great assessment! Now, if you had to choose one of the variance formulas to represent your data, which one would you choose? *Address this question.

[Candidate] I believe this depends on the application. I could see the two variances being applied in their own ways. If the analysis is performed at the user level, then it makes sense to use the first variance. But, if the analysis is performed at the country level, then use the country-level variance.

Assessment

The candidate is assessed across three attributes: statistics, data intuition and communication. Each attribute is rated based on the following scale:

Outstanding – Quick response with a sound solution.

Good – Minor mistakes, but converged toward a final solution that is sound.

Borderline – Required several hints before providing a sound solution.

Inadequate – Slow and incorrect response.

Based on the rubric above, the candidate receives the following remarks:

Statistics – Outstanding

The candidate demonstrated a strong grasp of statistics as she correctly recalls the variance formula and applied the formula to a sample dataset she created. She provided a comprehensive list of how the properties between the variance at the user vs country levels differ. Each of her points was statistically sound.

Data Intuition – Outstanding

The candidate demonstrated flexibility in applying the variance formula on a test dataset she created. Though this was not required to assess the variance at the user level vs country level, she demonstrates competency in manipulating the dataset to form a meaningful conclusion.

Communication – Outstanding

The candidate provided a structured response. She began with a clarifying question to ensure that she understands the problem. Then, she structured the response in the following order: recall the variance formula, test the formula on a sample dataset, and provide an analysis comparing the two.

[Amazon] Evaluate Variance

Problem

Unlock now by joining.