Last Updated: September 2021

Problem

Duration: 20 Minutes
Difficulty: Hard

An interviewer at Facebook asked:

What’s the shape of the distribution of comments on Facebook posts?
What statistical test would you use based on the distribution you specified?

Tips & Hints

This section provides tips and hints to help you practice this case problem, effectively. The practice tip section highlights how to practice this question. The solution hint section provides clues on how to solve this problem if you get stuck. Skip the solution hint if you want to try this problem on your own.

Practice Tip

To practice this problem effectively, set a timer for 20 minutes and explain your solution out loud to yourself or a practice buddy.
Make sure to use a whiteboard, sheet of paper, or word document to jot down your ideas. Jotting down your ideas will help structure your response.
Share your solution either on the course discussion or Slack channel to receive feedback from peers and instructors.
As you read the solution dialogue, sometimes you will see “*Address this question” after an interviewer asks a follow-up to the candidate. Pause the reading for a minute to respond to this follow-up as though you are the candidate.

Solution Hint

If you are struggling to frame the response, use the following structure to provide an effective response:

Clarifying Questions – Ask clarifying questions to address assumptions before diving into a solution.
Data – How would you describe the shape of the Facebook comments distribution? Why?
Statistical Test – What statistical test would you use? Why?

Solution

[Interviewer] What’s the shape of the distribution of comments on Facebook posts?

[Candidate] Thank you for the question. To address this problem, I would like to ask clarifying questions.

[Interviewer] Sure thing.

[Candidate] Do we care about posts on News Feed or within a product such as Groups?

[Interviewer] Good question. Let’s say that the focus of the problem is on News Feed.

[Candidate] Great. I believe there are two potential distributions – a normal distribution of comment counts or a distribution heavily skewed to the right. In any case, both are truncated at 0 given that a post cannot have a negative number of comments, and unbounded in the positive range.

[Interviewer] Okay, based on the choices you provide, one is normal and another is right-skewed. What do you believe would be the distribution of Facebook comments across posts?

[Candidate] I believe that the most likely distribution is right-skewed. The reason is that the bulk of posts would receive a low number of comments while only a handful would receive a high number of comments.

[Interviewer] Sounds good. Can you tell me anything else about the distribution you chose?

[Candidate] Yes, I suspect that the mean of the distribution would like somewhere between 1 to 5. Also, the distribution would contain outliers on the extreme. Finally, given that the distribution is right-skewed, the median would be higher than the mean.

Commentary: The candidate demonstrated strong data intuition when describing the shape of the distribution of comments. Indeed, the distribution is right-skewed with outliers that skew the mean to the right of the median.

[Interviewer] Sounds good. Now, here’s the second part of the question – What statistical test would you use based on the distribution you specified?

[Candidate] Just a quick follow-up question. Is the statistical test applied in an AB test where the comment size of control and treatment groups are compared?

[Interviewer] Yes.

[Candidate] There are two statistical tests to use. T-Tests for comparing two means or Wilcoxon-Whitney-Mann test (WWM). The former is parametric while the latter is non-parametric.

[Interviewer] Okay, which one would you use?

[Candidate] When the distribution is non-normal, we can use WWM. But, this is not necessary. Under large-sample size, even if the population distribution of the comments is non-normal, T-Test can work. This is because the T-Test is a statistical test on the sample mean. Given the Central Limit Theorem, as the sample size increases, the distribution converges toward normal. Hence, the type 1 error rate and CI are still functional.

In fact, if you apply a one-sample T-Test on data that follows an exponential distribution where the mean is 5, and you run a simulation of 1,000 tests with the significance level or the type 1 error rate set as 5%. Your result wouldn’t break as 95% of those tests would fail to reject while the remaining 5% of those tests would falsely reject.

So, the T-Test is sufficient here.

Assessment

The candidate is assessed across three attributes: statistics, data intuition and communication. Each attribute is rated based on the following scale:

Outstanding – Quick response with a sound solution.

Good – Minor mistakes, but converged toward a final solution that is sound.

Borderline – Required several hints before providing a sound solution.

Inadequate – Slow and incorrect response.

Based on the rubric above, the candidate receives the following remarks:

Statistical Methodology – Outstanding

The candidate demonstrated strong knowledge in statistical theory and applications. She bridged a statistical theory, CLT, and linked it to the T-Test. She is correct that the T-Test can be used as long as the sample size is large, even when the data is non-normal. One suggestion would be how she would know if the sample size is “large.” Typically, a sample size larger than 30 for a T-Test is considered large. For large online-scale testing, the sample size would be much larger in tens of thousands. Hence, would mitigate an issue of the non-normal distribution. Nonetheless, her statistical knowledge is sound.

Data Intuition – Outstanding

The candidate was highly specific when describing the distribution. She mentioned whether the distribution of comments is normal or skewed. In addition, provided details about the mean, median and outliers of the distribution.

Communication – Outstanding

The candidate asked questions to ensure that she understands the problem. In addition, the candidate explained her solution in a clear and comprehensive manner.

[Meta] Distribution of Comments

Problem

Unlock now by joining.