Last Updated: November 2021

Problem

Duration: 20 Minutes
Difficulty: Medium

An interviewer at Facebook asked:

How would you improve Facebook Messenger?

Tips & Hints

This section provides tips and hints to help you practice this case problem, effectively. The practice tip section highlights how to practice this question. The solution hint section provides clues on how to solve this problem if you get stuck. Skip the solution hint if you want to try this problem on your own.

Practice Tip

To practice this problem effectively, set a timer for 20 minutes and explain your solution out loud to yourself or a practice buddy.
Use a whiteboard, sheet of paper, or word document to jot down your ideas, which can help structure your response.
Share your solution either on the course discussion or Slack channel to receive feedback from peers and instructors.
As you read the solution dialogue, sometimes you will see “*Address this question” after an interviewer asks a follow-up to the candidate. Pause the reading for a minute to respond to this follow-up as though you are the candidate.

Solution Hint

A naive solution proposes ideas without backing them up based on user data. Think about how you would extract ideas based on the behavioral data on Messenger.
Approach the solution by in this procedure. First, elaborate on the business and product context. Second, two to three ideas that improve the product. Third, explain how you would validate whether to build the idea using data. Lastly, explain which idea to prioritize first.

Solution

[Interviewer] How would you improve Facebook Messenger?

Commentary: To address such a question on how to improve a product, it helps to familiarize yourself with the user experience and current features. Knowing what’s already on the existing product helps you identify potential bottlenecks and suggestions that can improve the feature. Otherwise, you risk proposing a solution that is redundant. So, prior to any product-case interviews, make sure to research the flagship products of a company interviewing you.

[Candidate] Thank you for the question. First of all, I would like to start by demonstrating my understanding of Facebook Messenger. This can serve as a starting point for us to identify a potential enhancement of the app.

[Interviewer] Sounds good.

[Candidate] I know that Messenger allows Facebook users to connect via messages, which are texts, images, videos, and such. The current app contains a couple of other features such as a (1) checkmark if a message is read, (2) “thumbs-up” button, and (3) group messaging capability.

These are features implemented to improve the user experience on Messenger. As the users engage the app more, the better it is for Facebook’s business in terms of active engagement and revenue in the long term.

Commentary: Note that the candidate demonstrated expertise from the gecko. She didn’t just ask, “What is the goal of improving Facebook Messenger?” There are moments when clearly identifying the goal of a problem is vital. For instance, when addressing a question, such as “How would you measure the user growth of a product?” It’s vital to address the goal in terms of who is the key audience for the metrics. For example, users could be executives, product managers, and engineers.

In this particular question, the aim of this problem statement is obvious. Improve Facebook Messenger because it helps the user experience. The candidate recognizes this and proceeds in demonstrating two key details:

Product Understanding – explains the current user experience on Messanger
Business Sense – explains how improving the user experience can serve Facebook’s business in the long term.

[Interviewer] Great, now how would you come up with ideas to improve Messenger?

[Candidate] I would start by investigating the current user behavior and identifying potential bottlenecks. To do so, can I assume that I have access to user event data such as messages?

[Interviewer] Yup, presume that you have access to any kind of data collected from users on Messenger.

[Candidate] Great, using the message data, I can identify how users currently engaging with one another. Finding message types that are frequent can inspire features that provide convenience to users.

For instance, given that users convey positive messages such as, “I like this,” “I love it,” “awesome,” adding a “thumbs up” button in the message box can help them express the sentiment much quicker.

Commentary: Note that the candidate proposes the idea of using data to conjure up ideas. The primary function of a data scientist is to make sound business and product decisions based on data. In a product question that involves improving a product, always mention how you would leverage data to validate a product idea.

[Interviewer] I do like the idea of analyzing messages the way you mention. But, how would you know which messages to take a look at? And, given that there are billions of messages, how would you conduct an analysis?

[Candidate] I would need to look at message types that frequently appear in the message history. To find the message types, I can apply natural language processing (NLP) to cluster messages into types. Without going in too in-depth on the methodology, for each message, I can embed the meaning in a vector using term-frequency and index document frequency (TF-IDF). I will then apply a clustering technique using either the K-Means or hierarchical clustering to group the messages.

Using the frequency of message types, I can see what kind of messages are often exchanged amongst users.

For instance, if users are sending messages such as, “here’s the address …” or “the address is at…”, these messages would be grouped into a cluster that conveys providing a location to someone.

If this cluster has a high volume of messages, perhaps a map from Google Maps or Apple Maps can be displayed in the Messenger app so the sender doesn’t have to copy the search result from the external app and paste it onto the message box. And, the receiver can see the location from the Messenger app itself.

Commentary: The candidate provided a solid response to how she would identify the types of messages to focus on given that there are billions of unique messages. She provided just enough technical details on how she would cluster messages.

Another technical approach could be using a different text-embedding technique such as Word2Vec, which is a neural network approach to representing a sparse text vector into a dense vector. Then, a clustering technique such as K-Means or hierarchical clustering can be applied to group the messages.

Note that, in terms of what technique she would use, unless the interviewer asks, it’s arbitrary as the focus isn’t the algorithmic methodology, rather a demonstration of product sense.

[Interviewer] Sounds good, can you come up with other ideas that can help the user experience based on the technique you mentioned?

[Candidate] One idea I have is looking at message types that involve creating group events. Based on my personal experience on Messenger, I sometimes coordinate event details including location and time with friends and save the details on an external calendar app. These extra steps waste time and take users away from Messenger and onto an external calendar app. I believe the friction point can be removed with an event group automatically generated upon a message with an event type is detected.

The existing Event feature on Facebook could be integrated. So, if users exchange event details in a message such as, “I will see you at 6 P.M. this Friday at the Hicks & Shaw bar.”

An NLP system can ask a user, “would you want to create an Event post?”, in the message box. If the user clicks yes, the details can be parsed and an event would be generated automatically for the friends in the conversation.

And, of course, I can validate whether this is an idea worth building based on how frequently such message type that involves creating an event among friends emerge from the NLP analysis as mentioned previously.

[Interviewer] Given the two ideas you proposed, the first, a location from a map app displayed in a conversation, and, the second, automatic creation of an event, how would you prioritize which feature to focus on?

[Candidate] I’d like to evaluate the ideas based on pros and cons, then provide a recommendation. I believe that the cons of both features are that they require (1) time and engineering efforts in building an NLP model that accurately detects the event types and (2) integration with other products internal or external to Messenger.

The pros are that they both can save time for users and get them to stay on the app which increases active engagement.

However, the second one is more meaningful than the first. While automatically displaying the map of a location might be useful information for users, automatically creating an event allows users to engage on the Facebook platform further.

Once an Event group is created, a user could invite other friends and engage in the event group more, which further increases the overall engagement of the platform. This ultimately increases Facebook’s key metric which is active minutes per user per day and increases advertisement revenue.

Commentary: A common question asked in a product data science interview involves how would you prioritize which feature to build. To address this question, you can approach this in terms of frameworks.

Pros & Cons
Effort Prioritization – Impact vs Effort

For each feature listed, provide a list of pros and cons. For instance, the benefit of enabling automatic location display from a map app is that users can save time. But, the cost could be the engineering effort required to build this in the form of NLP detection and integration with the maps app.

In addition to the pros & cons analysis, you could also evaluate in terms of impact & effort. The effort to build the product could be easy to medium in terms of time and technical challenge, but the impact could be low if users do not find the display meaningful.

On the other hand, the idea of building a feature that automatically detects an event created in a conversation and generates an Event post may have a similar level of effort as the map display feature. But, this second feature could have a higher impact as this provides additional ways users can engage across Facebook.

If the efforts are deemed similar, but the second idea has a higher potential impact than the first, then it might be worth pursuing the second idea first.

Assessment

The candidate is assessed across two attributes: product sense and communication. Each attribute is rated based on the following scale:

Outstanding – Quick response with a sound solution.

Good – Minor mistakes, but converged toward a final solution that is sound.

Borderline – Required several hints before providing a sound solution.

Inadequate – Incorrect response.

Based on the rubric above, the candidate receives the following remarks:

Product Sense – Outstanding

From the beginning to the end, the candidate demonstrated product expertise in addressing how to improve Facebook Messenger. In the beginning, she clearly established the business goal of the problem and demonstrated product knowledge. This shows that she understands that a business context is vital in a data science project and actively familiar with Facebook’s core product. In addition, when asked about how she would come up with ideas. Her idea of clustering messages and looking at message types that appear often was smart. Lastly, she understands how to prioritize ideas based on pros/cons and effort/impact.

Communication – Outstanding

She received an outstanding remark given that, overall, she provided answers that were comprehensive, clear, and focused. For instance, when asked about how she would utilize the message data to identify message types that emerge often, she explained a modeling solution that was easy to follow.

[Meta] How would you improve Facebook Messenger?

Problem

Tips & Hints

Unlock now by joining.