Product Sense Interview Questions

Sample Answer

Primary metric: incremental successful search sessions per exposed user. Justify it by tying to user value, a session is successful when it ends without quick reformulation and with strong downstream satisfaction signals. Supporting metrics: reformulation rate within $t$ minutes, long click rate or dwell time on clicked results, and query abandonment rate, each segmented by query class and confidence bucket. Add a safety check by tracking help center complaints or explicit bad answer reports, but keep it separate from the decision metric so you do not overreact to low volume anecdotes.

Sample Answer

You could optimize tips per trip, or you could optimize total driver earnings per online hour. Tips per trip wins only if the company goal is specifically gratuity capture and you can prove it does not change rider behavior, otherwise it is too narrow and can backfire via cancellations or lower ratings. Earnings per online hour wins here because it aligns to driver value and incorporates tradeoffs across fares, tips, and supply. Use guardrails like rider rating, cancellation rate, and trip completion to ensure you are not extracting tips at the expense of marketplace health.

Sample Answer

Step 1, restate the product goal: better discovery that increases sustained listening, not just scrolling. Step 2, pick a north star like listening minutes of successful sessions per user, where a successful session includes at least one intentional play and low skip rate. Step 3, add guardrails, for example skip rate, saves or likes per session, and long term retention like D7, because time spent can be inflated by browsing. Interpreting the readout, the layout may be increasing browsing friction or novelty clicks without improving content satisfaction, so you dig into session composition, skips, and saves by cohort and by module exposure.

Sample Answer

You could randomize at the rider level and measure cancellations per request, or randomize at the session level and measure cancellations per active session including those who never request. Rider-level per-request metrics can look great while hiding selection, because the warning can suppress marginal requests and change who enters the denominator. Session-level wins here because it captures the full funnel and lets you quantify tradeoffs: fewer cancellations versus fewer requests and completed trips. You still keep per-request cancellation as a secondary metric to diagnose whether the effect is prevention or just demand suppression.

Sample Answer

First, you define the hypothesis: autoplay previews increase content discovery, so play starts and long watch time should rise without hurting satisfaction. Then you randomize at the profile level and track a primary metric like weekly hours watched per profile, plus guardrails like % plays abandoned in the first 2 minutes and thumbs-down rate. To handle novelty, you look at the time path of effects by week, and you plan a longer holdout or a ramp with re-randomization checks to see if gains decay. For multiple outcomes, you pre-specify one primary, correct for multiple comparisons on secondary metrics, for example using Benjamini Hochberg on $p$-values, and interpret consistent directional changes rather than cherry-picking.

Sample Answer

This question is checking whether you can pick metrics that reflect user value, not just clicks, and whether you can anticipate validity threats when the UI changes what gets clicked. You run an A/B test with user-level randomization, but you do not treat raw CTR as the primary because layout changes induce position bias and can inflate clicks without improving satisfaction. You prioritize outcomes like successful search rate, query reformulation, and long-click or dwell time, with guardrails on latency and abandonment, and you validate instrumentation because the module can change what counts as an impression. For long-term trust, you add a holdout or long-term tracking for return rate and complaint signals, and you check heterogeneous effects on sensitive query classes.

Sample Answer

First I would sanity check definitions, DAU is typically any open, minutes require a play event, so a tracking or logging issue could create divergence. Next I would decompose minutes as $\text{minutes} = \text{DAU} \times \text{sessions per user} \times \text{minutes per session}$ and see which component moved. Then I would segment DAU growth by acquisition channel, country, platform, and new vs returning, a spike in low intent users from a notification or marketing push can raise DAU while lowering per user engagement. Finally I would check playback error rate, buffering, and app crashes, because a client bug can inflate opens while suppressing plays and minutes.

Sample Answer

This question is checking whether you can translate a metric movement into its components and then isolate which lever moved. Start from $$\text{Revenue} = \text{Impressions} \times \text{CPM}/1000$$ and confirm the arithmetic, flat revenue with those moves is plausible, so the real work is causal attribution. You should slice by placement, geo, device, and advertiser vertical to see whether impressions fell due to supply, policy, or ranking, and whether CPM rose due to auction pressure or pacing. Then evaluate quality by checking auction metrics like bid density, fill rate, and user outcomes like session time, because a CPM lift that comes from reduced inventory can be positive or harmful depending on engagement and long term demand.

Sample Answer

The standard move is to localize the break in the funnel and the blast radius, then correlate it with recent changes and operational signals. But here, the time and platform specificity matters because it screams client release, payments integration, or a backend dependency that iOS hits differently. In the first hour you should verify event instrumentation for checkout steps, check payment authorization and error codes, and compare iOS app versions to see if the drop aligns with a rollout. If errors spike at 7 pm, you escalate to engineering, consider a rollback or feature flag disable, and monitor recovery using a narrow iOS checkout success metric.

Sample Answer

This question is checking whether you can trade off a win metric against user experience and long term outcomes, without getting fooled by short term CTR. You should pick a primary objective like successful search sessions, then pair it with guardrails: p50 and p95 latency, pogo-sticking rate, dwell time, query reformulation, and long-click rate. At launch, segment by device and network, since the load time impact is not uniform and can flip the average treatment effect. Decide with an explicit constraint, for example only ship if $\Delta CTR > 0$ and $\Delta p95\_LCP \le 50\text{ms}$ while satisfaction proxies do not degrade.

Sample Answer

The standard move is to optimize for incremental gross order value and conversion, with checkout completion rate as the immediate win metric. But here, friction matters because a small increase in checkout time can cause drop-offs and support costs. You should track funnel metrics (checkout start to submit, time in checkout), incremental attach rate for paid add-ons, and quality guardrails like missing item reports, refunds, and customer support contact rate. Call it a win only if you see positive $\Delta$ revenue per checkout start without a statistically meaningful rise in cancellation, defects, or support per order.

Sample Answer

Get this wrong in production and you can spike bookings short term while driving host churn, lower acceptance, and worse guest experience, which is costly and slow to recover. The right call is to measure both sides of the marketplace: guest conversion and booking rate, plus host acceptance, cancellation, response time, and listing availability. You also need quality metrics like rebooking rate, complaint rate, and CS contacts, since Instant Book can increase mismatched expectations. Post-launch, watch for strategic host behavior like toggling eligibility or tightening rules, so segment by host type and track supply changes over multiple weeks.

Sample Answer

The standard move is to improve the funnel metric closest to revenue, like search conversion. But here, repeat bookings are down, so you should prioritize the lever that reduces bad stays and increases trust, which is listing quality, because it improves the core experience that drives habit. Support speed helps when things go wrong, but it does not prevent the failure and can even mask quality issues, weakening incentives for hosts to improve. You would validate by measuring changes in rebooking rate, complaint rate per stay, and refund rate, then tie it to a loop: better stays lead to better reviews, which attract better guests and hosts, which raises marketplace quality.

Sample Answer

Get this wrong in production and you either burn margin with incentives or degrade food quality and churn by over-batching. The right call is to quantify the cost per minute of ETA improvement for each lever and pick the one that improves the marketplace flywheel, not just the metric. Incentives can be a short-term stabilizer during peak imbalance, but batching is only safe if it does not raise lateness variance and item defect rates, so you set guardrails like p90 lateness, cancellation rate, and reorder rate within 14 days. You also segment by time of day and zone, because the optimal lever depends on supply elasticity and restaurant prep time bottlenecks.