Causal Inference Interview Questions

Sample Answer

ITT is the causal effect of assignment, not of exposure: $$\text{ITT} = \mathbb{E}[Y(1) - Y(0)]$$ where $1$ and $0$ index randomized assignment. Randomization gives you exchangeability, so $Z \perp (Y(1), Y(0))$, which is what makes ITT identifiable by the difference in observed means by assignment. Noncompliance does not break the causal interpretation of ITT because it is defined on assignment, not on what users actually experienced. What it does break is interpreting the estimate as the effect of actually seeing the new UI, which would require additional assumptions or an IV style estimand.

Sample Answer

You could target the ATE over all eligible users, or the ATT over users who actually receive autoplay exposure or behave as treated. ATE wins here because treatment is randomized, so $\mathbb{E}[Y(1) - Y(0)]$ is identified cleanly by the difference in means by assignment. ATT based on realized exposure is usually confounded because exposure is post-assignment and correlated with preferences and device constraints, so $\mathbb{E}[Y(1)-Y(0)\mid D=1]$ is not identified without extra structure. If you truly need an effect for compliers or engagers, you move to principal strata or LATE with an instrument, and you must state those assumptions explicitly.

Sample Answer

Step one, name the variables: $Z$ is assigned treatment, $D$ is the UI actually seen, and $Y$ is something like acceptance rate. Consistency says the observed outcome equals the potential outcome under the treatment actually received, so if you only record $Z$ but users can switch versions, $Y \neq Y(Z)$ is no longer guaranteed. Then your naive estimator, the difference in means by $Z$, remains an ITT for assignment, but it is not the effect of actually seeing the new screen. To fix it, you either enforce treatment delivery so $D=Z$, or you analyze ITT explicitly, or you use $Z$ as an instrument for $D$ and estimate a complier effect under exclusion and monotonicity.

Sample Answer

You could do propensity score matching to estimate an effect for drivers in the overlap region, or inverse probability weighting to reweight toward a target population like ATE. Matching wins here because uneven treatment rates and city level selection will likely create extreme weights and high variance under IPW unless overlap is excellent. With matching, you can enforce common support and calipers, then report the ATT on matched treated drivers, which is often the most policy relevant estimand for targeted rollouts. If you must do weighting, you would prefer stabilized weights and possibly a doubly robust estimator to reduce sensitivity.

Sample Answer

First, I would draw a quick DAG and list plausible common causes of render and watch time, like device type, network, time of day, tenure, and prior watch history. Next, I would define a strict pre treatment feature window so anything measured after the page request, like scroll depth or immediate interactions, is excluded to avoid conditioning on mediators. Then I would fit $e(x)$ with flexible structure, for example splines or gradient boosting, and check overlap plus balance after adjustment rather than trusting AUC. Finally, I would stress test by varying feature sets and showing balance and effect stability, because a well calibrated propensity model can still leave imbalance in key confounders.

Sample Answer

This question is checking whether you can diagnose failed adjustment and change strategy rather than forcing an answer. If stratification leaves large standardized mean differences, you should increase strata granularity, switch to matching with calipers, or use weighting with balance targeted estimation, and you should refit $e(x)$ with nonlinearities and interactions for variables like price and review score. You also consider trimming to common support if overlap is weak, then clearly restate the estimand because you are no longer estimating the full ATE. To stakeholders, you explain that if the propensity model cannot balance key confounders, the estimate is biased even if the code runs, and the dominant uncertainty is structural, not just sampling error.

Sample Answer

First, define the panel at city-week and mark $D_{it}=1$ once city $i$ is treated, then stays 1. Next, realize two-way fixed effects can mix comparisons across early and late adopters, and if effects change over event time, you can get negative weights and a misleading average. Then, estimate cohort-specific event-time effects using a staggered DiD estimator like Sun and Abraham style interactions, or compute group-time ATT as in Callaway and Sant'Anna. Finally, interpret results as $ATT(g,t)$ or event-time $\tau_k$, not as one global $\beta$.

Sample Answer

This question is checking whether you can diagnose when the canonical two-way fixed effects DiD breaks under staggered adoption and heterogeneous treatment effects. You look for contamination from using already-treated units as controls, dynamic effects that vary with event time, and anticipation effects that violate the no pre-treatment effect assumption. You fix it by estimating cohort-specific effects, reporting event-study dynamics with proper controls, and aggregating to a well-defined estimand like the weighted average of group-time ATTs. You also check whether there is policy spillover across countries, because interference invalidates standard DiD.

Sample Answer

This question is checking whether you can translate IV assumptions into who your estimate applies to. With encouragement IV, your estimate is the effect for compliers, users who switch to the new model because of the banner, not always-takers or never-takers. If compliers are more exploration-prone, the LATE can overstate impact for the broader base, so rolling out globally based on it can disappoint. You should recommend pairing IV with compliance segmentation, and clarify that the PM is choosing whether to target compliers or to predict general population impact.

Sample Answer

The standard move is to claim timing only changes purchases through app opens, then run 2SLS. But here, timing can directly affect purchases by shifting attention, competing notifications, or aligning with paydays or work breaks, so exclusion is fragile. You should test for direct effects on outcomes that should not respond to opening, like off-platform purchases or app-unrelated conversions, and check whether timing affects session composition or ad exposure even among non-openers. You also look for heterogeneous reduced-form effects by time zone, day-of-week, and prior notification sensitivity, since patterns consistent with circadian demand suggest exclusion failure.

Sample Answer

The standard move is to use local linear regression with a data-driven bandwidth, like Imbens Kalyanaraman or CCT, and report robust bias-corrected confidence intervals. But here, functional form matters because churn and offer targeting can create curvature, and high-order global polynomials often fake discontinuities and explode at the edges. You should run sensitivity: multiple bandwidths, local linear versus local quadratic, different kernels, and show the estimate is stable. If estimates flip sign or grow with bandwidth, you likely have misspecification or other policy changes near 0.80.

Sample Answer

Get this wrong in production and you attribute revenue changes to monetization when it is really selection into enabling, which leads to bad rollout decisions. The right call is: eligibility at 50 is a sharp assignment rule, but treatment uptake is imperfect, so you have a fuzzy RD. You estimate the intent-to-treat as the discontinuity in outcomes at 50, then estimate the local average treatment effect for compliers via $$\tau_{FRD} = \frac{\Delta Y}{\Delta D}$$ where $D$ is enabling monetization. You also report the first-stage jump in $D$ at 50 and make clear the LATE applies to creators near the cutoff whose enabling behavior is moved by eligibility.