Linear Algebra Interview Questions

Sample Answer

The projection of $v$ onto $u$ is $\text{proj}_u v = \frac{v \cdot u}{u \cdot u} u = \frac{11}{5}(1,2) = (2.2, 4.4)$. The residual is $v - \text{proj}_u v = (0.8, -0.4)$, and you can verify it is orthogonal to $u$ since $(0.8)(1) + (-0.4)(2) = 0$. This orthogonality is the defining geometric property: the residual is the shortest vector connecting $v$ to the line spanned by $u$, which is exactly why projections are foundational in least squares regression.

Sample Answer

You could do row reduction or compute determinants of all $\binom{4}{3} = 4$ maximal submatrices. Row reduction wins here because it is a single sequential process that takes $O(mn \cdot \min(m,n))$ operations and immediately reveals the rank, whereas checking all 3x3 subdeterminants requires computing four separate $3 \times 3$ determinants. With small integer entries, row reduction is especially clean since you can often spot zero rows quickly. State your approach out loud as you go, because interviewers at firms like Two Sigma value structured reasoning as much as the final answer.

Sample Answer

Start with the rank-nullity theorem: since $A$ is $5 \times 5$ with rank 3, you get $\text{nullity}(A) = 5 - 3 = 2$. Next, the column space of $A^T$ is the row space of $A$, and the dimension of the row space always equals the rank, so $\dim(\text{col}(A^T)) = 3$. Finally, for $Ax = b$ to have a solution for every $b \in \mathbb{R}^5$, you need the column space of $A$ to be all of $\mathbb{R}^5$, which requires rank 5. Since the rank is only 3, there exist vectors $b$ for which $Ax = b$ has no solution.

Sample Answer

You could compute $A^{-1}$ explicitly and multiply $A^{-1}b$, or you could use LU decomposition to solve $Ax = b$ via forward and back substitution. LU wins here because it is both faster ($O(n^2)$ per solve after an $O(n^3)$ factorization) and more numerically stable than forming the full inverse. The only scenario where explicitly computing $A^{-1}$ is justified is when you need to solve $Ax = b$ for many different right-hand sides $b$ and also need the inverse entries themselves, for example to read off portfolio sensitivities. In practice at a quant desk, you almost always prefer the factorization approach.

Sample Answer

Let's think through this. You have more unknowns (500) than equations (200), so the system is underdetermined. If a solution exists, the null space of $X$ is at least 300-dimensional, meaning infinitely many solutions exist. To pick one, you need an additional criterion. The most common choice is the minimum-norm solution $\beta^* = X^T(XX^T)^{-1}y$, which you can compute via the pseudoinverse from the SVD of $X$. In a quant context, you might instead impose $L_1$ or $L_2$ regularization (Lasso or Ridge), which effectively selects a unique solution by shrinking coefficients and improving out-of-sample stability.

Sample Answer

Start by noting that $|\lambda| \leq 1$ is always true for any stochastic matrix, so that alone tells you very little. The key issue is whether $\lambda = -1$ or other eigenvalues on the unit circle (besides $\lambda = 1$) exist. If $P$ has an eigenvalue of $-1$, the chain is periodic with period 2, and iterating $P^n$ will oscillate rather than converge. You need aperiodicity (no eigenvalues on the unit circle other than the simple eigenvalue at 1) combined with irreducibility to guarantee convergence. So the claim is incomplete: the second largest eigenvalue in absolute value must be strictly less than 1, which corresponds to the chain being both irreducible and aperiodic.

Sample Answer

This question is checking whether you understand the relationship between sample size, matrix rank, and the reliability of eigenvalue estimates. Your $500 \times 500$ sample covariance matrix has rank at most $\min(500, 250 - 1) = 249$, so at least 251 eigenvalues are exactly zero, and many of the nonzero eigenvalues are distorted by noise. In practice, the top eigenvalues are biased upward and the small ones are biased downward, a phenomenon described by the Marchenko-Pastur distribution from random matrix theory. You should either use shrinkage estimators (like Ledoit-Wolf), apply a factor model to reduce dimensionality before computing the covariance, or truncate the eigenvalue spectrum by retaining only eigenvalues that exceed the Marchenko-Pastur upper edge. Interviewers want to hear that you would never naively invert or trust the raw spectral decomposition in this $p > n$ regime.

Sample Answer

Start with what the interviewer is really testing: this question is checking whether you can distinguish between full-rank and rank-deficient least squares and pick the right tool. When $m < n$ (more features than observations), the design matrix $X$ is rank-deficient, so the QR decomposition of $X$ will have zero or near-zero entries on the diagonal of $R$, making back-substitution unstable. SVD decomposes $X = U \Sigma V^T$ and lets you compute the pseudoinverse by inverting only the non-negligible singular values, giving you the minimum-norm solution directly. The cost is $O(mn^2)$ for a thin SVD versus $O(mn^2 - n^3/3)$ for QR, so SVD is more expensive, but it is the only numerically reliable option when rank deficiency is present. In quant finance, this scenario arises constantly when you have a large factor zoo relative to your time series length.

Sample Answer

The standard move is to precompute the LU decomposition $A = LU$ (or $PA = LU$ with partial pivoting) at $O(n^3)$ cost once, then solve each new right-hand side with forward and back substitution at $O(n^2)$ per solve. But here, the structure of $A$ matters because if $A$ is symmetric positive definite, for example a covariance matrix, you should use Cholesky ($A = LL^T$) instead, which is about twice as fast to factor and is more numerically stable for that class of matrices. The key insight the interviewer wants is that you separate the expensive factorization phase from the cheap solve phase, and you choose the factorization that best matches the matrix structure.

Sample Answer

The standard move is to analyze projection and rotation separately, then compose. But here, the composition matters because the rotation lives in the 2D image of the projection, not in the full space. The projection kills one dimension (eigenvalue 0 for the normal direction), and the rotation by 45 degrees within the plane has eigenvalues $e^{\pm i\pi/4} = \frac{\sqrt{2}}{2} \pm \frac{\sqrt{2}}{2}i$, which are complex. So $T$ has eigenvalues $0, \frac{\sqrt{2}}{2} + \frac{\sqrt{2}}{2}i, \frac{\sqrt{2}}{2} - \frac{\sqrt{2}}{2}i$ and is not diagonalizable over the reals since the rotation component in the plane has no real eigenvectors (no vector in that plane maps to a scalar multiple of itself under a nonzero, non-180 degree rotation).

Sample Answer

Get this wrong in production and your regression coefficients explode or become garbage due to numerical instability, which in a trading system means wildly incorrect position sizing. The normal equations require forming $X^T X$ and solving $(X^T X)\beta = X^T y$, which squares the condition number: if $\kappa(X) = 10^6$, then $\kappa(X^T X) = 10^{12}$, pushing you past double precision limits. QR decomposition avoids this by directly computing the orthogonal projection $\hat{y} = Q Q^T y$ and solving $R\beta = Q^T y$, keeping the condition number at $\kappa(X)$. You should flag that near-collinear features (common with correlated financial signals) are exactly the regime where this distinction matters most.

Sample Answer

Get this wrong in production and you end up with a portfolio that is essentially leveraging estimation error, blowing up on transaction costs and taking unintended concentrated bets. The right call is to examine the condition number $\kappa(\Sigma) = \lambda_{\max}/\lambda_{\min}$ of your covariance matrix: when it is large, $\Sigma^{-1}$ amplifies noise in your expected return vector $\mu$, producing extreme weights $w^* = \Sigma^{-1}\mu$. You should explain that near-zero eigenvalues correspond to directions in asset space that are nearly collinear, and inverting them creates huge sensitivity. Practical fixes include truncating small eigenvalues, using a shrinkage estimator, adding a ridge penalty $\Sigma + \delta I$, or imposing explicit weight constraints.

Sample Answer

Using the top PCs sounds reasonable but they primarily capture well-known market and sector factors, which are already priced in and unlikely to predict next-day returns out of sample. Using the bottom PCs doesn't work because those directions correspond to the smallest eigenvalues of the sample covariance, meaning they are dominated by estimation noise and are unstable across time periods. That leaves a more nuanced approach: you should test intermediate PCs or, better yet, use a supervised method like partial least squares (PLS) or reduced-rank regression that finds linear combinations of returns maximally predictive of your target, rather than maximally explanatory of variance. You can validate by checking whether the candidate factors have stable loadings and out-of-sample predictive power using rolling windows.

Sample Answer

Most candidates default to choosing $k$ by looking at cumulative variance explained (e.g., 90%), but that fails here because in stat arb you trade the residuals, not the factors, so your goal is to remove systematic risk cleanly rather than compress variance. If $k$ is too small, systematic risk leaks into your residuals and your "alpha" signals are actually unhedged factor bets. If $k$ is too large, you absorb idiosyncratic signal into the factor structure and your residuals become too thin to trade. You should use the Marcenko-Pastur distribution or cross-validation on held-out time periods to identify the number of statistically significant eigenvalues, then verify that the resulting residual covariance matrix is approximately diagonal, which confirms the factors have absorbed the cross-sectional correlation structure.