Join Our 5-Week ML/AI Engineer Interview Bootcamp 🚀 led by ML Tech Leads at FAANGs
164. Bootstrap confidence intervals
GeneralBootstrap confidence intervals help you quantify uncertainty around an ML evaluation metric, like accuracy or MAE, by repeatedly resampling your dataset and recomputing the metric. In this problem, you’ll implement a simple bootstrap procedure to return a two-sided confidence interval for a metric computed on y_true and y_pred.
Requirements
Implement the function
The bootstrap CI is based on the percentile method:
where (m^{(b)}) is the metric computed on bootstrap resample (b).
Rules:
- For each bootstrap iteration, sample indices with replacement from
0..n-1, compute the metric on the resampled pairs, and store it. - Support
metric="accuracy"(treat values>= 0.5iny_predas class 1, else 0) andmetric="mae"(mean absolute error). - Use
np.random.default_rng(seed)for reproducible sampling. - Return the two bounds as a Python tuple
(lower, upper). - Use only NumPy and Python built-in libraries (no sklearn/scipy).
Example
Output:
| Argument | Type |
|---|---|
| seed | int |
| alpha | float |
| metric | str |
| y_pred | np.ndarray |
| y_true | np.ndarray |
| n_bootstrap | int |
| Return Name | Type |
|---|---|
| value | tuple |
Constraints
Use only NumPy; no sklearn/scipy.
Use np.random.default_rng(seed) for sampling.
Return Python list [lower, upper].
Hint 1
Convert y_true/y_pred to NumPy arrays and create an RNG with np.random.default_rng(seed) so resampling is reproducible.
Hint 2
In each bootstrap iteration, sample n indices with replacement via rng.integers(0, n, size=n), then compute the metric on y_true[idx] and y_pred[idx].
Hint 3
Store all bootstrap metric values, then return [np.quantile(vals, alpha/2), np.quantile(vals, 1-alpha/2)] as Python floats. For accuracy, binarize with (y_pred >= 0.5); for mae, use mean(abs(y_true-y_pred)).
Roles
Companies
GeneralLevels
Tags
Python Editor