[Google] Regression Model Metrics

Last Updated: April 2024

Problem#

How do you evaluate regression model metrics?

Solution#

1. Mean Squared Error (MSE) – Calculates the average squared difference between the actual values (y_i) and the predicted values (y_hat_i) for all samples. Lower MSE indicates a better fit. But, MSE is sensitive to outliers given the squaring errors amplifies the differences.

• Formula:MSE = 1/n * Σ(y_i - y_hat_i)^2where:
  • n = number of samples
  • y_i = actual value for sample i
  • y_hat_i = predicted value for sample i

2. Root Mean Squared Error (RMSE) – Square root of MSE. Makes the errors easier to interpret in the original units of your data, especially when dealing with non-negative targets (e.g., housing prices).

• Formula: RMSE = sqrt(MSE)

3. Mean Absolute Error (MAE) – Calculates the average absolute difference between actual and predicted values. Less sensitive to outliers compared to MSE.

• Formula: MAE = 1/n * Σ|y_i - y_hat_i|

4. R-squared (Coefficient of Determination) – Represents the proportion of variance in the target variable explained by the model. Ranges from 0 to 1, with a higher value indicating a better fit. However, it’s important to note that R² can increase by simply adding more features, even if they are not relevant.

• Formula: R² = 1 - Σ(y_i - y_hat_i)² / Σ(y_i - y_mean)² where y_mean is the average of the actual values.

5. Adjusted R-squared – A modification of R² that penalizes for adding more features to the model, helping to avoid overfitting. Generally considered a more reliable measure of fit compared to the standard R².

Choosing the Right Metrics

The choice of metrics depends on several factors:

• Problem Context: If the absolute magnitude of errors is crucial (e.g., predicting financial losses), MAE might be better. If understanding the proportion of variance explained is important, R² is a good choice.
• Outliers: If your data has outliers, MAE is often more robust than MSE or RMSE.
• Scale of the Target Variable: Metrics like MSE and RMSE are affected by the scale of your target variable. Consider normalization if the scale is significantly different from the range of predicted values.