In ridge regression, irreducible error as s increases?

In ridge regression, irreducible error as s increases?

• A. Increases
• B. Decreases
• C. Remains constant
• D. Fluctuates

Answer: (C)

Ridge regression changes the fit by shrinking coefficients to reduce variance, while the irreducible error comes from the randomness in the data-generating process y = Xβ + ε. That irreducible part is the variance of ε and cannot be explained away by any estimator or by adjusting regularization. So when you increase the regularization strength, the irreducible error term in the prediction error does not change—it stays the same. (If you were to change the actual noise level in the data-generating process, that irreducible error would of course change, but that's not what the regularization parameter does.)

Ridge regression changes the fit by shrinking coefficients to reduce variance, while the irreducible error comes from the randomness in the data-generating process y = Xβ + ε. That irreducible part is the variance of ε and cannot be explained away by any estimator or by adjusting regularization. So when you increase the regularization strength, the irreducible error term in the prediction error does not change—it stays the same. (If you were to change the actual noise level in the data-generating process, that irreducible error would of course change, but that's not what the regularization parameter does.)