Question

I have a linear model. I add a feature to the model and retrain it.

  1. If \(R^2\) increases, feature is significant
  2. If \(R^2\) decreases, feature is not significant
  3. Can’t say anything about significance with \(R^2\)

Answer

We can’t comment on significance of a feature with \(R^2\). The \(R^2\) is bound to increase (except collinear features, where it stays the same) as we add more and more features.

The significance of a feature can be interpreted from it’s p value, the lower the p value, the more significant it is.

Thanks for reading. If you find a correction or would like to know more, hit me up on Twitter and if you like the question, how about a tip: PayPal TipJar