Freedman’s paradox: a solution based on normalized entropy

Abstract

In linear regression models where there are no relationships between the dependent variable and each of the potential explanatory variables – a usual scenario in real-world problems – some of them can be identified as relevant by standard statistical procedures. This incorrect identification is usually known as Freedman's paradox. To avoid this disturbing effect in regression analysis, an info-metrics approach based on normalized entropy is discussed and illustrated in this work. As an alternative to traditional statistical methodologies currently used by practitioners, the simulation results suggest that normalized entropy is a powerful procedure to identify pure noise models.