The Geometry of Enhancement in Multiple Regression

Abstract

In linear multiple regression, “enhancement” is said to occur when R ²=b′r>r′r, where b is a p×1 vector of standardized regression coefficients and r is a p×1 vector of correlations between a criterion y and a set of standardized regressors, x. When p=1 then b≡r and enhancement cannot occur. When p=2, for all full-rank R _xx≠I, R _xx=E[xx′]=V Λ V′ (where V Λ V′ denotes the eigen decomposition of R _xx; λ ₁>λ ₂), the set $\boldsymbol{B}_{1}:=\{\boldsymbol{b}_{i}:R^{2}=\boldsymbol{b}_{i}'\boldsymbol{r}_{i}=\boldsymbol{r}_{i}'\boldsymbol{r}_{i};0<R^{2}\le1\}$ contains four vectors; the set $\boldsymbol{B}_{2}:=\{\boldsymbol{b}_{i}: R^{2}=\boldsymbol{b}_{i}'\boldsymbol{r}_{i}>\boldsymbol{r}_{i}'\boldsymbol{r}_{i}$; $0<R^{2}\le1;R^{2}\lambda_{p}\leq\boldsymbol{r}_{i}'\boldsymbol{r}_{i}<R^{2}\}$ contains an infinite number of vectors. When p≥3 (and λ ₁>λ ₂>⋯>λ _p ), both sets contain an uncountably infinite number of vectors. Geometrical arguments demonstrate that B ₁ occurs at the intersection of two hyper-ellipsoids in ℝ^p. Equations are provided for populating the sets B ₁ and B ₂ and for demonstrating that maximum enhancement occurs when b is collinear with the eigenvector that is associated with λ _p (the smallest eigenvalue of the predictor correlation matrix). These equations are used to illustrate the logic and the underlying geometry of enhancement in population, multiple-regression models. R code for simulating population regression models that exhibit enhancement of any degree and any number of predictors is included in Appendices A and B.