1
This final draft waspublished in International Journal of Research in Marketing, 33 (2016), 172-182. It is fully accessible to all users at libraries and institutions that have purchased a license. If you do not have access, please send an email to and we will gladly provide a copy of the published version.
Diagnosing Harmful Collinearity in Moderated Regressions: A Roadmap
Pavan Chennamaneni
Department of Marketing
University of Wisconsin-Whitewater
Whitewater, WI, 53190-1790
e-mail:
Phone: 262-472-5473
Raj Echambadi*
Department of Business Administration
University of Illinois at Urbana-Champaign
Champaign, IL61820.
Email:
Phone: 217-244-4189
James D. Hess
Department of Marketing and Entrepreneurship
University of Houston
Houston, TX 77204
Email:
Phone: 713-743-4175
Niladri Syam
Department of Marketing and Entrepreneurship
University of Houston
Houston, TX 77204
Email:
Phone: 713-743-4568
* Corresponding Author
March 2015
The names of the authors are listed alphabetically. This is a fully collaborative work.
1
Diagnosing Harmful Collinearity in Moderated Regressions
ABSTRACT
Collinearity is inevitable in moderated regression models. Marketing scholars use a variety of collinearity diagnostics including variance inflation factors (VIFs) and condition indices in order to diagnose the extent of collinearity in moderated models. In this paper, we show that VIF values are likely to misdiagnose the extent of collinearity problems while condition numbers do not accurately identify when collinearity is actually harmful to statistical inferences. We propose a new measure, C2, which diagnoses the extent of collinearity in moderated regression models. More importantly, this C2 measure, in conjunction with the t-statistic of the non-significant coefficient, can indicate the adverse effects of collinearity in terms of distorting statistical inferences and how much collinearity would have to disappear to generate significant results. The efficacy of C2 over VIFs and condition indices is demonstrated using simulated data and its usefulness in moderated regressions is illustratedin an empirical study of brand extensions.
Keywords: Collinearitydiagnostics,Variance inflation factors, Condition indices, Moderated models, Multiplicative interactions
- INTRODUCTION
Moderated regressions appear regularly in marketing. Consider a study of brand extensions where the buyer’s attitude towards the newer brand extensions(Attitude) are determined by quality perceptions of the parent brand (Quality), the perceived transferability of the skills and resources of the parent brand to the extension product (Transfer), and the interaction between Quality and Transfer (c.f. Aaker and Keller 1990). In order to test whether these relationships are significantly different from zero, the researcher fits the following moderated regression model:
Attitude = 0+ 1 Quality + 2 Transfer + 3Quality×Transfer + (1)
Owing to the potential for strong linear dependencies among the regressors, Quality and Transfer, and the multiplicative interaction termQuality×Transfer, there is always a fear that the presence of high levels of collinearity may lead to flawed statistical inferences. So the first question that confronts the researcher is whether the data are indeed plagued by collinearity and if so, the nature and severity of the collinearity. The researcher surveys the extant marketing literature and finds that that two rules of thumb -- values of variance inflation factors (VIFs), which are based upon correlations between the independent variables, in excess of 10, and values of condition indices in excess of 30 -- are predominantly used to judge the existence and strength of collinearity.
Low correlations or low values of VIFs (less than 10) are considered to be indicative that collinearity problems are negligible or non-existent (c.f. Marquardt 1970). However, VIF is constructed fromsquared correlations, VIF1/ (1-R2), and since correlations are not faithful indicators of collinearity, VIF can lead to misdiagnosis of collinearity problems.[1]Unlike VIFs, a high condition index (> 30) does indicate the presence of collinearity. However, the condition index by itself does not shed light on the root causes, i.e. the offending variables, of the underlying linear dependencies. In such cases wherein collinearity is diagnosed by the condition index, it is always good practice to examine the variance decomposition proportions (values greater than 0.50 in any row corresponding to a condition index greater than 30 indicates linear dependencies) to identify the specific variables that contributed to the collinearity present in the data (Belsley, Kuh and Welsh 1980).
Reverting back to the brand extension example, let us suppose that the t-statistic for the estimate of theinteraction coefficient3 is found to be 1.60, considerably below the critical value 1.96. In such a situation, the researcher faces the second critical question: did collinearity adversely affect the interaction coefficient in terms of statistical significance? While variance decomposition metrics do help identify the specific variables underlying the potential near-linear dependencies, they do not offer insight into whether collinearity adversely affects the significances of the variables. In other words, none of the current collinearity metrics including VIFs, condition indices, and variance decomposition proportions shed any light into whether collinearity is the culprit that caused the non-significance of the interaction coefficient.
If there was indeed a way to confirm collinearity as the culprit behind the interaction variable’s non-significance, the researcher is confronted with a third question: if the collinearity associated with the non-significant effect could be reduced in some meaningful way through collection of additional data, would the measured t-statistic increase enough to be statistically significant? Alternatively, suppose that the data on Quality and Transfer were constructed from a well-balanced experimental design, rather than from a survey, would this experiment lead to a reduction of collinearity sufficient enough to move the interaction effect to statistical significance? An answer to this question would enable the researcher to truly ascertain whether new data collection is needed or whether the researcher needs to focus her efforts elsewhere to identify the reasons for insignificant results. Unfortunately, existing collinearity diagnosis metrics including correlations, VIF, CI, or variance decomposition proportionsdo not provide any insight into this issue.
In this paper, we propose a new measure of collinearity, C2that reflects the quality of data to remedy the above mentioned problems. C2 not only diagnoses collinearity accurately but also indicate whether collinearity was the reason behind non-significant effects. More importantly, C2 can also indicate whether a non-significant effect would become significant if the collinearity in the data could be reduced, and if so, how much collinearity must be reduced to achieve this significant result.
2. COLLINEARITY IN MODERATED REGRESSION
Consider a moderated variable regression
Y= 01+1U+2V+3UV+,(2)
where U and V are ratio scaled explanatory variables in N-dimensional data vectors.[2] Because
the interaction term UV shares information with both U and V, there may be correlations and/or collinearity between these variables. Correlations refer to linear co-variability of two variables around their means. Computationally, correlation is built from the inner product of mean-centered variables. Collinearity, on the other hand, refers to the presence of linear dependencies between raw, uncentered values (Silvey 1969). Figure 1 illustrates two data sets with low and high correlation and collinearity.
Figure 1. Two Data Sets: and
In Figure 1, the scatter plot with dots exhibits no correlation between U and UV because their values form a symmetric ball around mean values. On the other hand, the raw values of the variables are entirely contained within the narrow dashed cone emanating from the origin, thus exhibiting substantial collinearity. The other data set with values indicated with crosses clearly exhibits correlation between U and UV because the scatter plot forms an ellipse around the mean values. However, these variables are not very collinear because their raw values point in almost every direction; the solid cone nearly fills the entire first quadrant. If the correlation of the × data approached 1.0 (perfect correlation) the solid cone would collapse to a vector (perfect collinearity). Perhaps this is why many scholars view “correlated” as a synonym for “collinear,” but properly, data may be collinear but not very correlated or very correlated and very collinear. Of these two distinct constructs, we focus on collinearity.
By construction, the term UV carries some of the same information as U and V and could create a collinearity problem, so let us look at how uniquethe values of the interaction term UV are compared to 1, V and V. The source of collinearity is clear but to measure its magnitude, one could regress UV on the three other independent variables in the auxiliary model,
UV=1+U+V+, (3)
and compute the ordinary least squaresprojection, =b01+b1U+b2V, from the OLS coefficients b0, b1, b2. How similar is the actual UV to theprojection The more similar they are, the higher the degree of collinearity, and if they are identical, then collinearity is perfect.
A measure of the similarity of UV to is the angular deviation of UV from its ordinary least squares projection found onthe plane defined by [1, U, V]. If this angle is zerothen the actual and predictedvectors pointprecisely in the same direction: there is perfect collinearity. Trigonometry tells us thatthe square of the cosine of this angle is the “non-centered” coefficient of determination,
, (4)
where e is the least squaresresidual vector of the auxiliary regression from (3). For this reason, the non-centered RN2 is a proper measure of collinearity.
RN2is related to the traditional coefficient of determination that contrasts the residual error to the mean-centered values of the dependent variable, . Of course, R2 is the square of the correlation between the projection of the dependent variable and the raw dependent variable, and may be expressed as
.(5)
Notice that R2 increases with RN2 and equals 1.0 when RN2=1.0. However, the value of R2 is also a decreasing function of the mean and anincreasing function of the varianceof the dependent variable in the auxiliary regression. Collinearity could be quite substantial but masked by a combination of large mean or small variance of U×V, so that small R2 falsely signals low collinearity.
This means that the variance inflation factor, computed from the traditional R2 of the auxiliary regression asVIF1/ (1-R2), is a confounded measure of collinearity despite itsgreat popularity. Specifically, weakness of the explanatory data U×V comes from three basic factors: a) lack of variability of regressors (small variance of UV), b) lack of magnitude of regressors data (smallmean,), and c) collinearity of the regressor with other explanatory variables (large RN2 or non-centered variance inflation factor, defined as VIFN 1/ (1-RN2)). VIF is a combination of all of these data weakness factors:
.(6)
Because of this, one dataset may have larger collinearity than another dataset although it has a smaller VIF value depending upon the mean () and variability () of the variables in the data.
3. DEVELOPING A NEW COLLINEARITY METRIC: C2
We propose a new collinearity metric for moderated regression models that is derived from the non-centered coefficient of determination and satisfies fivemajor criteria: a) the measure is based upon raw data rather than mean-centered data to avoid theproblems that affect correlation and VIF, b) it distinguishes collinearity from other sources of data weaknesses such as lack of variability of the exogenous variables and lack of magnitude, c) it is easily computed, d) it is easily interpreted, e) it helps distinguish collinearity that causes parameter insignificance from collinearity that has no effect on parameter significance. Specifically, RN2will be linearly transformed so that the rescaled score equals 0 when the collinearity is equivalent to a popular benchmark: a balanced experimental design.
A balanced design is used because collinearity in field studies occurs due to the uncontrollability of the data-generating mechanism (Belsley 1991, p. 8). Experiments, on the other hand, are appropriately controlled with spurious influences eliminated and hence collinearity must be less of a problem. The relative superiority of experimental designs for detecting interactions has been demonstrated elsewhere (see McClelland and Judd 1993). A balanced experimental design makes the causal variables U and V independent of one another, but because the moderator term U×V shares commonality with them, some degree of collinearity is present. As such, a well-balanced design makes an ideal benchmark against which the collinearity in the data can be compared. For pedagogical reasons, we will use a 2×2, two factor by two level experiment, but this will be generalized to a K×…×K, M factor by K level design and a continuous M factor design.
3.1 What is the level of collinearity in a well-balanced 2x2 experimental design?
In the case of experimental design with a sample of size N, the balanced design produces a design matrix [1, U, V, UV] where U and V take on values of 0 and 1 divided into blocks with N/4 subjects (there are four pairs of values of U and V in a 22 design). One such block is
.
Does this experimental design exhibit any collinearity amongst its variables?First, because collinearity refers to the degree that vectors point in the same direction relative to the origin, if there was no natural zero value in the experiment, collinearity cannot be uniquely specified. The vectors U and V above are slightly collinear because U’V=1 differs from zero, but if V could beeffect-coded to equal V0= (1, -1,1,-1)’ then U and V0 are orthogonal, U’V0=0. Therefore, we assume that in this benchmark experiment U and V havenatural zeroes. As an illustration, suppose that U is the number of advertisements that subjects are shown with the experimental condition, the control condition being no advertisements, a natural zero.
Second, it is straightforward to show that the collinearity angle, θ, between UV and the plane defined by [1, U, V] in the above experimental design is given by ==30. That is, the non-centered coefficient of determination in this experiment is RN2(experiment) = ¾. This collinearity in a balanced experimental design will be used as a baseline.[3]
3.2 Development of the C2 metric
Consider a linear transformation of the non-centered coefficient of determinationcomputed from the data ARN2(data)+B, whose parameters are chosen for easy interpretation. Specifically, suppose that A and B are scaled so that this score equals 0 when the collinearity is equal that of 22 balanced design experiment as seen aboveand equals 1 when RN2(data)= 1. Solving A¾+B=0 and A+B=1 gives and .We therefore define a collinearity score C2for a 2×2 baseline experiment as follows.
Definition of C2: In the moderated variable regression (1), a collinearity score for UV that equals 0 if the data came from a well-balanced experimental design and equals 1.0 if there is perfect collinearity within [1, U, V, UV] is given by
,(7)
where VIFN(data) is the non-centered variance inflation factor from regressing the actual data UV on 1, U and V as in (3).
The C2 measure for a two variable moderated regression can also be generalized for M variates U1,…,UM and all their two-way interactions such as UiUj. All one needs to do is replace RN2(experiment) in equation (7) by the generalized non-centered RN2. In addition, instead of an experiment with 2 levels, 0 and 1, the experiment could have K levels, 0, 1,…,K-1. Finally, the independent variables might be continuous on the unit interval [0, 1]. If the experimental design is well designed, the non-centered RN2, VIFN and C2 are given in Table 1. For example, if the data are continuous and there are M=4 variables, the appropriate C2 is 1-16/VIFN(data) for interaction terms and 1-40/VIFN(data) for the linear terms.
Table 1. RN2, VIFN and C2 for Well-Designed Experiments
Data Type / Range / Number of Variables / Non-Centered RN2 of Well-Designed Balanced Experiment / Non-Centered VIFN of Well-Designed Balanced Experiment / C2 based upon this Well-Designed Balanced ExperimentUi×Ujinteraction terms
Discrete / {0,1,2,…,
K-1} / M in
K×K×
…×K design / / /
Continuous / [0, 1] / M in
[0,1]×[0,1]… ×[0,1] design / / 16 /
Uilinear terms
Discrete / {0,1,2,…,
K-1} / M in
K×K×
…×K design / / /
Continuous / [0, 1] / M in
[0,1]×[0,1]… ×[0,1] design / / 4(3M-2) /
The collinearity score C2 is derived from the non-centered RN2 and therefore is not based upon correlations. Notice that C2=1 if and only if VIFN(data) = (or equivalently RN2=1). On the other hand, C2=0 does not say that all collinearity has been eliminated, only that it has been reduced to the same level as that found in a well-balanced experimental design. Using equation (5), one can express the collinearity score C2 in relation to the traditional (centered) variance inflation factor:
. (8)
As we have seen earlier, VIF is a measure that confounds three data weaknesses, collinearity, variability, and magnitude, but the term in denominator of equation (8) strips out the variability and magnitude, leaving C2as a pure measure of collinearity. If the traditional VIF approaches 1, C2may still be large.
Of course, while collinearity inflates standard errors in a moderated regression analysis, this does not mean that it is a serious problem. Collinearity is harmful only when coefficients that may otherwise be significant could lose statistical significance. The next section provides guidelines using the measure C2 to diagnose whether there is truly harmful collinearity.
4. WHEN IS COLLINEARITY HARMFUL?
The statistical significance of the estimator of coefficient 3 of the variable UV in the moderated regression (1) is typically evaluated by its t-statistic, which can be expressed in terms of the proposed collinearity score C2:
.(9)
(See Theil 1971, p. 166, for a comparable derivation). Although the t-statistic is determined by five other factors, numerosity (N), effect sizes (a3), residual errors (s), data magnitude (), and data variability (), equation (9) shows that one can tease apart the unique contribution of collinearity,as measured by C2, to the statistical significance of the estimator while holding the other factors constant.The condition index is a perfectly appropriate measure of collinearity, but there is no simple way to relate the condition index to the t-statistic of the interaction term, the way one can with the C2 measure of collinearity.
Recall that for a well-designed balanced experiment C2 equals zero. If there was a way to reduce the collinearity from its measured level C2to that of a well-balanced experiment, then the t-statistic t3would become. Suppose that a3 is insignificant because the measured t3 is below a critical value tcrit, but if the collinearity score was reduced to zero it would be significant because is above the critical value. This is equivalent to the condition