Using Principal Components Analysis in Program Evaluation: Some Practical Considerations

Abstract

Principal Components Analysis (PCA) is widely used by behavioral science researchers to assess the dimensional structure of data and for data reduction purposes. Despite the wide array of analytic choices available, many who employ this method continue to rely exclusively on the default options recommended in dominant statistical packages. This paper examines alternative analytic strategies to guide interpretation of PCA results that expand on these default options, including (a) rules for retaining factors or components and (b) rotation strategies. Conventional wisdom related to the interpretation of pattern/structure coefficients also is challenged. Finally, the use of principal component scores in subsequent analyses is explored. A small set of actual data is used to facilitate illustrations and discussion.
 


Despite the increasing popularity of confirmatory factor analytic (CFA) techniques, principal components analysis (PCA) continues to enjoy widespread use (Kellow, 2004; Thompson, 2004). Researchers who employ PCA are typically interested in (a) assessing the dimensional structure of a dataset (Dunteman, 1989) or (b) reducing a large number of variables into a smaller set of linear combinations (components) for subsequent analyses (e.g., multiple regression). For instance, an evaluator may have occasion to develop a new instrument and wish to ascertain the number and features of the underlying dimensions represented in the data. At other times an existing measure is modified or shortened and the sample data are used to explore the extent to which the structure of the original version has or has not been substantively altered (although CFA is a stronger method for this purpose). The PCA approach also is useful for creating new variables that are linear combinations of a set of highly correlated original variables. These new composite variables may then be used in subsequent analyses. As Stephens (1992) notes, “... if there are 30 variables (whether predictors or items), we are undoubtedly not measuring 30 different constructs, hence, it makes sense to find some variable reduction scheme that will indicate how the variables cluster or hang together” (p. 374). Use of PCA helps to solve at least two problems. First, the presence of multicollinearity (high inter-item or variable correlations) leads to inflated standard errors for the measured variables when conducting statistical analyses, which increases the probability of Type II errors (non-significance when a significant difference exists in the population). Second, when one is using a large set of variables to predict or explain another variable (or set of variables) as opposed to a smaller set of composites, one pays a price in terms of the degrees of freedom used in the analysis. All other things being equal, the more degrees of freedom expended the smaller the value of the omnibus test statistic (e.g., F) that results from the analysis (Stephens, 1992).
 
There are a number of important issues related to the data in hand that need to be addressed (e.g., linearity; absence of outliers) before invoking PCA, and readers are referred to Tabachnick and Fidell (2001) for an excellent overview of these considerations. Once PCA is determined to be appropriate, the analysis proceeds in a series of sequential steps―several options are available to researchers at each step. Too often researchers rely on the default options provided in the major statistical packages and fail to examine other options that may allow for fuller exploitation of the data. The purpose of the present paper is to briefly explore the options available to analysts with respect to (a) rules for retaining principal components and (c) rotation strategies. In addition, conventional wisdom related to the interpretation of pattern/structure coefficients is challenged on substantive grounds. Finally, we briefly explore how PCA may be used to derive component scores for further data analysis.