Evidence Based on the Internal Structure of the Instrument: Factor Analysis

  • D. Betsy McCoachEmail author
  • Robert K. Gable
  • John P. Madura


Up to this point, the relationship between affective characteristics and their translation into mathematical structure has been largely theoretical. Chapter 4 is a notable departure from previous chapters in that all the methodological approaches and techniques require data collected from a sample of participants from the instrument developer’s target group. The purpose of Chapter 4 is to introduce the most effective and accepted methods for understanding the internal structure of instruments developed to measure affective characteristics. The internal structure of an instrument is the empirically defined mathematical relationship between the proposed latent construct(s) and the items observed variables in the instrument. This mathematical relationship is commonly understood as dimensionality—and the empirical evidence needed to understand the dimensionality of the proposed instrument can only be supplied by a sample of “real” subjects. Chapter 4 largely addresses with a family of techniques known as factor analysis. The chapter specifically discusses two techniques commonly employed in the process of understanding the internal structure of an instrument. The chapter provides an overview of both exploratory factor analysis and confirmatory factor analysis and explains their utility and usage within the instrument development process.


Factor analysis Exploratory factor analysis Confirmatory factor analysis Constructs Content categories Pattern matrix Pattern coefficients Extraction Common factor analysis Principal component analysis Principal axis factoring Underextraction Overextraction Scree test Parallel analysis Minimum average partial procedure Factor rotation Orthogonal rotation Varimax rotation Oblique rotation Factor scores Respecification Loadings N:p ratio Interitem correlations Kaiser–Meyer–Olkin test Measure of sampling adequacy Eigenvalue Communalities Underidentified Overidentified Just identified Marker variable Parameter Root-mean-square error of approximation (RMSEA) Standardized root-mean-square residual (SRMR) Comparative fit index (CFI) Tucker lewis index (TLI) Absolute fit index Incremental fit index Information criteria Akaike information criterion (AIC) Bayesian information criterion (BIC) Correlated error 


  1. American Educational Research Association (AERA), American Psychological Association (APA) & National Council on Measurement in Education (NCME). (1999). The standards for educational and psychological testing. Washington: American Educational Research Association.Google Scholar
  2. Arrindell, W. A., & Van der Ende, J. (1985). An empirical test of the utility of the observations-to-variables ratio in factor and components analysis. Applied Psychological Measurement, 9(2), 165–178.CrossRefGoogle Scholar
  3. Bandalos, D. L., & Finney, S. J. (2010). Factor analysis: Exploratory and confirmatory. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 125–155). Florence: Routledge Education.Google Scholar
  4. Bollen, K. A., & Long, J. S. (1993). Testing structural equation models. Newbury Park: Sage.Google Scholar
  5. Briggs, N. E., & MacCallum, R. C. (2003). Recovery of weak common factors by maximum likelihood and ordinary least squares estimation. Multivariate Behavioral Research, 38(1), 25–56.CrossRefGoogle Scholar
  6. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press.Google Scholar
  7. Byrne, B. M. (2010). Structural equation modeling with AMOS: Basic concepts, applications and programming (2nd ed.). New York: Taylor and Francis Group.Google Scholar
  8. Cattell, R. B. (1966). Handbook of multivariate experimental psychology. Chicago: Rand-McNally.Google Scholar
  9. Cattell, R. B. (1978). The scientific use of factor analysis. New York: Plenum Press.CrossRefGoogle Scholar
  10. Coletta, A. J., & Gable, R. K. (1975). The content and construct validity of the Barth scale: Assumptions of open education 1. Educational and Psychological Measurement, 35(2), 415–425.CrossRefGoogle Scholar
  11. Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56(5), 754–761.PubMedCrossRefGoogle Scholar
  12. Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Hillsdale: Lawrence Erlbaum Associates.Google Scholar
  13. Conway, J. M., & Huffcutt, A. I. (2003). A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods, 6(2), 147–168.CrossRefGoogle Scholar
  14. Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research and Evaluation, 10(7), 1–9.Google Scholar
  15. Crawford, A. V., Green, S. B., Levy, R., Lo, W., Scott, L., Svetina, D., et al. (2010). Evaluation of parallel analysis methods for determining the number of factors. Educational and Psychological Measurement, 70(6), 885–901.CrossRefGoogle Scholar
  16. DiStefano, C. (2002). The impact of categorization with confirmatory factor analysis. Structural Equation Modeling, 9, 327–346.CrossRefGoogle Scholar
  17. DiStefano, Christine, Zhu, Min, & Mîndrilă, Diana (2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research and Evaluation, 14 (20), 1–11. Google Scholar
  18. Dolan, C. V. (1994). Factor analysis of variables with 2, 3, 5, and 7 response categories: A comparison of categorical variable estimators using simulated data. British Journal of Mathematical and Statistical Psychology, 47, 309–326.CrossRefGoogle Scholar
  19. Everitt, B. S. (1975). Multivariate analysis: The need for data, and other problems. British Journal of Psychiatry, 126, 237–240.PubMedCrossRefGoogle Scholar
  20. Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory factor analysis. New York: Oxford University Press.Google Scholar
  21. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(1), 272–299.CrossRefGoogle Scholar
  22. Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course. Greenwich: Information Age Publishing.Google Scholar
  23. Ford, J., MacCallum, R., & Tate, M. (1986). The application of factor analysis in psychology: A critical review and analysis. Personal Psychology, 39, 291–314.CrossRefGoogle Scholar
  24. Glass, G. V., & Maguire, T. O. (1966). Abuses of factor scores. American Educational Research Journal, 3, 297–304.CrossRefGoogle Scholar
  25. Gorsuch, R. L. (1997). Exploratory factor analysis: Its role in item analysis. Journal of Personality Assessment, 68(3), 532–560.PubMedCrossRefGoogle Scholar
  26. Gribbons, B. C., & Hocevar, D. (1998). Levels of aggregation in higher level confirmatory factor analysis. Structural Equation Modeling, 5(4), 377–390.CrossRefGoogle Scholar
  27. Guttman, L. (1953). Image theory for the structure of quantitative variates. Psychometrika, 18, 277–296.CrossRefGoogle Scholar
  28. Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416.CrossRefGoogle Scholar
  29. Hu, L. T., & Bentler, P. M. (1995). Evaluating model fit. In R. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 76–99). Thousand Oaks: Sage.Google Scholar
  30. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453.CrossRefGoogle Scholar
  31. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.CrossRefGoogle Scholar
  32. Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187–200.CrossRefGoogle Scholar
  33. Kaiser, H. F. (1960). Varimax solution for primary mental abilities. Psychometrika, 25, 153–158.CrossRefGoogle Scholar
  34. Kenny, D. A., Kaniskan, B., McCoach, D. B. (2011). The performance of RMSEA in models with small degrees of freedom. Unpublished paper, University of Connecticut.Google Scholar
  35. Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford.Google Scholar
  36. Lackey, N. R., Sullivan, J. J., & Pett, M. A. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Thousand Oaks: Sage.Google Scholar
  37. Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural analysis (4th ed.). Hillsdale: Erlbaum.Google Scholar
  38. Long, J. S. (1997). Regression models for categorical and limited dependent variables: Advanced quantitative techniques in the social sciences. Thousand Oaks: Sage.Google Scholar
  39. Lorenzo-Seva, U., Timmerman, M. E., & Kiers, H. L. (2011). The Hull method for selecting the number of common factors. Multivariate Behavioral Research, 46(2), 340–364.CrossRefGoogle Scholar
  40. MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99.CrossRefGoogle Scholar
  41. Marsh, H. W. (1987). Application of confirmatory factor analysis to the study of self-concept: First- and higher-order factor models and their invariance across groups. Psychological Bulletin, 97, 562–582.CrossRefGoogle Scholar
  42. McAnallen, R. R. (2010). Examining mathematics anxiety in elementary classroom teachers. (Doctoral Dissertation). Retrieved from DigitalCommons@UConn. (AAI3464333).Google Scholar
  43. McCoach, D. B. (2003). SEM isn’t just the Schoolwide Enrichment Model anymore: Structural Equation Modeling (SEM) in gifted education. Journal for the Education of the Gifted, 27, 36–61.Google Scholar
  44. Muthen, L. K., & Muthen, B. O. (2007). Mplus: The comprehensive modeling program for applied researchers (5th ed.). Los Angeles: Muthen & Muthen.Google Scholar
  45. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.Google Scholar
  46. Nunnally, J. C., Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). New York: McGraw-Hill.Google Scholar
  47. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, and Computers, 32(3), 396–402.PubMedCrossRefGoogle Scholar
  48. Patil, V. H., Singh, S. N., Mishra, S., Donavan, D. T. (2007). Parallel analysis engine to aid determining number of factors to retain [Software]. Retrieved from University of Kansas.
  49. Pett, M. A., Lackey, N. R., & Sullivan, J. J. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Thousand Oaks: Sage.Google Scholar
  50. Preacher, K. J., & Merkle, E. C. (2012). The problem of model selection uncertainty in structural equation modeling. Psychological Methods, 17(1), 1–14.PubMedCrossRefGoogle Scholar
  51. Rhemtulla, M., Brosseau-Liard, P. E., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373.PubMedCrossRefGoogle Scholar
  52. Rummell, R. J. (1970). Applied factor analysis. Evanston: Northwestern University Press.Google Scholar
  53. Slocum-Gori, S. L., & Zumbo, B. D. (2011). Assessing the unidimensionality of psychological scales: Using multiple criteria from factor analysis. Social Indicators Research, 102(3), 443–461.CrossRefGoogle Scholar
  54. Streiner, D. L. (1998). Factors affecting reliability of interpretations of scree plots. Psychological Reports, 83, 687–694.Google Scholar
  55. Thompson, B. (1992). A partial test distribution for cosines among factors across samples. In B. Thompson (Ed.), Advances in social science methodology (Vol. 2, pp. 81–97). Greenwich: JAI.Google Scholar
  56. Thompson, B. (1997). The importance of structure coefficients in structural equation modeling confirmatory factor analysis. Educational and Psychological Measurement, 57(1), 5–19.CrossRefGoogle Scholar
  57. Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington: American Psychological Association.CrossRefGoogle Scholar
  58. Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327.CrossRefGoogle Scholar
  59. Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: Honoring Douglas Jackson at seventy (pp. 41–71). Boston: Kluwer.CrossRefGoogle Scholar
  60. Velicer, W. F., & Fava, J. L. (1998). The effects of variable and subject sampling on factor pattern recovery. Psychological Methods, 3, 231–251.CrossRefGoogle Scholar
  61. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.PubMedCrossRefGoogle Scholar
  62. Zwick, W. R., & Velicer, W. F. (1986). Factor influencing five rules for determining the number of components to retain. Psychological Bulletin, 99, 432–442.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • D. Betsy McCoach
    • 1
    Email author
  • Robert K. Gable
    • 2
  • John P. Madura
    • 3
  1. 1.Educational Psychology DepartmentUniversity of ConnecticutStorrsUSA
  2. 2.Alan Shawn Feinstein Graduate SchoolJohnson and Wales UniversityStorrsUSA
  3. 3.Department of Educational PsychologyUniversity of ConnecticutStorrsUSA

Personalised recommendations