Various methods for determining unidimensionality are reviewed and the rationale of these methods is as sessed. Indices based on answer patterns, reliability, components and factor analysis, and latent traits are reviewed. It is shown that many of the indices lack a rationale, and that many are adjustments of a previous index to take into account some criticisms of it. After reviewing many indices, it is suggested that those based on the size of residuals after fitting a two- or three-parameter latent trait model may be the most useful to detect unidimensionality. An attempt is made to clarify the term unidimensional, and it is shown how it differs from other terms often used inter changeably such as reliability, internal consistency, and homogeneity. Reliability is defined as the ratio of true score variance to observed score variance. Inter nal consistency denotes a group of methods that are intended to estimate reliability, are based on the vari ances and the covariances of test items, and depend on only one administration of a test. Homogeneity seems to refer more specifically to the similarity of the item correlations, but the term is often used as a synonym for unidimensionality. The usefulness of the terms in ternal consistency and homogeneity is questioned. Uni dimensionality is defined as the existence of one latent trait underlying the data.
Get full access to this article
View all access options for this article.
References
1.
Andersson, C.G., Christoffersson, A., & Muthén, B. (1974). FADIV: A computer program for factor analysis of dichotomized variables (Report No. 74-1). Uppsala, Sweden: Uppsala University, Statistics Department .
2.
Anderson, T.W. (1959). Some scaling models and estimation procedures in the latent class model. In U. Grenander (Ed.), Probability and statistics (pp. 9-38). New York: Wiley.
3.
Andrich, D., & Godfrey, J.R. (1978-1979). Hierarchies in the skills of Davis' reading comprehension test, Form D: An empirical investigation using a latent trait model. Reading Research Quarterly, 2, 183-200.
4.
Armor, D.J. (1974). Theta reliability and factor scaling. In H. L. Costner (Ed.), Sociological methodology (pp. 17-50). San Francisco CA: Jossey-Bass.
5.
Bejar, I.I. (1980). A procedure for investigating the unidimensionality of achievement tests based on item parameter estimates. Journal of Educational Statistics, 17, 282-296.
6.
Bentler, P. (1972). A lower-bound method for the dimension-free measurement of internal consistency. Social Science Research, 1, 343-357.
7.
Birenbaum, M., & Tatsuoka, K. (1982). On the dimensionality of achievement test data . Journal of Educational Measurement. 19, 259-266.
8.
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of the EM algorithm. Psychometrika , 46, 443-459.
9.
Bock, R.D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179-197.
Carroll, J.B. (1945). The effect of difficulty and chance success on correlation between items or between tests. Psychometrika , 10, 1-19.
12.
Cattell, R.B. (1964). Validity and reliability: A proposed more basic set of concepts. Journal of Educational Psychology, 55, 1-22.
13.
Cattell, R.B. (1978). The scientific use offactor analysis in behavioral and life sciences. New York: Plenum .
14.
Cattell, R.B., & Tsujioka, B. (1964). The importance of factor-trueness and validity, versus homogeneity and orthogonality, in test scales. Educational and Psychological Measurement, 24, 3-30.
15.
Christoffersson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 40, 5-32.
16.
Cliff, N. (1977). A theory of consistency or ordering generalizable to tailored testing. Psychometrika, 42, 375-399.
17.
Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
18.
Divgi, D.R. (1980, April). Dimensionality of binary items: Use of a mixed model. Paper presented at the annual meeting of the National Council on Measurement in Education. Boston MA.
19.
Drasgow, F., & Lissak, R.I. (1983). Modified parallel analysis: A procedure for examining the latent-dimensionality of dichotomously scored item responses. Journal of Applied Psychology, 68, 363-373.
20.
Dubois, P.H. (1970). Varieties of psychological test homogeneity. American Psychologist, 25, 532-536.
21.
Etezadi, J. (1981). A general polynomial model for nonlinear factor analysis. Dissertation Abstracts International, 42, 4342a.
22.
Etezadi, J., & McDonald, R.P. (1983). A second generation nonlinear factor analysis. Psychometrika, 48, 315-342.
23.
Fraser, C. (1981). NOHARM: A FORTRAN program for non-linear analysis by a robust method for estimating the parameters of 1-, 2-, and 3-parameter latent trait models. Arrnidale, Australia: University of New England, Centre for Behavioural Studies in Education.
24.
Freeman, F.S. (1962). Theory and practice of psychological testing (3rd ed.). New York: Henry Holt.
25.
Fuller, E.L., & Hemmerle, W.J. (1966). Robustness of the maximum-likelihood estimation procedure in factor analysis. Psychometrika, 31, 255-266.
26.
George, A.A. (1979, April). Theoretical and practical consequences of the use of standardized residuals as Rasch model fit statistics. Paper presented at the annual meeting of the American Educational Research Association, San Francisco CA.
27.
Gage, N.L., & Damrin, D.E. (1950). Reliability, homogeneity, and number of choices . Journal of Educational Psychology, 41, 385-404.
28.
Gibson, W.A. (1959). Three multivariate models: Factor analysis, latent structure analysis, and latent profile analysis. Psychometrika , 24, 229-252.
29.
Gibson, W.A. (1960). Nonlinear factors in two dimensions. Psychometrika, 25, 381-392.
30.
Goodman, L.A., & Kruskal, W.H. (1979). Measures of association for cross classifications . New York: Springer-Verlag.
Gourlay, N. (1951). Difficulty factors arising from the use of the tetrachoric correlations in factor analysis. British Journal of Statistical Psychology, 4, 65-72.
33.
Green, B.F. (1956). A method of scalogram analysis using summary statistics . Psychometrika, 21, 79-88.
34.
Green, S.B., Lissitz, R.W., & Mulaik, S.A. (1977). Limitations of coefficient alpha as an index of test unidimensionality. Educational and Psychological Measurement , 37, 827-838.
35.
Greene, V.C., & Carmines, E.G. (1980). Assessing the reliability of linear composites. In K. F. Schuessler (Ed.), Sociological methodology (pp. 160-175). San Francisco CA: Jossey-Bass.
36.
Guilford, J.P. (1941). The difficulty of a test and its factor composition . Psychometrika, 6, 67-77.
37.
Guilford, J.P. (1965). Fundamental statistics in psychology and education (4th ed.). New York: McGraw-Hill.
38.
Gustafsson, J.E. (1980). Testing and obtaining fit of data to the Rasch model. British Journal of Mathematical and Statistical Psychology , 33, 205-233.
39.
Gustafsson, J.E., & Lindblad, T. (1978). The Rasch model for dichotomous items: A solution of the conditional estimation problem for long tests and some thoughts about item screening procedures (Report No. 67). Göteborg, Sweden: University of Göteborg, Institute of Education.
40.
Guttman, L. (1944). A basis for scaling qualitative data. American Sociological Review, 80, 139-150.
41.
Guttman, L. (1945). A basis for analyzing test-retest reliability . Psychometrika, 10, 255-282.
42.
Guttman, L. (1950). The principal components of scale analysis. In S. S. Stouffer (Ed.), Measurement and prediction (pp. 312-361). Princeton NJ: University Press.
43.
Haley, D.C. (1952). Estimation of dosage mortality relationships when the dose is subject to error (Report No. 15). Stanford CA: Stanford University, Applied Mathematics and Statistics Laboratory.
44.
Hambleton, R.K. (1969). An empirical investigation of the Rasch test theory model. Unpublished doctoral dissertation, University of Toronto, Canada.
45.
Hambleton, R.K. (1980). Latent ability scales: Interpretations and uses . New Directions for Testing and Measurement, 6, 73-97.
46.
Hambleton, R.K., Swaminathan, H., Cook, L.L., Eignor, D.R., & Gifford, J.A. (1978). Developments in latent trait theory: Models, technical issues, and applications. Review of Educational Research, 48, 467-510.
47.
Hambleton, R.K., & Traub, R.E. (1973). Analysis of empirical data using two logistic latent trait models. British Journal of Mathematical and Statistical Psychology, 26, 195-211.
48.
Harman, H.H. (1979). Modern factor analysis (3rd ed.). Chicago: University of Chicago Press.
49.
Hattie, J.A. (1979). A confirmatory factor analysis of the Progressive Achievement Tests: Reading Comprehension, reading vocabulary, and listening comprehension tests. New Zealand Journal of Educational Studies , 14, 172-188. (Errata, 1980, 15, 109.)
50.
Hattie, J.A. (1980). The Progressive Achievement Tests revisited. New Zealand Journal of Educational Studies, 15, 194-197.
51.
Hattie, J.A. (1981). A four stage factor analytic approach to studying behavioral domains. Applied Psychological Measurement, 5, 77-88.
52.
Hattie, J.A. (1984a). An empirical study of various indices for determining unidimensionality. Multivariate Behavioral Research, 19, 49-78.
53.
Hattie, J.A. (1984b). Some decision rules for determining unidimensionality . Unpublished manuscript, University of New England, Centre for Behavioural Studies, Armidale, Australia.
54.
Hattie, J.A., & Hansford, B.F. (1982). Communication apprehension: An assessment of Australian and United States data. Applied Psychological Measurement, 6, 225-233.
55.
Heise, D.R., & Bohrnstedt, G.W. (1970). Validity, invalidity, and reliability. In E. F. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology (pp. 104-129). San Francisco CA: Jossey-Bass.
56.
Hertzman, M. (1936). The effects of the relative difficulty of mental tests on patterns of mental organization. Archives of Psychology , No. 197.
57.
Hoffman, R.J. (1975). The concept of efficiency in item analysis. Educational and Psychological Measurement, 35, 621-640.
58.
Horn, J.L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185.
59.
Horn, J.L. (1969). On the internal consistency and reliability of factors. Multivariate Behavioral Research, 4, 115-125.
60.
Horrocks, J.E., & Schoonover, T.I. (1968). Measurement for teachers . Columbus OH: Merrill.
61.
Horst, P. (1953). Correcting the Kuder-Richardson reliability for dispersion of item difficulties. Psychological Bulletin,50, 371-374.
62.
Hulin, C.L., Drasgow, F., & Parsons, C. (1983). Item response theory: Applications to psychological measurement. Homewood IL: Dow & Jones Irwin.
63.
Humphreys, L.G. (1949). Test homogeneity and its measurement. American Psychologist, 4, 245. (Abstract).
Humphreys, L.G. (1956). The normal curve and the attenuation paradox in test theory. Psychological Bulletin,53, 472-476.
66.
Hutten, L. (1979, April). An empirical comparison of the goodness of fit of three latent trait models to real test data. Paper presented at the annual meeting of the American Educational Research Association, San Francisco CA.
67.
Hutten, L. (1980, April). Some empirical evidence for latent trait model selection. Paper presented at the annual meeting of the American Educational Research Association, Boston MA.
68.
Jackson, J.M. (1949). A simple and more rigorous technique for scale analysis. In A manual of scale analysis, Part II. Montreal: McGill University.
69.
Jöreskog, K.G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443-447.
70.
Jöreskog, K.G., & Sörbom, D. (1981). LISREL V: Analysis of linear structural relationships by the method of maximum likelihood. Chicago: National Educational Resources.
71.
Kaiser, H.F. (1968). A measure of the average intercorrelation. Educational and Psychological Measurement, 28, 245-247.
72.
Kaiser, H.F. (1970). A second generation little jiffy. Psychometrika, 35, 401-415.
73.
Kelley, T.L. (1935). Essential traits of mental life. Harvard Studies in Education, 26, 146-153.
74.
Kelley, T.L. (1942). The reliability coefficient. Psychometrika, 7, 75-83.
75.
Klich, L.Z., & Davidson, G.R. (1984). Toward a recognition of Australian Aboriginal Competence in cognitive formation. In J. Kirby (Ed.), Cognitive strategies and educational performance (pp. 155-202). New York: Academic Press.
76.
Koch, W.R., & Reckase, M.D. (1979, September). Problems in application of latent trait models to tailored testing (Research Report 79-1). Columbia MO: University of Missouri, Educational Psychology Department, Tailored Testing Research Laboratory.
77.
Kuder G.F., & Richardson, M.W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151-160.
78.
Laforge, R. (1965). Components of reliability. Psychometrika , 30, 187-195.
79.
Lam, R. ( 1980). An empirical study of the unidimensionality of Ravens Progressive Matrices. Unpublished master's thesis, University of Toronto, Canada.
80.
Lim, T.P. (1974). Estimation of probabilities of dichotomous response patterns using a simple linear model. Unpublished doctoral dissertation, University of Toronto, Canada.
81.
Linn, R.L. (1968). A Monte Carlo approach to the number of factors problem. Psychometrika, 33, 37-71.
82.
Loevinger, J. (1944). A systematic approach to the construction of tests of ability. Unpublished doctoral dissertation, University of California.
83.
Loevinger, J. (1947). A systematic approach to the construction and evaluation of tests of ability. Psychological Monograph, 61 (No. 4, Whole No. 285).
84.
Loevinger, J. (1948). The technique of homogeneous tests. Psychological Bulletin,45, 507-529.
85.
Lord, F.M. (1953). The relation of test score to the trait underlying the test. Educational and Psychological Measurement, 13, 517-548.
86.
Lord, F.M. (1958). Some relations between Guttman's principal components of scale analysis and other psychometric theory. Psychometrika , 23,291-296.
87.
Lord, F.M. (1980). Applications of item response theory to practical testing problems. New York: Erlbaum Associates.
88.
Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
89.
Lumsden, J. (1957). A factorial approach to unidimensionality. Australian Journal of Psychology, 9, 105-111.
90.
Lumsden, J. (1959). The construction of unidimensional tests. Unpublished master's thesis, University of Western Australia, Perth, Australia.
91.
Lumsden, J. (1961). The construction of unidimensional tests. Psychological Bulletin,58, 122-131.
92.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
93.
McDonald, R.P. (1962a). A general approach to nonlinear factor analysis . Psychometrika, 27, 397-415.
94.
McDonald, R.P. (1962b). A note on the derivation of the general latent class model. Psychometrika, 27, 203-206.
95.
McDonald, R.P. (1965a). Difficulty factors and nonlinear factor analysis . British Journal of Mathematical and Statistical Psychology , 18, 11-23.
McDonald, R.P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1-21.
101.
McDonald, R.P. (1976, April). Nonlinear and non-metric common factor analysis. Paper presented to the Psychometric Society , Murray Hill SC.
102.
McDonald, R.P. (1979). The structural analysis of multivariate data: A sketch of general theory. Multivariate Behavioral Research , 14, 21-38.
103.
McDonald, R.P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117.
104.
McDonald, R.P. (1982). Linear versus nonlinear models in item response theory. Applied Psychological Measurement, 6, 379-396.
105.
McDonald, R.P., & Ahlawat, K.S. (1974). Difficulty factors in binary data. British Journal of Mathematical and Statistical Psychology, 27, 82-99.
106.
McDonald, R.P., & Fraser, C. (1985). A robustness study comparing estimation of the parameters of the two-parameter latent trait model. Unpublished manuscript.
107.
Mosier, C.I. (1936). A note on item analysis and the criterion of internal consistency. Psychometrika, 1, 275-282.
108.
Mosier, C.I. (1940). Psychophysics and mental test theory: Fundamental postulates and elementary theorems. Psychological Review, 47, 355-366.
109.
Muthén, B. (1978). Contributions to factor analysis of dichotomous variables. Psychometrika, 43, 551-560.
110.
Muthén, B. (1981). Factor analysis of dichotomous variables: American attitudes toward abortion. In D. J. Jackson & E. F. Borgatta (Eds.), Factor analysis and measurement in sociological research: A multidimensional perspective (pp. 201-214). London: Sage.
111.
Muthén, B., & Christoffersson, A. (1981). Simultaneous factor analysis of dichotomous variables in several groups. Psychometrika , 46, 407-419.
112.
Nishisato, S. (1966). Minimum entropy clustering of test-items. Dissertation Abstracts International, 27, 2861B.
113.
Nishisato, S.( 1970a). Structure and probability distribution of dichotomous response patterns. Japanese Psychological Research, 12, 62-74.
114.
Nishisato, S. (1970b). Probability estimates of dichotomous response patterns by logistic fractional factorial representation. Japanese Psychological Research, 12, 87-95.
115.
Nishisato, S. (1971). Information analysis of binary response patterns . In S. Takagi (Ed.), Modern psychology and quantification method. Tokyo: University of Tokyo Press.
116.
Nishisato, S. (1980). Analysis of categorical data: Dual scaling and its applications. Toronto: University of Toronto Press, Canada.
117.
Novick, M.R., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1-13.
118.
Nunnally, J.C. (1970). Introduction to psychological measurement. New York: McGraw-Hill.
119.
Payne, D.A. (1968). The specification and measurement of learning. Waltham MA: Blaisdell.
120.
Raju, N.S. (1980, April). Kuder-Richardson Formula 20 and test homogeneity . Paper presented at the National Council for Measurement in Education, Boston MA.
121.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Nielson & Lydiche.
122.
Rasch, G. (1961). On general laws and the meaning of measurement in psychology. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, 4, 321-333.
123.
Rasch, G. (1966a). An individualistic approach to item analysis . In P. Lazarsfeld & N. V. Henry (Eds.), Readings in mathematical social science (pp. 89-108). Chicago: Science Research Association.
124.
Rasch, G. (1966b). An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 19, 49-57.
125.
Rasch. G. (1968). A mathematical theory of objectivity and its consequences for model construction. Paper read at the European Meeting of Statistics, Econometrics and Management Science, Amsterdam.
126.
Rasch, G. (1977). On specific objectivity: An attempt at formalizing the request for generality and validity of scientific statements. Danish Yearbook ofPhilosophy, 14, 58-94.
127.
Reckase, M.D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics , 4, 207-230.
128.
Rentz, R.R., & Rentz, C.C. (1979). Does the Rasch model really work? NCMEMeasurement in Education, 10, 1-11.
129.
Reynolds, H.T. (1977). The analysis of cross-classifications. New York: Free Press.
130.
Reynolds, T. (1981). Ergo: A new approach to multidimensional item analysis. Educational and Psychological Measurement, 41, 643-660.
131.
Rogers, H.J. (1984). Fit statistics for latent trait models. Unpublished master's thesis, University of New England, Armidale, Australia.
132.
Rosenbaum, P.R. (1984). Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49, 425-435.
133.
Ryan, J.P. (1979, April). Assessing unidimensionality in latent trait models. Paper presented at the annual meeting of the American Educational Research Association, San Francisco CA.
134.
Shostrom, E.L. (1972). Personal Orientation Inventory: An inventory for the measurement of selfactualization. San Diego CA: EdITS.
135.
Silverstein, A.B. (1980). Item intercorrelations, item-test correlations and test reliability. Educational and Psychological Measurement , 40, 353-355.
136.
Smith, K.W. (1974a). Forming composite scales and estimating their validity through factor analysis. Social Forces, 53, 169-180.
137.
Smith, K.W. (1974b). On estimating the reliability of composite indexes through factor analysis. Sociological Methods and Research , 4, 485-510.
138.
Spearman, C. (1927). The abilities of man: Their nature and measurement . London: Macmillan.
139.
Svoboda, M. (1972). Distribution of binary information in multidimensional space. Unpublished master's thesis, University of Toronto , Canada.
140.
Terwilliger, J.S., & Lele, K. (1979). Some relationships among internal consistency, reproducibility, and homogeneity. Journal of Educational Measurement , 16, 101-108.
141.
Thurstone, L.L. (1935). The vectors of the mind. Chicago : University of Chicago Press.
142.
Tucker, L.R., κoopman, R.F., & Linn, R.L. (1969). Evaluation of factor analytic research procedures by means of simulated correlation matrices. Psychometrika , 34, 421-459.
143.
Tucker, L.R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.
144.
van den Wollenberg, A.L. (1982a). A simple and effective method to test the dimensionality axiom of the Rasch model. Applied Psychological Measurement , 6, 83-91.
145.
van den Wollenberg, A.L. (1982b). Two new test statistics for the Rasch model. Psychometrika, 42, 123-140.
146.
Walker, D.A. (1931). Answer-pattern and score scatter in tests and examinations. British Journal of Psychology, 22, 73-86.
147.
Watkins, D., & Hattie, J.A. (1980). An investigation of the internal structure of the Bigg's study process questionnaire. Educational and Psychological Measurement, 40, 1125-1130.
148.
Wherry, R.J., & Gaylord, R.H. (1944). Factor pattern of test items and tests as a function of the correlation coefficient: Content, difficulty, and constant error factors . Psychometrika, 9, 237-244.
149.
White, B.W., & Saltz, E. (1957). Measurement of reproducibility. Psychological Bulletin,54, 81-99.
150.
Wise, S.L. (1982, March). Using partial orders to determine unidimensional item sets appropriate for item response theory. Paper presented at the annual meeting of the National Council for Measurement in Education, New York.
151.
Wise, S.L. (1983). Comparisons of order analysis and factor analysis in assessing the dimensionality of binary data. Applied Psychological Measurement, 7, 311-312.
152.
Wright, B.D. (1977). Solving measurement problems with the Rasch model . Journal of Educational Measurement, 14, 97-116.
153.
Wright, B.D., Mead, R.J., & Bell, S.R. (1979). BICAL: Calibrating items with the Rasch model (Research Report 23-B). Chicago IL: University of Chicago, Department of Education, Statistical Laboratory.
154.
Wright, B.D., & Panchapakesan, N. (1969). A procedure for sample-free item analysis. Educational and Psychological Measurement, 29, 23-48.
155.
Wright, B.D., & Stone, M.H. (1979). Best test design. Chicago: Mesa Press.
