This article provides readers with an accessible and minimally technical description of foundational issues surrounding the analysis of individual change—that is, the amount by which a given construct related to individuals changes over time. Of primary focus is an explanation of what is meant by construct commensurability, the requirement that longitudinal measures assess the same or comparable construct over time. Because construct commensurability is a key requirement for analyses of individual change, this article also describes various methods by which to empirically assess the commensurability of longitudinal measures. Based on the discussion of commensurability presented herein, implications for research practice are offered from a general validity theory perspective.
Aiken, L.S., West, S.G., & Millsap, R.E. ( 2008). Doctoral training in statistics, measurement, and methodology in psychology: Replication and extension of Aiken, West, Sechrest, and Reno’s (1990) survey of PhD programs in North America. American Psychologist , 63, 32-50.
2.
Aiken, L.S., West, S.G., Sechrest, L., & Reno, R.R. ( 1990). Graduate training in statistics, methodology, and measurement in psychology: A survey of PhD programs in North America. American Psychologist, 45, 721-734.
3.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. ( 1999). Standards for educational and psychological testing. Washington, DC: Author.
4.
Bejar, I.I. ( 1983). Introduction to item response models and their assumptions . In R. K. Hambleton (Ed.), Applications of item response theory (pp. 1-23). Vancouver, British Columbia, Canada: Educational Research Institute of British Columbia.
5.
British Columbia Ministry of Education. (2007 a). Grade 4 foundation skills assessment: Table of specifications . Retrieved October 15, 2007, from http://www.bced.gov.bc.ca/assessment/fsa/specifications/grade4/table_of_specs.pdf
6.
British Columbia Ministry of Education. (2007 b). Grade 7 foundation skills assessment: Table of specifications . Retrieved October 15, 2007, from http://www.bced.gov.bc.ca/assessment/fsa/specifications/grade7/table_of_specs.pdf
7.
Brown, T.A. ( 2006). Confirmatory factor analysis for applied research. New York: Guilford.
8.
Byrne, B.M. ( 1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Mahwah, NJ: Erlbaum.
9.
Campbell, D.T., & Fiske, D.W. ( 1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
Culhane, S.E., Morera, O.F., & Watson, P.J. ( 2006). The assessment of factorial invariance in need for cognition using Hispanic and Anglo samples. Journal of Psychology, 140, 53-68.
12.
Embretson, S.E. ( 2007). Construct validity: A universal validity system or just another test evaluation procedure? Educational Researcher , 36, 449-455.
Horn, J.L., McArdle, J.J., & Mason, R. ( 1983). When is invariance not invariant: A practical scientist’s look at the ethereal concept of factor invariance. Southern Psychologist, 1, 179-188.
16.
Jöreskog, K.G. ( 1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133.
17.
Kane, M.T. ( 1992). An argument-based approach to validity. Psychological Bulletin, 112, 527-535.
18.
Kolen, M.J., & Brennan, R.L. ( 2004). Test equating, scaling, and linking: Methods and practices (2nd ed.). New York: Springer-Verlag.
19.
Lissitz, R.W., & Huynh, H. ( 2003). Vertical equating for state assessments: Issues and solutions in determination of adequate yearly progress and school accountability. Practical Assessment, Research & Evaluation, 8. Retrieved June 9, 2009, from http://pareonline.net/getvn.asp?v=8&n=10
20.
Lissitz, R.W., & Samuelsen, K. ( 2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437-448.
21.
Lloyd, J.E.V., & Zumbo, B.D. ( 2007). The non-parametric difference score: A workable solution for analysing two-wave change when the measures themselves change across waves . Journal of Modern Applied Statistical Methods, 6, 413-420.
22.
Lloyd, J.E.V., Zumbo, B.D., & Siegel, L.S. ( 2009). When measures change over time: A workable solution for analysing change and growth across multiple waves. Journal of Educational Research & Policy Studies, 9, 81-100.
23.
Martineau, J.A. ( 2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability . Journal of Educational and Behavioral Statistics, 31, 25-62.
24.
McArdle, J.J. ( 2007). Five steps in the structural factor analysis of longitudinal data. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100: Historical developments and future directions (pp. 99-139). Mahwah, NJ: Lawrence Erlbaum.
25.
Meade, A.M., Lautenschlager, G.J., & Hecht, J.E. ( 2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279-300.
26.
Meredith, W. ( 1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525-543.
27.
Merenda, P.F. ( 2003). Measurements in the future: Beyond the 20th century. Psychological Reports, 92, 209-217.
28.
Messick, S. ( 1989). Validity. In R. L. Linn (Ed.),Educational measurement (3rd ed., pp. 13-103). New York: American Council on Education.
Novick, M.R. ( 1966). The axioms and principal results of classical test theory . Journal of Mathematical Psychology, 3, 1-18.
32.
Novick, M.R., & Lewis, C. ( 1967). Coefficient alpha and the reliability of composite measurements . Psychometrika, 32, 1-13.
33.
O’Sullivan, C., Scholderer, J., & Cowan, C. ( 2005). Measurement equivalence of the food related lifestyle instrument (FRL) in Ireland and Great Britain. Food Quality and Preference, 16, 1-12.
34.
Raudenbush, S.W. ( 2001). Toward a coherent framework for comparing trajectories of individual change. In L. Collins & A. Sayer (Eds.), Best methods for studying change (pp. 33-64). Washington, DC: American Psychological Association.
35.
Raudenbush, S.W., & Bryk, A.S. ( 2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
36.
Schaie, K.W., Maitland, S.B., Willis, S.L., & Intrieri, R.C. (1998). Longitudinal invariance of adult psychometric ability factor structures across 7 years. Psychology and Aging, 13, 8-20.
37.
Schumacker, R.E., & Lomax, R.G. ( 2004). A beginner’s guide to structural equation modeling (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
38.
Singer, J.D. ( 1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 24, 323-355.
39.
Singer, J.D., & Willett, J.B. ( 2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.
40.
Sireci, S.G. ( 1998). The construct of content validity. In B. D. Zumbo (Ed.), Validity theory and the methods used in validation: Perspectives from the social and behavioral sciences (pp. 83-117). Dordrecht, The Netherlands: Kluwer Academic Press.
41.
Sireci, S.G. ( 2007). On validity theory and test validation. Educational Researcher, 36, 477-481.
42.
Thurstone, L.L. ( 1947). Multiple factor analysis. Chicago : University of Chicago Press.
43.
Vacha-Haase, T., Ness, C.M., Nilsson, J., & Reetz, D. ( 1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education , 67, 335-341.
44.
Vaillancourt, T., Brendgen, M., Boivin, M., & Tremblay, R.E. (2003). A longitudinal confirmatory factor analysis of indirect and physical aggression: Evidence of two factors over time? Child Development, 74, 1628-1638.
45.
Vandenberg, R.J., & Lance, C.E. ( 2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 4, 4-70.
46.
Ward, R.M., Velicer, W.F., Rossi, J.S., Fava, J.L., & Prochaska, J.O. (1994). Factorial invariance and internal consistency for the decisional balance inventory-short form. Addictive Behaviors, 29, 953-958.
47.
Watkins, M.W., & Canivez, G.L. ( 2001). Longitudinal factor structure of the WISC-III among students with disabilities. Psychology in the Schools, 38, 291-298.
48.
Whitaker, B.G., & McKinney, J.L. (2007). Assessing the measurement invariance of latent job satisfaction ratings across survey administration modes for respondent subgroups: A MIMIC modeling approach. Behavior Research Methods, 39, 502-509
49.
Widaman, K.F. ( 1985). Hierarchically nested covariance structure models for research multitrait-multimethod data. Applied Psychological Measurement, 9, 1-26.
50.
Willett, J.B., Singer, J.D., & Martin, N.C. ( 1998). The design and analysis of longitudinal studies of development and psychopathology in context: Statistical models and methodological recommendations . Development and Psychopathology, 10, 395-426.
51.
Wu, A.D., Li, Z., & Zumbo, B.D. ( 2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research and Evaluation , 12, 1-26.
52.
Wu, C.H., Chen, L.H., & Tsai, Y.M. ( 2009). Longitudinal invariance analysis of the satisfaction with life scale. Personality and Individual Differences, 46, 396-401.
53.
Zumbo, B.D. ( 1999a). The simple difference score as an inherently poor measure of change: Some reality, much mythology. In B. Thompson (Ed.), Advances in social science methodology (Vol. 5, pp. 269-304). Greenwich, CT: JAI Press.
54.
Zumbo, B.D. ( 1999b). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
55.
Zumbo, B.D. ( 2007a). Validity: Foundational issues and statistical methodology . In C.R. Rao & S. Sinharay (Eds.), Handbook of statistics. Vol. 26: Psychometrics (pp. 45-79). Amsterdam: Elsevier Science.
56.
Zumbo, B.D. ( 2007b). Three generations of differential item functioning (DIF) analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.
