Despite the wide availability of statistical programs designed to deal with longitudinal data from a multilevel perspective, many applied researchers remain unfamiliar with the benefits of this methodology, particularly for the evaluation of interventions. The authors present an example of multilevel modeling as part of the analysis of evaluation data from an HIV intervention study. Strategies for understanding multilevel models using longitudinal (panel) data are demonstrated and discussed. The authors illustrate how multiple linear regression models provide a convenient conceptual background to understanding how hierarchical linear models can be developed and interpreted. Multilevel analysis results are compared and contrasted with typical approaches through general linear models for repeated-measures data. Analyses are presented using the SPSS and HLM 5 software.
Get full access to this article
View all access options for this article.
References
1.
AIDS Community Demonstration Projects. (1999). Community-level HIV intervention in 5 cities: Final outcome data from the CDC AIDS community demonstration projects. American Journal of Public Health, 89(3), 336-345.
2.
Arnold, C. L. (1992). An introduction to hierarchical linear models. Measurement and Evaluation in Counseling and Development, 25, 58-90.
3.
Bijleveld, C. C. J. H., & van der Kamp, L. J. T. (1998). Longitudinal data analysis: Designs, models, and methods. London: Sage Ltd.
4.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
5.
Cornman, D. H., Botticello, P. J., O’Connell, A. A., & Ataya, R. L. (2002, March). Project M: A sex-positive approach to risk reduction among HIV+ gay and bisexual men. Paper presented at the 14th National HIV/AIDS Up-Date Conference, San Francisco.
6.
Ethington, C. A. (1997). A hierarchical linear modeling approach to studying college effects. Higher Education: Handbook of Theory and Research, 12, 165-194.
7.
Fisher, J. D., & Fisher, W. A. (2000). Theoretical approaches to individual-level change in HIV risk behavior. In J. Peterson & R. DiClemente (Eds.), Handbook of HIV prevention (pp. 3-55). New York: Plenum.
8.
Fisher, J. D., Kimble Willcutts, D. L., Misovich, S. J. & Weinstein, B. (1998). Dynamics of sexual risk behavior in HIV-infected men who have sex with men. AIDS and Behavior, 2(2), 101-113.
9.
Frank, K. A. (1998). Quantitative methods for studying social context in multilevels and through interpersonal relations. Review of Research in Education, 23, 171-216.
10.
Games, P. A. (1990). Alternative analyses of repeated-measure designs by ANOVA and MANOVA. In A. von Eye (Ed.), Statistical methods in longitudinal research, Vol. 1: Principles and structuring change. (pp. 81-122). New York: Academic Press.
11.
Glanz, K., Lewis, F. M., & Rimer, B. (Eds.). (1997). Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
12.
Greenhouse, S. W., & Geiser, S. (1959). On methods in the analysis of profile data. Psychometrika, 24, 95-112.
13.
Hox, J. J. (2000). Multilevel analyses of grouped and longitudinal data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multilevel data (pp. 15-32). Mahwah, NJ: Lawrence Erlbaum.
14.
Huynh, H., & Feldt, L. S. (1976). Estimation of the Box correction for the degrees of freedom in randomized block and split-plot designs. Journal of Educational Statistics, 1, 29-82.
15.
Isaac, S., & Michael, W. B. (1995). Handbook in research and evaluation (3rd ed.). San Diego, CA: Educational and Industrial Testing Services.
16.
Johnson, R. A., & Wichern, D. A. (1998). Applied multivariate statistical analysis (4th ed.). Upper Saddle River, NJ: Prentice Hall.
17.
Jones, K., & Duncan, C. (1995). Individuals and their ecologies: Analysing the geography of chronic illness within a multilevel modelling framework. Health and Place, 1(1), 27-40.
18.
Keselman, H. J., Algina, J., & Kowalchuk, R. K. (2001). The analysis of repeated measures designs: A review. British Journal of Mathematical and Statistical Psychology, 54, 1-20.
19.
Kenny, D. A., Bolger, N., & Kashy, D. A. (2002). Traditional methods for estimating multilevel models. In D. S. Moskowitz & S. L. Hershberger (Eds.), Modeling intraindividual variability with repeated measures data: Methods and applications (pp. 1-24). Mahwah, NJ: Lawrence Erlbaum.
20.
Kirk, R. E. (1995). Experimental design (3rd ed.). Pacific Grove, CA: Brooks/Cole.
21.
Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. London: Sage Ltd.
22.
Lauby, J., O’Connell, A. A., Stark, M., & Adams, J. (1999, August). Analysis of cross-sectional surveys to evaluate a community-level HIV prevention intervention for women. Paper presented at the National HIV Prevention Conference, Atlanta, GA.
23.
Littell, R., Milliken, G., Stroup, W., & Wolfinger, R. (1996). SAS system for mixed models. Cary, NC: SAS Institute.
24.
MacCallum, R. C., & Kim, C. (2000). Modeling multivariate change. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multilevel data (pp. 51-68). Mahwah, NJ: Lawrence Erlbaum.
25.
MacCallum, R. C., Kim, C., Malarkey, W. B., & Kiecolt-Glaser, J. K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32(3), 215-253.
26.
Mantell, J. E., DiVittis, A. T., & Auerbach, M. I. (1997). Evaluating HIV prevention interventions. New York: Plenum.
27.
Maxwell, S. E., & Delaney, H. D. (1990). Designing experiments and analyzing data: A model comparison perspective. Belmont, CA: Wadsworth.
28.
Murray, D. M. (1998). Design and analysis of group-randomized trials. New York: Oxford University Press.
29.
O’Connell, A. A., Bol, L., & Langley, S. C. (1997). Evaluation issues and strategies for community-based organizations developing women’s HIV prevention programs. Evaluation and the Health Professions, 20(4), 428-454.
30.
O’Connell, A. A., McCoach, D. B., Levitt, H., & Horner, S. (2003, April). Modeling longitudinal ordinal response variables for educational data. Paper presented at the 84th AERA Annual Meeting, Chicago.
31.
Rasbash, J., Browne, W., Goldstein, H., Yang, M., Plewis, I, Healy, M., et al. (2002). A user’s guide to MLwiN (Version 2.1d). London: Institute of Education, University of London.
32.
Rasbash, J., Prosser, R., & Goldstein, H. (1991). ML3: software for three-level analysis, user’s guide. London: Institute of Education, University of London.
33.
Raudenbush, S. W. (2001). Toward a coherent framework for comparing trajectories of individual change. In L. M. Collins & A. G. Sayer (Eds.), New methods for the analysis of change (pp. 35-64.) Washington, DC: American Psychological Association.
34.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
35.
Raudenbush, S., Bryk, A., Cheong, Y. F., & Congdon, R. (2000). HLM 5: Hierarchical Linear and Nonlinear Modeling [Computer software]. Lincolnwood, IL: Scientific Software International.
36.
Rogosa, D., & Saner, H. (1995). Longitudinal data analysis examples with random coefficients models. Journal of Educational and Behavioral Statistics, 20(2), 149-170.
37.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
38.
Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 24, 323-355.
39.
Singer, J. D. (2002). Fitting individual growth models using SAS PROC MIXED. In D. S. Moskowitz & S. L. Hershberger (Eds.), Modeling intraindividual variability with repeated measures data: Methods and applications (pp. 135-170). Mahwah, NJ: Lawrence Erlbaum.
40.
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. Oxford, UK: Oxford University Press.
Stevens, J. P. (1996). Applied multivariate statistics for the social sciences (3rd ed.). Hillsdale, NJ: Lawrence Erlbaum.
44.
Tabachnick, B. G., & Fidell, L. S. (2001a). Computer assisted research design and analysis. Boston: Allyn & Bacon.
45.
Tabachnick, B. G., & Fidell, L. S. (2001b). Using multivariate statistics (4th ed.). Boston: Allyn & Bacon.
46.
Taris, T. W. (2000). A primer in longitudinal analysis. London: Sage Ltd.
47.
Thum, Y. M. (1997). Hierarchical linear models for multivariate outcomes. Journal of Educational and Behavioral Statistics, 22(1), 77-108.
48.
van der Kamp, L.J.T., & Bijleveld, C.C.J.H. (1998). Methodological issues in longitudinal research. In C.C.J.H. Bijleveld & L.J.T. van der Kamp (Eds.), Longitudinal data analysis: Designs, models, and methods (pp. 1-45). London: Sage Ltd.
49.
van der Leeden, R. (1998). Multilevel analysis of longitudinal data. In C.C.J.H. Bijleveld & L.J.T. van der Kamp (Eds.), Longitudinal data analysis: Designs, models, and methods (pp. 269-317). London: Sage Ltd.
50.
von Korff, M., Koepsell, T., Curry, S., & Diehr, P. (1992). Multi-level analysis in epidemiologic research on health behaviors and outcomes. American Journal of Epidemiology, 135(10), 1077-1082.
51.
Wallace, D., & Green, S. B. (2002). Analysis of repeated measures designs with linear mixed models. In D. S. Moskowitz & S. L. Hershberger (Eds.), Modeling intraindividual variability with repeated measures data: Methods and applications (pp. 103-134). Mahwah, NJ: Lawrence Erlbaum.
52.
Weinfurt, K. P. (2000). Repeated measures analysis: ANOVA, MANOVA, and HLM. In L. G. Grimm&P. R. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp. 317-361). Washington, DC: American Psychological Association.
53.
Weiss, C. H. (1998). Evaluation (2nd ed.). Upper Saddle River, NJ: Prentice Hall.
54.
Wong, G. Y., & Mason, W. M. (1985). The hierarchical logistic regression model for multilevel analysis. Journal of the American Statistical Association, 80(391), 513-523.
