Panel Attrition and External Validity in the Short-Term Follow-Up Study of Adolescent Substance Use

Abstract

The present study examines the relationship among grade, substance assessed; and type of measurement and statistical analysis employed in the detection of external validity threats due to panel attrition in the modal follow-up study found in adolescent substance use research. Six successive 2-year panel samples are examined The results indicate that dropouts demonstrate significantly higher baseline mean use of licit (tobacco and alcohol) and illicit (marijuana and hard drugs) substances when ANOVA is used, but only higher baseline use of illicit substances when chi-square analyses are used In addition, although mean use tests are no more sensitive to the increased variation in substance use that occurs during adolescence, they are more likely to reveal differences between dropouts and stayers in baseline use of licit substances. The results indicate (a) that findings from school-based, longitudinal studies almost certainly underestimate the level and extent ofsubstance use among adolescents; and (b) that alternative types of attrition analyses yield different effects regardless of grade, even in the short-term follow-up study. These findings are discussed for their implications to adolescent substance use research and for the establishment of standardized procedures to consider the relative risk of Type 2 to Type 1 error in attrition analyses.

Get full access to this article

View all access options for this article.

References

Ary, D.V. , and A. Biglan. 1988. Longitudinal changes in adolescent cigarette smoking behavior: Onset and cessation. Journal of Behavioral Medicine 11:361-82.

Ary, D.V. , A. Biglan , R. Glasgow , L. Zoref , C. Black , L. Ochs , H. Severson , R. Kelly , W. Weissman , E. Lichtenstein , P. Brozovsky , R. Wirt , and L. James. 1990. The efficacy of social-influence prevention programs versus "standard care": Are new initiatives needed? Journal of Behavioral Medicine 13:281-96.

Berberian, R.M. , W.D. Thompson , S.V. Kasl , L.C. Gould , and H.D. Kleber. 1976. The relationship between drug education programs in the Greater New Haven schools and changes in drug use and drug-related beliefs and perceptions. Health Education Monographs 4:327-76.

Biglan, A. , R. Glasgow , D.V. Ary , R. Thompson , H.H. Severson , E. Lichtenstein , W. Weissman , C. Faller , and C. Gallison. 1987. How generalizable are the effects of smoking prevention programs? Refusal skills training and parent messages in a teacher-administered program. Journal of Behavioral Medicine 10:613-28.

Biglan, A. , D. Hood , P. Brozovsky , L. Ochs , D. Ary , and C. Black. 1991. Subject attrition in prevention research. In Drug abuse prevention intervention research: Methodological issues , edited by C. G. Leukfeld and W. Bukowski. NIDA Research Monograph 107. Washington, DC: U.S. Government Printing Office.

Biglan, A. , H. Severson , D. Ary , C. Faller , C. Gallison , R. Thompson , R. Glasgow , and E. Lichtenstein . 1987. Do smoking prevention programs really work? Attrition and the internal and external validity of an evaluation of a refusal skills training program. Journal of Behavioral Medicine 10:159-71.

Blackford, L. 1977. Summary report surveys of students drug use. Unpublished manuscript, San Mateo Department of Public Health and Welfare, San Mateo, CA.

Cohen, J. 1962. The statistical power of abnormal-psychological research: A review. Journal of Abnormal and Social Psychology 65:145-53.

—. 1988. Statistical power analysis for the behavioral sciences . 2d ed. Hillsdale, NJ: Lawrence Erlbaum.

10.

—. 1990. Things I have learned (so far. American Psychologist 45:1304-12.

11.

Ellickson, P.L. , and R.M. Bell. 1990. Drug prevention in junior high: A multi-site longitudinal test. Science 247:1299-1305.

12.

Gigerenzer, G. , J. Swijtink , M. Porter , B. Daston , R. Beatty , and W. Kruger. 1989. The empire of chance: How probability theory changed social science. Cambridge: Cambridge University Press.

13.

Grady, K. , K. Gersick , D.L. Snow , and M. Kessen. 1986. The emergence of adolescent substance use. Journal of Drug Education 6:203-20.

14.

Hansen, W.B. , L.M. Collins , C.K. Malotte , C.A. Johnson , and J.E. Fielding. 1985. Attrition in prevention research. Journal of Behavioral Medicine 8:261-75.

15.

Hansen, W.B. , C.A. Johnson , B.R. Flay , J.W. Graham , and J. Sobel. 1988. Affective and social influences approaches to the prevention of multiple substance abuse among seventh grade students: Results from Project SMART. Preventive Medicine 17:135-54.

16.

Hansen, W.B. , N. Tobler , and J.W. Graham. 1990. Attrition in substance abuse prevention research: Ameta-analysis of 85 longitudinally followed cohorts. Evaluation Review 14:677-85.

17.

Johnston, L.D. , P.M. O'Malley , and J.G. Bachman. 1989. Drug use, drinking, and smoking: National survey results from high school, college, and young adult populations. National Institute on Drug Abuse (DHHS Publication ADM 89-1638). Washington, DC: U.S. Government Printing Office.

18.

Jurs, S.G. , and C.V. Glass. 1971. The effect of experimental mortality on the internal and external validity of the randomized comparative experiment . Journal of Experimental Education 40:62-66.

19.

Kandel, D. , and J. Logan. 1984. Patterns of drug use from adolescence to young adulthood: Periods of risk for initiation, continued use, and discontinuation . American Journal of Public Health 74:660-66.

20.

Newcomb, M.D. , and P.M. Bentler. 1988. Consequences of adolescent drug use: Impact on the lives of young adults. Newbury Park, CA: Sage.

21.

Reicken, H.W. , and R.F. Boruch. 1974. Social experimentation: A method for planning and evaluating social intervention. New York: Academic Press.

22.

Rhodes, J.E. , and L.A. Jason. 1988. Preventing substance abuse among children and adolescents. New York: Pergamon .

23.

Rossi, J.S. 1990. Statistical power of psychological research: What have we gained in 20 years? Journal of Consulting and Clinical Psychology 58:646-56.

24.

Sedlmeier, P. , and G. Gigerenzer. 1989. Do studies of statistical power have an effect on the power of studies? Psychological Bulletin 105:309-16.

25.

Snow, D.L. , K. Grady , and J.K. Tebes. 1987. Adolescent decision-making program: Report of a longitudinal school-based preventive intervention. Community Psychologist 20:13-14.

26.

Snow, D.L. , J.K. Tebes , M.W. Arthur , and R.C. Tapasak. Forthcoming. Panel attrition and external validity in adolescent substance use research. Journal of Consulting and Clinical Psychology.