Findings are reported related to the research methods and statistical techniques used in the 450 group quantitative studies examined as part of the literature review portion of the Division for Early Childhood Recommended Practices project. Twelve trained coders used an investigator-developed coding form to analyze studies across seven major dimensions: (a) sampling procedures, (b) variable selection, (c) variable definition, (d) measurement integrity, (e) treatment fidelity, (f) statistical analyses, and (g) magnitude-of effect reporting. Results suggested that the methodological integrity of the quantitative research used to inform recommended practices was not uniformly convincing and compelling. Implications are offered related to the strength of empirical support for recommended practices and the conduct and reporting of future research.
Get full access to this article
View all access options for this article.
References
1.
American Psychological Association (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
2.
American Psychological Association (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
3.
Bailey, D.B., & Simeonsson, R.J. ( 1990). Family Needs Survey. Chapel Hill, NC: University of North Carolina , FPG Child Development Institute.
4.
Campbell, D.T., & Stanley, J.C. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally .
5.
Carta, J.J., Greenwood, C.R., & Atwater, J. (1992). ESCAPE: Eco-behavioral System for Complex Assessments of Preschool Environments. Kansas City, KS: University of Kansas, Juniper Gardens Children's Project.
6.
Casto, G., & Mastropieri, M.A. (1986). The efficacy of early intervention programs: A meta-analysis. Exceptional Children , 52, 417-424.
7.
Dunn, L.M., & Dunn, L.M. ( 1981 ). Peabody Picture Vocabulary Test-Revised Form L (PPVT-R). Circle Pines, MN: American Guidance Service.
8.
Dunst, C.J., & Rheingrover, R. (1981). An analysis of the efficacy of infant intervention programs with organically handicapped children. Evaluation and Program Planning, 4, 287-383.
9.
Dunst, C.J., Trivette, C.M., & Cutspec, P.A. (2002, September). Toward an operational definition of evidence-based practices. Center-scope,1, 1-10.
10.
Fallon, M.A., & Harris, M.B. (1991). Training parents to interact with their young children with handicaps: Professional-directed and parent-oriented approaches . InfantlToddler Intervention, l, 297-313.
11.
Farran, D.C. (1990). Effects of intervention with disadvantaged and disabled children: A decade review. In S. J. Meisels & J. P Shonkoff (Eds.), Handbook of early childhood intervention (pp. 501-539). New York: Cambridge University Press.
12.
Farran, D.C. (2000). Another decade of intervention for children who are low income or disabled: What do we know now? In J. P. Shonkoff & S. J. Meisels (Eds.), Handbook of early childhood intervention (2nd ed., pp. 510-548). New York: Cambridge University Press.
Friedrich, W.N., Greenberg, M.T., & Crnic, K.A. (1983). A short form of the Questionnaire on Resources and Stress. American Journal of Mental Deficiency, 88, 42-48.
15.
Gay, L.R., & Airasian, P. (2000). Educational research: Competencies for analysis and application (6th ed.). Upper Saddle River, NJ: Merrill.
16.
Guess, D., Siegel-Causey, E., Roberts, S., Rues, J., Thompson, B., & Siegel-Caussey, D. (1990). Assessment and analysis of behavior state and related variables among students with profoundly handicapping conditions. JASH, 15, 211-230.
17.
Guralnick, M.J. (1997). Introduction: Directions for second-generation research. In M. J. Guralnick (Ed.), The effectiveness of early intervention (pp. xv-xvi). Baltimore : Paul H. Brookes.
18.
Heckenlaible-Gotto, M.J., & Choi, H.S. (1993). The use of research methods and statistics in TECSE, 1990-1993: A content analysis. (ERIC Document Reproduction Service No. ED379831 ).
19.
Kaufman, M. (1990). Foreword. In P Knoblock (Series Ed.) & R. Gaylord-Ross (Vol. Ed.), Issues and research in special education: Vol. 1. (pp. xi-xii). New York: Teachers College Press.
20.
Law, M. (2000). Strategies for implementing evidence-based practice in early intervention. Infants and Young Children, 13(2), 32-40.
21.
Lawson, S., & LaGaccia, S.S. ( 1990, November). A review of methodology choices used in journals targeting individuals with mental retardationldevelopmental disabilities: Implications for graduate research. Paper presented at the meeting of the Mid-South Educational Research Association , New Orleans, LA.
22.
LeLaurin, K., & Wolery, M. (1992). Research standards in early intervention: Defining, describing, and measuring the independent variable. Journal of Early Intervention, 16, 275-287.
23.
Mahoney, G., O'Sullivan, P., & Dennebaum, J. ( 1990). Maternal perceptions of early intervention services: A scale for assessing family-focused intervention. Topics in Early Childhood Special Education, 10(1), 1-15.
24.
McCartney, K., & Rosenthal, R. (2000). Effect size, practical importance, and social policy for children. Child Development, 71, 173-180.
25.
McWilliam, R.A. (2000a). New guidelines and standards for JEI authors and reviewers. Journal of Early Intervention, 23, 75-76.
26.
McWilliam, R.A. (2000b). Reporting qualitative studies. Journal of Early Intervention, 23, 77-80.
27.
National Research Council (2002). Scientific research in education. Committee on Scientific Principles for Education Research. R. J. Shavelson & L. Towne (Eds.). Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
Peters, M.T., & Heron, T.E. (1993). When the best is not good enough: An examination of best practice. Journal of Special Education, 26, 371-385.
30.
Sandall, S., McLean, M.E., & Smith, B.J. (2000). DEC recommended practices in early interventionlearly childhood special education. Longmont, CO: Sopris West.
31.
Shotts, C.K., Rosenkoetter, S.E., Streufert, C.A., & Rosenkoetter, L.I. (1994). Transition policy and issues: A view from states. Topics in Early Childhood Special Education, 14, 395-411.
32.
Shulman, S., & Margalit, M. (1985). Evaluation of research trends in special education journals ( 1979-1982). Studies in Educational Evaluation, 11, 75-81.
33.
Simeonsson, R. (1985). Efficacy of early intervention: Issues and evidence . Analysis and Intervention in Developmental Disabilities , 5, 203-209.
34.
Smith, B.J., McLean, M.E., Sandall, S., Snyder, P., & Broudy, A. (2000). Methods and activities used to produce the DEC recommended practices. In S. Sandall, M. E. McLean, & B. J. Smith (Eds.), DEC recommended practices in early intervention/early childhood special education (pp. 107-114). Longmont, CO: Sopris West.
35.
Smith, B.J., Strain, P., Snyder, P., Sandall, S., McLean, M.E., Ramsey, A., & Sumi, W.C. (2002). Analyses of the literature review that contributed to the DEC recommended practices: Nine years of EI/ECSE research. Journal of Early Intervention, 25, 108-119.
36.
Snyder, P. (2000). Guidelines for reporting results of group quantitative investigations. Journal of Early Intervention, 23, 145-150.
37.
Sparrow, S.S., Balla, D., & Cichetti, D. (1984). Vineland Adaptive Behavior Scales (survey form) . Circle Pines, MN: American Guidance Service.
38.
Strain, P.S., & Smith, B.J. (1986). A counter-interpretation of early intervention effects: A response to Casto and Mastropieri. Exceptional Children, 53, 260-265.
39.
Thompson, B. (2002a). Statistical, practical, and clinical: How many kinds of significance do counselors need to consider?Journal of Counseling and Development, 80, 64-71.
40.
Thompson, B. (Ed.). (2002b). Score reliability: Contemporary thinking on reliability issues. Newbury Park, CA: Sage.
41.
Vacha-Haase, T., Henson, R.K., & Caruso, J.C. (2002). Reliability Generalization: Moving toward improved understanding and use of score reliability. Educational and Psychological Measurement, 62, 562-569.
42.
Vacha-Haase, T., Kogan, L.R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions . ' Educational and Psychological Measurement, 60, 509-522.
43.
Wilkinson, L., & the Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations. Retrieved March 30, 2001, from http://www.apa.org/journals/amp/amp548594.htm1
44.
Wolery, M., & Bailey, D.B. (2002). Early childhood special education research. Journal of Early Intervention, 25, 88-99.
45.
Wolery, M., & Dunlap, G. (2001). Reporting on studies using single-subject experimental methods. Journal of Early Intervention, 24, 85-89.
