This article discusses critical issues related to conducting high-quality intervention research using experimental and quasi-experimental group designs. As researchers have learned more about teaching and learning, intervention studies have become more complex. The research community is struggling with ways to sensibly negotiate a balance between rigorous research designs that satisfy traditional laboratory standards of quality and interventions that are complex and flexible enough for conducting research in the real world of classrooms and schools. Rather than organizing the discussion around a list of resolute research standards, we weigh the pros and cons of making the many difficult choices involved in conducting intervention research. Our goal is to convey the sense that good designs must involve a series of balances and compromises that defy easily categorized solutions. Among the controversial areas discussed are the importance of defining the nature of the independent variable, the value of measuring implementation, and the improvement of the quality of quasi-experiments.
Get full access to this article
View all access options for this article.
References
1.
Ball, D.L. (1990). Reflections and deflections of policy: The case of Carol Turner. Educational Evaluation and Policy Analysis , 12, 247-259.
2.
Ball, D.L. (1995). Blurring the boundaries of research and practice . Remedial and Special Education, 16, 354-364.
3.
Beck, I.L., McKeown, M.G., Sandora, C., Kucan, L., & Worthy, J. (1996). Questioning the author: A yearlong classroom implementation to engage students with text. Elementary School Journal, 96, 385-414.
4.
Billups, L.H. (1997). Response to bridging the research-to-practice gap. Exceptional Children, 63, 525-526.
5.
Bottge, B., & Hasselbring, T.S. (1993). A comparison of two approaches for teaching complex, authentic mathematics problems to adolescents in remedial math classes. Exceptional Children, 59, 556-566.
6.
Brophy, J., & Good, T.L. (1986). Teacher behavior and student achievement. In M. Witrock (Ed.), The third handbook of research on teaching (pp. 328-375). New York: MacMillan.
7.
Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of LearningSciences, 2(2), 141-178.
8.
Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental validation of transactional strategies instruction with low-achieving second grade readers. Journal of Educational Psychology, 88, 18-37.
9.
Bryk, A.S., & Raudenbush, S.W. (1992). Hierarchical linear models . New-bury Park, CA: Sage.
10.
Campbell, D.T., & Erlebacher, A.E. (1970). How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful. In J. Hellmuth (Ed.), Compensatory education: A national debate (Vol. 3, pp. 185-233). New York: Brunner/Mazel.
11.
Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. In N. L. Gage (Ed.), Handbook of research on teaching (pp. 171-246). Chicago: Rand-McNally.
12.
Carnine, D. (1995). The professional context for collaboration and collaborative research. Remedial and Special Education, 16, 368-371.
13.
Carnine, D., Jones, E., & Dixon, R. (1994). Mathematics: Educational tools for diverse learners . School Psychology Review, 23, 406-427.
14.
Chamot, A.U., Keatley, C., & Mazur, A. (1999, April). Literacy development in adolescent English language learners: Project Accelerated Literacy (PAL). Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
15.
Charters, W.W., Jr. , & Jones, J.E. (1974, February). On neglect of the independent variable in program evaluation. Eugene : University of Oregon, Project MITT.
16.
Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago : Rand-McNally.
17.
Cooley, W.W., & Leinhardt, G. (1980). The instructional dimensions study. Education Evaluation and Policy Analysis, 2, 7-25.
18.
Cooper, H., & Hedges, L. V. (Eds.). (1994). The handbook of research synthesis. New York: Russell Sage Foundation.
19.
Denckla, M.B., & Rudel, R. (1976). Rapid 'automatized' naming (RAN): Dyslexia differentiated from other learning disabilities. Neuropsychologia,14, 471-479.
20.
Dimino, J., Gersten, R., Carnine, D., & Blake, G. (1990). Story grammar: An approach for promoting at-risk secondary students' comprehension of literature. Elementary School Journal, 91, 19-32.
21.
Dunst, C.J., & Trivette, C.M. (1994). Methodological considerations and strategies for studying the long-term effects of early intervention. In S. L. Friedman & H. C. Haywood (Eds.), Developmental follow-up: Concepts, domains, and methods. New York: Academic Press.
22.
Echevarria, J. (1995). Interactive reading instruction: A comparison of proximal and distal effects of instructional conversations. Exceptional Children, 61, 536-552.
23.
Elbaum, B., Vaughn, S., Hughes, M., & Moody, S.W. (1999). Grouping practices and reading outcomes for students with disabilities. Exceptional Children, 65, 399-415.
24.
Englert, C.S., Raphael, T.E., Anderson, L.M., Anthony, H.M., & Stevens, D.D. (1991). Making writing strategies and self-talk visible: Cognitive strategy instruction in regular and special education classrooms . American Educational Research Journal,28, 337-372.
25.
Englert, C.S., & Tarrant, K.L. (1995). Creating collaborative cultures for educational change. Elementary School Journal, 16, 325-336.
26.
Foorman, B.R., Francis, D.J., Fletcher, J.M., Schatschneider, C., & Mehta, P. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Psychology, 90(1), 37-55.
27.
Forness, S., & Kavale, K.A. (1987). De-psychologizing special education. In R. B. Rutherford & C. M. Nelson (Eds.), Severe behavior disorders of children and youth (pp. 2-24). Boston: College-Hill .
Fuchs, D., & Fuchs, L.S. (1998). Researchers and teachers working together to adapt instruction for diverse learners. Learning Disabilities Research & Practice, 13, 126-137.
30.
Fuchs, D., Fuchs, L.S., Mathes, P.G., & Lipsey, M.W. (in press). Reading differences between underachievers with and without learning disabilities: A meta-analysis. In R. Gersten, E. Schiller, S. Vaughn, & J. Schumm (Eds.), Research syntheses in special education. Mahwah, NJ: Erlbaum.
31.
Fuchs, D., Fuchs, L.S., Mathes, P.H., & Simmons, D.C. (1997). Peer-assisted strategies: Making classrooms more responsive to diversity. American Educational Research Journal , 34, 174-206.
32.
Fuchs, L.S., & Fuchs, D. (1994). Academic assessment and instrumentation. In S. Vaughn & C. Bos (Eds.), Research issues in learning disabilities: Theory, methodology, assessment, and ethics (pp. 233-242). New York: Springer Verlag.
33.
Fuchs, L.S., Fuchs, D., Bentz, J., Phillips, N.B., & Hamlett, C.L. (1994). The nature of student interactions during peer tutoring with and without prior training and experience. American Educational Research Journal, 31, 75-103.
34.
Fuchs, L.S., Fuchs, D., Kazdan, S.R., & Allen, S. (1999). Effects of peer-assisted learning strategies in reading with and without training in elaborated help giving. Elementary School Journal, 99, 201-220.
35.
Gage, N.L. (1978). The scientific basis of the art of teaching. New York: Teachers College Press.
36.
Gage, N.L. (1997). "The vision thing": Educational research and AERA in the 21st century part I: Competing visions of what educational researchers should do. Educational Researcher, 26(4), 18-21.
37.
Gall, M., Borg, W., & Gall, J. (1996). Educational research: An introduction (6th ed.). White Plains, NY: Longman.
Gersten, R. (1996). Literacy instruction for language-minority students: The transition years. Elementary School Journal, 96, 227-244.
40.
Gersten, R., & Baker, S. (1997). Design experiments, formative experiments, and developmental studies: The changing race of instructional research in special education. (Technical Report 98-1). Eugene, OR: Eugene Research Institute.
41.
Gersten, R., & Baker, S. (in press). What we know about effective instructional practices for English-language learners. Exceptional Children.
42.
Gersten, R., Carnine, D., & Williams, P. (1982). Measuring implementation of a structured educational model in an urban setting: An observational approach. Educational Evaluation and Policy Analysis, 4, 67-79.
43.
Gersten, R., Carnine, D., Zoref, L., & Cronin, D. (1986). A multifaceted study of change in seven inner city schools. Elementary School Journal, 86, 257-276.
44.
Gersten, R., & McInerney, M. (1997). What parents, teachers, and researchers view as critical issues in special education research (Technical Report 97-1). Washington, DC: American Institutes for Research.
45.
Gersten, R., Vaughn, S., Deshler, D., & Schiller, E. (1997). What we know about using research findings: Implications for improving special education practice. Journal of Learning Disabilities, 30, 466-476.
46.
Gersten, R., Williams, J., Fuchs, L., Baker, S., Koppenhaver, D., Spadorcia, S., & Harrison, M. (1998). Improving reading comprehension for children with disabilities: A review of research. Washington, DC: American Institutes for Research.
47.
Good, T.L., & Grouws, D.A. (1977). Teaching effects: A process product study in fourth grade mathematics classrooms. Journal of Teacher Education , 28, 49-54.
48.
Graham, S., & Harris, K.R. (1989). Components analysis of cognitive strategy instruction: Effects on learning disabled students' compositions and self-efficacy. Journal of Educational Psychology, 81, 353-361.
49.
Graham, S., & Harris, K. (1994). Cognitive strategy instruction: Methodological issues and guidelines in conducting research. In S. Vaughn & C. Bos (Eds.), Research issues in learning disabilities: Theory, methodology, assessment, and ethics (pp. 146-160). New York: Springer Verlag.
50.
Graham, S., MacArthur, C., & Schwartz, S. (1995). Effects of goal setting and procedural facilitation on the revising behavior and writing performance of students with writing and learning problems. Journal of Educational Psychology, 87, 230-240.
51.
Greeno, J.G., & Middle School Mathematics Through Applications Project Group. (1998). The situativity of knowing, learning, and research. American Psychologist, 53, 5-26.
52.
Greenwood, C.R., & Delquadri, J. (1988). Code for instructional structure and student academic response (CISSAR). In M. Hersen & A. S. Bellack (Eds.), Dictionary of behavioral assessment (pp. 120-122). New York: Pergamon.
53.
Gresham, F.M., Gansle, K.A., Noell, G.H., Cohen, S., & Rosenblum, S. (1993). Treatment integrity of school-based behavioral intervention studies: 1980-1990. School Psychology Review , 22, 254-272.
54.
Hall, G.E., & Loucks, S. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14, 264-276.
55.
Harter, S. (1985). The self-perception profile for children. Denver, CO: University of Denver.
56.
Hasselbring, T., Sherwood, B., & Bransford, J. (1986). An evaluation of the Mastering Fractions level-one instructional videodisc program. Nashville, TN: George Peabody College of Vanderbilt University, The Learning Technology Center.
57.
Hasselbring, T., Sherwood, R., Bransford, J., Fleenor, K., Griffith, D., & Goin, L. (1988). Evaluation of a level-one instructional videodisc program. Journal of Educational Technology Systems, 16, 151-169.
58.
Hunt, P., & Goetz, L. (1997). Research on inclusive educational programs, practices, and outcomes for students with severe disabilities. The Journal of Special Education, 31, 3-29.
59.
Kelly, B., Gersten, R., & Carnine, D. (1990). Student error patterns as a function of curriculum design. Journal of Learning Disabilities, 23, 23-32.
60.
Kennedy, M.M. (1991). Implications for teaching. In E. A. Ramp & C. S. Pederson (Eds.), Follow through: Program and policy issues (pp. 57-71). Washington, DC: U.S. Department of Education , Office of Education Research and Improvement.
61.
Kennedy, M.M. (1997). The connection between research and practice. Educational Researcher, 26(7), 4-12.
62.
Kline, F.M., Deshler, D.D., & Schumaker, J.B. (1992). Implementing learning strategy instruction in class settings: A research perspective. In M. Pressley, K. R. Harris , & J. T. Guthrie (Eds.), Promoting academic competence and literacy in school (pp. 361-406). San Diego: Academic Press.
63.
Klingner, J.K., & Vaughn, S. (1996). Reciprocal teaching of reading comprehension strategies for students with learning disabilities who use English as a second language . Elementary School Journal, 96, 275-293.
64.
Klingner, J.K., Vaughn, S., & Schumm, J.S. (1998). Collaborative strategic reading during social studies in heterogeneous fourth-grade classrooms. Elementary School Journal, 99, 3-22.
65.
Kornblet, A. (1997). Response to bridging the research-to-practice gap. Exceptional Children, 63, 523-524.
66.
Krathwohl, D.R. (1993). Methods of educational and social science research . White Plains, NY: Longman.
67.
Leinhardt, G. (1977). Program evaluation: An empirical study of individualized instruction. American Educational Research Journal, 14, 277-293.
68.
Leinhardt, G., Zigmond, N., & Cooley, W. (1981). Reading instruction and its effects. American Educational Research Journal, 18, 343-361.
69.
Losardo, A., & Bricker, D. (1994). A comparison study: Activity-based intervention and direct instruction. American Journal on Mental Retardation , 98, 744-765.
70.
Lovett, M.H., Borden, S.H., DeLuca, T., Lacerenza, L., Benson, N.J., & Brackstone, D. (1994). Treating the core deficits of developmental dyslexia: Evidence of transfer of learning after phonologically and strategy based reading training programs. Developmental Psychology, 30, 805-822.
71.
Lyon, G.R., & Moats, L.C. (1997). Critical conceptual and methodological considerations in reading intervention research. Journal of Learning Disabilities , 30, 578-588.
72.
Malouf, D.B., & Schiller, E.P. (1995). Practice and research in special education. Exceptional Children, 61, 414-424.
73.
McKinney, J.D., Osborne, S.S., & Schulte, A.C. (1993). Academic consequences of learning disability: Longitudinal prediction of outcomes at 11 years of age. Learning Disabilities Research & Practice,8, 19-27.
74.
National Center to Improve the Tools of Educators. (1998). Evaluation of research on educational approaches (EREA). Unpublished manuscript, University of Oregon.
75.
Newman, D. (1990). Opportunities for research on the organizational impact of school computers. Educational Researcher, 19, 8-13.
76.
O'Connor, R.E., & Jenkins, J.R. (1995). Improving the generalization of sound/symbol knowledge: Teaching spelling to kindergarten children with disabilities. The Journal of Special Education, 29, 255-275.
77.
O'Connor, R., Jenkins, J., Leicester, N., & Slocum, T. (1993). Teaching phonological awareness to young children with learning disabilities. Exceptional Children, 59, 532-546.
78.
O'Connor, R.E., Jenkins, J.R., & Slocum, T.A. (1995). Transfer among phonological tasks in kindergarten: Essential instructional content. Journal of Educational Psychology , 37(2), 202-217.
79.
Palincsar, A.S., & Brown, A.L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction , 1, 117-175.
80.
Peterson, L., Homer, A.L., & Wonderlich, S.A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477-492.
81.
Pressley, M., & El-Dinary, P.B. (1997). What we know about translating comprehension-strategies instruction research into practice. Journal of Learning Disabilities, 30, 486-488.
82.
Pressley, M., & Harris, K.R. (1994). Increasing the quality of educational intervention research. Educational Psychology Review, 6, 191-208.
83.
Reinking, D., & Pickle, J.M. (1993). Using a formative experiment to study how computers affect reading and writing in classrooms. In C. Z. Kinzer & D. J. Leu (Eds.), Examining central issues in literacy research, theory, and practice (pp. 263-270). Chicago: National Reading Conference.
84.
Richardson, V. (1994). Conducting research on practice. Educational Researcher, 23(5), 5-10.
85.
Richardson, V., & Anders, P. (1998). A view from across the Grand Canyon. Learning Disability Quarterly, 21, 85-97.
86.
Rosenberg, M.S., Bott, D., Majsterek, D., Chiang, B., Simmons, D., Gartland, D., Wesson, C., Fraham, S., Smith-Myles, B., Miller, M., Swanson, H.L., Bender, W., Rivera, D., & Wilson, R. (1994). Minimum standards for the description of participants in learning disabilities research. Remedial and Special Education , 15(1),56-59.
87.
Rosenshine, B., & Meister, C. (1994). Reciprocal teaching: A review of the research . Review of Educational Research, 64, 479-530.
88.
Sawyer, R.J., Graham, S., & Harris, K.R. (1992). Direct teaching, strategy instruction, and strategy instruction with explicit self-regulation: Effects on the composition skills and self-efficacy of students with learning disabilities. Journal of Educational Psychology, 84, 340-352.
89.
Scanlon, D., Schumaker, J.B., & Deshler, D.D. (1994). Collaborative dialogues between teachers and researchers to create education interventions: A case study. Journal of Educational and Psychological Consultation, (S1), 69-76.
90.
Scruggs, T.E., & Mastropieri, M.A. (1994). Issues in conducting intervention research: Secondary students. In S. Vaughn & C. Bos (Eds.), Research issues in learning disabilities: Theory, methodology, assessment, and ethics (pp. 130-145). New York: Springer Verlag.
91.
Simmons, D.C., Fuchs, D., Fuchs, L.S., Hodge, J.P., & Mathes, P.G. (1994). Importance of instructional complexity and role reciprocity to classwide peer tutoring. Learning Disabilities Research & Practice, 9, 203-212.
92.
Slavin, R.E. (1999). Rejoinder: Yes, control groups are essential in program evaluation: A response to Pogrow. Educational Researcher , 28(3), 36-38.
93.
Slavin, R., & Madden, N. (1995, April). Effects of Success for All on the achievement of English language learners. Paper presented at the annual meeting for the American Educational Research Association, San Francisco.
94.
Slocum, T.A., O'Connor, R.E., & Jenkins, J.R. (1993). Transfer among phonological manipulation skills . Journal of Educational Psychology, 85, 618-630.
95.
Stallings, J. (1975). Follow through program classroom observation evaluation . Menlo Park, CA: Stanford Research Institute.
96.
Stallings, J. (1980). Allocated academic learning time revisited, or beyond time on task. Educational Leadership, 9(11), 11-16.
97.
Swanson, H.L., & Hoskyn, M. (1998). Experimental intervention research on students with learning disabilities: A meta-analysis of treatment outcomes. Review of Educational Research, 68, 277-321.
98.
Troia, G.A. (1999). Phonological awareness intervention research: A critical review of the experimental methodology. Reading Research Quarterly, 34, 28-52.
99.
Vockell, E. L., & Asher, J. W. (Eds.). (1995). Educational research (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.
100.
Walberg, H.J., & Greenberg, R.C. (1998, April 8). The Diogenes factor: Why it's hard to get an unbiased view of programs like Success for All. Education Weekly, p. 52.
101.
Williams, J.P., Brown, L.G., Silverstein, A.K., & deCani, J.S. (1994). An instructional program in comprehension of narrative themes for adolescents with learning disabilities. Learning Disability Quarterly,17, 205-221.
102.
Williams, S.R., & Baxter, J.A. (1996). Dilemmas of discourse-oriented teaching in one middle school mathematics classroom. Elementary School Journal , 97, 21-38.
103.
Wong, B.Y.L., Butler, D.L., Ficzere, S.A., & Kuperis, S. (1997). Teaching adolescents with learning disabilities and low achievers to plan, write, and revise compare-contrast essays. Learning Disabilities Research & Practice, 12, 2-15.
104.
Woodward, J., & Gersten, R. (1992). Innovative technology for secondary learning disabled students: A multi-faceted study of implementation. Exceptional Children, 58, 407-421.
105.
Ysseldyke, J.E., & Christenson, S.L. (1987). The instructional environment scale: A comprehensive methodology for assessing an individual student's instruction . Austin, TX: PRO-ED.
