The enactment of No Child Left Behind (NCLB) has pushed to the forefront the ever-present need to conduct high quality research and translate our research findings to practice by emphasizing evidence-based practices for schools and related settings. Currently, much of the discussion surrounding evidence-based practices has focused on defining these practices based on the experimental designs used in special education research. Other issues related to conducting rigorous research, such as measurement and validity, have been noticeably absent from the discussion. In this paper, we suggest that shifts in our research, conceptually as well as practically, have impacted our measurement systems, ultimately affecting the validity of our research and the availability of evidence-based practices. The purpose of our paper is to discuss the role of measurement and examine the status of validity in research within the field of Emotional and Behavioral Disorders (EBD). In addition, we provide an effective blueprint for extending the empirical rigor and generalization of our research findings with the goal of improving the availability of evidence-based practices for children, families, and teachers.
Get full access to this article
View all access options for this article.
References
1.
AchenbachT. M. (1991). Manual for the child behavior checklist/4–18 and 1991 profile.Burlington, VT: University of Vermont Department of Psychiatry.
2.
CarrE. G. (1997). Invited commentary: The evolution of applied behavior analysis into positive behavior support.Journal of the Association for Persons with Severe Handicaps, 22, 208–209.
3.
ClarkeS., DunlapG., & StichterJ. P. (2002). Twenty years of intervention research in emotional and behavioral disorders: A descriptive analysis and a comparison with research in developmental disabilities.Behavior Modification, 25, 659–683.
4.
ConroyM. A., AsmusJ. M., LadwigC. N., SellersJ. A., & ValcanteG. (2004). The effects of proximity on the classroom behaviors of students with autism in general education settings.Behavioral Disorders, 29, 119–129.
5.
ConroyM., & StichterJ. P. (2003). The application of antecedents in the functional assessment process: Existing research, issues and recommendations.Journal of Special Education, 37, 15–25.
6.
Council for Exceptional Children. (2003). No child left behind act of 2001. Implications for special education in policy and practice: Title I improving the academic achievement of the disadvantaged.Washington, DC: Author.
7.
DunstC. J., TrivetteC. M., & CutspecP. A. (2002). Toward an operational definition of evidence-based practices.Centerscope, 1, 1–10.
8.
FornessS. R. (in press). Strengthening the research base in special education: The pursuit of evidence-based practice. In LandrumT., CrockettD., & GerberM. (Eds.), Strategies for achieving reform in special education.Mahwah, NJ: Lawrence Erlbaum Associates.
9.
FoxJ. J., & ConroyM. A. (1995). Setting events and behavior problems: An interbehavioral field analysis for research and practice.Journal of Emotional/Behavioral Disorders, 3, 130–140.
10.
GreshamF. M., & ElliottS. N. (1990). Social skills rating system manual.Circle Pines, MN: American Guidance Service.
11.
GreshamF. M., GansleK. A., & NoellG. H. (1993). Treatment integrity in applied behavior analysis with children.Journal of Applied Behavior Analysis, 26, 257–263.
12.
HarrisF. R., JohnstonM. K., KelleyC. S., & WolfM. M. (1964). Effects of social reinforcement on repressed crawling of a nursery school child.Journal of Educational Psychology, 55, 34–41.
13.
HornerR. H., SugaiG., ToddA. W., & Lewis-PalmerT. (in press). School-wide positive behavior support: An alternative approach to discipline in schools. In BambaraL., & KernL. (Eds.), Positive behavior support.New York: Guilford Press.
14.
HornerR. H., ToddA. W., Lewis-PalmerT., IrvinL. K., SugaiG., & BolandJ. B. (2004). The school-wide evaluation tool (SET): A research instrument for assessing school-wide positive behavior support.Journal of Positive Behavior Interventions, 6, 3–12.
15.
IrvinL. K., TobinT. J., SpragueJ. R., SugaiG., & VincentC. G. (under review). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions.Journal of Positive Behavior Interventions.
16.
KartubD., Taylor-GreeneS., MarchR., & HornerR. (2000). Reducing hallway noise: A systems approach.Journal of Positive Behavior Interventions, 2, 179–183.
17.
KauffmanJ. M. (1999). Commentary: Today's special education and its message for tomorrow.Journal of Special Education, 32, 244–255.
18.
KauffmanJ. M., & HallahanD. P. (1997). The illusion of full inclusion: A comprehensive critique of a current special education bandwagon.Austin, TX: Pro-Ed.
19.
KauffmanJ. M., & LloydJ. W. (1995). Inclusion of all students with emotional or behavioral disorders? Let's think again.Phi Delta Kappan, 76(7), 542–547.
20.
KernL., BambaraL., & FogtJ. (2002). Class-wide curricular modification to improve the behavior of students with emotional or behavioral disorders.Behavioral Disorders, 27, 317–326.
21.
KutashK., DuchnowskiA., KipS., OliveiraB., GreesonM., HarrisK., & SheffieldS. (2001, February 25–28). School reform efforts for children with emotional disturbances and their families. In A system of care for children's mental health: Expanding the research base. Proceedings of the 14th Annual Research Conference: Tampa, FL.
22.
LevinJ. R., O'DonnellA. M., & KratochwillT. R. (2003). Educational/psychological intervention research. In ReynoldsW., & MillerG. (Eds.), Handbook of psychology, vol. 7: Educational psychology (pp. 557–581). Hoboken, NJ: John Wiley & Sons.
23.
LewisT. J., HudsonS., RichterM., & JohnsonN. (2004). Scientifically supported practices in EBD: A proposed approach and brief review of current practices.Behavioral Disorders, 29, 247–259.
24.
LoY., LoeS. A., & CartledgeG. (2002). The effects of social skills instruction on the social behaviors of students at risk for emotional or behavioral disorders.Behavioral Disorders, 27, 371–385.
25.
LyonR., FletcherJ. M., ShawitzS. E., ShawitzB. A., TorgesonJ. K., WoodF. B., SchulteA., & OlsonR. (2001). Rethinking learning disabilities. In FinnC., RotherhamA., & HokansonC. (Eds.), Rethinking special education for a new century (pp. 259–288). City, ST: Thomas B. Fordham Foundation and the Progressive Policy Institute.
26.
MageauT. (2004). Determining “what works”: An interview with Dr. Grover Russ Whitehurst.T.H.E. Journal, 32–37.
27.
McCurdyL. L., MannellaM. C., & EldridgeN. (2003). Positive behavior support in urban schools: Can we prevent the escalation of antisocial behavior?Journal of Positive Behavior Interventions, 5(3), 158–170.
28.
McGinnisE., & GoldsteinA. P. (2004). Skillstreaming the elementary school child: New strategies and perspectives for teaching prosocial skills.Champaign, IL: Research Press.
29.
MessickS. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In WainerH., & BraunH. I. (Eds.), Test validity (pp. 33–45). Mahwah, NJ: Lawrence Erlbaum Associates.
30.
MetzlerC. W., BiglanA., RusbyJ. C., SpragueJ. R. (2001). Evaluation of a comprehensive behavior management program to improve school-wide positive behavior support.Education and Treatment of Children, 4, 448–479.
31.
MostertM. P., KauffmanJ. M., & KavaleK. A. (2003). Truth and consequences.Behavioral Disorders, 28, 333–347.
32.
National Association of State Mental Health Program Directors and The Policymaker Partnership for Implementing IDEA at National Association of State Directors of Special Education. (2002). Mental health, schools, and families working together for all children and youth: Toward a shared agenda. A concept paper sponsored by the U.S. Department of Education.
33.
NelsonJ. R., RobertsM., MathurS. R., & RutherfordR. B. (1999). Has public policy exceeded our knowledge base? A review of the functional behavioral assessment literature.Behavioral Disorders, 24, 169–179.
34.
NickersonA. B., & BrosofA. M. (2003). Identifying skills and behaviors for successful inclusion of students with emotional or behavioral disorders.Behavioral Disorders, 28, 401–409.
35.
OdomS. L. (2004). The RCT gold standard: Beware of the midas touch.Focus on Research: Newsletter of the CEC Division for Research, 17(1), 1–2.
36.
OdomS. L., BrantlingerE., GerstenR., HornerR., ThompsonB., HarrisK. (in press). Research in special education: Scientific methods and evidence-based practices.Education and Treatment of Children.
37.
OnwuegbuzieA. J. (2000, November 21). Expanding the framework of internal and external validity in quantitative research. Paper presented at the 00th Annual Meeting of the Association for the Advancement of Educational Research. Ponte Verde, Florida
38.
ParkerR. M. (2001). Power, control, and validity.Journal of Learning Disabilities, 23(10), 613–620.
39.
SassoG. (2001). The retreat from inquiry and knowledge in special education.Journal of Special Education, 24, 178–193.
40.
SassoG., ConroyM. A., StichterJ., & FoxJ. J. (2001). Slowing down the bandwagon: The misapplication of functional assessment for students with emotional and behavioral disorders.Behavioral Disorders, 26, 282–296.
41.
ScottT. M., & BarrettS. B. (2004). Using staff and student time engaged in disciplinary porcedures to evalute the impact of school-wide PBS.Journal of Positve Behavior Interventions, 6(1), 21–27.
42.
ScottT. M., BucalosA., LiaupsinC., NelsonC. M., JolivetteK., & DeSheaL. (2004). Using functional behavior assessment in general education settings: Making a case for effectiveness and efficiency.Behavioral Disorders, 29, 189–201.
43.
ShavelsonR. J., & TowneL. (Eds.). (2002). Scientific research in education.Washington, DC: National Academy Press.
44.
SidmanM. (1960). Tactics of scientific research: Evaluating experimental data in psychology.New York: Basic Books.
45.
SidmanM. (1988). Tactics of scientific research: Evaluating experimental data in psychology.Boston: Authors Cooperative Incorporated.
46.
SiegelA. E. (1964). Sidney Siegel: A memoir. In MessickS., & BrayfieldA. H. (Eds.), Decision and choice: Contributions of Sidney Siegel. New York: McGraw-Hill.
47.
SutherlandK. S., WehbyJ. H., & CopelandS. R. (2000). Effects of varying rates of behavior-specific praise on the on-task behavior of students with EBD.Journal of Emotional & Behavioral Disorders, 8, 2–8.
48.
SutherlandK. S., WehbyJ. H., & YoderP. J. (2002). Examination of the relationship between teacher praise and opportunities for students with EBD to respond to academic requests.Journal of Emotional & Behavioral Disorders, 10, 5–13.
49.
SugaiG., & HornerR. (2002). The evolution of discipline practices: School-wide positive behavior supports.Child & Family Behavior Therapy, 24(1–2), 23–50.
50.
U.S. Department of Education. (2002a). No child left behind: A desktop reference.Washington, DC.
51.
U.S. Department of Education. (2002b). No child left behind executive summary.Washington, DC.
WittJ. C., NoellG. H., LaFleurL. H., & MortensonB. P. (1997). Teacher use of interventions in general education settings: Measurement and analysis of the independent variable.Journal of Applied Behavior Analysis, 30, 693–696.
