Implementing Evidence-Based Practices in School Psychology: Excavation by De-Implementing the Disproved

Abstract

The scientist-practitioner model of practice is the most common approach to the profession of school psychology and embraces evidence-based practices as foundations of clinical practice. The focus on evidence-based practices involves not only using the preponderance of research to determine what works, but also how to implement these practices effectively. An important impediment to implementing innovative evidence-based practices is that interventions and practices that have been proved ineffective or of low value continue to be used in education and psychology. What are the issues that assist in discontinuing practices that are widely used, but have been disproved or are otherwise problematic? How can room be made for more effective, innovative, and evidence-based practices? This issue of the Canadian Journal of School Psychology is devoted to exploration of different forms of disproved, low value, or problematic practices, factors that keep these practices alive in schools, and how to best de-implement ineffective, low value, and problematic practices. If the scientist-practitioner model is to be defined largely by the implementation of evidence-based practices, then de-implementation will be a critical aspect in the evolution of the profession of school psychology.

Max Planck’s Principle, “Science progresses one funeral at a time” is morbid, but the point being that scientists do not tend to change their minds with new data. They hold on to old and even disproved ideas their entire lives. The problems may be worse in education. Education and psychology are considered by many practitioners to be arts as much as sciences. As such, there is no strong culture in education of having science directly influence the acquisition and elimination of practices. And the culture in psychology is mixed regarding the acceptance of science as a driving force. As such, change driven by science can be halting and resistant to innovations and improvements (McKay et al., 2018).

School psychology has embraced the relatively new field of implementation science (e.g., Forman et al., 2013). The issues and challenges of applying new and innovative practices with research support and implementing them in schools, classrooms, clinics, and counseling practices are the essence of evidence-based practice (Montini & Graham, 2015). Equally important is the topic of de-implementation, which is the divesting from ineffective, disproved, inefficient, low value, and even harmful educational and psychological practices that remain widely used (Hanrahan et al., 2015).

De-implementation has been considered in medicine, but the issues of education and psychology are different and possibly even more complex (Brownson et al., 2017). Practices in education must not only be effective, but also consistent with regulation, law, and risk-management practices; be financially and resource efficient; be in line with the values of community, parents, and other stakeholders; address entrenched financial interests such as contracts with curriculum developers and test publishers; be responsive to the local history and traditions of the school system; and be responsive to the unique needs and culture of the implementation site (Shernoff et al., 2017). Moreover, many entrenched educational and psychological practices are based on word-of-mouth myths that feel nice, but have no supporting evidence (e.g., learning styles). Disproving and divesting from ineffective practices have challenges, but divesting from myths that never had supporting evidence is a remarkably challenging proposition (Patey et al., 2018). Complicating matters further is that interventions in psychology and education may be effective for some clients, under some conditions, and at some times; but may not yet have research support or have inconsistent support. Therefore, knowing when de-implementation is warranted is another layer to the process. As challenging as de-implementation has been in medicine, de-implementation in school psychology is likely to prove more fraught and complex as the profession moves toward evidence-based practices.

Features of a Generic De-implementation Plan

There are three families of approaches to de-implementation of ineffective or inefficient practices. These are information and knowledge transmission, bottom-up approaches, and top-down approaches. Quite likely a combination of approaches will be required to create complete de-implementation of an ineffective or inefficient practice (Norton et al., 2017). There are few empirical studies of efforts to de-implement (e.g., Norton & Chambers, 2020). Most evidence is based on historical records. Even then, de-implementation of ineffective or harmful practice has been painful. The case of Ignaz Semmelweis in 1847 is the most extreme example. His work recommending against physicians transitioning from conducting autopsies to delivering babies without handwashing eventually saved hundreds of thousands of lives. However, many of his colleagues rejected him and his ideas completely. De-implementation requires not only knowledge and evidence, but its own set of strategic thinking and consultation skills. Perhaps Semmelweis may have had more immediate success had he not referred to his hesitant obstetrician colleagues as “irresponsible murderers.” He eventually died in a mental institution at the age of 47 (Nuland, 2004). There will always be resistance to change. However, the goal is to achieve evidence-based change away from ineffective practices and toward supported practices; preferably without experiencing professional rejection or the fate of Semmelweis.

Knowledge Transmission

The most traditional approach is to transmit knowledge that disproves the former practice and supports a new and more effective practice (Pinto & Witte, 2019). Professionals are unlikely to adopt innovation when there is an existing intervention that they are comfortable with, have been using for years, requires few resources, is established as part of professional culture or lore, is intuitive, and has become part of a classroom or other system (Prasad & Ioannidis, 2014). Debunking by providing strong peer-reviewed evidence demonstrating no positive or negative effects is the primary current method of de-implementation (Pinto & Park, 2019). There are mixed effects of de-bunking (Upvall & Bourgault, 2018). Knowledge does not always influence behavior. Also, many professionals have observed the disproved or low-value interventions in action and believe the interventions to be effective. Sometimes professionals are afraid to de-implement due to lack of any proven effective alternative (McKay et al., 2018). And other times, professionals may have a financial or reputational stake in a disproved practice and are unwilling to de-implement for these personal reasons.

Debunking can be effective, but requires a sustained and strategic effort. Effective debunking requires multiple sources, over a long period of time, and repeated often (Prusaczyk et al., 2020). There must be enough negative or alternative information in the educational mindspace that serves to change the entire culture of a profession (Prasad & Ioannidis, 2014). This is an extraordinarily large undertaking that cannot be conducted through an article, special issues of a journal, consultation, or a continuing education event. Multiple stage de-implementation strategies are required.

Bottom-Up Model

Bottom-up models involve influencing new professionals and professionals open to innovation. The primary stage of the bottom-up approach is to train the future generations of professionals in their respective fields to understand, but avoid, ineffective and harmful practices (Schlesinger & Grob, 2017). This is coupled with learning and integrating more effective and innovative practices as a method of building a new culture. Of course, a major problem is that this approach assumes those professionals training the next generation (i.e., university professors) have effectively de-implemented ineffective practices and have embraced effective alternatives. Often a major impediment to bottom-up change is that university professors are unwilling to de-implement their favorite practices (Shernoff et al., 2017). In addition, continuing education and in-service professional development can be effective and play a minor role in bottom-up change. Only a small percentage of professionals change their behavior toward de-implementing ineffective practices and adopting innovative practice due to a workshop (Pinto & Witte, 2019). However, those small numbers of professionals, when effectively supported, can provide a seed from which cultural change can grow (Pinto & Witte, 2019).

Top-Down Model

Top-down models involve mandated or strongly encouraged de-implementation or encouragement of practice. An example of a soft top-down approach is to have a professional association recommend best practices that exclude ineffective practices and encourage evidence-based practices (Upvall & Bourgault, 2018). In increasing rigidity of top-down de-implementation include private or government insurance companies reimbursing only for clinical practices proved effective, systems overtly favoring proven practices, area regulations that do not support disproved practices, and finally laws that support only practices proved to be effective. Top-down approaches are effective in changing behaviors, but restrict professional autonomy, impede innovation and may negatively influence the reputation of a given profession (Prasad & Ioannidis, 2014). Professionals are responsible for governing their own practices. Nonetheless, efforts to influence policy makers is one part of a de-implementation plan that cannot be ignored.

Overall Strategy

Education and psychology have reputations as fields that are less evidence-based than medicine and other professions (Patey et al., 2018). Implementation of innovation in these fields is certainly more complex in the field of school psychology than many fields. Effective de-implementation requires a strategic effort. The notion that professionals should stop using ineffective practices because it is self-evident based on research is likely to result in failed de-implementation. A strategy of wide-scale debunking; promotion of a more effective replacement practice; helping the professors and trainers to prepare the next generation in evidence-based practices and expressly rejecting ineffective practices; and influencing policy makers to create regulations, laws, and incentives to eschew practices that are consistently proved ineffective or harmful (Verkerk et al., 2018) is likely to prove most effective.

Purpose of the Special Issue

Although there is something to be said about hesitation to change and Max Planck’s Principle, there are signs that de-implementation is possible. Because de-implementation is among the most challenging aspects of developing a true evidence-based practice in school psychology, the creation of a framework of systems level consultation is necessary so that school psychologists have specific set of strategies targeted at de-implementing disproved ideas and creating a strong environment for evidence-based practices to flourish (van Bodegom-Vos et al., 2017).

This special issue of the Canadian Journal of School Psychology provides models of de-implementation, illustrations of examples of successful de-implementation, concepts and ideas in field where de-implementation is needed, or where specific challenges are present. Effective de-implementation requires nothing less than changing culture. As Peter Drucker is famous for saying, “Culture eats strategy for breakfast.” CJSP provides a multidisciplinary approach to the most advanced ideas on multiple topics related to de-implementation in a single issue. The goal is to provide frameworks, examples, and basic ideas in changing behaviors, culture, and professional activities to best support de-implementation in the practice of school psychology.

Brief Description of the Contributions to the Special Issue

“Reducing low-value practices: A functional-contextual consideration to aid in de-implementation efforts” by Ryan L. Farmer, Imad Zaheer, Gary Duhon, and Stephanie Ghazal (this issue) provides a clear, well-reasoned, and practical approach to reducing low-value practices. The application of operant learning theory provides a recognizable framework to reduce low-value practice and replace with high-value or evidence-based practices.

“De-implementing inappropriate accommodations practices” by Benjamin J. Lovett and Allyson G. Harrison (this issue) considers the complexity of providing accommodations to students requiring special considerations in assessment and the classroom. The focus is on reducing accommodations via considerations from functional behavior assessment.

“Why questionable assessment practices remain popular in school psychology: Instructional materials as pedagogic vehicles” by Ryan L. Farmer, Ryan J. McGill, Stefan Dombrowski, and Gary L. Canivez (this issue) review widely used texts and other instructional resources, which tend not to undergo a rigorous peer review process, and may serve as major vectors in the instruction and maintenance of the low-value practice of emphasizing profile analysis of intelligence test scores.

“Continued educational neuromyth belief in pre-service and in-service teachers: A call to de-implementation action for school psychologists” by Heather L. Craig, Gabrielle Wilcox, Erica M. Makarenko, and Frank P. MacMaster (this issue) presents results of a survey of teachers on the acceptance of disproved neuromyths in education and the practices engendered by these neuromyths. The authors suggest that making discussion and instruction in accurate educational neuroscience, directed largely by school psychologists, can provide alternative and high value pedagogical practices.

“Muddled measurement: A historical perspective on questionable practices in school psychology’s assessment of learning disabilities” by Eric Elias (this issue) provides a detailed review of the issues and problematic practices that have given rise to and maintained the use of the straight intelligence test score-achievement test score discrepancy as the primary approach to diagnosing specific learning disabilities. The consequences of this practice are discussed. In addition, the value of a widely accepted alternative perspective with far fewer problematic outcomes (response to intervention models) is discussed as a key component of de-implementation.