Abstract
It is important to understand the readiness of pre-service teachers (PSTs) to use evidence-based practices (EBP) upon entering the field. This study had two objectives: (1) pilot a measure of EBP previously validated with health professional students (EBP-KAP), and (2) investigate PSTs’ knowledge and perceptions of EBP. Fifty-seven PSTs completed the EBP-KAP. Participants’ self-reported knowledge of EBP was significantly greater than their attitudes toward and current use of EBP. PSTs in their first year of training had more positive attitudes toward EBP than PSTs in their second year. Implications for training and practice are discussed.
Evidence-based practice (EBP) has become a standard of care in psychology (see APA Presidential Task Force on Evidence-Based Practice, 2006); however, EBP has not yet become a standard of practice in the field of education. There have been increasing calls to close the gap between research and practice in education, as studies have identified a disparity between educational research and practice both at the policy level and in schools (see Anwer & Reiss, 2022; Broekkamp & van Hout-Wolters, 2007; Pegram et al., 2022). This discrepancy was most recently exemplified by the Ontario Human Rights Commission’s (2022) Right to Read Report, which described the poor literacy outcomes resulting from a lack of evidence-based reading instruction in the provincial curriculum. However, despite increasing awareness of the need for EBP in education and consultative support for teachers and future teachers, there is evidence to suggest an overall movement away from using EBP to inform instructional practices in the classroom (Kauffman, 2007).
Studies have investigated teachers’ perceptions and use of EBP and have found that, overall, teachers support the idea of using EBP to inform their instructional and classroom management practices (e.g., Stormont et al., 2011) but are reluctant to use EBP due to barriers such as locating and interpreting research (Kauffman, 2007) and a lack of training in implementation (Stormont et al., 2011; Freeman et al., 2014). Additionally, teachers may view researchers as being out of touch with the real world demands of the classroom environment and therefore see research findings as less important than personal experience when delivering curriculum and programming to students (Gore & Gitlin, 2004).
A study of British teachers found that, although most teachers supported the idea of using EBP, a lack of support and acceptance of EBP at the broader school level made it difficult to implement (Brown & Zhang, 2016). Other studies have identified additional barriers to EBP; for example, many teachers do not receive training in how to locate, interpret, and apply research findings as a component of their pre-service training, nor is knowledge of EBP a requirement of entering the teaching profession or maintaining professional credentials (Broekkamp & van Hout-Wolters, 2007). Additionally, teachers may view individual knowledge, personal experience, and creativity as more relevant to implementing programming or using various teaching strategies than adhering to empirical findings (e.g., Broekkamp & van Hout-Wolters, 2007). In contrast, professions such as psychology require practitioners to maintain continuing competency with research pertinent to their area of practice, and place significantly less emphasis on personal experiences or anecdotal evidence to inform them (see APA Presidential Task Force on Evidence-Based Practice, 2006).
Measuring EBP knowledge, attitudes, and practices
EBP related attitudes, knowledge, and skills have typically been measured using surveys that are specific to a particular field (e.g., medicine; see Ritchie et al., 2019) and few measures are transdisciplinary in nature (Shi et al., 2014). Studies investigating teachers’ EBP-related views have primarily used measures developed by researchers to test their specific hypotheses (e.g., Brown & Zhang, 2016), which limits cross-study comparisons over time. However, a questionnaire developed by Ritchie et al. (2019) measures EBP-related knowledge, attitudes, and practices across academic and professional practice settings, enabling researchers to compare EBP views within and across professions.
Pre-service teachers (PSTs) are an important population in which to study EBP knowledge, perceptions, and practices. It is important to understand their attitudes toward and willingness to use EBP to ensure that they are prepared to manage the frequently complex academic and behavioral needs of their students. Establishing a baseline during training could identify gaps in knowledge and could possibly lead to initiatives to address these gaps during both pre- and in-service training. Psychologists are multidisciplinary partners who can implement EBP, develop EBP capacity within schools, and provide consultative support (e.g., Newman & Rosenfield, 2019; Rosenfield, 2021); however, a shared common understanding between psychologists and teachers is essential to facilitate appropriate consultation practices and subsequent intervention implementation in schools. The goals of the current study were (1) to pilot a measure of EBP previously validated for use with health professional students and (2) to investigate PSTs’ current knowledge and perceptions of EBP and how frequently they use EBP to guide their instructional practices. Given the exploratory nature of the research no a priori hypotheses were made.
Method
Participants
Participants were 57 pre-service teachers enrolled in a two-year Bachelor of Education (BEd) program in Eastern Canada (first year n = 32, second year n = 25). Most participants were in the 20 to 25-year-old range (78.9%), Caucasian (89.5%), and female (80.7%).
Measures
Participant demographics
Basic demographic information was collected using an adaptation of the Background Information Questionnaire (BIQ; Curtis et al., 2014).
Evidence-based practice-knowledge attitudes and practices (EBP-KAP; Ritchie et al.,2019)
Attitudes and knowledge of evidence-based practices were assessed using the EBP-KAP. This measure consists of 23 items, rated on a 6-point Likert scale (1 = strongly disagree, 6 = strongly agree), and yields four subscales: (1) Knowledge (i.e., understanding of the key components of EBP); (2) Attitudes (i.e., positive and negative attitudes about EBP); (3) Information Retrieval Practices (i.e., extent to which respondent searches for empirical evidence to inform their practices); and (4) Professional Practice and Education (i.e., degree to which respondent regularly uses evidence in decision-making processes). This measure has moderately high internal reliability (α’s .74 to .84) and good factor structure with item loadings on each factor >0.4. For more information about the EBP-KAP, please contact the corresponding author.
Procedure
Once informed consent was obtained, participants completed confidential paper surveys that were counterbalanced among measures used as a part of a larger study. Following completion of the study, participants were entered into a draw to win one of three $50 gift cards to a local bookstore. The research received clearance from the University Research Ethics Board.
Results
Statistical Analyses
Positive bivariate correlations were found on all subscales. Correlations were small to moderate and significant (p < 0.05), demonstrating a substantial amount of unique variance for each subscale (see Table 1). Scores on the Knowledge subscale (Cronbach’s = .854) were significantly higher than scores on the Professional Practice and Learning subscale (Cronbach’s = .669), which were in turn significantly higher than scores on both Information Retrieval (= .821) and Attitudes (= .738) subscales. A within-subjects ANOVA was conducted to determine whether scores on each dimension were significantly different. The overall model was significant F(3, 53) = 12.04 p ≤ .001, η2 p = .41 (see Table 2).
Correlations Among Factors (N = 57).
Within-Subjects ANOVA Testing Differences Between Factor Scores (N = 57).
d = mean difference of listed subscales divided by pooled variance of those subscales.
Testing a Multivariate Model
A mixed between- and within-subjects MANOVA was conducted to examine differences in PSTs’ knowledge, practices, and attitudes about EBP (see Table 3). The overall model was significant (F(4, 52) = 1153.46, p ≤ 0.001, η2 p = .99). This standardized effect size is reported here in keeping with APA standards; however, it is important to note that effect sizes can easily become inflated in within-subjects designs, as participants are effectively compared against themselves, resulting in smaller error terms. Cohen’s d is reported in Tables 2 and 3 to facilitate cross-study comparisons. There was no significant interaction between year of study and student ratings on the EBP-KAP (F(4, 52) = 2.26, p = .075, η2 p = .148). There was a significant between-subjects effect of year of study on the Attitudes subscale, such that students in their first year of study had more positive attitudes toward EBP than students in their second year.
Between-Subjects MANOVA Testing Differences Between Factor Scores (N = 57).
*d = mean of year 2 – mean of year 1 divided by pooled variance of year 1 and 2 groups.
Discussion
The goal of this study was to gain a baseline of the attitudes, practices, and knowledge of PSTs using a transdisciplinary measure of EBP (EBP-KAP; Ritchie et al., 2019). Consistent with Ritchie et al. (2019), the EBP-KAP was an acceptable measure of evidence-based attitudes, knowledge, and practices as evidenced by moderately high internal reliability with this sample. Findings indicated that, regardless of their year of study, PSTs reported having more knowledge of EBP than did they use or see the benefits of EBP. This finding is consistent with previous findings from studies of practicing teachers and demonstrates a similar gap between PSTs’ self-rated knowledge and their implementation of EBP (e.g., Broekkamp & van Hout-Wolters, 2007). This may be due to whether evidence-informed curricula are taught within BEd programs (Freeman et al., 2014), a greater emphasis on learning through experience-informed practices or to inadequate generalization of concepts learned during preservice coursework to applied settings (Scheeler et al., 2009). This highlights the need to ensure that PST training includes both coursework based upon the best available evidence, as well as opportunities to apply knowledge and skills while receiving immediate performance feedback (Scheeler et al., 2009). PSTs may have overestimated their self-reported knowledge of EBP, which could account for some of the discrepancy between their knowledge and use of EBP. It is vital that PSTs receive training in how to locate, interpret, and apply research findings as a component of their training.
First-year students had significantly more positive attitudes toward EBP than second-year students. It is possible that second-year students, all of whom had completed practicum experiences by the time they participated in this study, viewed EBP as less relevant or applicable in the ‘real world’ classroom setting (see Broekkamp & van Hout-Wolters, 2007) or, conversely, may have struggled to apply the knowledge learned in coursework into their teaching practices during practicum and, thus, viewed it less favorably (Scheeler et al., 2009). Using an interprofessional collaboration model could be useful in developing both pre- and in-service teachers’ knowledge and use of EBP. School psychologists, who receive extensive training in evidence-informed interventions, could support teachers to learn and implement these interventions through teaching, modeling, and providing immediate feedback. Previous studies have shown that similar models of in-service teacher training positively affect intervention fidelity and increase teachers’ skill (see Kretlow & Bartholomew, 2010).
Limitations
Although this study provides important insight into PSTs’ understanding and self-reported use of EBP, it is important to acknowledge its limitations. First, the EBP-KAP measure has not previously been used in a sample of PSTs, therefore, the findings should be interpreted with caution until they can be replicated. Second, the study used a relatively small sample, which limits the generalizability of these findings. It is also unclear how familiar PSTs are with EBP or whether the concept was introduced and defined in their training. Finally, this study relied solely on self-report questionnaire data, which restricts participants’ ability to elaborate upon their responses.
Future Research
Continued research assessing reliability and validity of the EBP-KAP measure, especially in teacher populations, is warranted. Examples include assessment of construct validity using focus groups with experts, conducting confirmatory factor analyses from a range of professional training contexts, and assessment of predictive validity by correlating self-report EBP-KAP with behavioral observations. Finally, there is a paucity of research examining the connection between teacher and school psychology training and the interactions between trainees and staff in these disciplines. As noted by Margison and Shore (2009), if school psychologists are to consult and collaborate effectively with teachers, it is essential to understand teacher training and practice realities; this understanding is best developed as early as possible to ensure that professionals entering the field value professional pluralism (San Martin Rodriguez et al., 2005). This study was a small first step in this direction, but future research could examine the parallel trajectories from training to practice across school psychology and teacher training contexts.
Further research in this area has the potential to positively influence teacher training programs by providing in-depth data on the current state of teachers’ knowledge and skill with using EBP, as well as to detect gaps in these areas. It will also be incumbent upon researchers and universities to collaborate to use these findings to include teaching about EBP into teacher training, and to engage in interprofessional education in universities that train both teachers and school psychologists to develop collaborative relationships and understanding. As noted above, the findings of the Right to Read inquiry have highlighted the very real consequences of failure to implement EBP in schools. Engaging in interprofessional practice and productive consultation and developing a shared understanding of EBP could allow teachers and psychologists to work collaboratively to ensure the best possible outcomes for students.
Footnotes
Acknowledgements
The authors thank all the students who participated in this study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by a Scotia Scholar award from the Nova Scotia Health Research Foundation (now Research Nova Scotia) awarded to Ms. Noyes.
Ms. Noyes is currently a graduate student in School and Applied Child Psychology at Western University.
