An Exploratory Study of Pre-Service Teachers’ Evidence-Based Practice Related Knowledge,Attitudes,and Practices

Abstract

It is important to understand the readiness of pre-service teachers (PSTs) to use evidence-based practices (EBP) upon entering the field. This study had two objectives: (1) pilot a measure of EBP previously validated with health professional students (EBP-KAP), and (2) investigate PSTs’ knowledge and perceptions of EBP. Fifty-seven PSTs completed the EBP-KAP. Participants’ self-reported knowledge of EBP was significantly greater than their attitudes toward and current use of EBP. PSTs in their first year of training had more positive attitudes toward EBP than PSTs in their second year. Implications for training and practice are discussed.

Keywords

evidence-based practice pre-service teachers teacher training

Evidence-based practice (EBP) has become a standard of care in psychology (see APA Presidential Task Force on Evidence-Based Practice, 2006); however, EBP has not yet become a standard of practice in the field of education. There have been increasing calls to close the gap between research and practice in education, as studies have identified a disparity between educational research and practice both at the policy level and in schools (see Anwer & Reiss, 2022; Broekkamp & van Hout-Wolters, 2007; Pegram et al., 2022). This discrepancy was most recently exemplified by the Ontario Human Rights Commission’s (2022) Right to Read Report, which described the poor literacy outcomes resulting from a lack of evidence-based reading instruction in the provincial curriculum. However, despite increasing awareness of the need for EBP in education and consultative support for teachers and future teachers, there is evidence to suggest an overall movement away from using EBP to inform instructional practices in the classroom (Kauffman, 2007).

Studies have investigated teachers’ perceptions and use of EBP and have found that, overall, teachers support the idea of using EBP to inform their instructional and classroom management practices (e.g., Stormont et al., 2011) but are reluctant to use EBP due to barriers such as locating and interpreting research (Kauffman, 2007) and a lack of training in implementation (Stormont et al., 2011; Freeman et al., 2014). Additionally, teachers may view researchers as being out of touch with the real world demands of the classroom environment and therefore see research findings as less important than personal experience when delivering curriculum and programming to students (Gore & Gitlin, 2004).

A study of British teachers found that, although most teachers supported the idea of using EBP, a lack of support and acceptance of EBP at the broader school level made it difficult to implement (Brown & Zhang, 2016). Other studies have identified additional barriers to EBP; for example, many teachers do not receive training in how to locate, interpret, and apply research findings as a component of their pre-service training, nor is knowledge of EBP a requirement of entering the teaching profession or maintaining professional credentials (Broekkamp & van Hout-Wolters, 2007). Additionally, teachers may view individual knowledge, personal experience, and creativity as more relevant to implementing programming or using various teaching strategies than adhering to empirical findings (e.g., Broekkamp & van Hout-Wolters, 2007). In contrast, professions such as psychology require practitioners to maintain continuing competency with research pertinent to their area of practice, and place significantly less emphasis on personal experiences or anecdotal evidence to inform them (see APA Presidential Task Force on Evidence-Based Practice, 2006).

Measuring EBP knowledge, attitudes, and practices

EBP related attitudes, knowledge, and skills have typically been measured using surveys that are specific to a particular field (e.g., medicine; see Ritchie et al., 2019) and few measures are transdisciplinary in nature (Shi et al., 2014). Studies investigating teachers’ EBP-related views have primarily used measures developed by researchers to test their specific hypotheses (e.g., Brown & Zhang, 2016), which limits cross-study comparisons over time. However, a questionnaire developed by Ritchie et al. (2019) measures EBP-related knowledge, attitudes, and practices across academic and professional practice settings, enabling researchers to compare EBP views within and across professions.

Pre-service teachers (PSTs) are an important population in which to study EBP knowledge, perceptions, and practices. It is important to understand their attitudes toward and willingness to use EBP to ensure that they are prepared to manage the frequently complex academic and behavioral needs of their students. Establishing a baseline during training could identify gaps in knowledge and could possibly lead to initiatives to address these gaps during both pre- and in-service training. Psychologists are multidisciplinary partners who can implement EBP, develop EBP capacity within schools, and provide consultative support (e.g., Newman & Rosenfield, 2019; Rosenfield, 2021); however, a shared common understanding between psychologists and teachers is essential to facilitate appropriate consultation practices and subsequent intervention implementation in schools. The goals of the current study were (1) to pilot a measure of EBP previously validated for use with health professional students and (2) to investigate PSTs’ current knowledge and perceptions of EBP and how frequently they use EBP to guide their instructional practices. Given the exploratory nature of the research no a priori hypotheses were made.

Method

Participants

Participants were 57 pre-service teachers enrolled in a two-year Bachelor of Education (BEd) program in Eastern Canada (first year n = 32, second year n = 25). Most participants were in the 20 to 25-year-old range (78.9%), Caucasian (89.5%), and female (80.7%).

Measures

Participant demographics

Basic demographic information was collected using an adaptation of the Background Information Questionnaire (BIQ; Curtis et al., 2014).

Evidence-based practice-knowledge attitudes and practices (EBP-KAP; Ritchie et al.,2019)

Attitudes and knowledge of evidence-based practices were assessed using the EBP-KAP. This measure consists of 23 items, rated on a 6-point Likert scale (1 = strongly disagree, 6 = strongly agree), and yields four subscales: (1) Knowledge (i.e., understanding of the key components of EBP); (2) Attitudes (i.e., positive and negative attitudes about EBP); (3) Information Retrieval Practices (i.e., extent to which respondent searches for empirical evidence to inform their practices); and (4) Professional Practice and Education (i.e., degree to which respondent regularly uses evidence in decision-making processes). This measure has moderately high internal reliability (α’s .74 to .84) and good factor structure with item loadings on each factor >0.4. For more information about the EBP-KAP, please contact the corresponding author.

Procedure

Once informed consent was obtained, participants completed confidential paper surveys that were counterbalanced among measures used as a part of a larger study. Following completion of the study, participants were entered into a draw to win one of three $50 gift cards to a local bookstore. The research received clearance from the University Research Ethics Board.

Results

Statistical Analyses

Positive bivariate correlations were found on all subscales. Correlations were small to moderate and significant (p < 0.05), demonstrating a substantial amount of unique variance for each subscale (see Table 1). Scores on the Knowledge subscale (Cronbach’s = .854) were significantly higher than scores on the Professional Practice and Learning subscale (Cronbach’s = .669), which were in turn significantly higher than scores on both Information Retrieval (= .821) and Attitudes (= .738) subscales. A within-subjects ANOVA was conducted to determine whether scores on each dimension were significantly different. The overall model was significant F_{(3, 53)} = 12.04 p ≤ .001, η²_p = .41 (see Table 2).

Table 1.

Correlations Among Factors (N = 57).

	Attitudes	Practice-retrieval	Practice-application
Knowledge
r	0.438	0.445	0.551
p	≤0.01	≤0.01	≤0.01
Attitudes
r	1	0.315	0.492
p		0.02	≤0.01
Practice-retrieval
r		1	0.551
p			≤0.01

Table 2.

Within-Subjects ANOVA Testing Differences Between Factor Scores (N = 57).

Factor	M (95% CI)	SD	d*
Knowledge (Kn)	77.01 (73.73, 80.30)	12.27
Practice—Information Retrieval (PIR)	67.78 (63.85, 71.70)	14.58
Kn–PIR			2.52
Professional Practice and Learning (PPL)	70.46 (67.55, 73.36)	10.84
Kn–PPL			1.93
PIR–PPL			−0.75
Attitudes about EBP (Att)	68.78 (66.68, 70.89)	8.41
Kn–Att			2.56
PIR–Att			−0.29
PPL–Att			0.54

d = mean difference of listed subscales divided by pooled variance of those subscales.

Testing a Multivariate Model

A mixed between- and within-subjects MANOVA was conducted to examine differences in PSTs’ knowledge, practices, and attitudes about EBP (see Table 3). The overall model was significant (F_{(4, 52)} = 1153.46, p ≤ 0.001, η²_p = .99). This standardized effect size is reported here in keeping with APA standards; however, it is important to note that effect sizes can easily become inflated in within-subjects designs, as participants are effectively compared against themselves, resulting in smaller error terms. Cohen’s d is reported in Tables 2 and 3 to facilitate cross-study comparisons. There was no significant interaction between year of study and student ratings on the EBP-KAP (F_{(4, 52)} = 2.26, p = .075, η²_p = .148). There was a significant between-subjects effect of year of study on the Attitudes subscale, such that students in their first year of study had more positive attitudes toward EBP than students in their second year.

Table 3.

Between-Subjects MANOVA Testing Differences Between Factor Scores (N = 57).

Factor	df	df error	Year 1		Year 2
Factor	df	df error	M (95% CI)	SD	Mean (95% CI)	SD	p	$η_{p}^{2}$	d*
Knowledge	1	55	78.47 (74.12, 82.83)	9.83	75.56(70.63, 80.49)	14.89	.378	.014	−0.83
Practice – Information Retrieval	1	55	68.88 (63.68, 74.08)	14.08	66.67 (60.79, 72.55)	15.41	.574	.006	−0.58
Professional Practice and Learning	1	55	71.75 (67.90, 75.59)	10.19	69.17 (64.82, 73.52)	11.66	.378	.014	−0.78
Attitudes about EBP	1	55	71.95 (69.38, 74.52)	7.12	65.62 (62.02, 69.22)	8.72	.004	.142	−2.25

*d = mean of year 2 – mean of year 1 divided by pooled variance of year 1 and 2 groups.

Discussion

The goal of this study was to gain a baseline of the attitudes, practices, and knowledge of PSTs using a transdisciplinary measure of EBP (EBP-KAP; Ritchie et al., 2019). Consistent with Ritchie et al. (2019), the EBP-KAP was an acceptable measure of evidence-based attitudes, knowledge, and practices as evidenced by moderately high internal reliability with this sample. Findings indicated that, regardless of their year of study, PSTs reported having more knowledge of EBP than did they use or see the benefits of EBP. This finding is consistent with previous findings from studies of practicing teachers and demonstrates a similar gap between PSTs’ self-rated knowledge and their implementation of EBP (e.g., Broekkamp & van Hout-Wolters, 2007). This may be due to whether evidence-informed curricula are taught within BEd programs (Freeman et al., 2014), a greater emphasis on learning through experience-informed practices or to inadequate generalization of concepts learned during preservice coursework to applied settings (Scheeler et al., 2009). This highlights the need to ensure that PST training includes both coursework based upon the best available evidence, as well as opportunities to apply knowledge and skills while receiving immediate performance feedback (Scheeler et al., 2009). PSTs may have overestimated their self-reported knowledge of EBP, which could account for some of the discrepancy between their knowledge and use of EBP. It is vital that PSTs receive training in how to locate, interpret, and apply research findings as a component of their training.

First-year students had significantly more positive attitudes toward EBP than second-year students. It is possible that second-year students, all of whom had completed practicum experiences by the time they participated in this study, viewed EBP as less relevant or applicable in the ‘real world’ classroom setting (see Broekkamp & van Hout-Wolters, 2007) or, conversely, may have struggled to apply the knowledge learned in coursework into their teaching practices during practicum and, thus, viewed it less favorably (Scheeler et al., 2009). Using an interprofessional collaboration model could be useful in developing both pre- and in-service teachers’ knowledge and use of EBP. School psychologists, who receive extensive training in evidence-informed interventions, could support teachers to learn and implement these interventions through teaching, modeling, and providing immediate feedback. Previous studies have shown that similar models of in-service teacher training positively affect intervention fidelity and increase teachers’ skill (see Kretlow & Bartholomew, 2010).

Limitations

Although this study provides important insight into PSTs’ understanding and self-reported use of EBP, it is important to acknowledge its limitations. First, the EBP-KAP measure has not previously been used in a sample of PSTs, therefore, the findings should be interpreted with caution until they can be replicated. Second, the study used a relatively small sample, which limits the generalizability of these findings. It is also unclear how familiar PSTs are with EBP or whether the concept was introduced and defined in their training. Finally, this study relied solely on self-report questionnaire data, which restricts participants’ ability to elaborate upon their responses.

Future Research

Continued research assessing reliability and validity of the EBP-KAP measure, especially in teacher populations, is warranted. Examples include assessment of construct validity using focus groups with experts, conducting confirmatory factor analyses from a range of professional training contexts, and assessment of predictive validity by correlating self-report EBP-KAP with behavioral observations. Finally, there is a paucity of research examining the connection between teacher and school psychology training and the interactions between trainees and staff in these disciplines. As noted by Margison and Shore (2009), if school psychologists are to consult and collaborate effectively with teachers, it is essential to understand teacher training and practice realities; this understanding is best developed as early as possible to ensure that professionals entering the field value professional pluralism (San Martin Rodriguez et al., 2005). This study was a small first step in this direction, but future research could examine the parallel trajectories from training to practice across school psychology and teacher training contexts.

Further research in this area has the potential to positively influence teacher training programs by providing in-depth data on the current state of teachers’ knowledge and skill with using EBP, as well as to detect gaps in these areas. It will also be incumbent upon researchers and universities to collaborate to use these findings to include teaching about EBP into teacher training, and to engage in interprofessional education in universities that train both teachers and school psychologists to develop collaborative relationships and understanding. As noted above, the findings of the Right to Read inquiry have highlighted the very real consequences of failure to implement EBP in schools. Engaging in interprofessional practice and productive consultation and developing a shared understanding of EBP could allow teachers and psychologists to work collaboratively to ensure the best possible outcomes for students.

Footnotes

Acknowledgements

The authors thank all the students who participated in this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by a Scotia Scholar award from the Nova Scotia Health Research Foundation (now Research Nova Scotia) awarded to Ms. Noyes.

Ms. Noyes is currently a graduate student in School and Applied Child Psychology at Western University.

ORCID iD

Sara King

Author Biography

Amira Noyes, MA, is a doctoral student in School and Applied Child Psychology at Western University.

Krista C. Ritchie, PhD, is an associate professor in the Faculty of Education at Mount Saint Vincent University.

Sara King, PhD, RPsych, is an associate professor in the Faculty of Education and co-coordinator of the graduate program in School Psychology at Mount Saint Vincent University.

References

Anwer

Reiss

(2022). Linking research and practice in education: The views of expert researchers in the field. Journal of Education for Teaching. Advance online publication. https://doi.org/10.1080/02607476.2022.2055447

APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285.

Broekkamp

van Hout-Wolters

(2007). The gap between educational research and practice: A literature review, symposium, and questionnaire. Educational Research and Evaluation, 13(3), 203–220. https://doi.org/10.1080/13803610701626127

Brown

Zhang

(2016). Is engaging in evidence-informed practice in education rational? what accounts for discrepancies in teachers’ attitudes towards evidence use and actual instances of evidence use in schools? British Educational Research Journal, 42(5), 780–801. https://doi.org/10.1002/berj.3239

Curtis

D. F.

Hamilton

R. J.

Moore

D. W.

Pisecco

(2014). Are teachers’ beliefs related to their preferences for ADHD interventions? Comparing teachers in the United States and New Zealand. Australasian Journal of Special Education, 38(2), 128–149. https://doi.org/10.1017/jse.2014.12

Freeman

Simonsen

Briere

Macsuga-Gage

(2014). Pre-Service teacher training in classroom management: A review of state accreditation policy and teacher preparation programs. Teacher Education and Special Education, 37(2), 106–120. https://doi.org/10.1177/0888406413507002

Gore

J. M.

Gitlin

A. D.

(2004). [RE]visioning the academic-teacher divide: Power and knowledge in the educational community. Teachers and Teaching, 10(1), 35–58. https://doi.org/10.1080/13540600320000170918

Kauffman

J. M.

(2007). Conceptual models and the future of special education. Education and Treatment of Children, 30(4), 241–258. https://doi.org/10.1353/etc.2007.0024

Kretlow

Bartholomew

(2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education, 33(4), 279–299.

10.

Margison

J. A.

Shore

B. M.

(2009). Interprofessional practice and education in health care: Their relevance to school psychology. Canadian Journal of School Psychology, 24(2), 125–139. https://doi.org/10.1177/0829573509336537

11.

Newman

D. S.

Rosenfield

S. A.

(2019). Building competence in school consultation: A developmental approach. Routledge.

12.

Ontario Human Rights Commission. (2022). Right to read inquiry report. “Retrieved May 25, 2022, from https://www.ohrc.on.ca

13.

Pegram

Watkins

R. C.

Hoerger

Hughes

J. C.

(2022). Assessing the range and evidence-base of interventions in a cluster of schools. Review of Education, 10(1), e3336. https://doi.org/10.1002/rev3.3336

14.

Ritchie

Snelgrove-Clarke

Murphy

(2019). The 23-item evidence-based practice-knowledge attitudes and practices (23-item EBP-KAP) survey: Initial validation among health professional students. Health Professions Education, 5(2), 152–162. https://doi.org/10.1016/j.hpe.2018.09.004

15.

Rosenfield

(2021). Strengthening the school in school psychology training and practice. School Psychology Review. Advance online publication. https://doi.org/10.1080/2372966X.2021.1993032

16.

San Martin Rodriguez

Beaulieu

M.-D.

D’Amour

Ferrada-Videla

. (2005). The determinants of successful collaboration: A review of the theoretical and empirical studies. Journal of Interprofessional Care, 19, 132–147. https://doi.org/10.1080/13561820500082677

17.

Scheeler

Bruno

Grubb

Seavey

(2009). Generalizing teaching techniques from university to K-12 classrooms: Teaching preservice teachers to use what they learn. Journal of Behavioral Education, 18(3), 189–210. https://doi.org/10.1007/s10864-009-9088-3

18.

Shi

Chesworth

Law

Haynes

MacDermid

(2014). A modified evidence-based practice–Knowledge, attitudes, behaviour and decisions/outcome questionnaire is valid across multiple professions involved in pain management. BMC Medical Education, 14, 263.

19.

Stormont

Reinke

Herman

(2011). Teachers’ characteristics and ratings for evidence-based behavioural interventions. Behavioral Disorders, 37(1), 19–29.