Evaluation of Clinical Preceptor Training and Its Impact on Inter-Rater Reliability

Abstract

Background:

The consistency of clinical evaluations of respiratory therapy (RT) students by clinical preceptors is an area where research is needed. The purpose of this study is to evaluate if standardized clinical preceptor training will improve inter-relater reliability (IRR) scores for staff involved in evaluating the clinical skills of respiratory therapy students. We hypothesized that standardized clinical preceptor training will not significantly affect IRR scores for faculty members.

Methods:

Study design is a one-way repeated measures design evaluating mean IRR scores of faculty members involved with evaluating RT students before and after completion of training for clinical preceptors. We obtained IRB approval and used a purposive sampling approach. We could not blind or randomize this nonprobability sample due to the nature of the study design. We measured IRR by percentage agreement. We categorized scores as <70% = Needs Improvement; 70-79% = Good; 80-89% = Great; >89% = Excellent. We also assessed IRR scores by using a paired-sample t-test with an alpha level of <0.05 showing significance.

Results:

All faculty members met inclusion criteria and results showed that standardized training improved IRR scores for faculty members. Mean pre-training IRR scores for participants resulted in a score of 63% which was a 'Needs Improvement' score. Mean post-training IRR scores for participants resulted in a score of 78% which was a 'Good' score. Standardized preceptor training improved IRR scores by 15% and had a P value <0.05 showing that scores were significantly different.

Conclusions:

This study showed that standardized clinical preceptor training can improve IRR scores needed for accreditation. RT programs, and more importantly, RT students will benefit from standardized clinical preceptor training for all faculty members involved with clinical evaluations of students. This training will help improve the quality of clinical evaluations and guidance to RT students.

Pre/Post Training Mean IRR Scores