Abstract
Objective
The purpose of this study was to examine the retention of wilderness first aid (WFA) knowledge, self-efficacy beliefs, and skills over time in a sample of WFA course participants.
Methods
Seventy-two open enrollment (volunteer) WFA course participants were assessed at 4 months, 8 months, or 12 months after training. Changes in WFA knowledge and self-efficacy were assessed by written instruments after the course and at the follow-up interval (4, 8, or 12 months). The WFA skills were assessed by a scored medical scenario at the follow-up interval.
Results
As the time interval increased, WFA knowledge, self-efficacy, and skill proficiency decreased. The WFA knowledge and self-efficacy beliefs were not highly correlated with skill performance.
Conclusions
Without additional training, regular use of the course content, or efforts to refresh thinking on key topics, the ability of WFA students to effectively apply their learning will likely decrease as time from training increases. With respect to these WFA courses, student scores on written tests did not accurately reflect competence in performing practical skills related to a medical scenario. In addition, student self-confidence in the ability to perform such skills did not strongly correlate with actual skills and ability.
Introduction
Wilderness first aid (WFA) programs were developed for laypeople who may need to provide first aid in remote locations, in challenging weather, with limited support and equipment. Despite increasing interest in wilderness medicine certification and a growing number of training providers, no research has examined knowledge, self-efficacy (confidence in one's knowledge or skill), and skill retention after wilderness medicine courses.
A WFA course is an introductory, 16-hour course intended for non–medical professionals engaging in outdoor recreation or working in remote locations. The intended setting for use of WFA skills is an outdoor location where evacuations are primarily by walking or carryout with the assistance of local resources, and where local Emergency Medical Services and Search and Rescue access is expected in a timely manner (less than 8 hours). Examples include day hikes or overnight backpacking trips, stationary wilderness camps, weekend family activities, and front-country outdoor recreation. 1
The WFA course content typically focuses on performing a basic patient assessment to identify obvious injuries or abnormalities, and to assess basic and obvious signs, symptoms, and vital signs patterns. Course content includes initiation of first aid treatments (bleeding control, splints, wound care, spine immobilization, managing heat and cold), and conservative decisions on the need for, urgency of, and appropriate type of evacuation or intervention. It does not include complex medical assessments or diagnosis, or invasive treatments other than administration of epinephrine for anaphylaxis with an autoinjector.
Retention research has been conducted with medical professionals who were required to use their skills regularly amidst their occupational duties in hospital settings. 2 -4 Generally, findings indicate that written knowledge is retained longer than practical skill. 5 The skill retention rates for laypeople (similar to those who might take a WFA course) are similar to, or in many cases worse than, those of medical professionals. 6
Therefore, the intent of this study is to examine the retention of WFA knowledge, self-efficacy beliefs, and skills over time in a sample of WFA course participants through a quasiexperimental design examining groups 4 months, 8 months, or 12 months after course completion. Specifically, we tested the hypotheses that WFA knowledge, self-efficacy, and skill would decrease as the time interval from initial training increased. We also examined: 1) which skills existed at an acceptable (passing) level; 2) what knowledge was retained; and 3) the relationship between WFA knowledge, self-efficacy, and skill performance.
Methods
Participants and Logistics
The Wilderness Medicine Institute (WMI) of the National Outdoor Leadership School (NOLS) provides wilderness medicine training to both laypeople and medical professionals including outdoor recreationists, guides, trip leaders, medical students, and physicians. In all, 170 participants from 7 WMI WFA courses were invited to participate in this study. The courses were open to all interested parties and enrolled with mixed groups of high school and college students, and local nonstudents. All 7 WFA courses were taught over 2 consecutive days and were based in Lander or Riverton, Wyoming. Each course was taught by 1 of 3 senior WMI instructors accompanied by 1 of 3 experienced WMI assistant instructors. The study was reviewed and approved by the University of Utah Institutional Review Board, and subjects gave consent accordingly.
All enrolled participants were informed that they would be contacted once 4 months, 8 months, or 12 months after course completion and would be invited to return to WMI of NOLS headquarters in Lander, on a voluntary basis, to complete a written examination, self-efficacy questionnaire, and skills assessment. Participants in the courses who had medical training beyond WFA were not included in the study.
Course groups were assigned to testing intervals of 4, 8, or 12 months. To facilitate scheduling and testing, a window of plus or minus 1 week around the return date was allowed.
Measures
The 3 variables of primary interest in this study were WFA knowledge, WFA self-efficacy, and WFA skills. Wilderness first aid knowledge was assessed using a 25-question multiple-choice examination. The WFA self-efficacy scale was modeled after the recommendations of Bandura 7 for self-efficacy scale construction. The content coverage of the WFA self-efficacy scale is consistent with the WFA skill assessment (see 8 content areas below). Participants self-assessed their ability to perform the 8 WFA skill areas by rating 8 items on an 11-point scale ranging from 0% “cannot do it at all” to 100% “highly certain can do it.” The WFA self-efficacy scale was internally consistent, with a Cronbach's α = .92 for these data. Wilderness first aid skill was assessed with a checklist covering 8 specific skills reflecting the educational structure in which these topics were taught: 1) size up the scene; 2) initial assessment; 3) physical examination; 4) vital signs; 5) patient history; 6) manage injuries; 7) focused spine assessment; and 8) verbal report in SOAP (acronym for subjective, objective, assessment, plan) format.
At the completion of the WFA course, participants were given the WFA written examination and the WFA self-efficacy scale. At that time, they were invited to participate in the study and informed of their approximate testing dates. As their testing dates approached, students were contacted and scheduled for testing by WMI.
At their assigned testing session, participants initially completed a skills-based scenario similar to simulations in the original WFA. First aid materials identical to those provided during the WFA course were accessible. Fifty-nine of the 72 returnees were assessed using the same examiner and the same role-playing patient. The remaining 13 returnees were assessed by 3 additional examiners trained by the primary examiner. Training included scoring the scenario together with the primary examiner and discussing points of inconsistency until a common metric was established. Having multiple raters score a single scenario for consistency was familiar to the additional raters, as this technique, and the score sheet, are used in WMI instructor training.
After completing the skill assessment, the participants completed the same WFA knowledge test and self-efficacy measure used at the conclusion of their initial course. Students were not given any scenario performance feedback by the evaluators before completing the written instruments. Demographic information was collected at this time, and participants were asked if they had any additional training since their original course.
Data Analysis
Both WFA knowledge and WFA self-efficacy were analyzed using a 3 × 2 repeated measures analysis of covariance (ANCOVA) with group (4, 8, or 12 months) as the between-subjects effect and time (initial, follow-up) as the within-subjects effect. Whether their employment since initial course completion required first aid certification was included as a covariate to control for any employment-related refreshments of skill or knowledge.
A multivariate analysis of variance (MANOVA) was used to test the hypotheses related to the skills checklist. The 8 content areas were treated as interdependent dependent variables. The assigned group (4, 8, or 12 months) was the independent variable of interest. Given the multivariate nature and interdependence of the 8 content areas, discriminant function analysis was used to follow up a significant omnibus test. 8
The exploratory analysis further examined these data. Pass rates for the skill areas were assessed to see which skills were at the a priori determined acceptable level. Item level examination of the knowledge test was conducted, and correlations were used to examine the relationships between WFA knowledge, self-efficacy, and skills.
Results
Seventy-two of the students volunteered to participate in the study. These participants were 55% male, 45% female, and averaged 42 years of age. Groups did not differ significantly by average age (4-month group = 39.0 years; 8-month group = 44.6 years; 12-month group = 44.4 years). Data were cleaned and screened for univariate outliers before analysis.
WFA Knowledge and Self-Efficacy
The 4-, 8-, and 12-month groups were not significantly different from each other regarding WFA knowledge and self-efficacy beliefs at time 1 (initial; P > .05). The interaction terms were significant for both WFA knowledge (F1, 63 = 3.24, P = .046, partial η2 = .09, observed power = .60) and WFA self-efficacy (F1, 63 = 8.21, P = .001, partial η2 = .21, observed power = .95). Thus, the retention of WFA knowledge and self-efficacy depends upon group (4, 8, or 12 months). The difference between the 4-month and 8-month groups was not significant. However, the differences between the 8-month group and 12-month group were significant for knowledge (P = .03) and self-efficacy (P = .033). Both knowledge and self-efficacy were lower for the 12-month group compared with the 8-month group. The difference scores in Table 1 illustrate the larger drops in both WFA knowledge and self-efficacy for the 12-month group.
Wilderness first aid knowledge and self-efficacy by group
Range of possible scores for wilderness first aid (WFA) knowledge is 0 to 25; for WFA self-efficacy, it is 0 to 1.
WFA Skill Checklist
The initial MANOVA with the 8 skill areas comparing assigned group (4, 8, or 12 months) was significant (F16, 114, λ = .45, P < .001). Both follow-up discriminant functions were significant and warranted further interpretation (function 1 R c = .62, P < .001; function 2 R c = .53, P = .007).
Coefficients loading higher than .30 were considered part of the function for interpretation purposes (see Table 2). Function 1 primarily shows how the 4-month group differs from the 8-month and 12-month groups; showing higher scores on the vital sign, physical examination, and patient history skills, it also indicates lower scores (ie, negative scores) on sizing-up the scene and treatment of patient injuries. Follow-up analyses on function 1 found that the 4-month group was significantly higher on function 1 than both the 8-month group (P < .001) and 12-month group (P < .001).
Structure matrix
Function 1 primarily represents how the 4-month group is different from the 8-month and 12-month group;
Function 2 primarily shows how the 8-month group differs from the 4-month and 12-month groups and is primarily represented by the verbal SOAP reporting, but also included both patient history and spine assessment. Follow-up analyses on function 2 found that the 8-month group was significantly lower than both the 4-month group (P = .001) and 12-month group (P < .001).
Exploratory Data Analyses
The secondary purposes of this study were to explore the content participants were proficient in several months after the course and the relationship between WFA knowledge, self-efficacy, and skill. The percentage of study participants passing each of the WFA skill areas was examined (Figure 1). Treatment of injuries and the physical examination were completed at an acceptable level by 50% of the participants. Taking the vital signs and a comprehensive patient history were not completed at an acceptable level by any of the participants. Fewer than half of the participants were proficient in the scene size-up, initial assessment, and spine management. Interestingly, only 12.5% of the participants used a SOAP note memory aid, which was included with the first aid materials.
Passing skill areas. Note: because the verbal SOAP (subjective, objective, assessment, plan) reporting was not considered an essential skill and no pass/fail criteria were assessed, verbal SOAP is not presented here.
Regarding the WFA knowledge data, some questions were consistently missed more than others. Three of the questions were missed by more than 30% of the participants on retest: frostbite treatment, decision criteria for evacuation of head injury, and decision criteria for when to attempt to reduce a shoulder dislocation.
To explore the relationship between WFA knowledge, self-efficacy, and skill, the correlations between these variables were examined. The total skill score was correlated with the final (time 2) WFA knowledge and self-efficacy scores for each group. Skill performance as measured by the skill assessment was only weakly related to the WFA knowledge examination for all 3 groups (r2 ranged from a low of .01 for the 4-month group to a high of .09 for the 12-month group) and was virtually unrelated to WFA self-efficacy for all 3 groups (r2 ranged from a low of .01 for the 4-month group to a high of .07 for the 8-month group).
Discussion
In this study, WFA skills and knowledge deteriorated as the time interval from training increased, written tests did not accurately predict how well practical skills were performed, and self-efficacy beliefs did not correlate with actual skill performance. These results are consistent with those of previous studies that show first aid knowledge and skills deteriorate in the absence of repeated practice by lay providers. 9
In addition to general skill deterioration over time, participants across all groups demonstrated notably poor skill proficiency on what may, for a layperson, be novel and detailed skills: measuring vital signs, obtaining a patient history, and conducting the focused spine assessment (a selective spine immobilization protocol). A limited amount of instructional and practice time combined with the abundance of detail in the selective spine immobilization protocol, known as the focused spine assessment, may be an explanation for the overall poor performance of this skill across all groups. In contrast, splinting, perhaps a more familiar or common sense skill, was performed at a higher level by all groups.
The 3 most commonly missed written test questions ask about complex or novel information conveyed in WFA courses: frostbite treatment, decision criteria for evacuation of a head injury, and decision criteria for attempting to reduce a shoulder dislocation. Based on the initial knowledge test results, these criteria appeared to be understood at course completion. In a wilderness context, the layperson first aid provider may need to make an independent decision on the need for and urgency of treatment and evacuation. The WFA instructors should consider memory aids for these decisions in the event that communication with the Emergency Medical Services system is not available. 14
The finding that a high score on a written WFA test has little relation to skill performance in a WFA medical scenario is consistent with previous research 10 and suggests the need to dedicate WFA course time to hands-on skill practice while judiciously allocating time to learning lecture-based, written knowledge. Wilderness first aid is fundamentally a practical and applied set of knowledge and skills. Application should be the focus of the instruction.
In this study, a person's opinion of his or her skills and knowledge was not predictive of performance. Results in the present study were consistent with previous research on layperson confidence in relation to actual cardiopulmonary resuscitation skill and first aid skills. 12 Conversely, research on nonlaypersons has found self-efficacy beliefs to be generally predictive of performance. 13 It may be that nonlaypersons, such as medical students, nurses, or physicians, have more opportunity to calibrate the accuracy of the self-efficacy beliefs because they practice medical skills more frequently. This finding has implications for WFA instructors who may have course participants with a dangerously inflated belief that they will be capable of independently managing complex medical issues of outdoor activities, or conversely, participants with lower self-perceptions who might unnecessarily refrain from helping with first aid.
Our findings echo previous research calling for simplicity in first aid content and instruction to ensure proficiency later in time. 10 It may be appropriate to distill patient assessment skills down to the simplest form. For example, teaching WFA students to count the respiration rate may be more effective than discussing the quality of the respirations, and determining whether a patient is either conscious or unconscious may be more effective than teaching a more complex system (eg, the AVPU [alert, voice, pain, unresponsive] scale) to evaluate mental status.
Course content might also be reexamined. One of the challenges of a 16-hour course format is deciding what is practical and relevant to a layperson first aid provider in the wilderness context. Current research shows that the injuries most likely to occur in wilderness activities are minor athletic injuries, soft tissue injuries, and gastrointestinal issues. 11 This same research also shows that fractures, dislocations, frostbite, hypothermia—a full spectrum of injury and illness—can also occur. If WFA students are unlikely to encounter spinal injuries and are also unlikely to perform the skills well, it may be appropriate to remove a selective spine immobilization protocol from the WFA curriculum and focus on immobilization of any suspected spine injury. The time saved could then be devoted to reinforcing skills that the students may be likely to encounter and, most importantly, be capable of addressing.
Lastly, poor overall skill performance calls attention to the duration of WFA certification, which is commonly 2 to 3 years. Our study suggests that 2 years in the absence of a refresher course may be inappropriate. Research in the field of medical skill retention is comparable with these findings: skill levels decline significantly over time but can return to satisfactory levels after 1 refresher session, and they can be even higher after 2 refresher sessions. 10 The American Heart Association recommends that skills be practiced and assessed numerous times between the biannual basic life support and advanced cardiac life support refresher courses, although no minimum frequency has been established. 11
Ultimately, the WFA and other entry level wilderness medicine courses fill an important need for backcountry travelers. Teaching the basics of care for sprains and strains, wound care, backcountry hygiene, and gastrointestinal problems provides laypersons with the rudimentary skills to address the most common first aid issues. However, wilderness medicine educators and providers of wilderness programs need to realize the limits of such training. Without additional training, regular use of the course content, or efforts to refresh thinking on key topics, the ability of WFA students to effectively apply their learning will likely decay over time.
Implications for Research
Future researchers might find it interesting to explore the reasons for the difference in skill proficiency and investigate the manner in which the skills are taught. Further, it may be useful to look specifically at retention of isolated skills such as wound cleaning or splinting in the absence of other WFA content owing to the observation in the present study that novel skills may have overshadowed the performance of other WFA skills.
Study Limitations
Notable limitations of this study include the sample population, subjectivity of the scenario, use of raters, study measures, and study design. Although the study groups did not differ significantly by age, the overall sample population, at 42 years average age, is a bit older than the typical WFA course participant (34.5 years). Individual group composition may also explain the differences in group skill. The artificial nature of a scenario involving moulage and a simulated victim likely influence the actual performance of skills. Despite efforts to ensure consistent and equitable skill ratings, rater bias remains a possible confound and limitation of this study. The specific questions missed on the knowledge test could be attributed to question difficulty rather than lack of retention over time. It is also possible that the passing criteria in this study were exceedingly high. In addition, these results may not be applicable to other wilderness medicine programs owing to differences in course curriculum and instructors' skill or training.
Lastly, the quasiexperimental design itself was also a compromise, given the resources for the study. While every effort was made to create equitable and comparable groups, systematic assignment would have allowed the study to examine causality. In addition, testing and scoring each of the 170 potential study participants through a skill assessment upon WFA course completion would have provided a skill baseline and allowed for longitudinal comparisons similar to those made with results of WFA knowledge and self-efficacy. That was logistically impossible, but should be considered for future studies.
Conclusions
Without additional training, regular use of the course content, or efforts to refresh thinking on key topics, the ability for WFA students to effectively apply their learning will likely decrease as the time from course increases. Tests of knowledge or inquiries into confidence about ability to perform skills in a medical scenario should not be used to measure ability to perform a skill set that is fundamentally applied and practical. Lastly, course content on WFA courses should be examined for ways to best serve WFA graduates. That may mean focusing on the most common and treatable backcountry issues while deemphasizing infrequent or complex treatments that cannot be adequately addressed in a 16-hour course.
Footnotes
Acknowledgments
The authors would like to acknowledge Paige Healy, who served as the patient for the majority of the wilderness first aid scenarios. This study was made possible through financial and logistical support from the Wilderness Medicine Institute of the National Outdoor Leadership School.