Abstract
Introduction
The purpose of this study was to investigate the psychometric properties of the remote first aid self-efficacy scale (RFA SES). The RFA SES is a 30-item self-report scale developed in response to emerging evidence showing that self-efficacy is predictive of performance.
Methods
Trained alumni from Wilderness Medical Associates (WMA) training courses and less trained students from Laurentian University (LU) were recruited via email to complete an online questionnaire at 2 different periods (T1 and T2). The questionnaire included demographic questions, the 30-item RFA SES, the 10-item Connor-Davidson resilience scale (CD-RISC), and the 10-item generalized self-efficacy scale (GSES). Data analysis included assessment of the dimensionality, reliability, and validity of the scale.
Results
There were 448 alumni from WMA and 1106 students from LU who participated in the study. The RFA SES demonstrated a clear unidimensional structure. The mean interitem correlation was 0.75 at T1. Test-retest reliability (T1 to T2) was high for both the LU group (intraclass correlation [ICC]=0.90) and the WMA group (ICC=0.92). Moderate correlations were found between RFA SES and CD-RISC (r=0.42, P<0.001), a general measure of resilience, and the GSES (r=0.48, P<0.001), a general measure of self-efficacy. Wilderness Medical Associates participants showed higher mean scores than LU students at T1 (t [569]=16.2, P<0.001).
Conclusions
The RFA SES is a unidimensional, reliable, and potentially valid scale. Further research should focus on item reduction followed by additional tests of reliability and validity.
Introduction
First aid is the prehospital care provided by a bystander or victim, utilizing little to no medical equipment or support. 1 However, this first aid care may span hours and days in remote settings compared with minutes in urban settings. 2 This highlights the need for well-trained responders who are both competent and confident to respond. In an accident or medical emergency, remote populations may lack close and direct access to prehospital emergency medical services (EMS). 3 -5 In place of this health service, many remote contexts rely on trained members of the population who have taken a first aid course with curriculum relevant to the remote or wilderness context.4,6
Most remote first aid training courses rely on knowledge retention and skill competency for in-course student assessment. However, these assessments do not necessarily predict self-efficacy and likeliness to respond. 7 Self-efficacy is defined as “an individual’s belief in his or her capacity to execute behaviors necessary to produce specific performance attainments.” 7 A study with a population of trained wilderness first aid participants found that self-efficacy decreased over time, although neither self-efficacy nor knowledge retention was correlated to skill proficiency 1 y after the training was received 8 ; this study highlighted the importance of further understanding the implications of first aiders who were either overconfident or underconfident. Confidence is related to the construct of self-efficacy; however, self-efficacy is both the affirmation of one’s ability and the strength of their belief. 7 Comprehensive guidelines and recommendations for developing self-efficacy scales are available, and adhering to these recommendations would lead to a diverse array of items that reflect an effective self-efficacy scale. 9 The remote first aid self-efficacy scale (RFA SES) was developed according to these guidelines, 9 and the purpose of our study was to better understand the psychometric properties of the scale.
The RFA SES was designed for 2 primary purposes: 1) to evaluate of wilderness first aid and other types of emergency care training designed for remote geographies, communities, and worksites and 2) to facilitate reflection by training participants and providers so that they can self-assess their beliefs, confidence, and capacity to respond. 10 It was initially developed to evaluate community-based emergency care (CBEC) training programs. CBEC programs are designed for remote indigenous communities in Canada in which lay providers are trained in culturally appropriate first aid tools and techniques,4,11 The CBEC system aimed to train lay responders to build the skills, knowledge, and confidence to respond and help during an emergency in their community.11,12
Additionally, development of the RFA SES followed relevant guidelines for developing health, social, and behavioral scales, 13 which recommended 3 phases for scale development: item development, scale development, and scale evaluation. 13 The first 2 phases led to the development of the RFA SES, 10 and the final phase was the focus of our study to assess the psychometric properties of the scale in terms of dimensionality, reliability, and validity. 13 Dimensionality refers to the underlying relationships of items and how many dimensions or subfactors may exist in a scale. 13 Reliability refers to the internal consistency of items and how consistent responses are when repeated. 13 Validity refers to the degree that the scale measures the latent dimension that was intended (ie, remote first aid self-efficacy). 13
The purpose of this study was to answer the following questions: 1) What are the psychometric properties of the RFA SES? and 2) Is the RFA SES a valid and reliable tool for use in remote first aid training contexts?
Methods
This study was designed to assess the dimensionality, reliability, and validity of the RFA SES. The instrument used in this study was an online questionnaire that included informed consent, demographic questions, the 30-item RFA SES, the 10-item generalized self-efficacy scale (GSES), 14 and the 10-item Connor-Davidson Resilience Scale (CD-RISC). 15 For concurrent validity analysis, the GSES and CD-RISC were selected because they contain relevant construct similarities to the RFA SES. The CD-RISC measures individual resilience, and the GSES measures generalized self-efficacy, and each of these constructs was expected to be positively and moderately correlated with the RFA SES.
The development of the 30-item RFA SES occurred over several iterations, with expertise and feedback from emergency physicians, paramedics, wilderness first aid instructors/students, and health-based researchers with expertise in rural and remote contexts. 10 Each item of the RFA SES has a 100-point response range using an analog scale starting with “cannot do at all,” “moderately certain can do,” and “highly certain can do.” 10 Thus, higher scores reflect greater self-efficacy in performing first aid in remote contexts. See online Appendix A for a copy of the 30-item RFA SES.
Two different populations were invited to participate in the validation study: trained participants, currently holding a relevant training certification in wilderness emergency response, and a convenience sample of participants, mostly untrained in remote first aid. Convenience and purposive sampling techniques were used to obtain relevant samples from each population so that there was both a more relevantly trained and a less trained group. 16 This research was approved by the Laurentian University research ethics board (#6020617). Informed consent was obtained from all participants. The trained group comprised alumni from Wilderness Medical Associates (WMA) training programs in the United States and Canada, and a total of 39,348 were invited to participate. Participants in the less-trained group were current students enrolled at Laurentian University (LU) in Sudbury, Ontario, Canada. All 9091 active students at LU were invited to participate. The LU population was a convenience sample, and thus some participants may have had some level of first aid training. 16 Based on scale development recommendations, 13 the target sample size was a minimum of 200 responses from each of the 2 respective groups.
Data collection involved email distribution of a link to an online questionnaire, which was hosted on the Laurentian RedCap server. 17 WMA and LU administrators sent out the email as advised by the research team using their respective database of emails. The survey was available for 2 wk, and no reminder or incentive was offered to the WMA group at the request of management. The LU group received a reminder 1 wk before data collection was closed, and participants were offered a chance to win a $100 Amazon gift card if they participated. Data were collected at 2 different time periods 1 mo apart (T1 and T2) to facilitate test-retest reliability analysis.18,19 To ensure that participants’ anonymity was respected, a unique identifier was created by each participant responding to 3 personal questions, and this allowed our team to match the participant responses at T1 and T2.
Data analysis consisted of three steps: 1) assessment of dimensionality, 2) assessment of reliability, and 3) assessment of validity.7,8 Horn’s parallel analysis was used to assess the dimensionality of the 30-item RFA SES13,20 because it corrects for chance when using principle component analysis. 17 For reliability, the first step involved assessing the internal consistency reliability of scale items. 13 Then, test-retest reliability was assessed using intraclass correlations (ICC) between T1 and T2 to determine the stability of mean scale scores.13,21 Test-retest reliability measures the degree to which scale scores are stable and repeatable over time. 13 Before test-retest reliability analysis, participants who practiced first aid or completed a training course between T1 and T2 were removed from the analysis because a change in mean score could be expected if skills were learned or used in the intervening time period. Concurrent validity was analyzed using correlations between the mean scale scores of the RFA SES and both the GSES and CD-RISC. This analysis was used to assess the strength and direction of the relationship between the scores. Finally, mean scale scores for the LU and WMA groups were compared using an independent sample t test because it was expected that the WMA (more trained) group would have higher mean scores than the LU (less trained) group.
Results
In total, 1554 responses were received at T1; 1106 were from LU students, representing a 12% response rate, and the remaining 448 were from WMA alumni, representing a 1% response rate. Participants from T1 were invited to complete the scale again 1 mo later; 47% of the original 1554 respondents completed the RFA SES at T2. The sample characteristics are displayed in Table 1, and the standardized mean scale scores are displayed in Table 2.
Description of participants
Participants at T1 were respondents to the first time the online questionnaire was administered. All other descriptive statistics (sex, mean age, type of training) are related to the participants at T1.
Participants at T2 are a subset of the respondents at T1 who completed the questionnaire a second time approximately 1 mo later.
Urban first aid refers to standard first aid training in an urban setting with direct access to emergency medical services.
Remote first aid training refers to nonurban first aid training and includes wilderness first aid, wilderness advanced first aid, wilderness first responder, wilderness emergency medical technician, and wilderness emergency medical services courses.
Mean scale scores at T1 and T2 for the RFA SES, CD-RISC, and GSES
CD-RISC, Connor-Davidson resilience scale; GSES, generalized self-efficacy scale; LU, Laurentian University; RFA SES, remote first aid self-efficacy scale; WMA, Wilderness Medical Associates.
The 2 groups combined showed a clear unidimensional structure with no subscales or factors, with the first factor having an eigenvalue of 18.10. This structure was similar for both the LU group, with a first eigenvalue of 18.10, and the WMA group, with a first eigenvalue of 17.78. The eigenvalue for a second dimension was 1.17 for both groups combined, and for a third dimension, it was 0.89.
The interitem correlations (IICs) for the RFA SES were moderate to high; the mean IIC was 0.75, with a range of 0.53 to 0.84. Test-retest reliability of the RFA SES was assessed using an ICC of mean scale scores at T1 and T2. The ICCs were high for both the LU group (ICC=0.90) and the WMA group (ICC=0.92). There was a statistically significant increase in mean scale scores from T1 to T2 for the LU and WMA groups combined; however, it was only 1.4 points (n=571, P<0.001), which was not operationally meaningful based on the size of the scale and population. This change was driven by the LU population (P≤0.01) and not by the WMA population (P=0.30). An observable increase in scale scores also occurred for both the CD-RISC and GSES, although it was not statistically significant (CD-RISC, P=0.61; GSES, P=0.83). Scale reliability results for the RFA SES are summarized in Table 4.
Correlations between RFA SES, CD-RISC, and GSES (n=1554) a
CD-RISC, Connor-Davidson resilience scale; GSES, generalized self-efficacy scale; LU, Laurentian University; RFA SES, remote first aid self-efficacy scale; WMA, Wilderness Medical Associates.
All correlations were statistically significant (P<0.01)
To investigate the concurrent validity of the RFA SES, total scores were correlated with the CD-RISC and GSES at T1. The RFA SES was positively and moderately correlated with both scales, as indicated in Table 3. 22 The correlations were slightly higher for the WMA group. Mean RFA SES scores are presented in Table 2, and at T1, they were 18.1 points higher for the WMA participants than for the LU participants (t[569]=16.2, P<0.001).
Discussion
Dimensionality results indicated that the RFA SES was a unidimensional structure. In some ways, this was surprising because the scale items were very diverse, and yet, they could also be grouped in categories such as diagnosis and treatment. Nonetheless, the unidimensional structure does simplify scale utilization because there is no need to analyze or consider subscale scores. However, there is potential that in different populations, the RFA SES could demonstrate a multidimensional structure that is significant.
Reliability tests showed a high IIC (0.75), which suggests that there was internal reliability consistency and that all items were measuring the same construct; however, it also suggests that there may be item redundancy. 13 When items possess an IIC of 0.40 or higher, they are considered a less specific construct measure, and this is a potential indicator of redundancy. 23 This finding was not surprising for 2 reasons. First, the scale was created with more items before any psychometric testing because it was deemed easier to eliminate items than add items. Second, one of the reasons for the development of the scale was for it to be used as a participant reflection tool. It was anticipated that more items from diverse content and perspectives would stimulate more nuanced reflection about self-efficacy.
Another important indicator of scale reliability is understanding how it performs under similar circumstances and conditions with minimal variability. 24 Ideally, given that there was no planned intervention between T1 and T2 when participants completed the scale, it was expected that responses would remain relatively consistent over time. However, it was possible that other intervening factors between the periods could have influenced a participant’s responses. To control for this, 2 questions were asked during the second data collection: 1) Did you practice first aid during the 1-mo intervening period, and 2) Did you complete any first aid training during the 1-mo intervening period. Approximately 21% (n=156) of respondents at T2 indicated that they had either received first aid training or applied first aid during the intervening time period; these data were excluded. There was a slight positive drift in mean RFA SES scores (1.4 points) between T1 and T2 in both groups combined (LU and WMA). However, there was no statistically significant difference in mean scores in the more relevantly trained group (WMA).
Interestingly, although no inferential testing was completed, Table 2 shows a slight positive drift between T1 and T2 for the untrained group (LU) for both the CD-RISC and the GSES mean scores. This is believed to be caused by the relatively high scores of LU students for 2 reasons: 1) psychological factors related to lower age and 2) lack of relevant training and understanding of remote first aid training and contexts. The mean age of the LU and WMA groups was 27 y and 40 y, respectively. Thus, it appears that the younger LU group of university students seemed to have increased confidence over time. Two similar psychological constructs may explain this increase: overestimation and overplacement. 25 Overestimation is the construct of believing that you are better than you are, and, similarly, overplacement is the belief that you are better than others. 25 In addition, there is substantive evidence suggesting that overconfidence is pervasive in university students. 25 -27
The possible variability in confidence in the LU student group between T1 and T2 could have been based on a lack of, or inaccurate, beliefs about knowledge, skills, and experience related to RFA self-efficacy. For example, a recent study suggested that people with insufficient abilities, who lack knowledge of the material and awareness of their skills, are generally more overconfident. 26 However, it should be noted that 70% of the LU population was trained in some level of urban first aid; thus, the group was not untrained in first aid but just less trained in remote first aid.
Validity was assessed using a correlation matrix and an independent sample t test (2-way). The correlation analysis was used to assess the concurrent validity of the RFA SES by comparing the RFA SES scores with 2 other similar scale scores. As expected, all correlations between the RFA SES and the 2 other scale scores were positive, moderate, and ranged from 0.37 to 0.58, regardless of the group (LU, WMA, and LU and WMA combined). This supports the contention that the RFA SES is a construct-specific measure similar to but not overlapping strongly with the more generalized constructs of resilience and generalized self-efficacy. A correlation of 0.50 to 1.0 indicates a strong correlation between the measures; however, if the correlation is closer to 0.5, it indicates an association between constructs that are not identical. 28 Interestingly, the correlations between the CD-RISC and the GSES were higher, ranging between 0.68 and 0.70 across the 3 groups (LU, WMA, and LU and WMA combined). These higher correlations could be explained by the nature of the CD-RISC and GSES as generalized measures of similar constructs (resilience and self-efficacy) rather than the more specific construct measured by the RFA SES.
As expected, the trained WMA group had substantively higher mean RFA SES scores than the less trained LU group. The independent sample t test (2-tailed) comparing the scores at T1 confirmed that the difference was statistically significant. Additionally, the WMA group’s higher scores were observed at both T1 and T2 compared with those of the LU group. Comparing the scores in these 2 groups functioned as a form of sensitivity analysis because it was hypothesized that remote first aid self-efficacy would be higher in a more relevantly trained group (WMA). Thus, the magnitude of the difference now provides a benchmark for what may be expected in future studies. However, the 2 groups were different populations, and as noted earlier, the LU student group was not completely untrained and may have been overconfident in their responses, which could suggest that the 18.1-point difference in scores may be an underestimation of the difference.
Interestingly, as indicated in Table 2, there were slightly higher scale scores for the WMA group at T1 and T2 for both the CD-RISC and GSES; however, the difference was not found to be significant. Nonetheless, there is some evidence that this could be due to a higher maturity level in the WMA group due to the higher age. 30 In a North American population, the mean CD-RISC score is 80.2, 29 and the mean GSES score is 72.5. 14 The CD-RISC mean scores in our study were comparable, ranging from 78.9 to 83.2 (see Table 2). However, the GSES scores were noticeably lower, ranging from 64.9 to 67.8. The lower GSES scores could be explained by the young population.
Scale test-retest reliability results for LU and WMA participants (n=571) a
LU, Laurentian University; RFA SES, remote first aid self-efficacy scale; WMA, Wilderness Medical Associates.
The number of participants (n=571) is less than at T2 (n=727) because 21% of the respondents were removed for the test-retest analysis if they received training or practiced first aid between T1 to T2.
Limitations
There were several limitations to our study. First, the LU population was a convenience sample selected because it was assumed that they would likely have a lower level of first aid training and even less likely have wilderness first aid training. However, >60% of the participants had first aid training, and 9% were trained in wilderness first aid. This means that when comparing the WMA group to the LU group, it was a comparison between a more relevantly trained (WMA) and less relevantly trained group (LU students). A second limitation was that the RFA SES was not assessed as an evaluation or reflection tool. Third, there was also a possibility of selection bias in the 2 samples (LU and WMA), especially due to the low response rate, especially in the WMA group (1%).
Conclusions
The analysis from our study suggests that the 30-item RFA SES is unidimensional, reliable, and concurrently valid. The scale’s IIC was high, suggesting that all items measured the same construct. Future research should focus on developing and validating a short-form version. Test-retest reliability confirmed the stability of the scale over a 1-mo period. Concurrent validity of the scale revealed positive and moderate correlations when measuring related but different constructs: a general scale of self-efficacy and a general scale of resilience. Finally, there were substantively higher scores in a more highly trained group (in wilderness first aid) than a less trained group, suggesting that the RFA SES can differentiate between relevant first aid training levels. Future studies should explore additional reliability, validation, and utility within the context of different types of wilderness first aid training courses, and a time series design would be appropriate to identify changes in remote first aid self-efficacy before and after a training course and then into the future at regular periods. 8
Footnotes
Acknowledgements
Acknowledgments: The authors thank Wilderness Medical Associates and Laurentian University for providing participants for this study. Additionally, our study was supported by colleagues at the Centre for Research in Occupational Safety and Health and the Centre for Rural and Northern Health.
Author Contributions: study concept and design (JD, SR, BO, JL); data acquisition (JD, SR, BO, JL); data analysis (BO, JD, SR); drafting and critical revision of the manuscript (all authors); approval of final manuscript (all authors).
Financial/Material Support: This study was made possible through financial support from the Center for Research in Occupational Safety and Health through a Seed Grant.
Disclosures: None.
Supplementary data
Supplementary material associated with this article can be found in the online version at
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
