Abstract
Introduction
Alcohol is the cause of many visits to hospital and emergency departments (ED) (Bakke et al., 2016; Blow et al., 2010; Xx., 2021). Caring for alcohol-intoxicated patients requires different types of skills from nurses who work in this context (Clarke et al., 2015, Xx., 2021; Hellum et al., 2016; Warren et al., 2012). When it comes to this patient group, EDs around the world deal with the same kind of situations. The somatic symptoms of ED patients are usually managed, whereas dealing with alcohol use disorder is often missing. When a patient is alcohol-intoxicated, it is not known whether the person is a user, abuser or alcohol dependent before they are evaluated in some form of intervention by health care professional
Different kinds of nursing skills or nursing staff's competencies have been assessed in various care areas by creating specific instruments. Measuring these skills is a way of determining the level of preparedness and competence and assessing whether training or further education is needed (Araújo et al., 2016; Bing-Jonsson et al., 2015; Lin et al., 2009; Meretoja et al., 2004). However, no instrument measuring the skills to care for acutely ill alcohol-intoxicated patients in ED settings was found to be available for use. That is why development and validation of an instrument of this kind is important. It also corresponds to the aims of the World Health Organization (WHO) as one objective of the WHO's alcohol strategy is to increase awareness of the complexity of alcohol-related problems and promote the development of effective interventions to reduce and prevent problems exacerbated by alcohol use (World Health Organization, 2022). The development and validation of the Nurses’ Skills to Care for Alcohol-Intoxicated Patients in Emergency Department (NSCAIP-ED) instrument was initiated to meet this need.
The aim of the study was to report the development and validation of the NSCAIP-ED instrument.
Methods
A mixed-methods design was used in the development process of the NSCAIP-ED. It included both deductive and inductive research methods, such as email surveys, group interviews, an integrative literature review and personal interviews. After a pilot study of the instrument, a descriptive quantitative survey was conducted with the instrument to measure the self-evaluated skills of Finnish ED nursing staff concerning the care of acutely ill alcohol-intoxicated patients in the year 2016. The survey was sent to all Finnish central hospitals and university hospitals EDs. The target group was nursing staff (N = 1220) in the EDs. A total of 252 individuals responded. The descriptive data of this survey have been reported in a separate article (Mäkelä et al., 2020).
For the instrument validation, a psychometric analysis was executed. The evaluation of content validity was based on the views of substance specialised experts in the research group considering instruments’ scales and items. Usability of the NSCAIP-ED was assessed based on earlier research (Mäkelä et al., 2020). Construct validity was evaluated using factor analysis. Cronbach's alpha values were calculated when evaluating internal consistency.
Instrument development
Stages of the instrument development were as follows. First, three Delphi rounds were arranged to obtain professionals’ opinions. Round 1 was an email survey (n = 6) and round 2 was group interviews (n = 6). Five nurses and a social worker working in an ED participated in the group interviews. Round 3 was a group interview of the expert panel, which included two nurses and a nursing manager of one ED, a team leader of sobering unit, a project designer of the special skills project and a nursing director of development and evaluation. The aim of the second phase was to conduct an integrative literature review (Hakala et al., 2017). Then, six personal interviews were performed with nurses working in the ED. After every phase, the findings were processed and the instrument was generated based on them. The instrument consisted of seven scales and 49 items. The scales and items were based on competence requirements that were defined at the beginning of the instrument development with the expert panel, and they were also commented on by physician specialising in substance abuse care. Finally, the research group that was established for the instrument’s content evaluation agreed that all the items were adequate to the instrument. This research group consisted of a substance abuse treatment specialised nurse and physician, a nursing manager of the ED and a director of the research group who represented scientific nursing research competence (Figure 1).
Development of the NSCAIP-ED instrument.
The NSCAIP-ED instrument contained seven scales: (1) alcohol risk use evaluation skills (11 items); (2) interaction skills (6 items); (3) ethical skills (7 items); (4) counselling and education skills (7 items); (5) security and safety skills (9 items); (6) teamwork and network skills (5 items); and (7) follow-up treatment skills (4 items). The instrument consisted of a total of 49 5-point Likert-type items (Table 1). The participants self-evaluated their skills using the following scale: 1 = extremely poor; 2 = reasonably poor; 3 = reasonably good; 4 = good; and 5 = extremely good. The background factors included in the instrument were nursing staff's organisation, position, work experience in healthcare, emergency care, psychiatric nursing and intoxicant care, and participation in alcohol use-related training and substance abuse-related training (Hakala et al., 2020; Mäkelä et al., 2020).
The Nurses’ Skills to Care for Alcohol-Intoxicated Patients in Emergency Department survey.
Note. Values are given as n (%) unless otherwise indicated.
Cronbach’s alpha values have been published before (Mäkelä et al., 2020). bThe lower and upper quartiles (Q1–Q3).
*The article has been updated to reflect the changes.
ED nurses (n = 11) working in a district hospital that did not belong to the target group of the final survey participated in a pilot study. The pilot study was conducted to ensure the feasibility of the instrument. Pre-testing the questions helps to ensure that the statements are meaningful when answered by the target population before the development of the instrument is completed. The purpose of pre-testing is to remove incorrect phrasing and inconsistencies from the instrument and thus make it as comprehensible as possible (Boateng et al., 2018). In this phase, feedback was collected to find out whether the participants understood the statements in the same way and find out their thoughts around answering the questionnaire. The participants suggested a few improvements, but they mainly felt that the NSCAIP-ED was easy to use: it took a reasonable 20 min to complete and the layout was suitable. No items were removed and no new ones were added after the pre-testing phase; a few corrections were made to the structures of the existing scales based on the feedback received (Hakala et al., 2020). After the pilot study, the final survey with the NSCAIP-ED instrument was conducted (Mäkelä et al., 2020).
Sample/participants
In this validation study, we used the data collected with the NSCAIP-ED instrument in 2016. It was a cross-sectional survey that was conducted on all Finnish ED nursing staff who worked in a central hospital or university hospital (N = 1220, n = 252). These hospitals were chosen as the target group because of the similarity of their EDs. The target organisations covered the whole of Finland geographically. The aim of that study was to describe nursing staff's self-assessed skills concerning the care of acutely ill alcohol-intoxicated patients in EDs (Mäkelä et al., 2020). According to “the rule of thumb”, at least five participants would be sufficient for each item on the scale for the statistical results to be reliable (Bryman & Cramer, 2005; Rattray & Jones, 2007). The NSCAIP-ED instrument has 49 items; therefore, the number of participants must be at least 245. This criterion was met in this study because 252 ED nursing staff members responded to the survey.
Instrument validation and data analysis
The validity and reliability of the NSCAIP-ED instrument were examined as comprehensively as possible and are reported in this article. The purpose of the validity examination was to find out whether the instrument measured what it was intended to measure (De Vet et al., 2011). Reliability means stability, where the instrument always reliably measures the same thing (DeVellis, 2017). The examination of content validity began with the observation of face validity; then, construct validity was clarified using confirmatory factor analysis (CFA). CFA is a factor analysis that is commonly used when the instrument structure is based on a theoretical framework. With CFA, it is possible to find out how well the data fit this framework (De Vet et al., 2011). The model fit indicators used were the chi-square test, the root mean square error of approximation (RMSEA < 0.08), comparative fit index (CFI > 0.95), Tucker-Lewis index (TLI > 0.90) (Xia & Yang, 2019) and the standardised root mean square residual (SRMR < 0.08). In this study, Cronbach alphas were considered when evaluating internal consistency. Acceptable alpha values are in the range of 0.70–0.90, and values <0.60 are not accepted (DeVellis, 2017).
The data that were collected to assess the NSCAIP-ED instrument's validity were analysed statistically. To describe the data, frequencies, percentages, medians and quartiles were given. CFA and Cronbach's alpha values were used to evaluate the validity and internal consistency of the instrument. The analyses were performed using SPSS version 28 for Windows (IBM, Armonk, NY, USA) and CFA was carried out using Mplus version 8.7 (Muthén & Muthén, Los Angeles, CA, USA).
Ethical considerations
Ethical guidelines and legislations were followed throughout the study (World Medical Association, 2018). The study was approved by the Ethics Committee for Human Sciences of the University of Turku, Finland (Decision 30/2015, 8 September 2015). Permission to conduct the survey was asked accordingly from the target organisations (Mäkelä et al., 2020). Answering the questionnaire was optional for the health professionals who participated in the study, and they were informed about the research by email before they decided to take part. Responding to the survey was considered as consent to participate in the study (Finnish National Board on Research Integrity TENK, 2019). The data were treated in confidence and by using reliable statistical methods. All the stages of instrument development and validation have been reported as comprehensively as possible.
Results
Descriptive statistics
The participants of the NSCAIP-ED survey (N = 1220, n = 252) were nursing professionals who worked in a Finnish ED. The total response rate for the study was 20.7%. Most of them (85.3%) were nurses with a mean working experience more than 10 years (49.2%). Around two-thirds (88.0%) of the participants did not have experience from intoxicant care and over half (59.0%) had not had substance abuse-related training during their career (Mäkelä et al., 2020).
Almost all participants (n = 252) responded to all items of the NSCAIP-ED instrument. The items with the most missing answers were “37. I know how to act in a violent situation” (n = 247) and “33. I know how to use security services” (n = 247), which also had the most “extremely good” answers (n = 159). Medians were in the range of 2.00–5.00. The sections “Ethical skills” and “Security and safety skills” both included three items that had the highest quartiles of the results (Q1–Q3 = 4.00–5.00). The lowest quartiles (Q1–Q3 = 1.00–2.00) and the most “extremely poor” responses were in the sections “Alcohol risk use evaluation skills” and “Follow-up treatment skills” (Table 1).
Evaluation of internal consistency was based on Cronbach's alpha values (Mäkelä et al., 2020). All scales of the NSCAIP-ED instrument had acceptable Cronbach's alpha values (range = 0.866–0.912) (Table 1).
Confirmatory factor analysis
The factor structure was examined using CFA considering 49 items of the instrument divided into seven factors. Model fit indicators showed that this factor model can be applied to the data of this survey with mostly acceptable fit (chi-square test p < 0.001; RMSEA 0.079; CFI 0.923; TLI 0.918; SRMR 0.084).
Discussion
The purpose of the NSCAIP-ED instrument is to measure nursing staff's self-evaluated skills concerning the care of acutely ill alcohol-intoxicated patients in EDs. In line with this purpose it was tested in the ED environment by the nursing staff working there. It has been reported earlier that ED nursing staff's skills to care for alcohol-intoxicated patients may be incomplete (Hakala et al., 2021; Iqbal et al., 2015; Warren et al., 2012). No similar instrument was found that could produce knowledge about the skills needed in caring for alcohol-intoxicated patients, thus pointing out the need for an instrument of this kind. Validation was necessary to establish if the NSCAIP-ED instrument could be commonly used. Based on the results of validation, this instrument measures well what it is meant to measure, and the results also showed good reliability.
Face validity evaluation was based on the opinions of the pilot group and the research group that were established for the instrument’s content evaluation as well as on the results of an earlier study in which the research group stated that NSCAIP-ED instrument is usable and reliable in measuring nursing staff's skills concerning the care mentioned above (Mäkelä et al., 2020). The aim of face validity is to determine the usability of the instrument and the functionality of the layout (De Vet et al., 2011; Udina, 2012). As an indication of the usability of the NSCAIP-ED instrument, earlier use, for example, pointed out that results of instruments’ scales were consistent with the background information of the respondents. Based on the background information, almost half of the participants had more than 10 years of work experience in emergency care, but 88% of the respondents had not worked in intoxicant care. The basic nursing skills, such as ethical skills, were at a better level than alcohol risk use evaluation skills or follow-up treatment skills (Mäkelä et al., 2020). This is relevant because even if there has been work experience as an emergency nurse, there is not necessarily competence on intoxicant care specific skills. These results support the usability of the instrument and show that the content is functional.
The quality of the NSCAIP-ED instrument was investigated by evaluating the implementation of the Delphi method, a literature review and interviews with professionals that were used in the development process of the instrument. Therefore, the opinions and views of various experts were taken into account as fully as possible when developing the instrument. After a pilot study, minor changes to the NSCAIP-ED were made to ensure that the structure of the instrument was feasible, relevant to the context and credible.
The response scale consisted of a Likert-type scale throughout the instrument, which facilitates responding and could reduce erroneous responses. This increased the value of the instrument’s validity. A Likert-type scale has been found to work well, but even when using a scale of this kind the respondent must be clearly informed of what their answer means at any point on the scale, and the same scale should be used throughout the instrument from start to finish (Krosnick & Presser, 2009). When using NSCAIP-ED, the respondent was left with no ambiguity as to what it meant to answer the statements because the answer options were the same from beginning to end (1 = extremely poor, 2 = reasonably poor, 3 = reasonably good, 4 = good and 5 = extremely good).
Responding to the NSCAIP-ED survey was executed electronically, which enabled respondents to reply at the time that was most convenient for them, and it was easy for them to return the answers. The good features of electronic data collection are that the data are often in an immediate format, data collection is inexpensive, participants are easy to reach and respondents prefer to answer questions related to sensitive topics online compared to face-to-face interviews (Boateng et al., 2018).
To verify content validity that is as inclusive as possible, two different scientific methods were used in the instrument development process. A literature review was conducted to build an evidence-based instrument and to define all concepts properly. Interviews with experts were used to ensure that the content and structure of the instrument was adequate and usable. Hence, the approach was both deductive and inductive; such a combination of both approaches is considered clearer and more comprehensive than development processes utilising just one method. Theoretical knowledge can be collected deductively, and practical and contextually relevant information can be collected inductively (Boateng et al., 2018).
Construct validity should be assessed to find out how consistent the results of the instrument are in relation to the phenomenon under study. The results obtained with the instrument should be an accurate description of the subject under investigation (De Vet et al., 2011). Construct validity was tested statistically with the CFA and it mostly showed acceptable fit. These results support the notion that the factor structure of the NSCAIP-ED instrument is the ability to detect ED nursing staff's skills to care for alcohol-intoxicated patients, but it should be tested further with larger sample sizes. The size of the sample participating in a study may affect the results of CFA. Despite this, CFA should be chosen if the instrument is based on theory (Prudon, 2015). The instrument in this study was produced based on scientific literature and professional knowledge. That is why CFA was an undisputed choice as a method to give reliable information about the instrument.
Internal consistency was examined by calculating Cronbach's alpha values. The core idea of Cronbach's alpha test is that instruments are reliable when all their sections are congruent and the variables correlate proportionately with each other (Boateng et al., 2018; DeVellis, 2017). The NSCAIP-ED instrument’s scales had acceptable Cronbach's alpha values, which indicates good internal consistency. The results of this study show that all items were answered evenly, supporting the notion that respondents did not get frustrated with the number of items when answering the questionnaire.
The reliability of the instrument can be improved and the incidence of measurement error reduced by limiting the variation due to the measurement situation; for example, by measuring a certain thing from all participants at the same time in the same way and by making sure that the correct use of the instrument is understood (De Vet et al., 2011). The originally planned sample of respondents was quite large because it was a national survey and covered all Finnish hospital districts. Thus, because all participants could not be contacted individually by the researchers, a cover letter with clear instructions for responding was attached to the survey. This means that all participants received the same instructions. Responding took place over a period of time predetermined by the researchers, so the timing of responses coincided fairly well. The aim was to avoid scenarios such as a change in participants’ responses due to various significant events, seasons or longer-term training, and thus reduce the incidence of measurement error.
The instrument and the survey results were only handled by members of the research group who had the same knowledge and skills to use it. The interpretability of the results means that the quantitative results obtained by the instrument can be converted into a qualitative form that can be generally understood (De Vet et al., 2011; Streiner & Kottner, 2014). Based on the NSCAIP-ED survey that was conducted, all the results could be analysed and discussed in an article (Mäkelä et al., 2020).
Alcohol is a costly societal problem. It is crucial to be able to measure the skills to care for alcohol-intoxicated patients so that ED managers know when the staff needs more training and education (Mäkelä et al., 2022). These skills include different kinds of competencies that are included in the structure of the NSCAIP-ED instrument. ED nursing staff's positive attitude, knowledge and proper use of the skills mentioned above provide better care for patients who are alcohol abusers or at risk of problematic alcohol use (Clarke et al., 2015; Hakala et al., 2021). Patient care where these skills are implemented can be very effective and reduce patients’ alcohol use or even get them to give up alcohol altogether (Koivunen et al., 2017).
Limitations
There are some limitations in our study because the NSCAIP-ED survey was conducted in one country and the number of participants was limited (N = 1220, n = 252). With a larger sample of participants, the data could have been split and different kinds of statistical analyses could have been conducted to verify the construct validity of this instrument. After all, we were able to execute relevant statistical testing and analyses gave mostly reliable results. The NSCAIP-ED survey that was conducted was in Finnish, which limited the use of the instrument at the time; however, translation of the instrument into English for further use was started in this article (Table 1), and the instrument should be tested internationally.
A gold standard that could have been compared to the instrument was not found, so criterion validity could not be clarified. In the future, with test–retest study design, reliability and responsiveness could be tested more comprehensively. The missing gold standard also affected the evaluation of the instrument’s content validity because there was no other instrument to parallel it. However, members of the pilot group and the research group that were established for the instrument’s content evaluation and earlier literature supported the assumption that the NSCAIP-ED instrument’s content validity was good (Mäkelä et al., 2020; Hakala et al., 2021; Mäkelä et al., 2022).
The low participation rate might influence the validation and, in the future, the NSCAIP-ED instrument must be further tested with larger data. This may be due to the length of the survey but there was no mention in the pre-testing phase that the survey would be too long. There is a difference between the measurements of skills and the measurement of self-evaluated skills. The instrument is now based on participants’ self-evaluation but there is a possibility of developing a knowledge test to measure nurses’ skills to care for alcohol-intoxicated patients in the ED. Comparing the results of the knowledge test with the self-evaluation would be interesting and complements the information about the NSCAIP-ED instrument’s validity. This will be taken into account when planning further studies.
Conclusion
The NSCAIP-ED is a feasible and reliable instrument that can be used when measuring nurses’ skills to care for alcohol-intoxicated patients in the ED context. This instrument could be useful for nursing managers in EDs for evaluating their nursing staff's self-evaluated skills in the care area mentioned and for designing continuing education based on the results. It is important to have interventions that can improve skills of this kind. The NSCAIP-ED could also be tested in other nursing environments or with nursing students when there is a need to find out their self-evaluated skills to care for patients influenced by alcohol. This way, necessary training could be offered at the right time and it could raise the standard of care in the long run. Nursing staff's relevant caring skills will offer comprehensive and good quality treatment for patients.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study is funded by the Satakunta Hospital District (Government Research Financing 81902, Decision 57/2016, 82003 Decision 135/2017, 84012 Decision 158/2019, 850 Decision 412/2020) and the Hospital District of Southwest Finland (Government Research Financing 81902, Decisions 7/2015 and 15/2016).
Correction (November 2023):
Table 1 has been corrected in the article.