Abstract
Background:
The most important functional factor of hospitals is having a preparedness plan for dealing with disasters and emergencies such as road traffic injuries. There is a gap in the design and the development of a valid and reliable tool to evaluate the levels of hospital preparedness during road traffic injuries with mass casualty.
Objectives:
The aim of this study was to explore a valid and reliable tool for measurement of hospital preparedness in road traffic injuries with mass casualties.
Methods:
This study was conducted in two phases, each comprising different steps: tool design and judgmental evidence. In the first step, the determination of the content domain, sampling from content (item generation), and the formation of the tool and in the second stage, validity and reliability of the tool were performed.
Results:
From the 139 items that were finally set, nine dimensions including command and control, infrastructure and medical equipment, information and communication systems, surge capacity, triage and medical services, safety and security, human resources management, coordination and cooperation, and training and exercise were identified. Content validity index and content validity ratio of the tool were 0.97 and 0.98, respectively. The reliability of the tool was 0.89 with the kappa coefficient, respectively.
Conclusion:
The tool has sufficient reliability and validity for measuring hospital preparedness in road traffic injuries with mass casualties. Thus, this tool can be used for assessing the preparedness of hospitals for better planning, preparedness, and response to road traffic injuries with mass casualties.
Introduction
Approximately 1.35 million people die each year as a result of road traffic injuries (RTIs) and 93% of which occur in low- and middle-income countries. RTIs are the main leading cause of death for children and young adults aged 5–29 years. 1 Between 20 and 50 million more people suffer non-fatal injuries, and many of them suffer disability due to their injuries. 1 RTIs are responsible for the largest proportion of unintentional injuries and mortalities in the world. 2 RTIs cause significant economic losses for individuals, families, and nations. These losses are due to the cost of treatment as well as the lost productivity for those who have been killed or disabled as a result of the injuries. 3 Regarding the considerable financial consequences of RTIs, the World Health Organization (WHO) reported that injuries and mortality caused by RTIs cost 5% of the gross domestic product (GDP) in low- and middle-income countries. 4 Without increased efforts and activities, RTIs are predicted to become the seventh leading cause of death by 2030. 5
Healthcare facilities preparedness is critical to reduce the risk and negative impact of disasters and emergencies. Hospitals should provide effective and continuous medical services during disasters and emergencies. 6 At the time of RTIs with mass casualties, hospitals are the first place where injured people are taken to, and thus, hospitals will be overcrowded and incapable. 7 Therefore, hospitals should be prepared for providing healthcare services in RTIs through context-based planning.7,8
According to the Hospital Safety Index developed by the WHO, the most important functional factors of hospitals are having a preparedness plan for dealing with emergencies and disasters. It is necessary that hospitals be ready before the emergencies and disasters from all aspects. 3 Evaluation of hospital preparedness can lead to identifying the current preparedness gaps which may limit the hospital response at the time of RTIs.9,10 Various dimensions, including structural, non-structural, and functional preparedness; personnel preparedness; and disaster plan implementation, have been considered for evaluating hospital preparedness for emergencies and disasters.11–13 Although there is an all-hazard approach to disaster and emergency planning, some incidents are different and need more attention than others; furthermore, having a specific contingency plan is essential. There is a gap in designing and developing a valid and reliable tool to evaluate the levels of hospital preparedness during RTIs with mass casualties. Filling this gap, the present study aimed to design and validate a tool for evaluating hospital preparedness in RTIs with mass casualties.
Methods
This was a mixed method study which was conducted from October 2017 to September 2018 in two stages of tool design as well as validity and reliability measurements.
Stage 1: tool design
In this stage, domains, sub-domain, and appropriate items were identified and generated using literature review and panels of experts. Accordingly, relevant studies with the subject of hospital preparedness in emergencies and disasters were searched through the Scopus, Web of Science, PubMed, ScienceDirect, Google Scholar, MEDLIB, IranDoc, Magiran, and SID databases. Searching for the articles was based on several key terms, including hospital preparedness, hospital readiness, disaster preparedness, mass casualty incidents, Road Traffic Injury, and Road Traffic Accidents, which were searched individually or in combination, using either OR/AND. In this stage, relevant studies in the domain of hospital preparedness in emergencies and disasters and especially in the road safety field were examined in relevant data sources, and all validated evaluation tools for hospital preparedness in emergencies and its dimensions were collected and examined. Panels of experts were another source of data collection for tool design. Four expert panels were held with the participation of eight experts who had knowledge and experience in the fields of health in disasters and emergencies. The data extracted from the literature review and panels of experts were used to design the first draft of the tool.
Stage 2: tool validity and reliability
This stage required the confirmation of a group of experts in order to show the validity of items and the entire tool. In this stage, the tool was examined by experts and items were developed to assess the variable of interest.
First, the tool was sent to 16 experts via email and in person. In the end, 14 experts judged the tool. The tool was examined by a group of three experts on emergency medicine, five faculty members in health in emergencies and disasters, three head nurses of the emergency department, and three experts on hospital disaster risk management.
Face validity
To examine face validity, the item impact score test was utilized. The 10 experts were first asked to rate the importance of each item on a 5-point Likert-type scale from “Totally important” (5), “Important” (4), “Moderately important” (3), “Slightly important” (2), and “Not at all important” (1). Next, the impact scores were calculated using the following formula
The impact score of an item must not fall below 1.5 for it to have face validity. Therefore, only items with an impact score of above 1.5 were considered as having an acceptable level of face validity.
Content validity
The content validity of the tool was examined based on Lawshe’s 14 content validity ratio (CVR) and to examine content validity index (CVI), Waltz and Bausell’s 15 CVI. To determine the CVR, 14 experts were asked to rate the items on a scale of “It is essential,” “It is useful but not essential,” or “It is not essential.” Then, based on the following formula, CVR was calculated: CVR = (Ne − N/2)/(N/2), in which Ne is the number of panelists indicating “essential” and N is the total number of panelists. Based on Lawshe’s table, in order to determine the minimum value of CVR, items with a CVR of above 0.51 were considered significant. 16
To determine the CVI, experts expressed the relevance, clarity, and simplicity of each item on a 4-point Likert-type scale. Experts indicated the relevance of each item using 1 (“Not relevant”), 2 (“Somewhat relevant”), 3 (“Quite relevant”), or 4 (“Highly relevant”). Similarly, simplicity was rated 1 (“Not simple”), 2 (“Relatively simple”), 3 (“Simple”), or 4 (“Completely simple”), and clarity was rated 1 (“Not clear”), 2 (“Relatively clear”), 3 (“Clear”), or 4 (“Completely clear”). Then, based on the following formula, CVI was calculated: CVI = (Number of judges who rated 3 or 4/Total number of judges). The minimum acceptable value for CVI is 0.79, and if the CVI of an item was below this value, it would be eliminated.16,17
Tool reliability
To measure the reliability of the tool, the kappa coefficient was used. The kappa coefficient assesses the agreement between scores of observers, referees, or raters on a topic. It is used when a tool was filled by two or more interviewers on similar populations, and a general consensus regarding the tool is to be achieved. 18 To assess inter-rater reliability, the tool was given to 10 independent experts. These experts were different from experts who participated in previous face and content validity. Experts administered the tool independently at different times by referring to one pilot hospital. The evaluators were blinded to the results of others, and they were not allowed to discuss the case with each other and were not allowed to exchange information.
Results
Tool design
In the first stage, based on the literature review and expert panels, 74 items were extracted. After deciding upon the items, the tool was developed in three sections: (1) general characteristics of the hospital; (2) various items on nine dimensions (command and control, infrastructure and medical equipment, information and communication systems, surge capacity, triage and medical services, safety and security, human resources management, coordination and cooperation, and training and exercise); and (3) a checklist of specific resources and equipment of trauma patient management in six dimensions (airway management, circulation, head, neck, and spinal cord trauma, chest and abdomen, limbs trauma, and burns and wounds). In the item generation step, 122 items were generated and 74 items were obtained from the review of sources and similar tools were combined. After reviewing the items, repetitious and overlapping items were removed. Finally, the preliminary tool had 174 items on nine dimensions of hospital preparedness as well as a checklist of specific resources and equipment of trauma patient management on six dimensions.
Face and content validity
To determine face validity, the tool was given to eight experts familiar with the topic and experienced in the field, including two experts on emergency medicine, one head nurse of the emergency department, one expert on hospital disaster risk management, and four experts on health in emergencies and disasters. They were asked to judge the simplicity, importance, and clarity of the items. Based on their opinion, some items were adjusted in terms of simplicity, clarity, and importance. In addition, the impact score of the items was calculated. The overall mean total impact score was 4.6. The results showed that six items have an impact score of below 1.5 and were, therefore, eliminated.
To determine content validity in the first round, 20 of the 168 items were eliminated. These items had a CVR of below 0.51 and were eliminated based on Lawshe’s table. Moreover, some eliminated items were combined with the remaining ones and edited based on the opinions of experts. Based on the opinions of experts in the first round, 148 of the remaining items were adjusted and were assessed to determine CVI. In this round, eight items with a CVI of below 0.70 were eliminated. Also, five items with a CVI of 0.70–0.79 were adjusted and edited based on expert opinion. After adjustment, the tool with 140 items was emailed to the panel members to determine validity in the third round. In the third round, one item with a CVI of below 0.70 was eliminated, and in the end, 139 items remained in the tool. Finally, Total CVR for the whole tool was 0.97.and also total CVI (average of CVIs of all items) for the whole tool was 0.98. Eventually, after assessing the face and content validity of the tool, 139 items including 78 items of the original tool and 61 items of the checklist of the specific resources and equipment of trauma patient management remained. (Tables 1 and 2).
Values of CVR, CVI, and impact score of items in the checklist.
CVR: content validity ratio; CVI: content validity index; RTIs: road traffic injuries; ICU: intensive care unit; ATLS: Advanced Trauma Life Support; ICS: incident command system; MOA: memorandum of agreement.
Values of CVR, CVI, and impact score of items in the checklist of the specific resources and equipment of trauma patient management.
CVR: content validity ratio; CVI: content validity index; CT: computed tomography; MRI: magnetic resonance imaging; ABG: arterial blood gas.
In the final checklist for each item, a 3-point Likert-type scale was used (no = 0, somewhat = 1, yes = 2). In order to measure the hospital’s preparedness, three levels of good (with a preparedness of 66%–100%), moderate (with a preparedness of 34%–65%), and poor (with a mean of 0%–33%) were considered.
Reliability results of tool
Reliability of the tool was measured with percentage agreement and estimation of Fleiss kappa coefficient for all dimensions and total items. The estimated kappa coefficient was 0.89, which was classified as the strong level of agreement. 18 Fleiss kappa coefficient of nine dimensions and total checklist are shown in Table 3.
Fleiss kappa coefficient of nine dimensions and total checklist.
Discussion
The development and validation of the tool for evaluation of hospital preparedness in RTIs with mass casualty was an important finding of the present study. Developing such a valid and reliable tool was conducted for the first time in the present research. Various tools have been designed for evaluating hospital preparedness in disasters and emergencies. Studies on hospital preparedness are generally focused on the use of the tool proposed by the WHO which is based on the “all hazards” approach and assesses the safety of hospitals in structural, non-structural, and functional dimensions. One limitation of this tool is the lack of measuring psychometric measures (reliability and validity criteria) for the tool.19,20 Another weak point is its inability to measure all structural, non-structural, and functional dimensions of hospital preparedness in disasters and emergencies. 12 Studies have mostly focused on the structural and non-structural dimensions, neglecting functional ones. 21 The present tool has focused on all three dimensions of preparedness and its subdimensions. This tool was designed based on nine dimensions of hospital preparedness, including command and control, infrastructure and medical equipment, information and communication systems, surge capacity, triage and medical services, safety and security, human resources management, coordination and cooperation, and training and exercise. The extracted dimensions are more expansive and precise than similar tools for evaluating hospital preparedness in RTIs with mass casualties.3,12,22 Also, the tool included the part of specialized resources and equipment for managing trauma patients in RTIs with mass casualties, which was ignored in previous tools. The presence of these resources and equipment in hospitals is essential for managing trauma patients and affects hospital preparedness for managing injury cases in RTIs with mass casualties.
In the present tool, items such as designing, reviewing, and notifying the response plan for RTIs; attainment of financial resources; and appropriate organizing by Activation of the incident command system (ICS) are also included. Considering the importance of intra- and inter-organizational coordination and collaboration, necessity of focusing on information and communication systems and promoting cooperation with supportive units were supposed as the other criteria for evaluating hospital preparedness in RTIs.
Limitations
The current tool does not take into account all aspects of hospital preparedness, for example, the hospital’s preparedness for structural safety has not been evaluated, but with expert advice, all aspects of preparedness associated with road traffic accidents have been extracted.
Conclusion
Findings showed that the tool has sufficient reliability and validity for evaluating hospital preparedness in RTIs with mass casualties. Assessing hospital preparedness can inform policymakers, planners, managers, and hospital staff about their current preparedness status. In addition, hospital preparedness assessment can lead to the identification of weaknesses which may be covered by improving hospital capacities for effective response at the time of emergencies and disasters. The provision of specialized and efficient human resources, stress management strategies, continuous education, and exercises for promoting the preparedness of staff can be considered as important factors for successful performance at the time of emergencies and disasters. The existence of a standard triage system and relevant guidelines, establishment of infrastructures and medical equipment and tools required in RTIs, familiarity with and use of various techniques for enhancing capacity, and effective assessment and evaluation of resources and facilities can be assumed as other important factors for hospital preparedness.
Footnotes
Acknowledgements
We would like to thank all experts for their assistance and guidance in this research.
Author contributions
All authors contributed equally to the manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Availability of data
The data sets generated during the current study are available from the corresponding author
