Abstract
Introduction:
The Full Outline of Unresponsiveness score coma scale is a recently introduced coma scale. The objectives of this study were to assess the interrater reliability of the Full Outline of Unresponsiveness score coma scale when physicians and nurses in the emergency department apply the Full Outline of Unresponsiveness score on patients clinically suspected to have acute stroke and to look for any association between Full Outline of Unresponsiveness score coma scale and in-hospital mortality.
Methods:
Prospective study of 105 patients clinically suspected to have acute stroke recruited in an emergency department in a 4-month period. The Full Outline of Unresponsiveness score coma scale and Glasgow Coma Scale of each patient were assessed by one doctor and one nurse independently. The interrater reliability between physicians and nurses using the Full Outline of Unresponsiveness score and Glasgow Coma Scale score was assessed. The association between the Full Outline of Unresponsiveness score coma scale and in-hospital mortality was analysed using logistic regression, controlled for age, sex and diagnosis.
Results:
Full Outline of Unresponsiveness score had a good interrater reliability when applied to patients suspected to have acute stroke (kappa = 0.742, 95% confidence interval = 0.626–0.858). This was comparable to Glasgow Coma Scale score with a kappa = 0.796 (95% confidence interval = 0.694–0.898). For every 1-point increase in Full Outline of Unresponsiveness score coma scale, a reduction in in-hospital mortality was observed with an odds ratio of 0.76 (95% confidence interval = 0.63–0.91, p = 0.003), controlled for age, sex and diagnosis.
Conclusion:
The Full Outline of Unresponsiveness score may be a tool that can be used by emergency department doctors and nurses in assessing clinical stroke patients.
Introduction
Coma is the alteration in consciousness that results from various pathological conditions affecting cerebral function. The coma scales provide a clinical tool for rapid bedside assessment that aids communication, guides patient management and helps clinical research. 1 The most widely used coma scale at present is the Glasgow Coma Scale (GCS). However, GCS has its limitation, including failure to assess the verbal component in patients who are intubated, aphasic, aphonic or with trauma to vocal apparatus or language barrier.1,2 The GCS has been criticized for mathematical skew and bias towards the motor score. 3 It also does not provide neurological details like brainstem reflexes. 4
Because of these shortcomings, a new coma score, the FOUR (Full Outline of UnResponsiveness) score, was introduced in 2005. 4 It was developed and validated initially in neuro-intensive care unit. The FOUR score comprises four components (eye, motor, brainstem and respiration) (Appendix 1). Each component has a score ranging from 0 to 4. Higher scores represent better conscious status. The total FOUR score ranges from 0 to 16. It was claimed that the new score could overcome the limitations of GCS and provide greater neurological details. By incorporating the components of brainstem reflexes and breathing patterns, the FOUR score is able to recognize various stages of herniation. 4 The absence of verbal component in FOUR score avoids the difficulty in assessing patients with communication problems. By assessing eye blinking and tracking to command, the FOUR score is able to detect the minimal sign of consciousness and differentiate between locked-in syndrome and vegetative state.4,5 Each component of FOUR score has a maximum score of 4. The total score is not skewed to any of the four components.
The FOUR score was shown to have good to excellent interrater reliability. 4 The FOUR score was also shown to predict in-hospital death and poor neurological outcomes. 4 Further studies on FOUR score have also been performed in foreign countries and have been applied to patients with altered conscious level,6–8 traumatic brain injury, 9 stroke 2 and neurological diseases.4,5,10–12 There were a few studies performed in emergency departments (EDs).6,10,12
Since the FOUR score appears to be a good coma scale and was validated in overseas studies, this study aimed at assessing how the FOUR score performs in our locality. The objective of our study was to validate the FOUR score in patients suspected to have acute stroke in an ED. We assessed the interrater reliability when the nurses and doctors applied the FOUR score to these patients. We also tried to investigate whether the FOUR score could predict the in-hospital mortality.
Methods
Study setting and design
This prospective study was conducted in an ED of a district hospital in Hong Kong. The data were collected from 19 August 2013 to 20 December 2013.
Patient inclusion
All adult patients aged 18 or above, who presented with clinical features suggestive of acute stroke and with symptom onset within 1 week, were included. Consecutive sampling was used. The triage nurse enrolled the patients who were eligible for this study at the triage station. All the nurses in the ED had clinical experience in identifying possible stroke patients. Cincinnati Prehospital Stroke Scale that helps to identify possible stroke patients was introduced in the briefing sessions to the nurses before the study.
Data collection
Before starting the study, all physicians and nurses in the ED attended briefing sessions on the FOUR score. Written definitions of both the FOUR score coma scale and the GCS were provided in the study forms (Appendix 1). Illustrative diagrams to demonstrate the FOUR score coma scale were included in the study forms. For each eligible patient, the patient was rated independently by a doctor and a nurse, using both the FOUR score coma scale and the GCS. The ratings by the doctor and nurse were recorded on two separate study forms. The assessment by the doctors and nurses was performed within 1 h of ED arrival.
Outcome measures
There were two outcome measures. The first one was the interrater reliability of FOUR score coma scale and GCS when physicians and nurses applied both scales to patients clinically suspected to have acute stroke in the ED. The second one was to assess whether the FOUR score coma scale correlated and predicted in-hospital mortality. The final diagnosis, in-hospital clinical course and final outcome of each patient were traced using a centralized computer database of the hospital.
Data analysis and statistical analysis
The statistical analysis was performed with the Statistical Package for Social Sciences (IBM SPSS Statistics 19). For both the FOUR score and GCS score, the overall weighted kappa scores were calculated to determine the interrater reliability between the nurses’ rating and doctors’ rating. Weighted kappa (κw) >0.8 is considered as excellent agreement; κw between 0.6 and 0.8 suggests good observer agreement; κw between 0.4 and 0.6 suggests fair to moderate agreement; and κw of 0.4 or less is considered poor. Spearman’s correlation coefficient between the FOUR score and the GCS score was calculated to assess construct validity. Cronbach’s α was calculated for each score to assess internal consistency. We investigated the prediction of in-hospital mortality by the FOUR score, as well as the GCS score, using logistic regression model controlling for age, sex and diagnosis (ischaemic stroke, haemorrhagic stroke or non-stroke case). The accuracy of the FOUR score coma scale and the GCS to predict in-hospital mortality was assessed using the area under the receiver operating characteristic curve (AUROC).
Results
Patient characteristics
During the study period, 117 sets of study forms were distributed and received. A total of 12 sets of study forms had incomplete data to calculate the FOUR score or the GCS score and were excluded. Overall, 105 patients clinically suspected to have acute stroke were included. The average age of patients was 68.3 years (range, 18–96 years). The admission rate was 94.3% (Table 1). Of those 105 patients, 78 patients (74.3%) were confirmed to have acute stroke, of which, 67 patients (63.8%) had ischaemic stroke and 11 patients (10.5%) had haemorrhagic stroke. The final diagnosis of the patients is shown in Table 2. For the distribution of FOUR score and GCS score, the scores were negatively screwed (Figure 1(a) and (b)). In total, 71.4% patients had FOUR score of 16 and 65.2% had GCS score of 15.
Comparisons of different groups of patients.
FOUR: Full Outline of Unresponsiveness; GCS: Glasgow Coma Scale.
The median of FOUR score and the median of GCS score were based on both doctors’ and nurses’ rating.
Interrater reliability between doctors’ rating and nurses’ rating.
FOUR: Full Outline of Unresponsiveness; GCS: Glasgow Coma Scale; CI: confidence interval.
(a) Distribution of FOUR score of the enrolled 105 patients rated by doctors and nurses and (b) distribution of GCS score of the enrolled 105 patients rated by doctors and nurses.
Interrater reliability
Of the 105 patients suspected to have acute stroke, the total FOUR score had a good interrater reliability (κw = 0.742, 95% confidence interval (CI) = 0.626–0.858) and was comparable to that of GCS score (κw = 0.796, 95% CI = 0.694–0.898) (Table 2). The total FOUR score was found to have good interrater reliability on confirmed stroke cases (n = 78, κw = 0.733, 95% CI = 0.600–0.866).
Validity and internal consistency
Spearman’s correlation coefficients between FOUR score and GCS score were 0.871 (p < 0.001) for doctors’ rating and 0.914 (p < 0.001) for nurses’ rating, suggesting significant positive correlation between the FOUR score and the GCS score. Cronbach’s α for the FOUR score rated by doctors was 0.843 and that for the FOUR score rated by nurses was 0.868. Therefore, FOUR score had good internal consistency.
In-hospital mortality prediction by the FOUR score
Of the 105 patients suspected to have acute stroke, 13 patients (12.4%) died during stay in hospital. Of the 13 patients who died, 6 suffered from ischaemic stroke, 3 suffered from haemorrhagic stroke and 4 were non-stroke patients (Table 3).
Diagnosis of the enrolled 105 patients who were clinically suspected to have acute stroke.
When analysing the 105 patients suspected to have acute stroke using logistic regression, for every 1-point increase in total FOUR score, there was an estimated 23% reduction in the odds of the in-hospital mortality (odds ratio (OR) = 0.77, 95% CI = 0.66–0.90, p = 0.001). When statistical adjustment was performed to control for age, sex and diagnosis (ischaemic stroke, haemorrhagic stroke and non-stroke cases), there was an estimated 24% reduction in the odds of the in-hospital mortality (OR = 0.76, 95% CI = 0.63–0.91, p = 0.003) for every 1-point increase in total FOUR score. When analysis was performed on the 78 patients confirmed to have acute stroke, for every 1-point increase in total FOUR score, there was an estimated 10% reduction in the odds of the in-hospital mortality (OR = 0.90, 95% CI = 0.71–1.13, p = 0.352), after adjusting for age, sex and the type of stroke. However, that result was not statistically significant (Table 4).
Prediction of in-hospital mortality by the FOUR score/GCS score using logistic regression model.
FOUR: Full Outline of Unresponsiveness; GCS: Glasgow Coma Scale.
Adjusted for age, sex and diagnosis (ischaemic stroke/haemorrhagic stroke/± non-stroke cases).
The accuracy of FOUR score and GCS score to predict in-hospital mortality was analysed by AUROC. For the 105 patients suspected to have acute stroke, the AUROC for FOUR score rated by doctors was 0.758 (95% CI = 0.599–0.916, p = 0.003), the AUROC for FOUR score rated by nurses was 0.758 (95% CI = 0.598–0.918, p = 0.003), the AUROC for GCS score rated by doctors was 0.744 (95% CI = 0.579–0.910) and the AUROC for GCS score rated by nurses was 0.752 (95% CI = 0.587–0.917) (Figure 2). Therefore, the accuracy of FOUR score to predict in-hospital mortality of patients suspected to have acute stroke in ED was similar to that of GCS score.
The receiver operating characteristic (ROC) curves of FOUR score and GCS score, rated by doctors and nurses. The area under receiver operating characteristic curve (AUROC) reflects the performance of the scoring to predict in-hospital mortality.
Discussion
Our study demonstrated good interrater reliability (κw = 0.742, 95% CI = 0.626–0.858) when ED doctors and nurses applied the FOUR score to patients clinically suspected to have acute stroke. Our study also showed acceptable predictive power of FOUR score to predict in-hospital mortality in that group of patients (AUROC = 0.758 (95% CI = 0.599–0.916, p = 0.003)). For every 1-point increase in total FOUR score, there was an estimated 24% reduction in the odds of the in-hospital mortality (OR = 0.76, 95% CI = 0.63–0.91, p = 0.003), after adjusting for age, sex and diagnosis.
In our study, the doctors and nurses in the ED only received simple briefing sessions about the FOUR score before the study. They did not receive specialized neurological training. However, the result of our study showed good interrater reliability of the FOUR score. Therefore, this score may be used by healthcare workers without specialized neurological training.
Concerning the interrater reliability, we analysed the agreement of FOUR score and GCS score rated by ED doctors and ED nurses. There was blinding between the raters on the rating of FOUR score in our study. The doctors and nurses rated the FOUR score independently on two different sets of study form. However, we failed to ensure blinding between the raters on the rating of GCS score. GCS score was documented on the ED clinical notes shared by doctors and nurses. The doctors could see the GCS score rated by the triage nurses on the clinical notes. This might increase the interrater agreement of the GCS score. A separate study team composed of raters not directly involved in clinical management of the patients will be required in order to ensure adequate blinding of the GCS scores and FOUR scores. This is difficult to achieve with limited resource.
Concerning the in-hospital mortality, we adjusted the results to age, sex and diagnosis (ischaemic stroke, haemorrhagic stroke and non-stroke cases). However, there were many other factors that influenced mortality, for instance, the premorbid status of the patients. During analysis, we regarded the non-stroke cases as a single diagnostic group. However, this group comprised various diagnoses.
We enrolled 105 patients clinically suspected to have acute stroke at triage station in ED. Only 74.3% patients were confirmed to have acute stroke. Therefore, we cannot directly generalize the results to stroke patients. Although the analysis on the suspected stroke cases yielded statistically significant results, subgroup analysis on the confirmed stroke case did not yield statistically significant result on the ability of the FOUR score to predict mortality, this is probably the result of inadequate sample size of confirmed stroke cases. The distribution of FOUR score and GCS score did not have a normal distribution and was negatively skewed. In fact, a large proportion of patients had FOUR score of 16 (71.4%) and GCS score of 15 (65.2%). This would increase the interrater agreement because most validation studies perform worse in patients with marked decline in consciousness. Another consequence of the small number of patients with low GCS/FOUR score was that it became difficult to demonstrate the advantage of FOUR score over GCS score in patients with low GCS. In a systematic review in which pooled analysis of four prospective studies on FOUR score was performed, FOUR score was able to further characterize the patients with GCS score of 3. In the group of patients with GCS 3, FOUR score ≤1 had 84% chance of mortality; while the mortality was only 44% in a FOUR score ≥2. 13 The ability of FOUR score to further characterize the patients with very low GCS can be explained by the greater neurological details provided by FOUR score. However, the problem of negatively skewed distribution of FOUR score is unavoidable when the study is performed in the setting of ED, unless we only recruit patients with altered mental status alone. And in reality, many patients with minor stroke seen in ED do not have a change in conscious level.
There are some practical problems associated with the use of FOUR score to assess patients suspected to have acute stroke in the ED. Since FOUR score assesses greater neurological details, it is more time consuming to use when compared with GCS. Also, performing corneal reflexes in conscious patients is not comfortable and may not be necessary as the majority of patients with suspected stroke are fully conscious. Most of the patients who are intubated are often given full sedation and paralytic agents. There will be difficulty in assessing breathing rate above the ventilator rate. This results in a falsely low rating of its respiratory component.
Limitations
The results of our study were similar to the results of overseas studies. However, ours was a single centre study, and the results may not be generalizable. Future larger study, preferably a multi-centre study, is required to explore the advantages of FOUR score over GCS score. The study should include larger number of patients with lower GCS score in order to look for any advantage of FOUR score over GCS score in providing greater neurological details and further characterizing the patients with very low GCS score. Proper blinding of both the FOUR scores and GCS scores between the raters is suggested to ensure accurate measurement of interrater reliability.
Conclusion
FOUR score is a recently developed and validated coma scale with some advantages over GCS score. Our study was able to demonstrate good interrater reliability when ED doctors and ED nurses applied the FOUR score to patients suspected to have acute stroke independently. We also demonstrated the ability of FOUR score to predict in-hospital mortality in that group of patients. The results were similar to the results of other studies performed in foreign countries. We would suggest further larger studies with inclusion of larger number of patients with low GCS/FOUR scores and with proper blinding of the scores between raters.
Footnotes
Appendix
Definition of FOUR score and Glasgow Coma Scale score.
| FOUR score | GCS score |
|---|---|
| Eye response 4 = eyelids open or opened, tracking, or blinking to command 3 = eyelids open but not tracking 2 = eyelids closed but open to loud voice 1 = eyelids closed but open to pain 0 = eyelids remain closed with pain |
Eye response 4 = eyes open spontaneously 3 = eye opening to verbal command 2 = eye opening to pain 1 = no eye opening |
| Motor response 4 = thumbs-up, fist or peace sign 3 = localizing to pain 2 = flexion response to pain 1 = extension response to pain 0 = no response to pain or generalized myoclonus status |
Motor response 6 = obeys commands 5 = localizing pain 4 = withdrawal from pain 3 = abnormal flexion response to pain 2 = extension response to pain 1 = no motor response |
| Brainstem reflexes 4 = pupil and corneal reflexes present 3 = one pupil wide and fixed 2 = pupil or corneal reflexes absent 1 = pupil and corneal reflexes absent 0 = absent pupil, corneal and cough reflex |
Verbal response 5 = oriented 4 = confused 3 = inappropriate words 2 = incomprehensible sounds 1 = no verbal response |
| Respiration 4 = not intubated, regular breathing pattern 3 = not intubated, Cheyne–Stokes breathing pattern 2 = not intubated, irregular breathing 1 = breaths above ventilator rate 0 = breaths at ventilator rate or apnoea |
FOUR: Full Outline of Unresponsiveness; GCS: Glasgow Coma Scale.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.