Sage Journals: Discover world-class research

Abstract

Numerous existing Pediatric Early Warning Scores (PEWs) have varying degrees of reliability and validity, which are used in variety diseases of patients. This study is a prospective diagnosis study which involved the pediatric nurse evaluation of patient status using modified Pediatric Early Warning Score (NU-PEWS) until the patient was discharged or transferred to PICU. A total of 824 pediatric patients were admitted, 407 participants were enrolled in this study. The NU-PEWS demonstrated the most accurate cut-off point at greater than 3, with 90.5% sensitivity and 89.1% specificity. The receiver operating characteristic (ROC) indicated positive results in the general medical condition (ROC 0.958), gastrointestinal, respiratory, and hematologic diseases (ROC 0.94-0.97) whereas lowest in neurological disease (ROC 0.843). This study validated that NU-PEWS has good performance in detecting deteriorating patients, and that prediction utilizations are good in almost every subgroup of disease, with the exception of neurological disease.

Keywords

Pediatric Early Warning Score PEWS children clinical deterioration

Introduction

In many hospitals, clinical deterioration of hospitalized pediatric patients increases mortality by approximately 5% to 15%.¹ Numerous tools have been developed to assist clinicians in identifying patients who are rapidly deteriorating. The Pediatric Early Warning Score (PEWS) system is a pediatric trigger tool that is used in several countries worldwide. Different circumstances and resource constraints will influence the variables of the scoring benchmarks in each hospital.^1,2 The majority of PEWs are based on vital signs and clinical assessments, such as heart rate, respiratory rate, and level of consciousness.^3-5

The PEWS system can be applied to pediatric patients with varying diseases. However, according to the previous study of Early Warning Scores (EWS) in adults, various diseases related to neurological disorders, cardiovascular disorders, and sepsis may require different or additional scoring variables.^6-8 Furthermore, when the predictive value of EWS was compared across a range of disorders, the results are inconsistent.^8-10 As a result, we hypothesize that different diseases may require distinct scoring in order to accurately predict the trend toward intensive care unit transfer.

There were few case studies involving the use of pediatric early warning scores in different disease subgroups. In Thailand, varieties of pediatric trigger tools have been implemented in many provincial hospitals.^11-13 Pediatric Early Warning Score, Pediatric Advanced Warning Score, and Pediatric Clinical Alert Score are examples of scoring systems that detect deteriorating patients. The existence of numerous PEWS with varying levels of reliability and validity has resulted in a lack of clarity regarding how these scoring systems compare to one another. In July 2020, the pediatricians of Naresuan University Hospital—a tertiary hospital in the lower northern region of Thailand—established a modified PEWS system named NU-PEWS. This is a study to validate the NU-PEWS in assessing hospitalized pediatric patients, and to determine how the scores differ when used for specific diseases.

Methods

This prospective study enrolled pediatric patients aged 1 month to 15 years that were admitted to Naresuan University Hospital between November 2020 and August 2021. The study’s limitations include patients that were referred from other hospitals, transfer from emergency room to pediatric intensive care units (PICUs), and patients admitted to PICUs for elective postoperative care.

Pediatricians at Naresuan University created a modified pediatric early warning score called NU-PEWS, which consists of consciousness, heart rate, capillary refill time, blood pressure, respiratory rate, retraction, oxygen saturation, and body temperature. Each item was scored between 0 and 3, resulting in a cumulative score between 0 and 20 (See Table 1). This scoring system was developed using monitoring equipment, patient characteristics, and nursing skill at the pediatric ward. Scores in respiratory retraction and oxygen saturation were adjusted for more appropriate and straightforward use.

Table 1.

Naresuan University Pediatric Early Warning Score.

Item	0	1	2	3
Consciousness	Good consciousness Alert	Drowsiness Response to voice	Response to pain	Unresponsive Seizure
Heart rate (bpm)
>1 month-1 year	80-180	>180-220	<80	>220
>1-10 years	60-140	>140-180	<60	>180
>10 years	60-100	>100-150	<60	>150
Capillary refill (seconds)	<2	2-3	>3
Respiratory rate (bpm)	1 month-1 year: 30-50>1-2 years: 20-50>2-5 years: 20-40>5-13 years: 15-30>13 years: 10-20	1 month-2 years: >50>2-5 years: >40>5-13 years: >30>13 years: >20	Apnea1 month-1 year: <30>1-5 years: <20>5-3 years: <15>13 years: <10
Retraction	No	1 site	2 sites	3 sites
Blood pressure (mmHg)	Normal		BP ≥ Percentile 95th	SBP• 1 month-1 year: <70• 1 year: <[(2 × age) + 70]Or BP ≥ (Percentile 99th) + 5Age in year
SpO2 (%)	>94	90-94Or below baseline 1%-5%	<90%Or below baseline >5%
Body temperature (°C)	36-37.9	>37.9-38.5	<36 or >38.5

To ensure the accuracy of the evaluation, nurses were trained on how to assess each NU-PEWS item, including vital signs, respiratory retraction, consciousness, and capillary refill time. Then, the nurses’ ability to perform the NU-PEWS assessment was reevaluated by a well-trained senior nurse and physician. At the beginning of the study, four trained nurses conducted 30 recordings to determine inter-rater reliability. Subsequently, the trained pediatric nurses performed NU-PEWS evaluation every 4 hours until the patient was discharged or transferred to PICU. Data collection includes age, sex, diagnosis, underlying diseases, NU-PEWS results, and the reason for PICU admission. The study’s primary objective was to detect clinical deterioration necessitating PICU admission. The conditions that demanded PICU admission in this study were comatose state, status epilepticus, respiratory distress requiring high flow nasal cannula, non-invasive ventilation, and mechanical ventilation, hypotension requiring inotropic medication, emergency hypertension, and other conditions needing intensive management such as renal failure, liver failure, diabetic crisis, and gastrointestinal bleeding.

This study was approved by Naresuan University Institutional Review Board (No. P3-0117/2563) and was conducted in accordance with the Declaration of Helsinki (as revised in 2013). A sample size of at least 400 subjects was required to estimate the population mean.

Analysis

In statistical analysis, categorical variables were presented in percentage and the continuous variables with non-normal distributions were expressed in median range. The intraclass correlation coefficient (ICC) revealed a 95% confidence interval (CI) which indicates consistency or absolute agreement. Validity of NU-PEWS were analyzed using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) with their respective 95% confidence intervals (CI). The NU-PEWS results that prompt PICU transfers were measured using receiver operating characteristic (ROC). The ROC analysis plots all sensitivity versus (1-specificity) at selected cut-offs points by placing each pair of sensitivity and (1-specificity) in ROC space.

Result

During the initial phase, this study showed very good inter-rater reliability with intraclass coefficient of 0.948 -1.00. There were 824 pediatric admissions during the study period, but only 407 were enrolled due to the following limitations: PICU admissions, referrals from other hospitals, elective cases and incomplete data. (See Figure 1)

Figure 1.

Flow diagram of enrollment process.

The 407 patients had a median age of 2.1 years, and 54.7% were male. Fifty-seven percent of the patients were between the ages of 1 and 5 years old, 21.1% were under 1 year old, and 13.3% had comorbidities. Hematologic disease was the most common comorbidity (33.3%), followed by respiratory disease (15.9%). Respiratory disorders (51.6%), gastrointestinal disorders (25.6%), and infectious diseases (13.5%) were the most frequently occurring clinical diagnoses.

The prevalence of unplanned PICU admissions occurred in 20 cases (4.9%) mostly aged 5 to 10 years (35.0%), and was associated with respiratory distress diagnosis (55.0%), altered consciousness (25.0%), and infectious diseases (30.0%). The majority of the patients (55%) had comorbidities, with 11 cases involving hematologic disease (54.5%) (See Table 2)

Table 2.

Demographic and Characteristic Data.

Patient characteristics	Total (n = 407) (n, %)	PICU (n = 20) (n, %)
Age (year) Median (range)	2.1 (0.1-14.8)	2.9 (0.3-14.0)
Age (year) group
<1	86 (21.1)	6 (7.0)
1-5	232 (57.0)	5 (2.2)
5-10	68 (16.7)	7 (10.3)
10-15	21 (5.2)	2 (9.5)
Sex (male)	223 (54.7)	9 (45.0)
PICU transfer	20 (4.9%)	20 (100)
Diagnosis
Neurological disease	22 (5.4)	5 (22.7)
Respiratory disease	210 (51.6)	11 (5.3)
Gastrointestinal disease	104 (25.6)	1 (1.0)
Cardiovascular disease	8 (2.0)	0 (0.0)
Genitourinary disease	6 (1.5)	0 (0.0)
Hematologic disease	19 (4.7)	3 (15.8)
Infectious disease	55 (13.5)	6 (10.3)
Others	14 (3.4)	2 (13.3)
Underlying disease
Yes	54 (13.3)	11 (55.0)
Neurological disease	5 (7.9)	0 (0.0)
Cardiovascular disease	8 (12.7)	2 (18.2)
Respiratory disease	10 (15.9)	1 (9.1)
Gastrointestinal disease	3 (4.8)	0 (0.0)
Genitourinary disease	2 (3.2)	0 (0.0)
Hematologic disease	21 (33.3)	6 (54.5)
Infectious disease	3 (4.7)	1 (9.1)
Others	11 (17.5)	4 (36.3)

The NU-PEWS demonstrated the most accurate cut-off point in overall general medical conditions at more than 3, with a sensitivity and specificity of 90.5% (95% CI 89.8-91.2) and 89.1% (95% CI 88.4-89.8), respectively. The performance of NU-PEWS revealed a high degree of accuracy in all patients (ROC 0.958; 95% CI 0.930-0.986). NU-PEWS showed excellent performance in gastrointestinal, respiratory, and hematologic diseases due to its high sensitivity, specificity, and accuracy. (ROC 0.94-0.97). The scoring system also exhibited acceptable accuracy in the infectious disease subgroup, but was less accurate in neurological diseases. A cut-off value of greater than 4 illustrates increased sensitivity and specificity in gastrointestinal, hematologic, and neurological diseases (See Tables 3 and 4).

Table 3.

Validity of NU-PEWS in General Medical Conditions and Disease Subgroups.

Scores	General medical conditions				RS		GI		Heme		NS		ID
Scores	Sens (%)	Spec (%)	PPV (%) (95% CI)	NPV (%) (95% CI)	Sens (%)	Spec (%)	Sens (%)	Spec (%)	Sens (%)	Spec (%)	Sens (%)	Spec (%)	Sens (%)	Spec (%)
≥2	100	66.3	0.9 (0.7-1.1)	100 (100)	100	61.4	100	72.8	100	74.4	100	58	100	78.2
≥3	90.5	89.1	2.4 (2.0-2.7)	99.9 (99.9-100)	100	86.1	100	94.5	100	88.6	80	88.6	83.3	94
≥4	71.4	97.1	6.9 (6.3-7.5)	99.9 (99.8-99.9)	66.7	96.1	100	99.2	100	97.8	80	95.6	77.8	98.5
≥5	33.3	99.4	14.0 (13.2-14.8)	99.8 (99.7-99.9)	33.3	99.1	0	99.9	33.3	100	40	99.3	83.3	98.7

Abbreviations: RS, respiratory disease; GI, gastrointestinal disease; Heme, hematologic disease; NS, neurological disease; ID, infectious disease; Sens, sensitivity; Spec, specificity; PPV, positive predictive value; NPV, negative predictive value; 95% CI, 95% confidence interval.

Table 4.

The Area Under the Receiver Operating Characteristic (ROC) in General Medical Conditions and Disease Subgroups.

Disease	ROC	95% CI
General medical conditions	0.958	0.930-0.986
Respiratory disease	0.930	0.924-0.936
Gastrointestinal disease	0.972	0.960-1.000
Hematologic disease	0.943	0.929-0.958
Neurological disease	0.843	0.647-1.000
Infectious disease	0.886	0.723-1.000

Optimal cut-off point ≥3.

Discussion

Early identification and management of patients at risk of clinical deterioration are essential to facilitate initial intervention. PEWS is a tool for early detection in the hospital and emergency department which has numerous cut-off points and a high degree of validity in detecting clinical deteriorations.^2,5,14,15 For optimal discrimination, the early warning score should be highly sensitive while also having a high receiver operating characteristic (ROC), both positive and negative predictive values.^16-18

NU-PEWS is an early warning score which has a higher ROC, sensitivity and specificity than other PEWS.^19-22 The Thammasat Pediatric Early Warning Score (TPEWS), which is used in Thammasat Hospital—a tertiary hospital—has a similar ROC, sensitivity, and specificity, but a different cut-off point in this study.¹¹ NU-PEWS has a cut-off point at 3 which is lower than bedside PEWS²⁰ and TPEWS¹¹ in terms of overall general medical condition. Different item descriptions in the score, the capacity of monitoring equipment and nursing care in pediatric inpatient wards, and the indication for intensive care admission may result in varying cut-off points, necessitating the adjustment of PEWS according to the hospital situation.

PEWS was studied in a limited number of pediatric patients in various subgroups.^21,23,24 Limited literature reviews^8,10 were also performed amongst adults, which revealed heterogeneity in the predictive performance of the early warning score across diseases, despite the fact that the current study demonstrated superior performance across all disease subgroups (ROC 0.843 -0.972). Alhmoud et al¹⁰ conducted a large systematic review of early warning scores in adults and revealed that while stroke, cardiac, and renal diseases had the best predictive performance, enhanced accuracy was demonstrated in gastrointestinal, hematologic and respiratory diseases (ROC 0.93 -0.972), despite lowest in neurological disease in this study. This result contradicts the findings of Dean et al²³ who reported excellent EWS in neurological diseases (ROC 0.93). For improved disease discrimination particularly in neurology, a threshold greater than 4 has a higher ROC, sensitivity, specificity and other predictive values than a threshold of 3 (ROC 0.877, sensitivity 80%, specificity 95.6%, PPV 18.2, NPV 99.7). Neurological disease evaluation in children can be challenging because it requires specific critical alarms such as irritability, and training for abnormal neurological signs detection. Modifications to the clinical item and scoring system in the PEWS, as well as intensive training for nurses, may be necessary to increase predictability when using the PEWS.

Limitation

The findings in this study are subject to a number of important limitations. First, there is a lack of predictive performance for EWS in cardiovascular diseases because the majority of the patients are newborns and are transferred from another hospital. Second, data on the patient’s quality of care, management, and intervention were not collected.

Further research should be conducted to evaluate the outcomes of the score comparison between cut-off points greater than 3 and 4 in various subgroups of disease with the goal of reducing morbidity, mortality, and length of hospital stay. Certain diseases, such as cardiovascular disease, may require a different implementation of PEWS.

Conclusion

This study showed that NU-PEWS is an effective tool at detecting patients who have deteriorated beyond a threshold score of 3. Prediction utilization is high in the overall medical conditions and disease subgroups except in neurological disease. The early warning system should be established with a high degree of sensitivity while maintaining a reasonable degree of specificity. Adjusting specific critical alarms may be necessary to optimize PEWS performance in neurological diseases.

Footnotes

Acknowledgements

We would like to thank Miss Kornthip Jeephet, Statistics Technical Officer, Research Center, Faculty of Medicine, Naresuan University, for her assistance with the statistical analysis. We are grateful to Ms. Daisy Jimenez Gonzales and Mrs. Judely Marish Cruz Cañete, lecturers, Faculty of Medicine, Naresuan University, for proofreading, editing, and checking English accuracy.

Author Contributions

Chutima Phuaksaman: contributed to conception and design; contributed to analysis and interpretation; drafted manuscript; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy. Paphawadee Sukboonthong: contributed to conception; contributed to acquisition; drafted manuscript; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring itegrity and accuracy.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Statement

This study was approved by Naresuan University Institutional Review Board (No. P3-0117/2563). The parents of all study participants gave both verbal and written informed consent before study enrollment.

ORCID iD

Chutima Phuaksaman

References

Rosman

Karangwa

Law

Monuteaux

Briscoe

McCall

. Provisional validation of a pediatric early warning score for resource-limited settings. Pediatrics. 2019;143(5):e20183657. doi:10.1542/peds.2018-3657

Seiger

Maconochie

Oostenbrink

Moll

. Validity of different pediatric early warning scores in the emergency department. Pediatrics. 2013;132(4):e841-e850. doi:10.1542/peds.2012-3594

Miranda

Camargo

Nascimento Sobrinho

, et al. Translation and adaptation of a pediatric early warning score. Rev Bras Enferm. 2016;69(5):888-896. doi:10.1590/0034-7167-2015-0096

Monaghan

. Detecting and managing deterioration in children. Paediatr Nurs. 2005;17(1):32-35. doi:10.7748/paed2005.02.17.1.32.c964

Chapman

Wray

Oulton

Pagel

Ray

Peters

. ‘The score matters’: wide variations in predictive performance of 18 paediatric track and trigger systems. Arch Dis Child. 2017;102(6):487-495. doi:10.1136/archdischild-2016-311088

Singkwang

. Development of the modified early warning signs of increased intracranial pressure (MEWS of IICP) guidelines in postoperative brain tumor patients, Buddhachinaraj Phitsanulok Hospital. Buddhachinaraj Med J. 2020;37(2):136-148.

Somana

. Effects of modified early warning score on the level of sepsis progression and mortality among hospitalized medical patients. Int J Evid Based Healthc. 2016;14:S33. doi:10.1097/01.XEB.0000511674.58163.77

Hester

Youn

Trifilio

, et al. The modified early warning score: a useful marker of neurological worsening but unreliable predictor of sepsis in the neurocritically ill: a retrospective cohort study. Crit Care Explor. 2021;3(5):e0386. doi:10.1097/CCE.0000000000000386

Liu

Carey

, et al. Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit. JAMA Netw Open. 2020;3(5):e205191. doi:10.1001/jamanetworkopen.2020.5191

10.

Alhmoud

Bonnici

Patel

Melley

Williams

Banerjee

. Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review. BMJ Open. 2021;11:e045849. doi:10.1136/bmjopen-2020-045849

11.

Chaiyakulsil

. Validation of thammasat pediatric early warning score for prediction of pediatric intensive care unit admission. Thammasat Med J. 2020;20(3):215-225.

12.

Tanaree

. The efficacy of the pediatric early warning score: PEWS in the pediatric ward at Kamphaeng Phet Hospital. Thai J Pediatr. 2016;55(3):196-201.

13.

Tangsricharoen

Kanjanavanit

. Performances of the pediatrics early warning score (PEWS) to predict early deaths within 24 hours and causes of death in pediatric intensive care unit. J Nakornping Hosp. 2018;9(2):28-40.

14.

Trubey

Huang

Lugg-Widger

, et al. Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children: a systematic review. BMJ Open. 2019;9(5):e022105. doi:10.1136/bmjopen-2018-022105

15.

Skaletzky

Raszynski

Totapally

. Validation of a modified pediatric early warning system score: a retrospective case-control study. Clin Pediatr. 2012;51(5):431-435. doi:10.1177/0009922811430342

16.

Romero-Brufau

Huddleston

Naessens

, et al. Widely used track and trigger scores: are they ready for automation in practice? Resuscitation. 2014;85(4):549-552. doi:10.1016/j.resuscitation.2013.12.017

17.

Oliveira

AECD

Machado

Santos

EDD

Almeida

ÉBD

. Alarm fatigue and the implications for patient safety. Rev Bras Enferm. 2018;71(6):3035-3040. doi:10.1590/0034-7167-2017-0481

18.

Winters

Cvach

Bonafide

, et al. Technological distractions (part 2): A summary of approaches to manage clinical alarms with intent to reduce alarm fatigue. Crit Care Med. 2018;46(1):130-137. doi:10.1097/CCM.0000000000002803

19.

Fuijkschot

Vernhout

Lemson

Draaisma

Loeffen

. Validation of a Paediatric Early Warning Score: first results and implications of usage. Eur J Pediatr. 2015;174(1):15-21. doi:10.1007/s00431-014-2357-8

20.

Parshuram

Duncan

Joffe

, et al. Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children. Crit Care. 2011;15(4):R184. doi:10.1186/cc10337

21.

McLellan

Gauvreau

Connor

. Validation of the Children's Hospital Early Warning System for critical deterioration recognition. J Pediatr Nurs. 2017;32:52-58. doi:10.1016/j.pedn.2016.10.005

22.

Miranda

JOF

Camargo

Nascimento Sobrinho

Portela

Monaghan

. Accuracy of a pediatric early warning score in the recognition of clinical deterioration. Rev Lat Am Enfermagem. 2017;25:e2912. doi:10.1590/1518-8345.1733.2912

23.

Dean

Fenix

Spaeder

Levin

. Evaluation of a pediatric early warning score across different subspecialty patients. Pediatr Crit Care Med. 2017;18(7):655-660. doi:10.1097/PCC.0000000000001176

24.

Gawronski

Ciofi degli Atti

Di Ciommo

, et al. Accuracy of Bedside Paediatric Early Warning System (BedsidePEWS) in a pediatric stem cell transplant unit. J Pediatr Oncol Nurs. 2016;33(4):249-256. doi:10.1177/1043454215600154