Abstract
Objectives:
To develop and validate a new clinical prediction model for screening patients at risk for obstructive sleep apnea–hypopnea syndrome (OSAHS).
Methods:
This study used 2 data sets to develop and validate the model. To build the model, the first data set comprised 892 patients who had diagnostic polysomnography (PSG); data were assessed by multivariate logistic regression analysis. To validate the new model, the second data set comprised 374 patients who were enrolled to undergo overnight PSG. Receiver operating characteristic analysis and all predictive parameters were validated.
Results:
In the model development phase, univariate analysis showed 6 parameters were significant for prediction apnea–hypopnea index ≥15 events/hour: male sex, choking or apnea, high blood pressure, neck circumference >16 inches (female) or 17 inches (male), waist circumference ≥80 (female) or 90 cm (male), and body mass index >25 kg/m2. Estimated coefficients showed an area under the curve of 0.753. In the model validation phase, the sensitivity and specificity were approximately 93% and 26%, respectively, for identifying OSAHS. Comparison with the Epworth Sleepiness Scale score of ≥10 and STOP-Bang score ≥3 showed sensitivity of 42.26% and 56.23%, respectively, for detecting patients at risk.
Conclusions:
This new prediction model gives a better result on identifying patients at risk for OSAHS than Epworth Sleepiness Scale and STOP-Bang in terms of sensitivity. Moreover, this model may play a role in clinical decision-making for a comprehensive sleep evaluation to prioritize patients for PSG.
Introduction
Obstructive sleep apnea–hypopnea syndrome (OSAHS) is a common sleep disorder in adults that is characterized by partial or complete collapsibility of the upper airway during sleep. The current worldwide prevalence of OSAHS is 2% and 4% in middle-age men and women, respectively, and is strikingly higher in for older adults. 1 Similarly, the prevalence is 1.9% to 4.8% in the middle-age Thai population. 2 Moreover, an even higher prevalence is estimated for bariatric patients (35%-94%) and stroke patients (60%-70%).3,4
Patients with OSAHS typically have repetitive periods of hypoxemia, hypercapnia, and microarousal following the respiratory events. This cycle leads to sleep fragmentation, poor sleep quality, and increased sympathetic hyperactivity. These patients may also develop symptoms of excessive daytime sleepiness, neurocognitive impairment, and mood changes that interfere quality of life and increase risks of motor vehicle accidents or work difficulties. 5 Importantly, untreated OSAHS is correlated with multiple cardiovascular consequences, including systemic hypertension, pulmonary hypertension, heart failure, arrhythmia, diabetes, and stroke.6-8 However, many individuals with OSAHS remain undiagnosed. 9 Moreover, patients with OSAHS may have an increased perioperative risk. 10 Several studies have shown that patients with OSAHS have an almost 2-fold higher use of health care resources, including prolonged hospital stay, physician fees, and prescribed medications.11,12 Effective treatment for patient with OSAHS gradually decreases the associated health care costs and improves overall quality of life. 13
Overnight polysomnography (PSG) in an attended setting remains the gold standard for diagnosis of OSAHS. 14 However, the use of PSG has several limitations, including a prolonged waiting time, depending on the availability of often-insufficient resources, and the lack of well-trained sleep technicians. The portable sleep study has become popular for diagnosis of patients with uncomplicated OSAHS. However, this study is also inconvenient, especially in lower socioeconomic areas, and the need for adequate skills to score the sleep remains problematic.
Based on these sleep study limitations, the use of sleep questionnaires seems a more simple and easy approach to identify patients at risk. First, Epworth Sleepiness Scale (ESS) questionnaire is one of the most popular tools for evaluating the level of daytime sleepiness, and it was recently noted to be useful in monitoring treatment response.15,16 Second, although the Berlin Questionnaire is a simple tool to assess patients at risk for OSAHS in the primary care setting, it unfortunately has an unacceptably high false-negative rate for screening.17,18 Third, the STOP-Bang Questionnaire—which considers snoring, tiredness, observed apnea, high blood pressure, body mass index (BMI), age, neck circumference (NC), and male sex—is an effective instrument that plays a role in ruling out OSAHS in surgical patients in the perioperative setting.19,20 However, the parameters and cutoff values for this questionnaire may not be applicable for Asian people. Moreover, most of these screening tools are used in the tertiary hospital setting as a part of comprehensive sleep evaluation by a sleep specialist. To address the importance of recognizing the undiagnosed OSAHS population and the associated health care costs, this study aims to create a reliable instrument to help clinicians in identify at-risk patients.
Patients and Methods
Study Population
A prospective cohort study was performed from June 2018 to June 2020, at the Otolaryngology Outpatient Department and Sleep Laboratory Center of Songklanagarind Hospital, a tertiary hospital in Southern Thailand. This study protocol was approved by the Human Research Ethics Committee, Faculty of Medicine, Prince of Songkla University (REC 61-039-13-1), and an informed consent form was obtained from each patient. The study enrolled adult patients’ age 18 to 65 years with suspected OSAHS. Exclusion criteria were pregnancy, psychological disease, and current use of sedative or antipsychotic drugs. Patient data collected were age, sex, OSAHS-associated symptoms, ESS and STOP-Bang scores, and physical examination, including blood pressure, BMI, NC, waist circumference (WC), and Friedman tongue position. 21 The NC was measured at the Adam’s apple level, and WC was measured at the umbilical level with the patient in the standing position at the end of expiration. The Friedman tongue position was assessed in the upright position with mouth opening without tongue protrusion. All patients completed self-screening questionnaires the same day as undergoing the sleep study. Patients with incomplete data were excluded from the study.
Sleep Studies
Type I PSG (Compumedics E series, Compumedics) was conducted and comprised electroencephalography, electrooculography, chin and leg electromyography, electrocardiography, thermistors and nasal pressure transducer for oronasal airflow, thoracic and abdominal belts for respiratory efforts, pulse oximetry for oxyhemoglobin concentration, tracheal microphone for snoring, and sensors for sleeping position. Data for these parameters recordings were scored manually using standard criteria. 22 Apnea was scored by a decrease in the thermistor signal ≥90% of the pre-event baseline for ≥10 seconds. The definition of hypopnea was a 30% decrease in nasal pressure signals for ≥10 seconds associated with ≥3% desaturation or an arousal. The apnea–hypopnea index (AHI) was calculated by the number of apnea and hypopnea events/hour of total sleep time. 14 An AHI score ≥15 events/hour was considered to definitively diagnose OSAHS. 23
Statistical Analysis
Baseline data were reported as number (percentage), mean (standard deviation), or median (interquartile range). Comparison between groups was performed with χ2 or Fisher exact test for nominal variables, and Mann-Whitney U test for ordinal variables. The 2 data sets, the model development phase and validation phase, were also independent variables. For the model development phase, all variables from the first data set with P < .05 on univariate analysis were simultaneously considered in multivariate analysis. Multiple logistic regression analysis was used to construct a model to predict obstructive sleep apnea (OSA) by using the estimated coefficients of each variables. The optimal cutoff value was determined by using the cost–benefit analysis. The disease prevalence and the cost ratio of untreated moderate to severe OSA were applied in this calculation. Receiver operating characteristic (ROC) curve analysis was then used to assess the accuracy of model predictability, and an area under curve (AUC) was estimated.
For the model validation phase, all variables from the second data set were used. Scores were compared for ESS and STOP-Bang to predict an AHI score ≥15. The sensitivity, specificity, likelihood ratio, positive predictive value (PPV), negative predictive value (NPV), and AUC were analyzed. All statistical analysis was performed using Epidata software (version 3.1) and R (version 3.5.1). A value of P < .05 was considered to be statistically significant.
For easy accessibility and data collection, the development model was created in the smartphone application (app) “sleepmore”; this prototype software was developed at Engineering Faculty of Prince of Songkla University. All enrolled participants could download from Google Play store without charge.
Results
Development Phase
The study enrolled 892 patients who met inclusion criteria and had complete diagnostic PSG data. Mean age was 49.33 ± 13.07 years; 604 (67.7%) patients were male. Mean BMI was 29.66 ± 8.20 kg/m2; approximately 73% were overweight (BMI >25 kg/m2). Choking sensation or apnea was reported by 400 (44.84%) patients. Daytime sleepiness assessed by ESS score showed 443 (49.66%) patients had scores ≥10. Hypertension was found in 671 (75.22%) patients. Based on AHI score, 126 (14.1%) patients were classified as non-sleep apnea (AHI <5), and 766 (85.87%) patients had OSA (AHI ≥5). Of the OSA group, 205 (23%) patients had mild OSA, 164 (18.4%) patients had moderate OSA, and 397 (44.5%) patients had severe OSA. Approximately 73% of OSA group were overweight versus 27% of non-OSA patients.
Univariate analysis showed 6 variables had statistically significance for prediction of AHI ≥15 events/hour: male sex, choking or apnea, high blood pressure or hypertension, NC >16 inches (female) or >17 inches (male), WC ≥80 cm (female) or ≥90 cm (male), and BMI >25 kg/m2; these parameters considered in the multiple logistic regression in the following step. The coefficients, odd ratios, and corresponding 95% CIs were estimated (Table 1). Male versus female sex had approximately 3.65 (95% CI: 2.64-5.06) times higher odds for prediction. Patients with versus without choking sensation or apnea had an odd ratio of 1.57 (95% CI: 1.15-2.13). A significant relationship for predicting OSA was shown for NC, with >16 inches (female) or >17 inches (male) having an odd ratios of 5.06 (95% CI: 2.75-9.32) compared to patients with smaller NC values. Patients with BMI >25 kg/m2 and WC of ≥80 cm (female) or ≥90 cm (male) also had odds ratios of 1.73 (95% CI: 1.16-2.59) and 1.80 (95% CI: 1.16-2.81), respectively, compared with patients who had lower BMI and WC values. Therefore, the model was developed using the estimated coefficients of the 6 significant variables (Table 2). The cost–benefit analysis was used to determine the optimal cutoff score by using the prevalence of 11.4% and the cost ratio of 0.0528. The scores of individuals were summed and categorized into 2 groups, low and high suspicion for OSAHS, by using a cutoff point at 1.5831 with sensitivity of 93.76% (95% CI: 91.43-95.62) and specificity of 29.31% (95% CI: 24.46-34.53). From ROC curve analysis, the AUC of this model was 0.753.
Factors Associated With Moderate to Severe OSA (AHI ≥15) in Multiple Logistic Regression Analysis for 892 Patients.
Abbreviations: AHI, apnea–hypopnea index; BMI, body mass index; DBP, diastolic blood pressure; HT, hypertension; NC, neck circumference; OSA, obstructive sleep apnea; SBP, systolic blood pressure; WC, waist circumference.
Factors and Scores Used in the Prediction Model and Cutoff Point.
Abbreviations: BMI, body mass index; DBP, diastolic blood pressure; HT, hypertension; NC, neck circumference; SBP, systolic blood pressure; WC, waist circumference.
Validation Phase
This phase used a new data set of 374 patients with complete diagnostic sleep study data. Mean age and BMI were 44.1 ± 12.5 years and 31.1 ± 9.2 kg/m2; 194 (51.9%) patients were men. Choking sensation or apnea was present in 214 (57.2%) patients; daytime sleepiness with ESS score ≥10 was found in 160 (42.8%) patients. An AHI ≥15 was found in 265 (70.86%) of 374 patients versus 109 (29.14%) patients with AHI <15. Age, NC, WC, and BMI were significantly different between 2 groups (Table 3). Predictive parameters for AHI ≥15 are shown in Table 4. Sensitivity and specificity were 92.83% and 25.69%, respectively, for detecting OSAHS, and the PPV and NPV were 75.23% and 59.57%, respectively.
Baseline Demographic Data in the Model Validation Phase for 374 Patients.
Abbreviations: AHI, apnea–hypopnea index; BMI, body mass index; BP, blood pressure; ESS, Epworth sleepiness scale; IQR, interquartile range; SD, standard deviation.
Model Performance With ESS and STOP-Bang Questionnaire Score Comparison to Predict AHI ≥15 in 374 Patients.
Abbreviations: AHI, apnea–hypopnea index; ESS, Epworth sleepiness scale.
Comparison With of ESS Score and STOP-Bang Questionnaires
Data for the ESS score and STOP-Bang questionnaire scored responses were used to calculate their performance for predicting AHI ≥15. An ESS score ≥10 identified OSAHS patients with a sensitivity of 42.26% and specificity of 55.96%. The STOP-Bang scores ≥3 predicted OSAS with a sensitivity and specificity of 56.23% and 53.21%, respectively (Table 4).
Discussion
Although many studies reported that untreated OSAHS is associated with unfavorable long-term cardiovascular outcomes, most individuals with OSAHS remain undiagnosed, in part because of the lack of public awareness and the low level of education that remain problematic in developing countries. Moreover, many basic health insurance systems do not cover the medical expenses of sleep studies and continuous positive airway pressure treatment. Overall the availability of sleep studies may still be limited to the tertiary hospital setting, which makes it difficult for people to access this health care service. Importantly, the high-risk populations, especially those who are employed as public drivers, have been ignored in identifying sleep disorders. Studies have confirmed a 2 to 3 times increased risk of motor vehicle collisions among untreated OSA patients. 24 Moreover, OSAHS patients with AHI ≥15 were significantly affected by neurobehavioral issues overall. 25 Today, the increasing prevalence of obesity worldwide will gradually affect the development of OSAHS and associated metabolic consequences.
The 6 predictive factors in this model did not much differ from those in other studies, which reflects that these variables are important factors in occurrence of this condition. In this new model, however, the cutoff point was changed to achieve better predictability. In the developing phase, the model showed good predictability (AUC: 0.753). Daytime hypersomnolence can be found in approximately 20% of the population overall and can also occur in other sleep disorders, including sleep deprivation or insomnia. 26 In the present study, an ESS score ≥10 could identify patients at risk for moderate to severe OSA with the moderate sensitivity of 42% and specificity of 56%, relevant to previous studies. 20 Also, many studies have confirmed that ESS scores did not correlate with OSA severity; however, daytime sleepiness evaluated by ESS had a role in following patients after treatment. 16 Because of this role, hypersomnolence may be ignored in some OSA cases, whereas some patients may be asymptomatic despite having severe OSA. For these reasons, subjective symptoms may be unreliable predictors of OSA.
Surprisingly, STOP-Bang questionnaire was confirmed to have consistently high performance in OSA screening in surgical patients with a sensitivity of 93% to 100% on meta-analysis. 27 However, the questionnaire showed moderate sensitivity of 56% and specificity of 53% in identifying AHI ≥15 in the present study, possibly due to different study populations study and cutoff values for each parameter. In addition, this new model used only 6 variables, fewer than other questionnaires. Moreover, the morphometric model was reported to have very high predictive performance in detecting patients with OSAS, with a sensitivity of 98% and specificity of 100%. 28 Another prediction model using the artificial intelligence system also showed favorable results with as high as 88% sensitivity and 97% specificity. 29 However, these models had limitations because they did not simplify assessment and may be quite complicated to use in a routine practice setting.
Smartphone use in medicine has become increasingly popular among patients as a part of screening tools that patient can use themselves. Some of apps for evaluating sleep and sleep apnea have been developed. For more accurate results, some require a different external device connected to the smartphone, including a snoring detector or single-channel electroencephalography or wristwatch. Many apps monitor snoring recording to predict OSA, varying in sensitivity from 64% to 96%, with PSG. 30 Because the accuracy and reliability of these results vary depending on the study population, the clinical utility of smartphones and apps remains questionable.
The new prediction model, using the sleepmore app, is an easily accessible and user-friendly tool to screen for OSAHS with sensitivity as high as 93%, and negative results can rule out the condition. Because of low specificity, the model yields a high rate of false-positive results. However, it may be used as a part of comprehensive sleep evaluation for decision-making in clinically uncertain cases. Moreover, these factors and their cutoff points are adjustable for the Asian population, which has a lower rate of obesity and a different craniofacial structure compared with the Western population. Importantly, parameters used in apps have high reliability based on the objective versus symptom-based data collection. This approach may be helpful in clinical decision-making to prioritize at-risk patients for PSG. An app-based approach can also reduce the cost and unnecessary waiting time for PSG. Moreover, such as app may play a role in proactive health service systems by raising public awareness of sleep disorders, especially OSAHS. Patient with early detection and early treatment will improve their quality of life and prevent their cardiovascular risks. The limitation of this study was that the selection bias leads to show the high prevalence (approximately 70%) of patients with AHI of ≥15, compared with the previous literatures reporting the prevalence of moderate to severe OSA of 25% in middle-aged general population. 31 Thus, it could not represent the general population as well.
In conclusion, the routine requirement for PSG to diagnose OSAHS regardless of clinical probability may be costly and time-consuming. The use of screening instruments, as a part of clinical decision-making, can identify patients for whom PSG should be prioritized.
Footnotes
Acknowledgments
The authors would like to thank the Faculty of Medicine, Prince of Songkla University for funding, Miss Jirawan Jayuphan for her assistant in data analysis, Mr. Tharit Wichiankoo and Mr. Thanaphon Toprasit for application development and data gathering.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Medical Research Council (grant number 61-039-13-1).
