Sensitivity of the SNNOOP10 list in the high-risk secondary headache detection

Abstract

Aim

To evaluate the diagnostic accuracy of the SNNOOP10 list in the detection of high-risk headaches.

Methods

Patients that visited the Hospital Clínico San Carlos (Madrid) emergency department due to headache that were allocated to a Manchester Triage System level between critical and urgent were prospectively included but retrospectively analysed. A researcher blind to the patients’ diagnosis administered a standardised questionnaire and afterwards a neurologist blind to the questionnaire results diagnosed the patient according to the International Classification of Headache Disorders. The primary endpoint was to assess the sensitivity of the SNNOOP10 list in the detection of high-risk headaches. Secondary endpoints included the evaluation of the sensitivity, specificity, positive predictive value, negative predictive value and area under the curve of each SNNOOP10 item.

Results

Between April 2015 and October 2021, 100 patients were included. Patients were 44 years old (inter-quartile range: 33.6–64.7) and 57% were female. We identified 37 different diagnoses. Final diagnosis was a primary headache in 33%, secondary headache in 65% and cranial neuralgia in 2%. There were 46 patients that were considered as having high-risk headache. Patients from the primary headache group were younger and more frequently female. Sensitivity of SNNOOP10 list was 100% (95% confidence interval: 90.2%–100%). The items with higher sensitivity were neurologic deficit or disfunction (75.5%), pattern change or recent onset of the headache (64.4%), onset after 50 years (64.4%). The most specific items were posttraumatic onset of headache (94.5%), neoplasm in history (89.1%) and systemic symptoms (89%). The area under the curve of the SNNOOP10 list was 0.66 (95% CI: 0.55–0.76).

Conclusion

The red flags from the SNNOOP10 list showed a 100% sensitivity in the detection of high-risk headache disorders.

Keywords

Headache disorders secondary emergency service hospital cerebrovascular disorders diagnosis neoplasms

Introduction

Headache disorders are among the most frequent and prevalent neurological diseases (1). Headache is also one of the most common reasons for seeking care at an emergency department (ED) in the United States (US) (2), being consistently the fourth to fifth reason for consult globally (3,4). The main priority in the ED is detecting those patients that might have a life-threatening cause of headache (5), a proportion that varies between 2% and 18% (6 –11) depending on the studies. Mortality due to some of causes of secondary headache, such as subarachnoid haemorrhage, intracerebral haemorrhage or bacterial meningitis varies from 12% to 50% (12 –14), and the prognosis might be highly influenced by the accuracy of diagnosis and timely treatment (15).

Diagnosis of secondary headache disorders is based on the presence of red flags, which are intended to reduce wasted and unnecessary referrals, through better stratification of patients (16,17). There are several proposals, most of them based on consensus of experts, with limited prospective validation (18 –21). In 2019, the Secondary Headache Special Interest Group of the International Headache Society compiled the main red flags under the acronym “SNNOOP10” (22). To date, these criteria have not been validated. The aim of this study was to evaluate if secondary headache disorders can be correctly identified by using the SNNOOP10 list (22) in a cohort of patients that visited the ED with headache complaints and had high probability of secondary headache.

Methods

This was an observational analytic study with prospective cohort design. The study was done according to the standards for reporting of diagnostic accuracy studies (STARD) guidelines (23). The study population consisted of patients visiting the ED of the Hospital Clínico San Carlos de Madrid, Spain, a third-level, academic, public hospital, partly covering the metropolitan area of Madrid, with a population of 375,000 covered in this area. Recruitment was done between 20 April 2015 and 20 October 2016.

Eligibility criteria

The inclusion criteria were: 1) patients visiting the ED because of headache or if headache was one of the main symptoms at that time; 2) patients not being directly admitted to the critical box of the ED; 3) patients allocated to a Manchester Triage System (MTS) (11) level between 1-“critical” and 3-“urgent”; 4) patients older than 18 years old. Patients were excluded if 1) the primary reasons to consult were skull or cervical trauma; 2) that were unable to precisely describe the characteristics of the headache because of disorders of consciousness, speech disorders or cognitive impairment; 3) that had already participated in the study.

Recruitment and sampling

Sampling method was by probabilistic sampling, all the potential patients that came in during the shifts of the research team were screened. Each researcher worked approximately four shifts per month, for a total of 96 shifts. Based on a prior study conducted in the same hospital, the number of patients with an MTS score between 1–3 in a six-month period was 175, accounting for almost a patient per day (11). Every time that a patient visited the ED because of headache and had an MTS score between 1–3, the ED nurses alerted the research team who recruited the patient.

Index test

The research team was divided in two sections. The first group of researchers included four physicians, that approached the patients during their ED stay. They explained the purpose of the study and invited him/her to participate. If the participant agreed, they administered a standardised questionnaire that included a list of red flags, covering the 15 items from the SNNOOP10 list (22), and clarified the meaning of each specific red flag, if necessary.

The SNNOOP10 list is a proposal from the secondary headache special interest group of the International Headache Society which summarises the 15 main red flags, based on a literature review and expert opinion. The full SNNOOP10 list is available in the online Supplementary Table 1. At the time we analysed the results of our study, it had not yet been validated (22). Since the study was done prior to the publication of the SNNOOP10 list (2019), the original study questionnaire prospectively assessed 28 different red flags, however, for the purpose of the present study, we will just retrospectively analyse the 15 variables that are included in the SNNOOP10 list. The researchers were completely blind to the clinical information and were not allowed to ask anything beyond what was listed in the questionnaire. Furthermore, the outcome of the questionnaire was not shown to the evaluating neurologist and did therefore not alter the clinical evaluation and care of each patient.

Reference test

Afterwards, the second group, formed by two neurologists DGA and NGG with specific interest in headache disorders, reviewed the discharge report and the medical file of the patient. Then all patients were invited to a three-month follow-up either by in-person consultation or by phone. If needed, additional imaging, blood tests or other tests were conducted. The patients were classified by using the ICHD-3 beta (24), and once published, according to ICHD-3 criteria (25). Additionally, patients were classified as having high-risk headaches (HRH) if the cause of the headache: a) could threaten the patient’s life (e.g., central nervous system infections, subarachnoid haemorrhage); b) could cause significant morbidity, either persistent (e.g., vision loss) or paroxysmal (e.g., seizures); or c) could mimic a condition fulfilling criteria for a or b (e.g., first episode of hemiplegic migraine or thunderclap headache). The patients’ classification as having HRH was done by the two neurologists ALG and JPE, based on all the available information, the results of the investigations and the three-month follow-up evaluation, which was in-person when they deemed it necessary. Both neurologists were completely blind to the results of the questionnaire. Figure 1 summarises the study design.

Figure 1.

Study design summary. Created with BioRender.com.

Studied variables

Besides the presence of SNNOOP10 items, we analysed demographic and clinical variables. The first were sex, age, country of birthplace; and the clinical variables included headache intensity, presence of photophobia, phonophobia, osmophobia, nausea, vomiting, and aggravation by physical activity.

Ethics

The study was approved by the local ethics committee board (CP14/425-E). All participants were informed about the study, agreed to participate and signed informed consent form. The study was done in accordance with the Declaration of Helsinki principles (26).

Study endpoints

The primary endpoint was to evaluate the sensitivity of the SNNOOP10 list in the detection of high-risk headaches (HRH) in a series of patients that visited the ED because of headache and had high risk of secondary headache (MTS score 1–3).

As secondary endpoints, we analysed the diagnostic accuracy of each SNNOOP10 item, including sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio and area under the curve (AUC) in receiver operating characteristic model (ROC). We also compared the frequency and type of SNNOOP10 items between the main groups of the ICHD, primary headache disorders, secondary headaches and neuralgias, and patients with and without HRH.

Statistics

Qualitative data was presented as frequencies and percentages, with 95% confidence interval (95% CI). In the description of qualitative data, the selected measures of central tendency were mean if the distribution was normal and median if not. Measures of dispersion were range (R) and standard deviation (SD) or inter-quartile range (IQR) depending on the distribution. In the contrast of hypothesis, Chi-squared test was used to compare qualitative variables and if the number of subjects per group was <30, then Fisher exact test was employed. To compare between qualitative and quantitative variables, Student t-test or Mann-Whitney U-test were used, and if qualitative variable had more than two categories, then ANOVA was used. We estimated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR). We created a logistic regression model with all the variables and a multivariate model that included all the variables with a p-value of 0.3 or less. We calculated area under the curve by receiver operating characteristic curves and represented them graphically. Statistical signification threshold was set in 0.05 and multiple comparisons were managed by exact test and multivariate modelling. Sample size was not estimated in advance. Missing data was managed by complete-case analysis. Statistical analysis was conducted using the Statistical Package for Social Sciences version 26.0 software (SPSS, Inc., Chicago, IL, USA).

Data availability statement

The full datasets are available to other authors upon request to the corresponding author.

Results

During the study period, 100 patients fulfilled eligibility criteria, aged 44 years median (IQR: 33.6–64.7), 57% were women. The final diagnosis was primary headache in 33 cases (33%), secondary headache in 65 (65%) and cranial neuralgia in 2 (2%). Among the secondary headache patients, 46 (46%) had an HRH. Supplementary Figure 1 shows the frequency of diagnosis according to the ICHD-3 group. The total number of diagnosis was 37. There were 45 patients that were classified as high-risk headache. Supplementary Table 2 shows the frequency of specific diagnosis coded by ICHD-3 and the high-risk specific diagnoses.

Patients from the primary headache group were younger and more frequently female (67 vs. 51%) than patients with secondary headache. Hypersensitivity to stimuli was more frequent in primary headache group. Table 1 compares demographic and clinical variables between the primary and secondary headache groups. Patients with HRH were older and had less frequent hypersensitivity to stimuli. Table 1 compares demographic and clinical variables between standard headache and high-risk headaches.

Table 1.

Demographic and clinical variables comparison between the main categories of the ICHD and between non-high risk and high-risk headache patients. The neuralgia group was omitted in the statistical comparison between primary and secondary headache patients.

Variable	Primary (n = 33)	Secondary (n = 65)	Neuralgia (n = 2)	p-value	Non high-risk (n = 55)	High-risk headaches (n = 45)	p-value
Female sex	22 (66.7%)	33 (50.8%)	2 (100%)	0.134^†	36 (65.5%)	21 (46.7%)	0.070^†
Mean age	39.0 (13.5)	56.4 (21.5)	50.3 (20.7)	<0.001^‡	43.4 (17.9)	58.8 (21.0)	<0.001^‡
Photophobia (n = 55)	26 (78.8%)	28 (43.1%)	1 (50%)	0.001^†	37 (67.3%)	18 (40%)	0.009^†
Phonophobia (n = 46)	21 (63.6%)	25 (38.5%)	0 (0%)	0.021^†	31 (56.4%)	15 (33.3%)	0.027^†
Osmophobia (n = 18)	11 (33.3%)	6 (9.2%)	1 (50%)	0.005^†	14 (25.5%)	4 (8.9%)	0.038^†
Nausea (n = 53)	19 (57.6%)	33 (50.8%)	1 (50%)	0.669^†	32 (58.2%)	21 (46.7%)	0.315^†
Vomiting (n = 32)	7 (21.2%)	25 (38.5%)	0 (0%)	0.112^†	14 (25.5%)	18 (40%)	0.136^†
Aggravation by physical activity (n = 35)	10 (30.3%)	25 (38.5%)	0 (0%)	0.506^†	19 (34.5%)	16 (35.6%)	1.000^†
Mean intensity	8.3 (1.8)	7.6 (1.7)	7.8 (1.7)	0.133‡	8.1 (1.5)	7.5 (2.1)	0.119^‡

†Fisher exact test. ‡ANOVA.

Sensitivity of SNNOOP10 list

All the patients included in the study had at least one positive SNNOOP10 item. The sensitivity of the SNNOOP10 list for HRH detection was 100% (95% CI: 90.2–100%). The mean frequency of SNNOOP10 items in the primary headache patients was 3.75 (1.7), being minimum 1 and maximum 7; in the secondary headache group the mean was 4.7 (1.6), range 1–8; and both patients of the neuralgia group had four positive items. The number of positive SNNOOP10 items in the secondary group was significantly higher than in the primary headache group (p = 0.008).

Frequency and type of SNNOOP10 items in the ICHD groups

Frequency of SNNOOP10 items 5 (older age) and 6 (pattern change) were more frequent in the secondary headache group. Ocular pain, trigeminal autonomic features and medication overuse headache were more frequent in the primary headache group. Table 2 shows the frequency of SNNOOP10 items in primary headache disorders, secondary headache patients and cranial neuropathies.

Table 2.

Frequency of SNNOOP10 items in the main categories of the ICHD and between patients with and without high-risk headaches. The neuralgia group was omitted in the statistical comparison between primary and secondary groups.

Item	Primary (n = 33)	Secondary (n = 65)	Neuralgia (n = 2)	p-value^Ø	Non high-risk (n = 55)	High-risk headaches (n = 45)	p-value
S1 Systemic symptoms including fever (n = 13)	2 (6.1%)	11 (16.9%)	0 (0%)	0.208	6 (10.9%)	7 (15.6%)	0.558
S2 Neoplasm in history (n = 15)	3 (9.1%)	11 (16.9%)	1 (50%)	0.372	6 (10.9%)	9 (20%)	0.264
S3 Neurologic deficit or dysfunction (n = 63)	21 (63.6%)	42 (64.6%)	0 (0%)	1.000	29 (52.7%)	34 (75.6%)	0.023
S4 Onset of headache is sudden or abrupt (n = 46)	16 (48.5%)	30 (46.2%)	2 (100%)	0.834	24 (43.6%)	24 (53.3%)	0.422
S5 Older age (after 50 years) (n = 43)	5 (15.2%)	38 (58.5%)	1 (50%)	<0.001	15 (27.3%)	29 (64.4%)	<0.001
S6 Pattern change or recent onset of headache (n = 61)	16 (48.5%)	45 (69.2%)	1 (50%)	0.051	33 (60.0%)	29 (64.4%)	0.683
S7 Positional headache (n = 45)	17 (51.5%)	28 (43.1%)	0 (0%)	0.521	25 (45.5%)	20 (44.4%)	1.000
S8 Precipitated by sneezing, coughing, or exercise (n = 38)	10 (30.3%)	28 (3.1%)	0 (0%)	0.275	21 (38.2%)	17 (37.8%)	1.000
S10 Progressive headache and atypical presentations (n = 65)	20 (60.6%)	45 (67.2%)	2 (100%)	0.498	37 (67.3%)	30 (66.7%)	1.000
S11 Pregnancy or puerperium (n = 0)	0	0	0	–	0 (0%)	0 (0%)	1.000
S12 Painful eye with autonomic features (n = 17)	9 (27.3%)	7 (10.8%)	1 (50%)	0.046	13 (23.6%)	4 (8.9%)	0.063
S13 Posttraumatic onset of headache (n = 13)	2 (6.1%)	11 (16.9%)	0 (0%)	0.208	3 (5.5%)	10 (22.2%)	0.017
S14 Pathology of the immune system (n = 17)	3 (9.1%)	14 (21.5%)	0 (0%)	0.163	9 (16.4%)	8 (17.8%)	1.000
S15 Medication Overuse Headache (n = 8)	7 (21.2%)	0 (0%)	1 (50%)	<0.001	8 (14.8%)	0 (0%)	0.007

Frequency and type of SNNOOP10 items in the HRH group

All the participants from the HRH group had at least one positive item in the SNNOOP10 list, having a minimum of 2 and maximum of 8 items. On the other hand, in the standard group, all patients had at least one positive item, as well, the minimum being 1 and the maximum 8. The mean number of positive SNNOOP10 items in the standard group was 4.0 (1.8), compared with 4.9 (1.4) in the HRH group (p = 0.009). Frequency of items 3 (neurological deficit), 5 (older age), and 13 (posttraumatic onset) was higher in the HRH group. Supplementary Table 3 shows the frequency of each SNNOOP10 item between the standard group and the high-risk headache group.

Presence of SNNOOP10 items in the ICHD-3 groups

The group with a higher mean number of items was cranial autonomic cephalalgias, and the lower number was tension-type headache group. Supplementary Figure 2 shows the mean number of SNNOOP10 items per ICHD-3 group. Table 2 shows the frequency of each SNNOOP10 item per ICHD-3 group.

Diagnostic accuracy of the SNNOOP10 items

In the detection of HRH, the most sensitive items were neurologic deficit or dysfunction, progressive headache, change in the pattern and older age. The most specific items were posttraumatic onset, immune-compromised state, history of cancer, and systemic symptoms. Table 3 shows the sensitivity (S), specificity (E), positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR) of each SNNOOP10 item. The item with the higher AUC was number 5, age of onset >50 years, with AUC: 0.67. Most of the items were close to AUC: 0.5. Figure 2 shows the receiver operating characteristic curves of each of the different SNNOOP10 items. Table 4 depicts the values of the area under the curve in the receiver operating characteristic model. The combination of all the SNNOOP10 variables showed an area under the curve of 0.66 (95% CI: 0.55–0.76), with a standard error of 0.054 (p = 0.006), showed in Figure 3.

Table 3.

Diagnostic accuracy of SNNOOP10 items.

Item	Se	Sp	PPV	NPV	PLR	NLR
S1 Systemic symptoms including fever (n = 13)	15.5% (6.9–30.0)	89.0% (77.0–95.4)	53.8% (26.1–79.6)	56.3% (45.3–66.8)	1.42 (0.51–3.94)	0.94 (0.83–1.07)
S2 Neoplasm in history (n = 15)	20.0% (10.0–35.0)	89.1% (77.0–95.4)	60.0% (32.9–82.5)	57.6% (46.5–68.1)	1.83 (0.70–4.76)	0.89 (0.77–1.04)
S3 Neurologic deficit or dysfunction (n = 63)	75.5% (60.1–86.6)	47.2% (33.8–61.1)	53.9% (41.0–66.4)	70.0% (52.8–83.5)	1.43 (1.06–1.93)	0.52 (0.29–0.89)
S4 Onset of headache is sudden or abrupt (n = 46)	53.3% (38.0–68.1)	56.3% (42.4–69.4)	50.0% (35.4–64.5)	50.0% (35.4–64.5)	1.22 (0.81–1.83)	.083 (0.59–1.16)
S5 Older age (after 50 years) (n = 43)	64.4% (48.7–77.7)	72.7% (58.8–83.4)	65.9% (49.9–79.1)	71.1% (57.5–82.3)	2.36 (1.45–3.83)	0.49 (0.32–073)
S6 Pattern change or recent onset of headache (n = 61)	64.4% (48.7–77.7)	40.0% (27.3–54.1)	46.7% (34.1–71.3)	57.8% (40.1–73.3)	1.07 (0.79–1.45)	0.88 (0.57–1.39)
S7 Positional headache (n = 45)	44.4% (29.9–59.8)	54.5% (40.6–64.8)	44.4% (29.9–59.8)	54.5% (40.6–67.8)	0.97 (0.63–1.51)	1.01 (0.76–1.36)
S8 Precipitated by sneezing, coughing, or exercise (n = 38)	37.7% (24.1–53.4)	61.8% (47.7–74.2)	44.7% (28.9–61.5)	54.8% (41.7–67.3)	0.98 (0.59–1.63)	1.01 (0.51–1.27)
S10 Progressive headache and atypical presentations (n = 65)	66.7% (50.9–75.6)	32.7% (21.0–46.8)	44.7% (32.7–57.3)	54.5% (36.9–71.1)	0.99 (0.75–1.30)	1.01 (0.62–1.65)
S12 Painful eye with autonomic features	8.8% (2.8–22.1)	76.4% (62.7–86.3)	23.5% (7.8–50.2)	50.6% (39.5–61.6)	0.37 (0.13–1.07)	1.19 (1.07–1.32)
S13 Posttraumatic onset of headache (n = 13)	22.2% (11.7–37.4)	94.5% (83.9–98.6)	76.9% (45.9–93.8)	59.7% (48.7–69.9)	4.07 (1.19–13.91)	0.82 (0.70–0.96)
S14 Pathology of the immune system (n = 17)	17.8% (8.5–32.6)	83.6% (70.7–91.7)	47.0% (23.8–71.4)	55.4% (44.1–66.2)	1.08 (0.45–2.58)	0.98 (0.85–1.13)
S15 Medication overuse headache (n = 8)	0% (0–9.6)	85.2% (72.3–92.9)	0% (0–40.2)	50% (29.5–60.5)	0 (0–0)	1.00 (0.80–1.25)

NLR: Negative Likelihood ratio; NPV: Negative Predictive Value; PLR: Positive Likelihood Ratio; PPV: Positive Predictive Value; Se: Sensitivity; Sp: Specificity.

Figure 2.

Receiver Operating Characteristics Curve of each SNNOOP10 item.

Table 4.

Area under the curve of each SNNOOP10 item.

Item	Area	Standard error	Sign.	Lower bound	Upper bound
S1 Systemic symptoms including fever (n = 13)	0.521	0.058	0.724	0.406	0.635
S2 Neoplasm in history (n = 15)	0.542	0.058	0.468	0.428	0.656
S3 Neurologic deficit or dysfunction (n = 63)	0.621	0.056	0.037	0.511	0.731
S4 Onset of headache is sudden or abrupt (n = 46)	0.539	0.058	0.507	0.425	0.652
S5 Older age (after 50 years) (n = 43)	0.676	0.055	0.002	0.569	0.783
S6 Pattern change or recent onset of headache (n = 61)	0.53	0.058	0.609	0.416	0.644
S7 Positional headache (n = 45)	0.506	0.058	0.917	0.392	0.62
S8 Precipitated by sneezing. coughing. or exercise (n = 38)	0.51	0.058	0.857	0.396	0.625
S9 Papilledema	0.5	0.058	1.000	0.386	0.614
S10 Progressive headache and atypical presentations (n = 65)	0.504	0.058	0.95	0.39	0.618
S11 Pregnancy or puerperium (n = 0)	0.5	0.058	1.000	0.386	0.614
S12 Painful eye with autonomic features	0.423	0.057	0.187	0.311	0.535
S13 Posttraumatic onset of headache (n = 13)	0.581	0.058	0.164	0.467	0.695
S14 Pathology of the immune system (n = 17)	0.524	0.058	0.683	0.41	0.638
S15 Medication overuse headache (n = 8)	0.426	0.057	0.203	0.314	0.538

Figure 3.

Receiver Operator Characteristics Curve of the pooled SNNOOP10 list.

Determinants of high-risk headaches

The items that significantly showed association with HRH were older age, posttraumatic onset, neurologic deficit or disfunction, and neoplasm in history. Painful eye and cranial autonomic symptoms had a lower odd of associating HRH. Supplementary Table 4 shows univariate logistic regression model of SNNOOP10 items in the prediction of high-risk headache. In the multivariate analysis, only older age and neurologic deficit remained statistically significant (Table 5).

Table 5.

Multivariate logistic regression model of SNNOOP10 items in the determination of high-risk headache.

Item	OR	95% CI	p-value
S2 Neoplasm in history (n = 15)	1.76	0.50–6.15	0.37
S3 Neurologic deficit or dysfunction (n = 63)	2.71	1.04–7.05	0.041
S4 Onset of headache is sudden or abrupt (n = 46)	1.38	0.56–3.95	0.47
S5 Older age (after 50 years) (n = 43)	2.82	1.12–7.12	0.028
S12 Painful eye with autonomic features	0.38	0.10–1.467	0.161
S13 Posttraumatic onset of headache (n = 13)	3.62	0.83–15.68	0.086
S2 Neoplasm in history (n = 15)	1.76	0.50–6.15	0.37

Discussion

In this study, we tested if high-risk headaches were adequately detected by using the SNNOOP10 red and orange flag list in patients with high probability of having a secondary headache disorder. For this purpose, a series of patients were evaluated during their ED stay and prospectively followed-up afterwards by neurologists with expertise on headache medicine. The main findings of this study are that all the patients that visited the ED because of headache and a high-risk headache, were detected by using the SNNOOP10 list. The sensitivity of the combination of all the red and orange SNNOOP10 flags was 100%. However, most of the red flags also showed a low specificity. Still, diagnostic tests owing to detect a potentially life-threatening condition must be more sensitive than specific (27,28), and this study shows the challenges of diagnosing headache patients based on the history of the patients alone. In order to evaluate sensitivity, we selected patients based on the presence of some red flags after being evaluated in the ED triage area. This could represent a selection bias, as the frequency of secondary headache patients was 65%, much higher than the 2–18% described in other studies that evaluated the general epidemiology of headache in the ED (6 –10,29). This could influence the positive predictive value and negative predictive value, yet specificity seemed equally low. The potential reasons for the low specificity of the items are diverse. First, the number of secondary headache disorders is profuse. ICHD-3 lists 118 different secondary causes of headache (25). Second, the same red flag might lead to many different causes of headache. A systematic review found 119 different causes of thunderclap headache published in literature up to 2014 (30). Third, a single disorder can present with different phenotypes. In the case of Cerebral Venous Sinus Thrombosis (CVST), headache is not constant, as 12% of the patients do not experience it over the course of the disease (31 –33), whereas in 22-45% of the cases, it is the only neurological symptom (34,35).

Not every secondary headache disorder is equally menacing (36,37). In the present study we focused on high-risk headaches, a group which also include some primary headache disorders that might mimic a secondary cause. These patients should be managed as a potential secondary headache sufferers, and only an after appropriate study, these headaches can be classified as primary headache disorders, such as primary thunderclap headache or atypical aura (25). In the present study we tested the sensitivity of the SNNOOP10 list in a selected population with a higher probability of having a secondary headache. The higher the sensitivity, the higher the negative predictive value. However, future studies should also evaluate the specificity of the different red flags, to see its applicability in epidemiological studies and other settings.

In this study we observed that the systematic use of red and orange flags adequately detected all patients with secondary headache and HRH patients. The implementation of red flag lists might be facilitated by the use of abbreviated proposal, as the SNNOOP10 criteria (22). In the case of other standardised triage systems, such as Andorran-Spanish triage system and Manchester Triage System (37), the degree of expertise of the nurse influenced the assigned classification, due to the need of a basic neurological examination, including the assessment of meningeal irritation signs.

Studies analysing the true diagnostic accuracy of diagnostic items, such as the SNNOOP10 list, should anticipate all these factors. In our study, we found 37 different diagnoses among 100 patients. Second, frequency of secondary diagnosis per each different SNNOOP10 criterion might be highly variable. Studies that analysed frequency of subarachnoid hemorrhage (SAH) in all the patients with thunderclap onset of the headache vary between 6.2% and 17% (38 –40). Therefore, the sample size estimation must anticipate a high number of different secondary headache disorders, so the needed sample could be remarkable to ensure that the primary endpoint can be completed. Other red flag validation studies have included up to 589 patients (29). Our study was clearly underpowered for many of the secondary endpoints, as comparing the frequency of red flags within the different categories, being many of the analyses not statistically significant. Each study should be designed to answer a single question and in all the exploratory variables, the absence of evidence is not evidence of absence (41).

In our sample, most of the items from SNNOOP10 list seemed to be more frequent in the secondary or HRH groups, however in some cases, such as pattern change, progressive worsening or precipitation by sneezing, frequency seems to be equal. Diagnosis of headache is still based on history and examination (42), and some patients might not fully understand some questions or misinterpret them. The main reasons for visiting the ED because of headache are usually intensity, treatment resistance and fear about having a severe condition (43).

In line with that, many primary headache patients might have atypical characteristics, or even red flags (44,45), in particular in the ED setting (46). Migraine headache might present with transient neurological symptoms (47), sudden onset (48), onset after 50 years (49), change on the pattern (43), worsening by sneezing, coughing or exercising (45), cranial autonomic symptoms (50). However, by definition, primary headache disorder diagnosis can be done only once the headache is not better accounted for by any other headache disorder (25). In the case of primary headaches, the use of red flags may be complemented by the evaluation of green flags, defined as signs or verbal cues pointing towards a primary headache diagnosis. Recently, a list of five green flags was proposed, with some of these may be particularly relevant in the ED setting, as the onset of the headache during childhood, the presence of close family members with the same headache phenotype and the cessation of the headache more than one-week ago (51). Nonetheless, the presence of any red or orange flag, obligates the clinician to rule out a secondary headache. To complete the equation, further studies should evaluate the negative predictive value of the SNNOOP10 list should in non-selected samples.

The present study has several flaws. First, it is a single centre study, so the studied population might not be representative of other settings. Spanish healthcare system is public and subsidised by government, which could affect the profile of patients that seek attention in the ED, however, access to medical attention is free and patients do not need to be referred from primary care. Second, the sample size is limited. We did not calculate sample size in advance and given the vast number of different secondary and high-risk headache disorders, to have a representative number of different entities, the number of included patients should ensure a sufficient proportion of each of the main secondary headache disorders, based on their prevalence in the ED and the desired precision or margin of error, which should be calculated based on the specific study primary endpoint. Third, the study was designed to evaluate the sensitivity of the SNNOOP10 list in the detection of patients with HRH, so most of the secondary endpoints should be interpreted cautiously, including the frequency of red flags in patients with primary headaches and the PPV and NPV, which could be affected by the prevalence of secondary headaches in the sample. Fourth, validation of SNNOOP10 list was not done and items were verbally administered, which may decrease but do not exclude the possibility of misinterpretation, future studies may validate it when administered in paper or digitally. Some causes of secondary headache were underrepresented due to the eligibility criteria, which excluded patients with cranial or cervical trauma and patients with altered level of consciousness, altered mental status or speech disorders. Given our study design, only patients that visited the ED during the shifts of the research team were evaluated for eligibility, however, shifts occurred in a random basis, which could minimise the risk of selection bias.

Based on the present findings, we suggest that future studies should investigate the relative weight of each red flag, if all of them are equally valid or some of them should be re-considered. It should be evaluated if the red flags lists can be self-administered by patients, even at home. Finally, a prospective study with two arms could be done in order to compare if the use of SNNOOP10 list impacts the prognosis of patients and modifies endpoints associated with ED management of patients, in terms of number of imaging studies done, percentage of diagnosed patients, ED stay duration and percentage of neurology consults.

Conclusion

In our sample, the SNNOOP10 list showed a 100% sensitivity in the detection of high-risk headaches in patients seeking for medical attention in the emergency department due to headache. The vast number of causes of secondary headache, and the absence of items with perfect sensitivity and specificity suggests that the SNNOOP10 list should be further validated and implemented in clinical practice by the combination of all its items.

Clinical implications

Headache is a leading reason for presenting to the emergency department. Detection of severe causes is based on red flags, which are insufficiently studied.

We evaluated for the first time the SNNOOP10 list, a proposal from the Secondary Headache Special Interest Group of the International Headache Society.

All patients with a serious cause of headache had at least one ref flag included in the SNNOOP10 list, suggesting its potential usefulness.

Until the development of headache-specific biomarkers, detection of high-risk headaches can be done by the evaluation of red flags, being the SNNOOP10 list a possible approach with high sensitivity.

Supplemental Material

sj-pdf-1-cep-10.1177_03331024221120249 - Supplemental material for Sensitivity of the SNNOOP10 list in the high-risk secondary headache detection

Supplemental material, sj-pdf-1-cep-10.1177_03331024221120249 for Sensitivity of the SNNOOP10 list in the high-risk secondary headache detection by David García-Azorín, Jaime Abelaira-Freire, Nuria González-García, Esther Rodriguez-Adrada, Henrik Winther Schytz, Mads Barloese, Ángel Luis Guerrero, Jesús Porta-Etessam and Francisco Javier Martín-Sánchez in Cephalalgia

Footnotes

Acknowledgements

The authors thank the Emergency Department crew for their great support in the study. The statistical analysis was conducted by DGA.

Data Availability Statement

All data obtained and/or analysed are available from the authors upon reasonable request.

Declaration of conflicting interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: DGA, HWS are members of the Secondary Headache Special Interest Group of the International Headache Society. Authors declare no other conflicts of interest or disclosures relevant to the manuscript.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

David García-Azorín

Henrik Winther Schytz

Mads Barloese

Ángel Luis Guerrero

Jesús Porta-Etessam

References

GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018; 392: 1789–1858.

Burch

Rizzoli

Loder

The prevalence and impact of migraine and severe headache in the United States. Figures and trends from government health studies. Headache 2018; 58: 496–505.

Burch

Loder

, et al. The prevalence and burden of migraine and severe headache in the United States. Updated statistics from government health surveillance studies. Headache 2015; 55: 21–34.

Smitherman

Burch

Sheikh

, et al. The prevalence, impact, and treatment of migraine and severe headaches in the United States: a review of statistics from national surveillance studies. Headache 2013; 53: 427–436.

Giamberardino

Affiaitati

Costantini

, et al. Acute headache management in emergency department. A narrative review. Intern Emerg Med 2020; 15: 109–117.

Goldstein

Camargo

Jr Pelletier

, et al. Headache in United States emergency departments: demographics, work-up and frequency of pathological diagnoses. Cephalalgia 2006; 26: 684–690.

Davidsen

Gulbrandsen

, et al. Acute headache: a prospective diagnostic work-up of patients admitted to a general hospital. Eur J Neurology 2008; 15: 1293–1299.

Sahai-Srivastava

Desai

Zheng

Analysis of headache management in a busy emergency room in the United States. Headache 2008; 48: 931–938.

Dermitzakis

Georgiadis

Rudolf

, et al. Headache patients in the emergency department of a Greek tertiary care hospital. J Headache Pain 2010; 11: 123–128.

10.

Munoz-Ceron

Marin-Careaga

Peña

, et al. Headache at the emergency room: Etiologies, diagnostic usefulness of the ICHD 3 criteria, red and green flags. PLoS ONE 2019; 14: e0208728.

11.

García-Azorín

Abelaira-Freire

, Rodriguez-Adradaet al. Study about the Manchester Triage System subtriage in patients that visited the Emergency Department due to Headache. Neurología 2020; S0213–4853.

12.

Stegmayr

Eriksson

Aspllund

Declining mortality from subarachnoid hemorrhage: changes in incidence and case fatality from 1985 through 2000. Stroke 2004; 35: 2059–2063.

13.

Sacco

Marini

Toni

, et al. Incidence and 10-year survival of intracerebral hemorrhage in a population-based registry. Stroke 2009; 40: 394–399.

14.

Thigpen

Whitney

Messonnier

, et al. Bacterial meningitis in the United States, 1998–2007. N Eng J Med 2011; 364: 2016–2025.

15.

Nye

Ward

TN.

Clinic and emergency room evaluation and testing of headache. Headache 2015; 55: 1301–1308.

16.

Barton

CW.

Evaluation and treatment of headache patients in the emergency department: A survey.

Headache1994; 34: 91–94.

17.

Dodick

DW.

Diagnosing secondary and primary headache disorders. Continuum (Minneap Minn) 2021; 27: 572–585.

18.

la Cour Karottki

Ashina

Updates in the diagnostic approach of headache. Curr Pain Headache Rep 2021; 25: 80.

19.

Maizels

Headache evaluation and treatment by primary care physicians in an emergency department in the era of triptans. Arch Intern Med 2001; 161: 1969–1973.

20.

Relja

Granato

Capozzoli

, et al. Nontraumatic headache in the Emergency Department: a survey in the province of Trieste. J Headache Pain 2005; 6: 290–300.

21.

Friedman

Hochberg

Esses

, et al. Applying the International Classification of Headache Disorders to the emergency department: An assessment of reproducibility and the frequency with which unique diagnosis can be assigned to every acute headache presentation. Ann Emerg Med 2007; 49: 409–419.

22.

Remmers

Schytz

, et al. Red and orange flags for secondary headaches in clinical practice. SNNOOP10 list. Neurology 2019; 92: 134–144.

23.

Bossuyt

Reitsma

Bruns

, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015; 351: h5527.

24.

Headache Classification Committee of the International Headache Society (IHS). The International Classification of Headache Disorders, 3rd edition (beta version). Cephalalgia 2013; 33: 629–808.

25.

Headache Classification Committee of the International Headache Society (IHS). The International Classification of Headache Disorders, 3rd edition. Cephalalgia 2018; 38: 1–211.

26.

Good clinical practice: consolidate guideline. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) adopts Consolidated Guideline on Good Clinical Practice in the Conduct of Clinical Trials on Medicinal Products for Human Use. Int Dig Health Legis 1997; 48: 231–234.

27.

Campbell

Klugar

Ding

, et al. Diagnostic test accuracy: methods for systematic review and meta-analysis. Int J Evid Based Healthc 2015; 13: 154–162.

28.

Chassé

Fergusson

DA.

Diagnostic accuracy studies. Semin Nucl Med 2019; 49: 87–93.

29.

Locker

Thompson

Rylance

, et al. The utility of clinical features in patients presenting with nontraumatic headache: an investigation of adult patients attending an emergency department. Headache 2006; 46: 954–961.

30.

Devenney

Neale

Forbes

RB.

A systematic review of causes of sudden and severe headache (Thunderclap Headache): Should lists be evidence based?

J Headache Pain 2014; 15: 49.

31.

Sparaco

Feleppa

Bigal

ME.

Cerebral venous thrombosis and headache – a case series. Headache 2015; 55: 806–814.

32.

Damak

Crassard

Wolff

, et al. Isolated lateral sinus thrombosis. a series of 62 patients. Stroke 2009; 40: 476–481.

33.

Wasay

Kojan

Dai

, et al. Headache in cerebral venous thrombosis: Incidence, pattern and location in 200 consecutive patients. J Headache Pain 2010; 11: 137–139.

34.

Cumurciuc

Crassard

Sarov

, et al. Headache as the only neurological sign of cerebral venous thrombosis: a series of 17 cases. J Neurol Neurosurg Psychiatry 2005; 76: 1084–1087.

35.

Chou

DE.

Secondary headache syndromes. Continuum (Minneap Minn) 2018; 24: 1179–1191.

36.

Friedman

Lipton

RB.

Headache emergencies: diagnosis and management. Neurol Clin 2012; 30: 43–59.

37.

Gomez-Angelats

Miró

Bragulat Baur

, et al. Triage level assignment and nurse characteristics and experience. Emergencias 2018; 30: 163–168.

38.

Landtblom

Fridriksson

Boivie

, et al. Sudden onset headache: a prospective study of features, incidence and causes. Cephalalgia 2012; 22: 354–360.

39.

Morgestern

Luna-Gonzales

Huber

, et al. Worst headache and subarachnoid hemorrhage: prospective, modern computed tomography and fluid analysis. Ann Emerg Med 1998; 32: 297–304.

40.

Perry

Stiell

Silviotti

MLA

, et al. Clinical decision rules to rule out subarachnoid hemorrhage for acute headache. JAMA 2013; 310: 1248–1255.

41.

Altman

Bland

JM.

Statistics notes: Absence of evidence is not evidence of absence. BMJ 1995; 311: 485.

42.

Godwin

Cherkas

Panagos

, et al. Clinical policy: critical issues in the evaluation and management of adult patients presenting to the emergency department with acute headache. Ann Emerg Med 2019; 74: e41–e74.

43.

Minen

Loder

Friedman

Factors associated with emergency department visits for migraine: an observational study. Headache 2014; 54: 1611–1618.

44.

Long

Koyfman

Benign headache management in the emergency department. J Emerg Med 2018; 54: 548–468.

45.

Kelman

Pain characteristics of the acute migraine attack. Headache 2006; 46: 942–953.

46.

Zodda

Procopio

Gupta

Evaluation and management of life-threatening headaches in the emergency department. Emerg Med Pract 2019; 21: 1–20.

47.

Eriksen

Thomsen

Olesen

Implication of clinical subtypes of migraine with aura. Headache 2006; 46: 286–297.

48.

García-Azorín

González-García

Abelaira-Freire

, et al. Management of thunderclap headache in the emergency room: A retrospective cohort study. Cephalalgia 2021; 41: 711–720.

49.

Kelman

Migraine changes with age: impact on migraine classification. Headache 2006; 46: 1161–1171.

50.

Barbanti

Aurilia

Dall’Armi

, et al. The phenotype of migraine with unilateral cranial autonomic symptoms documents increased peripheral and central sensitization. A case series of 757 patients. Cephalalgia 2016; 36: 1334–1340.

51.

Pohl

García-Azorín

, et al. Green Flags and headache: A concept study using the Delphi method. Headache 2021; 61: 300–309.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.74 MB