Abstract
Background and objective:
We aimed to validate the diagnostic accuracy of appendicitis, its severity, its description, and the surgical approach, including open or laparoscopic appendectomy and diagnostic laparoscopy, in the Danish National Patient Registry (DNPR) against information from the electronic medical records.
Methods:
A random sample of 1046 patients of all ages and sexes recorded in the DNPR from the Capital Region of Denmark during 2010–2015 was investigated. Patients’ admission had to include a discharge code for appendicitis (K35−K379) according to the International Classification of Disease version 10 (ICD-10) alone or in combination with a surgical code for appendectomy or the surgical code for a diagnostic laparoscopy. We calculated the positive predictive values (PPVs) with 95% confidence intervals.
Results:
Data from a total of 1018 patients were available for data analysis. The ICD-10 codes for appendicitis resulted in a good PPV of >95% when combined with a surgical code. ICD-10 codes combined with the surgical codes could discriminate between uncomplicated and complicated appendicitis with some caution as the PPVs ranged from 68% to 93%. Only the surgical code for laparoscopic appendectomy yielded a good PPV (99%) for the actual surgical approach. The surgical code for a diagnostic laparoscopy alone did not cover a normal diagnostic laparoscopy.
Conclusions:
The diagnostic accuracy of appendicitis and appendectomy in routinely collected administrative health data is good for ICD-10 codes for appendicitis in combination with a surgical code and especially for laparoscopic appendectomy. Uncomplicated and complicated appendicitis can be distinguished with some caution.
Summary
High diagnostic accuracy of appendicitis and laparoscopic appendectomy in Danish National Patient Registry. Uncomplicated and complicated appendicitis also can be identified though with some caution.
Context and Relevance
This original validation study of the codes for appendicitis, appendectomy, and diagnostic laparoscopy against the operative report included 1018 Danish patients of all ages and both sexes. Appendicitis and laparoscopic appendectomy could be accurately identified through the registry. Also, patients treated for uncomplicated and complicated appendicitis were possible to discriminate. The findings are highly important for future register-based studies utilizing data from Nordic health registers.
Introduction
In the 21st century, the incidence of appendicitis is 120 per 100,000 person-years in Northern Europe. 1 Thus, appendectomy for appendicitis is one of the most frequently performed acute surgical procedures. In the Danish National Patient Registry (DNPR), 2 admissions of patients suffering from appendicitis can be identified by either diagnostic codes according to the International Classification of Disease version 10 (ICD-10) codes 3 in the Danish version 4 or surgical codes according to the Nordic Medico-Statistical Committee (NOMESCO) Classification of Surgical Procedures. 5 Register-based studies are important for the investigation of outcomes after admissions for diseases and surgeries; however, the validity of the results relies on the quality of the registry coding. 6 Validation studies of the codes for appendicitis and open appendectomy have been previously evaluated in the Danish setting; however, these validation studies were limited either by outdated surgical coding practice 7 or by a lack of validation of the specific ICD-10 codes’ texts. 8 The Danish version of the ICD-10 codes is used in the DNPR 9 and includes the visual appearance of the appendix and intra-abdominal findings, for example, gangrenous appendicitis (K358B) or appendicitis with generalized peritonitis (K352). 4 A recent nationwide study validated the ICD-10 codes and surgical codes for appendicitis and appendectomy against the histopathology reports, but no information was retrieved from the medical records and operative reports. 10 Hence, this study does not reflect the surgeon’s intraoperative grading of appendicitis. 10 Furthermore, no study has validated a normal or negative diagnostic laparoscopy, that is, when appendicitis is suspected but not found during a diagnostic laparoscopy. If a normal appendix is surgically removed, it may be referred to as a negative appendectomy. Normal appendices are generally left in situ in Denmark when found during laparoscopy. 11 A nationwide, register-based, Swedish study found patients undergoing a negative appendectomy to have a higher short- and long-term risk of mortality than patients undergoing open or laparoscopic appendectomy for uncomplicated appendicitis. 12 Therefore, it is relevant for Public Health to identify both these patients and those with different severity of appendicitis for Danish register-based studies to explore this further.
The study aimed to validate the diagnostic and/or surgical codes for (1) appendicitis, (2) the severity of appendicitis, (3) the ICD-10 code text describing appendicitis, and (4) the surgical approach, including open appendectomy, laparoscopic appendectomy, and normal diagnostic laparoscopy, in the DNPR against the electronic medical records (gold standard) to facilitate disease and population identification in future register-based studies.
Methods
Protocol, reporting, and approvals
A protocol including all planned tables was written in advance. The study was reported according to the reporting guideline for assessing the quality of validation studies of health administrative data. 13
The study was approved by the Danish Data Protection Agency (HGH-2016-105 and P-2020-761), by the Danish Patient Safety Authority (3-3013-2134/1 and 3-3013-1960/1), and by each head of department from all involved hospital departments (n = 30) in the Capital Region of Denmark. According to Danish legislation, no ethical committee approval was required, nor informed consent from patients.
Patients
In DNPR, 2 which has national coverage, all registrations include the unique civil registration number of the patients, the ICD-10 codes 3 for the discharge diagnoses, the dates of admission and discharge, and the surgical codes according to NOMESCO, 5 if surgery was performed. For this study, the Danish Health Data Authority extracted a random sample of unique patients of all ages and sexes. The patients were eligible if they were recorded in the DNPR with an admission to a hospital in the Capital Region of Denmark between 1 January 2010 and 31 December 2015, with the ICD-10 codes 3 and/or surgical codes (NOMESCO) 5 described in detail in the following sections and in Supplementary File Table 1.
Appendicitis in DNPR range from a two-digit ICD-10 code level (e.g. K35) to a four-digit ICD-10 code level (e.g. K358C) 4 and some ICD-10 codes are more frequently used than others. Six different criteria for discharge diagnosis and/or surgical codes were applied for the inclusion of patients. The first two criteria were designed to capture the typical patient admitted for appendicitis. Thus, patients had to be recorded with an ICD-10 code for appendicitis (K35−K358C or K37−K379) in the DNPR, where only the two-digit ICD-10 code level was used; thus, any characters after the two first digits were truncated (K35* and K37*). This resulted in 200 patients with the ICD-10 codes (K35* or K37*), and 200 patients with both ICD-10 codes (K35* or K37*) and surgical appendectomy codes (JEA00 or JEA01) (see Supplementary File Table S1). For the third and fourth criteria, up to 25 patients coded with at least one of the 11 specific ICD-10 codes in DNPR for different types of appendicitis (K352 generalized peritonitis, K353 localized peritonitis, etc.) and up to 25 patients coded with both ICD-10 and surgical appendectomy codes (JEA00 or JEA01) (see Supplementary File Table S1). The fifth criterion was designed to capture up to 25 patients undergoing surgery coded with JEA10 (appendectomy with drainage) in the DNPR, which was rarely used. The sixth criterion captured 100 patients undergoing surgery coded with JAH01 (diagnostic laparoscopy) in DNPR. For both the fifth and sixth criteria, there was no specific ICD-10 code required. In total, this resulted in 1046 patients (Supplementary File Table S1). All patients identified above had to be unique; thus, no patient could be selected more than once.
Categories and data collection from the operative report
Four different validations for DNPR data were planned: first, the overall two-digit ICD-10 code level (K35*or K37*) implying appendicitis (yes/no); second, the specific two- to four-digit ICD-10 code level implying the severity of appendicitis that included uncomplicated appendicitis (yes/no) and complicated appendicitis (yes/no); and third, the specific ICD-10 code text of the two- to four-digit ICD-10 code level and its consistency with the description of the appendix and/or the intra-abdominal findings (e.g. K352 generalized peritonitis: yes, generalized peritonitis/no; K353 localized peritonitis: yes, localized peritonitis/no). For these three validations, two different analyses were conducted: one analysis where only the ICD-10 codes were mandatory, and another analysis where both ICD-10 and the surgical codes were mandatory. The final and fourth validation was the surgical approach based on surgical codes. In this validation, the surgical codes included JEA00 (open) appendectomy (yes/no), JEA01 laparoscopic appendectomy (yes/no), JEA10 appendectomy with drainage (yes/no), and JAH01 diagnostic laparoscopy (yes, normal diagnostic laparoscopy no).
We compared the discharge ICD-10 codes and surgical codes registered in the DNPR with extracted data from the gold standard that was preferably the operative report or another part of the medical record (e.g. admission or discharge report) or during the relevant admission. No data from the histopathology report were collected. Data were extracted and typed into a predefined Excel sheet twice by a medical doctor, who had previously worked in general surgical departments. The data extraction from the electronic medical record was performed blinded to the recorded ICD-10 and surgical codes in the DNPR. The definitions used to categorize data were based on a previous publication 14 and are specified in the outcome tables. Any doubt or conflicts regarding categorization during data extraction were solved within the author group. No revalidation was planned.
Statistical methods
Categorical variables were reported by numbers and proportions in percentage, and continuous variables were reported by median, range, and interquartile range (IQR), if not normally distributed. Accuracy was measured by the estimated positive predictive value (PPV) that was calculated as follows
For each PPV, the corresponding 95% confidence interval (CI) was calculated using the method for binomial proportions. Data were analyzed with SAS® Studio 3.8 (Enterprise Edition; SAS Institute Inc., Cary, NC, USA).
Results
Patients
In total, this validation study included 1046 patients (Fig. 1). For 28 patients, data could not be retrieved from the medical records; thus, these patients were not analyzed (Fig. 1). Table 1 reports the characteristics of the validation cohort. Most of the patients had been admitted to a department of general surgery (72%). The patients had a median age of 30 years (range 0–97 years, IQR 17–53 years) and sexes were almost equally distributed with 52% female patients. The number of patients analyzed for each validation below is presented in Fig. 1.

Overview of the total number of patients, the number of patients available for analyses, and the number of patients for each validation. ICD-10: International Classification of Disease version 103 Danish version. 4
Characteristics of the analyzed patients in the validation cohort presented as number (%) or median (range).
Including mixed surgical and medical departments; internal medicine departments (internal, geriatrics, hematology, infectious medicine, endocrinology, medical gastroenterology, acute medicine, cardiology, and pulmonary medicine and cardiology); anesthesia; urology; and not classified.
Two-digit ICD-10 code level for appendicitis
The results of the two-digit ICD-10 code level for appendicitis are presented in Table 2. For acute appendicitis (K35*), both the ICD-10 code with and without surgical codes yielded good PPVs of 95% and 98%, respectively. For unspecified appendicitis (K37*), only the ICD-10 code in combination with the surgical codes resulted in a good PPV of 99%.
Validation of the two-digit ICD-10 code level for appendicitis relying on either ICD-10 codes only or ICD-10 codes in combination with the surgical codes (JEA00 appendectomy or JEA01 laparoscopic appendectomy, Nordic Medico-Statistical Committee). 5
ICD-10: International Classification of Disease version 103 Danish version4; PPV: positive predictive value; CI: confidence interval.
“appendicitis yes” was defined as the surgeon describing either appendicitis present or intraoperative appearance of the appendix consistent with appendicitis. 14
“appendicitis no” was defined as the surgeon describing either a normal appendix with no visible changes 14 or another condition.
any characters after the two first digits of the ICD-10 code were truncated, thus a two-digit ICD-10 code level.
The severity of appendicitis
The PPV for uncomplicated appendicitis (K358, K358A, K358C, K37, and K379) ranged from 24% to 88% and 68% to 88% without and with surgical codes, respectively (Fig. 2). In general, all ICD-10 codes’ texts containing “unspecified” had the lowest PPVs for uncomplicated appendicitis ranging from 24% to 75% (Supplementary File Table S2). The PPVs for complicated appendicitis (K352, K353, and K358B) ranged from 78% to 93% (see Fig. 2).

Positive predictive values in percentage of the severity of appendicitis: uncomplicated (gray) or complicated (black) according to the definition of the ICD-10 code only (circle) or the ICD-10 code combined with a surgical code (square), respectively. The 95% confidence intervals are marked with error bars. On the x-axis, the two- to three-digit ICD-code levels are in bold and levels under these are in normal text. The surgical codes include JEA00 appendectomy or JEA01 laparoscopic appendectomy from Nordic Medico-Statistical Committee. 5 The exact data are presented in Supplementary File Table S2.
Specific ICD-10 code text
The results of the specific three- to four-digit ICD-10 code for the exact description of the appendix or operative findings are presented in Supplementary File Table S3, and the PPVs in Fig. 3. The specific ICD-10 code texts alone matched the majority of the description in the operative report for K353B periappendicular abscess, K358A inflamed appendix, and K358B gangrenous appendix with PPVs of 81%, 60%, and 71%, respectively. However, for the remaining specific ICD-10 code texts, the accordance with the operative report was poor. The PPVs did not improve; for example, the PPV and 95% CI did not become more precise when the specific ICD-10 code texts were combined with surgical codes (Fig. 3).

Positive predictive values in percentage of the description of appendicitis according to the definition of the ICD-10 code only (circle) and ICD-10 code combined with a surgical code (square), respectively, the 95% confidence intervals are marked with error bars. The surgical codes include JEA00 appendectomy or JEA01 laparoscopic appendectomy from Nordic Medico-Statistical Committee. 5 The exact data and definitions based on the operative report are presented in Supplementary File Table S3.
The surgical approach
Laparoscopic appendectomy, appendectomy, or drainage
The surgical code for laparoscopic appendectomy (JEA01) had a high PPV of 99% (95% CI, 97%–99%) (Table 3). The surgical code for both open appendectomy (JEA00) and appendectomy with drainage (JEA10) had lower PPVs of 51% and 56%, respectively (Table 3).
Validation of the specific surgical approach for appendectomy according to Nordic Medico-Statistical Committee 5 surgical codes for patients suffering from appendicitis, who were treated surgically.
The numbers in bold are the true positives. The numbers in italics are the number of patients classified as having both an appendectomy and an additional surgical procedure such as an umbilical hernia repair (n = 8), thus overlapping numbers that are not counted in the total column. PPV: positive predictive value; CI: confidence interval; ICD-10: International Classification of Disease version 103 Danish version. 4
Primary or converted procedure with an incision centered at McBurney’s point.
Any approach and incision with the addition of drainage.
Converted procedures and other incisions that are needed for a laparoscopic appendectomy or open appendectomy centered at McBurney’s point (e.g. laparotomy).
Diagnostic laparoscopy
The validation of the surgical code for diagnostic laparoscopy (JAH01) included a total of 153 patients (Fig. 1). In the operative report, a normal or negative diagnostic laparoscopy was only described for 23 patients, thus resulting in a poor PPV of 15% (Supplementary File Table S4). Instead, this surgical code covered converted (n = 46) and other procedures (n = 28). We explored whether the PPV improved with the addition of one or more criteria about the performing department, additional surgical codes, and specific discharge diagnosis (see Supplementary File Table S4). The PPV improved; however, the 95% CI was wider and a few true positives were missed.
Discussion
In this validation study, we found that the best way to identify patients with appendicitis in DNPR was by using a combination of the two-digit ICD-10 code level for appendicitis (K35* and K37*) in combination with the surgical codes for appendectomy (JEA00 or JEA01). This resulted in very good PPVs of 98%–99%. Discriminating between uncomplicated and complicated appendicitis was possible when combined with the surgical codes. The PPV for the surgical code for laparoscopic appendectomy was very good. A diagnostic laparoscopy covered various procedures and not a normal or negative diagnostic laparoscopy; however, PPV improved when more criteria were added.
We found a very good PPV for appendicitis of 98%–99% when the two-digit ICD-10 code levels K35* and K37* were combined with either of the surgical codes for appendectomy (JEA00 or JEA01). This is better than the reported PPV of 90 (96% CI, 89–90) found in a recent Danish validation study, 10 where the gold standard was the histopathology report and not the operative report as in our study. Decrease in PPV for appendicitis when the histopathology report versus the operative report was used was also seen in an American validation study of ICD-10 codes. 15 The choice of either of these two gold standards can be advocated for. The histopathology report is an excellent and rather objective gold standard if the actual and final diagnosis of the patient is sought. However, the operative report provides insight into the course of treatment chosen by the operating surgeon. In Denmark, the median length of stay is 16 h and 17 h for adults 16 and children, 17 respectively. Therefore, the histopathology report will not affect the treatment course as this is typically not completed until days after the operation when the patients have already been discharged. Another Danish validation study that also consulted medical records had a similar good PPV of 99% (95% CI, 98%–100%) as in our study. 9
In our validation study, we investigated the severity of appendicitis and the description of the operative findings in depth. Regarding the severity of appendicitis, uncomplicated and complicated appendicitis can be discriminated in Danish routinely collected administrative health data with some caution. The combination of specific two- to four-digit ICD-10 codes level and surgical codes (JEA00 or JEA01) resulted in PPVs for uncomplicated and complicated appendicitis ranging from 68% to 88% and 78% to 93%, respectively. This was similar to a reported PPV for the severity of appendicitis of 88% (95% CI, 82%–93%) from the Danish validation study of 262 medical records. 9 Unfortunately, this study did not specify if the validation was based on diagnostic ICD-10 codes alone or a combination of ICD-10 codes and surgical codes. Our results warrant that identifying patients in DNPR by diagnostic codes for appendicitis only should be done with some caution (Table 2), for example, when identifying patients with appendicitis treated with antibiotics only. Regarding the description of the operative findings, our study is the only Danish validation study that has investigated the specific ICD-10 code texts with the description in the operative report. We do not recommend these specific ICD-10 code texts to discriminate between the operative findings as most of these texts had rather poor PPVs and only a few resulted in a moderate PPV of >60%.
We found a very good PPV for laparoscopic appendectomy of 99% (95% CI, 97%–99%), which has become the dominating surgical approach for both adults 16 and children 17 in Denmark. This is in line with the results of two previous Danish validation studies on the surgical approach for appendectomy. The earliest Danish validation study of appendectomies performed in 1977–1988 reported a very good PPV for open appendectomy of 99%, which was the predominant surgical approach back then. 7 However, the validated surgical code in that study 7 was from an old coding system, the Danish Classification of Operations and Treatments, 18 and the validation was biased as it was conducted in a cohort consisting of women undergoing gynecological surgery. A second validation study conducted in Denmark, 1998–2007, also found a very good PPV of 99% (95% CI, 98%–100%); however, this applied to the correctness of either of the operative technique, thus either open or laparoscopic appendectomy and not the specific approach. 9
In our validation study, a diagnostic laparoscopy registered in DNPR covered various procedures and not only a normal or negative diagnostic laparoscopy (PPV 15% (95% CI, 10%–22%)). A Swedish nationwide register-based study also had difficulties identifying these patients undergoing a normal or negative diagnostic laparoscopy and ended up not analyzing these. 19 However, we argue that with the addition of more criteria, the PPV improved; thus, these patients can be identified although with some caution.
The strengths of the present validation study were that all categories were predefined in a protocol and that it was reported according to the relevant guideline. 13 This is the largest Danish validation of appendicitis and appendectomy based on the operative report and the first for diagnostic laparoscopy. We chose to conduct the study from a random sample to increase the generalizability of the validation. 20 Furthermore, the sample was derived from the largest Danish region and from all types of departments to ensure proper detection of PPV for both ICD-10 and surgical codes. The validation cohort consisted of patients of all ages and sexes to avoid bias, and we extracted data twice from the operative report. However, this validation study also has some limitations. Although more than 1000 patients were included in the validation cohort, each validation consequently had lower number of patients, resulting in somewhat broad 95% CIs. Furthermore, data from after 2016 were not collected, as the data quality from this timepoint and onwards for years was questioned due to the implementation of an electronic medical records system in the Capital Region of Denmark. However, Statistics Denmark has now assured that the challenges with data quality have been resolved 21 ; thus, register-based studies in DNPR can safely use the results of our validation study. We only planned on reporting one measure of diagnostic accuracy, the PPV, and not the negative predictive value, sensitivity, specificity, or likelihood ratios. These other measures of diagnostic accuracy could not be applied to our cohort due to the design of our study. Furthermore, our validation cohort does not have the same prevalence of appendicitis as a general cohort from DNPR 13 as the criteria set for the sample included ICD-10 codes and surgical codes for appendicitis and appendectomy, respectively. Furthermore, the planned validation of a diagnostic laparoscopy was inaccurate and post hoc criteria had to be applied.
In conclusion, both the two-digit ICD-10 code level for appendicitis (K35* and K37*) in combination with a surgical code and the surgical code JEA01 laparoscopic appendectomy resulted in very good PPVs and can be used for register-based studies using Danish routinely collected administrative health data through DNPR. Uncomplicated and complicated appendicitis can be discriminated with some caution and JAH01 diagnostic laparoscopy may be used to identify a normal or negative diagnostic laparoscopy if more criteria are added.
Supplemental Material
sj-docx-1-sjs-10.1177_14574969221148078 – Supplemental material for Validity of the coding for appendicitis, appendectomy, and diagnostic laparoscopy in the Danish National Patient Registry
Supplemental material, sj-docx-1-sjs-10.1177_14574969221148078 for Validity of the coding for appendicitis, appendectomy, and diagnostic laparoscopy in the Danish National Patient Registry by Siv Fonnes, Rune Erichsen and Jacob Rosenberg in Scandinavian Journal of Surgery
Footnotes
Author contributions
All authors made a substantial contribution to the concept or design of the work, and the acquisition, analysis, or interpretation of data. S.F. drafted the article, and R.E. and J.R. revised it critically for important intellectual content. All authors have read and approved the final article, and take public responsibility for it.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Data availability statement
Due to Danish legislation, data cannot be shared through a public data repository. For enquiries on data, please contact the corresponding author.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
