Longitudinal rates of hospital adverse events that contributed to death in Norway and Sweden from 2013 to 2018

Abstract

Objectives

In this paper, we explore and compare types and longitudinal trends of hospital adverse events in Norway and Sweden in the years 2013–2018 with special reference to AEs that contributed to death.

Design

Acute care hospitals in both countries performed medical record reviews on randomly selected medical records from all eligible admissions.

Analysis: Comparison between Norway and Sweden of linear trends from 2013–2018, and percentage rates of admissions with at least one AE according to types and severities.

Setting

Norway and Sweden have similar socio-economic and demographic characteristics, which constitutes a relevant context for cooperation, comparison and mutual learning. This setting has promoted the use of GTT to monitor national rates of AEs in hospital care in the two countries.

Participants

53 367 medical records in Norway and 88 637 medical records in Sweden were reviewed.

Results

13.2% of hospital admissions in Norway and 13.1% in Sweden were associated with an AE of all severities (E-I). 0.23% of hospital admissions in Norway and 0.26% in Sweden were associated with an AE that contributed to death (I). The differences between the two countries were not statistically significant.

Conclusions

There were no significant differences in overall rates (E-I) of AEs in Norway and Sweden, nor in rates of AEs that contributed to death (I). There was no significant change in AEs or fatal AEs in either country over the six-year time period.

Keywords

Adverse events global trigger tool measurement leadership mortality

Introduction

Patient safety efforts have increased since the Institute of Medicine (IOM) ascribed an annual incidence of 44 000–98 000 deaths to hospital adverse events in the US in their seminal report To err is human in 2000.¹ The estimates were reproduced in countries like Australia, New Zealand, Denmark, Canada, The Netherlands and Sweden.^2–7 Some studies and reports have claimed that the IOM report underestimated the true incidence of death caused by adverse events.^8,9 Other studies suggest the estimates have created an exaggerated attention to mortality from adverse events, relative to the small opportunities for improvement identified by such studies. One study found that most patients that died from preventable adverse events were very ill with a mean life expectancy estimated to six months.^10,11 However, 40% of deaths where patients had experienced problems in care were considered preventable with higher quality healthcare.¹⁰

Although efforts to address risks and low-quality care within healthcare systems have increased, few scientific studies have compared risks between healthcare systems. Such comparative studies have a potential to reveal unaddressed system causes of adverse events.^12,13 As a standardised measurement, the Global Trigger Tool (GTT) provides opportunity to study levels of harm across national healthcare systems.^14–18 Inter-rater reliability (IRR) of the GTT for identifying presence and severity of AEs is moderate to substantial between independent internal reviewer teams, and fair to moderate between independent external reviewer teams.^19,20 IRR improves significantly with practical training before the reviewing starts.²¹

The GTT method has been applied in all acute care hospitals in Norway since 2010 and in Sweden since 2013.^15,16 As far as we know, these were the first countries to compare national rates of AEs based on the results from the GTT method.¹⁷ In 2013 Norwegian hospitals had significantly higher rates of AEs related to surgery, while Swedish hospitals had significantly higher AE rates related to pressure ulcers, falls and ‘other’ AEs. In both countries AEs that contributed to death were estimated to be present at 0.3% of all admissions.¹⁷

In the current study, we retrospectively explore and compare how the rate and the distribution of AEs with special reference to AEs contributing to death vary over a six-year period (2013–2018) in the two countries.

Methods

The GTT method is an internationally recognised and standardised procedure for medical record review to identify and estimate rates and severity of AEs among adult patients in hospital admissions.¹⁸ Both Norway and Sweden use the GTT method to measure AEs in health care and the results are analysed on an annual basis.^15,16 In both countries the GTT method is based on randomly selected medical records reviewed by trained GTT teams consisting of nurses and physicians. For more details we refer to our previous study in 2013, and the handbook for description of the GTT record review method.^17,18 This study was designed to compare types and longitudinal trends of hospital adverse events of all severities and those that contributed to death in Norway and Sweden in the years 2013–2018.

Sampling

In both countries, data came from medical records reviewed from hospital admissions during the period January 2013 to December 2018. Eligible medical records were completed records from admissions of patients 18 years and above with in-hospital stays that lasted at least 24 hours. Admissions in psychiatric or rehabilitation departments were excluded. Lists of all eligible admissions were retrieved locally from the electronic administrative system in each hospital, and a random selection of the required number of records was done.

Classification and reporting of data

Identified AEs were classified according a 5-point severity scale (Table 1). An AE could only be categorised to one grade of severity. In this study, we analyse the compounded severity category E-I, which include all AEs, and the severity category I, which include AEs that contributed to patient’s death during the hospital stay or within 30 days after the patient was discharged.

Table 1.

Categories used to assess the severity of adverse events (AEs).

E	AE contributed to temporary harm to the patient which required intervention
F	AE contributed to temporary harm to the patient which required prolonged hospitalisation
G	AE contributed to permanent patient harm
H	Intervention was required to sustain life
I	AE contributed to patient death

AEs were also classified according to type, based on a list which Norway copied from Sweden in 2010, when the national monitoring in Norway started. In Sweden the national monitoring of AEs started in 2013, and a few more AE types were added, like for example sepsis, distended urinary bladder and clostridium difficile infection. In total, 23 types of AEs were specified in Norway and 27 in Sweden (Table 2). On the Norwegian and Swedish lists of AE types 19 categories were identical. We divided AE types into six groups based on both mutual and diverging categories for AEs between the countries (Table 2). When AE rates were compared, all six groups of AEs were analysed. When comparing AE type rates for severity category I, AEs classified as falls and pressure ulcers were excluded since the number of cases was very small. Norway had only five AEs with falls and one with pressure ulcer, while Sweden had 14 falls and one pressure ulcer. In Sweden AEs were also categorised as being preventable or not.¹⁶

Table 2.

Groups of AE types based on original lists of AE types from Sweden and Norway.

Groups of AE types	Original lists of AE types
Groups of AE types	Sweden	Norway
Hospital-acquired infection	Clostridium difficile infection	–
	Central line infection	Central line infection
	Infection other	Infection other
	Pneumonia	Pneumonia
	Postoperative wound infection	Postoperative wound infection
	Sepsis	–
	Urinary tract infection	Urinary tract infection
	Ventilator-related pneumonia	Ventilator-related pneumonia
Pressure ulcer	Pressure ulcer	Pressure ulcer
Adverse drug event	Adverse drug event	Adverse drug event (including blood and iv fluid therapy)
Surgical complication	Other surgical complication	Other surgical complication
	Wrong-side surgery	Wrong-side surgery
	Organ injury	Organ injury
	Postoperative haemorrhage /haematoma	Postoperative haemorrhage /haematoma
	Reoperation	Reoperation
	–	Postoperative respiratory AE
Falls	Falls	Falls
Falls	–	Fractures
Other	Allergic reaction	Allergic reaction
	Anaesthesia-related AE	–
	Haemorrhage – not surgery	Haemorrhage all types
	Skin or vessel	–
	Medical equipment	Medical equipment
	Postpartum AE	Postpartum AE
	Thrombosis/embolus	Thrombosis/embolus
	Other AE	Other AE (delayed diagnosis)
	–	Deterioration of chronic condition
	Organ failure	–
	Neurological AE	–
	Distended urinary bladder	–

When AEs were reported, one admission could have more than one AE, of different severities and types and if relevant, one AE could be categorised as more than one type. For example, one AE may be categorised with the types postoperative bleeding and reoperation. For this study, presented rates of admissions with AE types are therefore not mutually exclusive for AEs with severity category E-I. However, for analysis of severity category I, only one AE of one type was selected, based on a clinical interpretation when several types had been reported for the same AE. If an AE was reported both as a surgical complication and a hospital-acquired infection (Table 2), the AE was categorised as a hospital-acquired infection, assuming the surgical complication to be a postoperative infection.

GTT teams in both countries reported the total number of admissions that the investigated records had been randomly selected from. This was used for weighting the team results, when calculating national estimates. A database was constructed in Sweden, which also collected data on hospital type, medical specialty, patient’s gender, age, length of hospital stay, assessment of preventability of AE, if the patient had a surgical procedure during the admission, or if care had been provided “off-site”, i.e. in a unit not specialized for the patient’s medical condition.¹⁶

Settings

In Norway all acute care hospitals participated in the review from 2013 to 2018, with few exceptions. During the period some hospitals merged and, hence, the number of teams were reduced. Some hospitals also changed the number of teams during this period. Number of hospitals and GTT teams are accounted for in Table 3. The GTT teams selected at least 10 medical records bimonthly, which two trained registered nurses (RN) first reviewed separately. They then compared their results and validated them with a physician.

Table 3.

Number of hospitals and GTT teams that participated in the reviews from 2013 to 2018, and number of medical records reviewed in each country.

	N hospitals		N GTT teams		N medical records reviewed
Year	Norway	Sweden	Norway	Sweden	Norway	Sweden
2013	21	60	45	60	10 986	19 927
2014	23	60	36	60	8 804	18 629
2015	24	59	35	59	8 887	13 770
2016	24	57	34	57	8 390	12 590
2017	21	56	30	56	7 430	12 235
2018	24	55	33	55	8 870	11 486

In Sweden, 55–60 acute care hospitals participated from 2013 to 2018. Each hospital had its own review team. During 2013–2014 the minimum monthly number of randomly selected admissions reviewed was 40 for university hospitals, 30 for the central county council hospitals and 20 for the small hospitals. From 2015 onwards the number of reviewed records was reduced with 50%. A RN searched the records for triggers. Then a team with one or more RN and one physician, assessed records with positive triggers, in order to identify possible AEs and classify them.

Reviewers in both countries were employed in the hospitals where they did the review. All reviewers had clinical experience although some RN now worked in the quality department.

Statistical analysis

Annual AE rates were calculated per country as weighted averages of annual AE rates for all GTT teams participating in the review for the given year. Each team was assigned a weight equal to the number of eligible admissions that the teams investigated records had been randomly selected from, divided by the total number of eligible admissions for all teams, that investigated records had been randomly selected from. For each team, we added up all the annual data and assigned a weight reflecting the proportion of all eligible admissions for the six-year period that investigated records for the team had been randomly selected from. This was used to calculate weighted averages for national AE rates for all years combined (2013–2018). AEs divided into types and severities were not symmetrically distributed, so non-parametric confidence intervals were calculated using 10,000 bootstrap simulations.²² The probability of selecting an observation was equal to the weight of the team. Bootstrap for the national annual AE rates was performed by randomly sampling with replacement from the annual AE rates for all teams participating in the review for the given year. To reflect variation over time within GTT teams, bootstrap for the national AE rates for all years together was performed in two steps. Firstly, a team was randomly sampled with replacement and then it was sampled within the annual AE rates for the team. For all analyses, the significance level was set to 5%. The data preparation and calculations were performed using Microsoft Excel V.2016 and the R statistical software.²³

Ethics

The Ministry of Health and Care Services in Norway concluded that GTT could be applied as a means of quality assurance at the hospitals within the framework of national regulation, provided that data reported from the hospitals to the Norwegian Directorate of Health were anonymous, which they were. The study was thus not affected by the Norwegian Health Research Act. In Sweden, the record review was conducted in accordance to national regulation regarding the use of medical records and seen as a part of quality improvement initiatives in the hospitals. Personal identification numbers were not entered into the national database.

Results

Norway: A total of 53,367 records, randomly selected from 3,594,995 admissions in 21–24 hospitals were reviewed. This is equivalent to 1.5% of all eligible admissions.

Sweden: A total of 88,637 medical records, randomly selected from 7,201,781 admissions in 55–60 hospitals were reviewed. This is equivalent to 1.2% of all eligible admissions.

There was no significant change in AEs or fatal AEs in either country over the six-year time period.

Severity category E-I

In the samples, a total of 6 862 hospital admissions with at least one AE across all severity categories were identified in Norway and 10 568 in Sweden. Figure 1 presents estimated national annual rates of hospital admissions with one or more AEs in severity category E-I from 2013 to 2018, with associated 95% confidence intervals (CI). Linear trends for the annual AE rates in the two countries during the study period (Figure 1) show no significant change (p = 0.43 in Norway and p = 0.57 in Sweden).

Figure 1.

Estimated admission rates with at least one adverse event in severity category E-I, in Norway and Sweden from 2013 to 2018. Error bars indicate 95% confidence intervals. Long-dashed lines show the non-significant linear trends for the study period.

There were no significant differences in the estimated rates in Figure 1, nor in the estimated rates for all years together (2013–2018) between Norway (13.2% (95% CI 12.0–14.8)) and Sweden (13.1% (95% CI 11.5–14.9)).

In Table 4, we compare rates of different AE types in severity category E-I for all years together (2013–2018) in Norway and Sweden. There were significantly higher rates of surgical complications in Norwegian hospitals compared to Swedish hospitals. Swedish hospitals had significantly higher rates of ‘other’ AEs and pressure ulcer.

Table 4.

Rates (%) of hospital admissions with at least one adverse event (AE) according to type, severity category E-I combined, in Norway and Sweden in 2013–2018.

Type of adverse event	Norway – mean (95% CI)	Sweden – mean (95% CI)	Sweden minus Norway difference (95% CI)
Hospital-acquired infection	6.02% (5.42, 6.93)	5.32% (4.60, 6.10)	−0.70 (−1.88, 0.23)
Surgical complication	3.56% (3.13, 4.06)	2.09% (1.72, 2.52)	−1.47 (−2.09, −0.86)
Other	2.36% (2.02, 2.84)	4.61% (3.73, 5.55)	2.26 (1.24, 3.23)
Adverse drug event	2.03% (1.55, 2.50)	1.50% (1.11, 1.85)	−0.53 (−1.15, 0.05)
Pressure ulcer	0.72% (0.50, 0.98)	1.17% (0.95, 1.48)	0.45 (0.13, 0.84)
Falls	0.70% (0.47, 0.99)	0.75% (0.59, 0.92)	0.05 (−0.28, 0.33)

Severity category I

In the following analyses, only one AE type is registered for AEs that contributed to patient’s death (severity category I). A total of 136 hospital admissions with an AE in severity category I were identified in Norway and 227 in Sweden. Figure 2 presents national annual rates of hospital admissions with one AE in severity category I from 2013 to 2018, with associated 95% confidence intervals (CI). AE rates in severity category I have been declining slightly during the study period in Norway and increasing slightly in Sweden (Figure 2). None of the trends are significant (p = 0.14 in Norway and p = 0.46 in Sweden).

Figure 2.

Estimated admission rates with an adverse event in severity category I, in Norway and Sweden from 2013 to 2018. Error bars indicate 95% confidence intervals. Long-dashed lines show the non-significant linear trends for the study period.

There were no significant differences in the estimated rates in Figure 2, nor in the estimated rates for all years together (2013–2018) between Norway (0.23% (95% CI 0.15–0.32)) and Sweden (0.26% (95% CI 0.17–0.37)).

In Table 5, rates of different AE types in severity category I for all years together in Norway and Sweden are compared. For AEs that contributed to patient’s death, there was no significant difference between Norway and Sweden in the rates of hospital admissions with different AE types.

Table 5.

Rates (%) of hospital admissions with an adverse event (AE) according to type, severity category I, in Norway and Sweden in 2013–2018.^a

Type of adverse event	Norway – mean (95% CI)	Sweden – mean (95% CI)	Sweden minus Norway (95% CI)
Hospital-acquired infection	0.09% (0.04, 0.15)	0.13% (0.07, 0.20)	0.04 (–0.05, 0.12)
Other	0.06% (0.02, 0.10)	0.07% (0.03, 0.13)	0.02 (–0.05, 0.08)
Surgical complication	0.05% (0.02, 0.08)	0.03% (0.00, 0.06)	–0.03 (–0.07, 0.03)
Adverse drug event	0.03% (0.01, 0.06)	0.02% (0.00, 0.06)	–0.01 (–0.05, 0.03)

^aAdmissions with AE types falls and pressure ulcers of severity I, are not included in Table 5 since the number of these AEs was very small. Norway had only five AEs with falls and one with pressure ulcer, while Sweden had 14 falls and one pressure ulcer.

Preventability

Already from the introduction of the GTT method in Sweden the GTT teams have assessed the preventability of each identified AE, in addition to the regular categorisation of AEs. In Sweden 60.9% of AEs in severity categories E-H were considered preventable. 39.6% of AEs in severity category I were considered preventable and patients in this category were significantly older. In the Swedish material the mean patient age for AEs in category I was 78.5 years and the mean age in category E-I was 70.1. The mean patient age for all records reviewed was 67.2 years.

Discussion

The two national studies compared in this study reviewed in total 142,004 medical records, constituting to our knowledge the largest medical record review study to date. The Harvard Medical Practice Study reviewed 30,121 medical records, and the North Carolina study reviewed 2, 341 medical records.^14,24 Approximately 13% of hospital admissions eligible for the GTT study in both Norway and Sweden were associated with an AE (E-I) and 0.2–0.3% of hospital admissions were associated with an AE that contributed to death (I). There were no significant differences in the annual rates between the countries. There were statistically significant higher rates of surgical complications of severity E-I in Norwegian hospitals compared to Swedish hospitals. Swedish hospitals had significantly higher rates of ‘other’ AEs and pressure ulcers. This is in line with our previous study on AE rates in Norway and Sweden based on data from 2013 which found similar results.¹⁷ The AE types with different rates between Norway and Sweden, had not been changed, and were thus not affected by alterations mentioned previously.

In Norway and Sweden the rates of AE types of the most serious AEs, that contributed to death, follow the same pattern (Table 5). In both countries hospital-acquired infections are the most frequent type of AE, while adverse drug events are less frequent. Swedish data did not show any significant decline in rates of AEs in the category that contributed to death (I category). The slightly declining trend of AEs that contributed to death in Norway was not significant and could be result of normal variation within the material. However, a different analysis in the Norwegian GTT report for 2019, included data from two more years, from 2012 to 2019, and showed a statistically significant decline (p = 0.005).²⁵ The Swedish part of the study accounted for preventability and found that approximately 40% of the AEs that contributed to death in Sweden are considered preventable. From earlier research we know that high age and surgical procedures are associated with a higher incidence of AEs.¹⁶ Patients that die in hospital experience seven times the rate of AEs compared to those that do not die.²⁶ Reviewing inpatient deaths with the GTT may therefore provide new insight in causes of adverse events that contribute to death.

Most previous national studies of adverse events have been based on the Harvard Medical Practice Study methodology (HMPS) which does not include AEs that only cause temporary harm requiring interventions.²⁷ Such AEs are equivalent to category E in the GTT method. The GTT method was published in 2007 and few national or regional health system studies have been based on its methodology.²⁷ One national GTT study from the US looked at 785 admissions in 2008. The study sampled only admissions of Medicare beneficiaries and found that 1.5% of the admissions were associated to an AE that contributed to death.⁸ Adverse drug events, especially bleedings related to anticoagulation therapy, dominated causes of death in the US nationwide study. In the present study adverse drug events was the least frequent cause of AEs that contributed to death. The difference in the contribution of adverse drug reactions to AEs that contributed to death, may partly be explained by new medical options for anticoagulation treatment, that were introduced after the US study.²⁸ Comparisons between the two studies is also problematic since the study population in the US study of Medicare beneficiaries was very small and selected agewise.

Another GTT study across eight acute care US hospitals from 2007 to 2011 did not report percentage of admissions associated with death but found that the most common types of hospital-acquired AEs were related to surgery, medication and hospital associated infections.²⁹ The latter resonates with our findings. A third study applied the GTT method, to study medical records from admissions in 2002 to 2007 in 10 randomly selected hospitals in North Carolina. The study found that 0.6% of admissions were associated with AEs that contributed to death.^14,30 It did not find evidence of improvement, in spite of extensive policy efforts to promote patient safety.

A systematic review of GTT studies found that about 30–35% of all admissions are associated with AEs.²⁷ That is substantially higher numbers than that found in our study.

We have earlier concluded that GTT can serve as a usable tool for patient safety surveillance not only nationally but also between countries.¹⁷ We have previously argued that geographical aspects and different practices between Norway and Sweden regarding centralization of surgical procedures may explain the differences in AE rates related to surgical procedures, while differences between the countries in the outcome of nursing care may be related to differences in the organisation of departments, workload and nursing competencies. By comparing hospital admissions associated with AEs in the US GTT studies and in our study it seems that rates of AEs that contributed to death are three to five times higher in US hospitals compared to Norwegian and Swedish hospitals. Even if some of the differences can be explained by study design, a comparison is of great interest as a starting point for exploration of structural differences in the health care sector between the Nordic countries and the USA that may have an impact on patient safety issues.

Strengths and limitations

A particular strength to our study is that it is performed longitudinally (six years), to study trends over time. The sample is large (142 004 medical records) and randomly selected from all acute care hospitals, which reduces, but does not exclude the possibility that the sample is skewed regarding age and gender. Since the study does not correct for age, and high age is associated with a higher incidence of AEs, the trend over time could be influenced by an increase in population age.¹⁶ However, the AE rates both increase and decrease over the years for both countries. Since none of the trends are significant one could argue that they are within normal variation.

In retrospective record review studies, a potential weakness is poor documentation quality. Only documented AEs can be identified. The true number of AEs is thus probably higher than found. In addition, the GTT method has a time limit of 20 minutes, which means that not all AEs in reviewed records are identified. This matter is less relevant for our study, since AEs that contribute to death are less likely to be overlooked.

IRR was not assessed between GTT teams in this study, which is a limitation. Although previous studies have found IRR to be moderate to substantial between independent internal GTT reviewer teams, we cannot be sure that this also accounts for our study.^19,20 A strength is that the results have been available for hospitals and national health authorities to use for continual improvement efforts during all the study years.²⁵

Conclusions

Overall rates of AEs in Norway and Sweden are similar, as well as rates of AEs that contributed to death. This is expected considering similarities between the two countries’ healthcare systems. Differences in the rates of several types of AEs (surgical complications, ‘other’ AEs and pressure ulcer) require further analysis and provide opportunity for Norwegian and Swedish governing bodies to address patient safety issues further. There was no significant change in AEs or fatal AEs in either country over the six-year time period.

Footnotes

Data sharing statement

The data are administered by the Norwegian Directorate of Health and by the Swedish Association of Local Authorities and Regions (SALAR). No additional data are available.

Acknowledgement

We thank the referees for valuable comments, which have led to great improvements to the quality of the paper.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Ellen Tveter Deilkås

Øystein Flesland

References

Kohn

Corrigan

Donaldson

MS.

To err is human: building a safer health system. Washington: National Academies Press, 2000.

Wilson

Runciman

Gibberd

, et al. The quality in Australian health care study. Med J Aust 1995; 163: 458–471.

Davis

Lay-Yee

Briant

, et al. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J 2002; 115: U271.

Schioler

Lipczak

Pedersen

, et al. Incidence of adverse events in hospitals. A retrospective study of medical records. Ugeskr Laeger 2001; 163: 5370–5378.

Baker

Norton

Flintoft

, et al. The Canadian adverse events study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004; 170: 1678–1686.

Zegers

de Bruijne

Wagner

, et al. Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 2009; 18: 297–302.

Soop

Fryksmark

Koster

, et al. The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care 2009; 21: 285–291.

Levinson

DR.

Adverse events in hospitals. National incidence among medicare beneficiaries. Washington: Department of Health and Human Services, Office of Inspector General, 2010.

Classen

Resar

Griffin

, et al. ‘ Global trigger tool' shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff (Millwood) 2011; 30: 581–589.

10.

Hogan

Healey

Neale

, et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf 2012; 21: 737–745.

11.

Shojania

KG.

Deaths due to medical error: jumbo jets or just small propeller planes?

BMJ Qual Saf 2012; 21: 709–712.

12.

Drösler

Klazinga

Romano

, et al. Application of patient safety indicators internationally: a pilot study among seven countries. Int J Qual Health Care 2009; 21: 272–278.

13.

OECD. Health at a Glance 2013: OECD indicators. Paris: OECD, 2013.

14.

Landrigan

Parry

Bones

, et al. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med 2010; 363: 2124–2134.

15.

Deilkas

Bukholm

Lindstrom

, et al. Monitoring adverse events in Norwegian hospitals from 2010 to 2013. BMJ Open 2015; 5: e008576.

16.

Nilsson

Borgstedt-Risberg

Soop

, et al. Incidence of adverse events in Sweden during 2013–2016: a cohort study describing the implementation of a national trigger tool. BMJ Open 2018; 8: e020833.

17.

Deilkås

Risberg

Haugen

, et al. Exploring similarities and differences in hospital adverse event rates between Norway and Sweden using global trigger tool. BMJ Open 2017; 7: e012492.

18.

Griffin

Resar

IHI global trigger tool for measuring adverse events. IHI innovation series ed. Cambridge, MA: Institute for Healthcare Improvement, 2009.

19.

Mevik

Griffin

Hansen

, et al. Is inter-rater reliability of global trigger tool results altered when members of the review team are replaced? Int J Qual Health Care 2016; 28: 492–496.

20.

Sharek

Parry

Goldmann

, et al. Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients. Health Serv Res 2011; 46: 654–678.

21.

Classen

DCM

Lloyd

RCP

Provost

, et al. Development and evaluation of the institute for healthcare improvement global trigger tool. J Patient Saf 2008; 4: 169–177.

22.

Efron

Tibshirani

RJ.

An introduction to the bootstrap. London: Chapman & Hall, 1993.

23.

R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2020.

24.

Brennan

Leape

Laird

, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard medical practice study I. N Engl J Med 1991; 324: 370–376.

25.

Helsedirektoratet. Pasientskader i norge 2019 målt med global trigger tool. Oslo: The Norwegian Directorate of Health, 2020.

26.

Haukland

Mevik

von Plessen

, et al. Contribution of adverse events to death of hospitalised patients. BMJ Open Qual 2019; 8: e000377.

27.

Hibbert

Molloy

Hooper

, et al. The application of the global trigger tool: a systematic review. Int J Qual Health Care 2016; 28: 640–649.

28.

Connolly

Ezekowitz

Yusuf

, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med 2009; 361: 1139–1151.

29.

Kennerly

Kudyakov

da Graca

, et al. Characterization of adverse events detected in a large health care delivery system using an enhanced global trigger tool over a five-year interval. Health Serv Res 2014; 49: 1407–1425.

30.

Makary

Daniel

Medical error – the third leading cause of death in the US. BMJ 2016; 353