Sage Journals: Discover world-class research

Abstract

Aims

SMARTEST is a register-based randomized clinical trial (RRCT) that compares dapagliflozin to metformin in early-stage type 2 diabetes. The primary outcome includes progression of microvascular complications based on data from the Swedish National Diabetes Register (NDR). In this sub-study, the aim was to validate microvascular complication variables in the NDR against electronic health records (EHRs).

Methods

Data were extracted from EHRs of 276 SMARTEST participants with a median observation period of 3 years in the Uppsala, Örebro and Sörmland counties and compared with NDR data. Agreement was determined for all corresponding data entries as well as for progression of microvascular complications after randomization.

Results

The agreement for all corresponding data entries was 98.9% (Intraclass Correlation Coefficient 0.999) for creatinine and eGFR, 95.1% for albuminuria, 91.6% for foot-at-risk and 98.2% for retinopathy status (Kappa 0.67-0.91). The agreement for progression of microvascular complications was 98.0% for CKD stage, 98.9% for albuminuria grade, 96.3% for foot-at-risk grade and 99.6% for retinopathy grade progression (Gwet’s AC₁ 0.96-1.00).

Conclusion

Microvascular complication variables in the NDR show good agreement with EHR data. The use of a well-established national health care registry, exemplified by the NDR, for endpoint collection in RRCTs such as SMARTEST is supported by this study.

Keywords

Type 2 diabetes microvascular complications diabetes register validation register-based randomized clinical trial endpoint

Key messages

• Data on microvascular in the Swedish National Diabetes Register (NDR) display good agreement with data from electronic health records

• The ascertainment of the NDR was particularly good for retinopathy and nephropathy

• Within nephropathy, there was 98.9% concordance for eGFR and 95.1% for albuminuria

• For diabetic foot problems, the concordance was lower but acceptable

• The NDR is suitable for use in register-based clinical trials

Introduction

Chronic hyperglycemia in type 2 diabetes (T2D) leads to micro- and macrovascular complications that greatly impact the longevity, morbidity and quality of life for people with diabetes.^1–3 SGLT2i have consistently been shown to effectively prevent the development and progression of several diabetes complications when used as add-on treatment for T2D in high-risk individuals.^4–8 In the ongoing SMARTEST study, conducted in Sweden, the SGLT2i dapagliflozin is compared to metformin for treatment of early-stage T2D.⁹ The primary composite endpoint includes development or progression of micro- and macrovascular diabetes complications and are based on information obtained from national registers. The RRCT study design thus adopted is a cost-effective alternative to the traditional RCT, and has been used to an increasing extent in several medical fields.¹⁰ While observational register studies are common within the field of diabetes,^11,12 the SMARTEST study is to our knowledge the first RRCT to be conducted within this field.

In the SMARTEST study, the Swedish National Diabetes Register (NDR) is the source of several endpoints, including development or progression of microvascular complications. This national quality register includes approximately 90% of all patients with diabetes in Sweden and while it was originally launched to monitor quality in Swedish diabetes care, it has also been extensively used in research.¹³ In similarity with many other registers, the validity of data in the NDR has not been ascertained, which could have consequences for the interpretability of the SMARTEST study.

In this study, the aim was to evaluate the concept of the RRCT by validating clinical data in the NDR. This was done by investigating the agreement between data captured in the NDR and electronic health records (EHRs), specifically regarding the development or progression of microvascular complications, for a subset of the SMARTEST study population.

Methods

SMARTEST is a nationwide study with more than 30 study sites. The participants are randomized 1:1 to dapagliflozin or metformin and are then followed in regular care. Periodical withdrawals from national registers, including NDR, are made for endpoint collections. Adults with T2D according to WHO criteria with less than 4 years duration and BMI 18.5–45 kg/m² that are either drug naive or are receiving monotherapy for their diabetes are included. Exclusion criteria include more than 4 weeks treatment with insulin, GLP1-RA, SGLT2i or any combination of glucose-lowering medications, HbA1c > 70 mmol/mol on monotherapy and >80 mmol/mol for drug naïve, established cardiovascular disease, renal failure (eGFR<60 mL/min/1.73 m²), pregnancy or breastfeeding.

This sub study to SMARTEST was conducted in the Uppsala, Örebro and Sörmland counties. For participants included in the SMARTEST study and residing within the respective county, data was manually extracted from EHRs between 7 April 2021 and 30 March 2022 and compared to data extracted from the NDR on 8 June 2022. Participants were generally included in the order of inclusion in the SMARTEST study in each county, with some exceptions due to work distribution among the study personnel. The participants had consented to data collection from EHRs when signing the written informed consent to the SMARTEST study, which has received approval from the Swedish Ethical Review Authority (DNR 2019-01747).

Data extraction from EHR

Data regarding diabetes complications from the date of diagnosis for each participant were documented in a structured eCRF using RedCap¹⁴ hosted by Uppsala University. The data extraction was performed by qualified health personnel with no scientific interest in the SMARTEST study at the Uppsala University Hospital, the Clinical Research Center at Örebro and at Mariefred Primary Health Care Center in Sörmland. Documented variables included creatinine, UACR, foot examinations (monofilament test, tuning fork test, pulse palpation), fundus examination (retinopathy status and stage) and corresponding dates for these examinations and blood tests. Each participant was reviewed from the date of his/her diabetes diagnosis and onwards, except for fundus examinations, that were reviewed from 18 months before randomization and onwards. The participant’s study-ID number was used for identification and no other identifying information were entered in the eCRF.

Data extraction from the NDR

Health care units that manage patients with T2D are expected to report individual clinical data for these patients at least annually to the NDR. Within the framework of the SMARTEST study, withdrawals from the NDR are made every 3 months and stored in a central study database, operated by the Uppsala Clinical Research Center. In this sub study, NDR data withdrawn on 8 June 2022 were obtained from this database for the included participants. These data included creatinine, eGFR (MDRD¹⁵ and CKD-EPI¹⁶ equations), albuminuria grade, numerical value of UACR, foot-at-risk grade, status and grade of retinopathy. In the NDR, creatinine is reported as a continuous variable from which eGFR is calculated based on age and gender. Albuminuria is graded in 0–3 (0 = no albuminuria, 1 = normalized albuminuria (after treatment), 2 = microalbuminuria, 3 = macroalbuminuria) and since 2019 also optionally as numerical value in mg/mmol. Foot-at-risk grade is reported as 1-4 (1 = healthy foot 2 = signs of neuropathy/angiopathy, 3 = previous history of severe foot complications and 4 = ongoing severe foot complications) based on patient history and clinical examination. Retinopathy is reported as status (no = 0, yes = 1) and grade 1-4 (1 = mild, 2 = moderate, 3 = severe and 4 = proliferative retinopathy). For foot-at-risk grade and retinopathy, the date on which the examinations were made is documented, usually specified as the first day of the month. For all register entries, the date of entry is documented.

Data management

A scheme of data flow and management is provided in Figure 1. Data from EHR and the NDR were split into four categories: creatinine/CKD stage, UACR, foot-at-risk and retinopathy. For each of these categories, data from EHRs and the NDR were merged and sorted by study-ID and date. Data from the NDR were then linked to and paired with data from EHRs based on date of examinations and/or register entry and for continuous variables also the numerical value of the variable. Only NDR data that had been registered or were referring to source data within the observational period were included. Once data had been merged, sorted and paired, the four datasets were inspected and in cases of very discrepant information in EHRs versus the NDR, a targeted reextraction of EHRs was undertaken, after which data was confirmed, corrected, removed or added. Importantly, all authors remained blinded to information on randomized study treatment since this was not included in the analysis datasets.

Figure 1.

Data flow and management Solid boxes and arrows represent location and flow of EHR (electronic health records) data. Dashed boxes and arrows represent location and flow of NDR (the Swedish National Diabetes Register) data. ^aExtracted on 8 June 2022^bExtracted manually between 7 April 2021 and 31 March 2022.

Classification of data on microvascular complications

Creatinine was handled as a continuous variable for the purpose of validation and was not processed further. For albuminuria, categorization of EHR data was made based on the following algorithm: normal was defined as UACR<3 mg/mmol; microalbuminuria was defined as an incident UACR of 3–30 mg/mmol; macroalbuminuria as an incident UACR >30 mg/mmol. Due to lack of impact for the SMARTEST study, no differentiation was made between no albuminuria and normalized albuminuria (after treatment) which were merged into the same category. For NDR data, the reported albuminuria grade or the transformed albuminuria grade, based on the numerical UACR and by the same algorithm as for EHR data, was utilized. In cases where both an albuminuria grade and a numerical value had been entered; supremacy was considered for the numerical value.

For foot-at-risk, guidelines from the NDR and the Swedish Association of Local Authorities and Regions¹⁷ were adopted. Thus, grade 1 was defined as the absence of abnormalities in monofilament test, tuning fork test or pulses and no manifest diabetes-related foot lesions, e.g. ulcers. Grade 2 was defined as abnormal findings in at least one of monofilament test, tuning fork test or pulses. Grade 3 was defined as signs of neuropathy or angiopathy according to grade 2 plus history of ulcers, amputation, deformities, or skin fissures. Grade 4 was defined as ongoing ulcers, osteopathy or neuropathic pain. For NDR data, the foot-at-risk grade was utilized without further processing.

Retinopathy status (yes/no) was based on the fundus examination in EHR and on the retinopathy status and grade variables in the NDR. The retinopathy grades (mild, moderate, severe and proliferative) provided in the fundus examination of EHR and the NDR were utilized without further processing.

Definition of microvascular events: Development or progression of microvascular complications

Preliminary definitions of microvascular events in the SMARTEST study were adopted.

Thus, progression of CKD stage was defined as a decline in eGFR, calculated with the 2009 CKD-EPI formula,¹⁶ from above to below 60 mL/min/m² (CKD stage 3) observed >12 months after randomization that was either repeatedly observed on at least one consecutive non-identical observation or the last available observation. The rationale behind the 12-month limit is the well-described temporary drop in eGFR that occurs immediately after treatment initiation with SGLT2-inhibitors and remains up to 12 months compared with placebo.^8,18,19

Development or progression of albuminuria, foot complications and retinopathy were defined as the progression to a higher stage (i.e. from no albuminuria to micro-/macroalbuminuria or from micro-to macroalbuminuria) from the highest of the inclusion visit assessment and the last NDR entry or EHR data, respectively, within 18 months that was either repeatedly observed on at least one consecutive and non-identical observation OR the last available observation OR, for albuminuria, normalized after initiation of RAAS-blockade therapy. Observations were only counted as events if > 30 (>90 for retinopathy) days from randomization to date of examinations (foot-at-risk, retinopathy) or register entry (albuminuria). For foot-at-risk and retinopathy, the date of examinations was converted to the 15^th of the month, if registered as the 1^st of that month. The analysis of microvascular events was restricted to participants with a follow-up time of more than the above specified time limits, i.e. to participants at risk to have the events.

Statistical analysis

General aspects of data quality were presented descriptively. This included source specific data distribution and overlap, completeness (percentage of EHR data transferred to the NDR), uniqueness (percentage of NDR data that could be linked to EHR data), duplicate and extraneous NDR data, reporting rate (number of unique NDR entries per patient year) and timeliness between corresponding NDR (date of entry or examination utilized as applicable) and EHR entries.

Validation was carried out for corresponding entries in the NDR and EHRs as well as for microvascular events based on NDR versus EHR data. EHR data were considered the golden standard. For the continuous variable creatinine, the exact percentual agreement in the NDR versus EHRs and the single measure type ICC, using a two-way mixed model with an absolute agreement definition²⁰ was determined. For categorical variables, the exact agreement and Cohen’s kappa²¹ (κ, for dichotomous variables) or Cohen’s weighted kappa with linear weights²² (κ_w, for polytomous ordered variables) were determined. The following cutoffs for interpretation of kappa values have been proposed: 0.01-0.20 = poor, 0.21-0.40 = fair, 0.41-0.60 = moderate, 0.61-0.80 = good, 0.81-1 = very good.²³ In cases with low prevalence of some categories, Gwet’s AC₁,²⁴ that provides more robust estimates in this setting,²⁵ was used instead. In addition, for dichotomous variables, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were determined. Excel for Mac version 16.66.1 (Microsoft corporation, Redmond, WA) was used for data handling. ICC was calculated in SPSS v28.0 (IBM corp., Armonk, NY). Kappa and AC₁ estimates were calculated in R Statistical Software v 4.2.2 (R Core Team, 2021) using the irrCAC R Package v1.0 (Gwet 2019). The AC₁ estimates were assessed in terms of the Altman benchmarking scale²³ presented above, using the function altman.bf (). Descriptive plots were constructed in Graph Pad Prism version 9.1.0 (GraphPad Software, San Diego, Ca).

Results

Participant characteristics

This subset of SMARTEST study participants included 276 individuals with a combined observational period of 776 patient-years (median 3 years per participant), of which 271 patient-years were after randomization in the SMARTEST study (median 13 months per participant). They were included in study centers within the defined geographical area, from 5 September 2019 to 7 February 2022. The majority were men (180; 65.2%). At inclusion in the SMARTEST study, they were generally overweight/obese (BMI 30.8 ± 5.1 kg/m²), had early-stage type 2 diabetes (duration 21 ± 15 months) with good glycemic control (HbA1c 45.7 ± 8.0 mmol/mol) and low prevalence of diabetes complications (albuminuria 9.8%, neuropathy/angiopathy 3.6%, retinopathy 9.4%). Detailed information on clinical characteristics at inclusion in the SMARTEST study is provided in Table 1.

Table 1.

Participant characteristics (n = 276) at inclusion in a regional subset of the SMARTEST study. Data presented as a n (%) or mean ± SD. Treatment refers to before inclusion.

Variable	n = 276
Gender, male/female (%)	180/96 (65.2/34.8)
Age, years	61 ± 10
T2D duration, months	21 ± 15
BMI, kg/m²	30.8 ± 5.1
Systolic blood pressure, mmHg	139 ± 16
Diastolic blood pressure, mmHg	84 ± 10
HbA1c, mmol/mol	45.7 ± 8.0
Creatinine μmol/l	74.5 ± 13.6
eGFR, ml/min/1.73 m²	88.8 ± 14.1
Total cholesterol, mmol/l	4.8 ± 1.1
LDL, mmol/l	2.98 ± 0.98
HDL, mmol/l	1.29 ± 0.34
Triglycerides, mmol/l	2.04 ± 1.26
Nephropathy, none/micro-/macroalbuminuria (%)	249/26/1 (90.2/9.4/0.4)
Neuropathy/Angiopathy, no/yes (%)	266/10 (96.4/3.6)
Retinopathy^a, no/yes (%)	229/26 (83.0/9.4)
Retinopathy grade, mild/mod/sev/prol (%)	22/3/1/0 (8.0/1.1/0.4/0)
Metformin, n (%)	236 (85.5)
Drug naïve, n (%)	40 (14.5)
Antihypertensive treatment, n (%)	185 (67.0)
ACEI, n (%)	53 (19.2)
ARB, n (%)	93 (33.7)
Lipid-lowering agent, n (%)	117 (42.4)

^aBaseline assessment missing for 21 subjects. mod = moderate, sev = severe, prol = proliferative, ACEI = Angiotensin-Converting-Enzyme Inhibitors, ARB = Angiotensin II Receptor Blockers.

General data characteristics

Table 2 displays the distribution of data in the NDR and EHRs. NDR completeness (percentage of EHR entries transferred to the NDR) was 54.7% for creatinine, 55.3% for albuminuria, 66.6% for foot-at-risk and 86.7% for retinopathy status. NDR uniqueness (percentage of NDR entries that could be linked uniquely to EHR entries) was 66.3% for creatinine, 72.5% for albuminuria, 66.8% for foot-at-risk and 44.8% for retinopathy status, the remaining NDR data consisting predominantly of duplicate NDR entries (Table 2). Unreported, missing data from EHR did not differ from EHR data that had been reported to the NDR, with the exception of foot-at-risk, where the proportion of neuropathy/angiopathy was significantly higher among reported data (Supplement S1). Out of the valid NDR entries as retinopathy = yes, there was an associated retinopathy stage in 43.3% (13/30), but this reporting rate increased during the observational period, exceeding 80% by 2022 (Supplement, S2). The reporting rate of unique NDR entries per patient year was 1.09 for creatinine, 0.71 for albuminuria, 0.86 for foot-at-risk and 0.36 for retinopathy. The median number of total unique entries per participant was 3 for creatinine, 2 for albuminuria grade, 2 for foot-at-risk grade and 1 for retinopathy. The number of participants that had at least one unique NDR entry was 264 (95.7%) for creatinine, 237 (85.9%) for albuminuria, 216 (78.3%) for foot-at-risk and 237 (85.9%) for retinopathy. Counting only unique entries after randomization, the equivalent numbers were 200 (72.5%) for creatinine, 151 (54.7%) for albuminuria, 161 (58.3%) for foot-at-risk and 116 for retinopathy (42.0%) (Supplement S3 and S4). The median time between EHR and NDR entry was 7 days for creatinine, 6 days for albuminuria and 0 days for foot-at-risk and retinopathy.

Table 2.

Data distribution for entries in the NDR and EHRs.

	Creatinine	Albuminuria	Foot-at-risk	Retinopathy
EHR, total	1548	996	1007	323
EHR + NDR^a	846	551	671	280
NDR, total	1276	760	1004	625
NDR, duplicate entries^b	315	145	277	330
NDR, extraneous entries^c	115	64	56	15

^aNDR and EHR data that could be linked uniquely,

^brepeated entries without new information appearing in EHR,

^cNDR entries that could not be linked to available EHR data.

Agreement of corresponding NDR and EHR entries

The agreement was 98.9% (ICC 0.999, 95% CI 0.999-0.999, p < 0.001) for a total of 846 corresponding creatinine entries, 95.1% (κ_w 0.80, 95% CI 0.72-0.87, p < 0.001) for the 551 corresponding albuminuria grade entries, 91.6% (κ 0.67, 95% CI 0.59-0.76, p < 0.001) for the 606 corresponding foot-at-risk grade entries and 98.2% (κ 0.91, 95% CI 0.84-0.99, p < 0.001) for the 280 corresponding retinopathy status entries. Table 3 displays crosstabulations of entries in the NDR versus EHRs with associated sensitivity, specificity, PPV and NPV, where applicable. For the 13 corresponding data on retinopathy grade (11 mild, 2 moderate), the agreement was 100% (κ 1.00, 95% CI 1.00-1.00, p < 0.001).

Table 3.

Categorical classification of microvascular complications in corresponding NDR and EHR entries.

^aEHR datapoints with an assessment of foot-at-risk grade but with no information on clinical examinations were not included in the analysis. Neuro = neuropathy, Angio = angiopathy, PPV = positive predictive value, NPV = Negative predictive valueAgreement of microvascular events fulfilment according to the NDR versus EHRs.

The agreement between the two sources was assessed for events of development or progression of microvascular complications. It was 98.0% (AC₁ 0.98, 95% CI 0.95-1.00, p < 0.001) for CKD stage, 98.9% (AC₁ 0.99, 95% CI 0.97-1.00, p < 0.001) for albuminuria, 96.3% (AC₁ 0.96, 95% CI 0.93-0.99, p < 0.001) for foot-at-risk and 99.6% (AC₁ 1.00, 95% CI 0.99-1.00, p < 0.001) for retinopathy progression. All these agreement estimates were classified as very good, according to Altman’s benchmarking scale.²³ Table 4 displays crosstabulations of microvascular events according to the NDR versus EHRs with associated sensitivity, specificity, PPV and NPV.

Table 4.

Microvascular events based on data in the NDR versus EHRs.

PPV = positive predictive value, NPV = negative predictive value.

Discussion

In this study, we have validated variables pertaining to microvascular complications in the NDR against source data in EHRs. We found that the agreement between corresponding NDR and EHR entries was good concerning albuminuria and foot-at-risk grade and very good concerning creatinine and retinopathy status. Since this was a sub study to the SMARTEST study, we also evaluated the agreement between NDR and EHR data regarding fulfilment of microvascular events as preliminarily defined in the SMARTEST study. In this context, the agreement was very good for all events: progression of CKD-stage, albuminuria grade, foot-at-risk grade and retinopathy stage. The completeness of NDR data was adequate, entries based on novel information in EHRs occurring at average more than annually for creatinine, slightly less than annually for albuminuria and foot-at-risk and more than once every three years for retinopathy; all in keeping with Swedish national guidelines.²⁶ In general, the assessable date of NDR data was timely with respect to appearance in EHRs; the delay was equal to or less than a week for all variables at median. Although clinical data in the NDR have been previously validated, this is the first publication of the validity of microvascular complication variables in the NDR. The findings reinforce the use of the NDR for observational studies and support its use as source data in RRCTs such as the SMARTEST study.⁹

General data characteristics

The missingness of data in the NDR with respect to EHRs was overall quite substantial; around 45% for creatinine and albuminuria, 33% for foot-at-risk grade, while only 13% for retinopathy status. Corresponding retinopathy grades were only available for 43% of all retinopathy entries but increased during the observational period, probably reflecting increasing awareness of the variable since its recent introduction in 2018. Creatinine and UACR may be obtained for reasons other than routine surveillance of diabetes complications by other health care units that do not report to the NDR. This is far less likely regarding foot examinations and highly implausible regarding fundus examinations. Moreover, diagnosis of albuminuria is often, and should be, a congregated assessment based on several repeated measurements. Previous measurements that have not coincided with the reporting of the albuminuria grade may therefore have been factored into the assessment notwithstanding the absence of a temporal association. Human errors in the documentation or transfer of EHR data or patient objection to data transfer may also explain some data loss to the NDR. Importantly, routines regarding frequency, completeness and method of reporting to the NDR differ among health care providers. For example, around one-third of all health care units still employs manual reporting of NDR data and automated data extraction, when implemented, is usually limited to the last available data for each variable. Obviously, a less selective automated extraction process, standardized and employed by all health care providers would overcome many potential sources of data loss. However, such a procedure may have ethical and legal issues relating to personal integrity.

Notwithstanding possible causes of the observed missingness, there were no substantial numerical or proportional differences between missing and reported data, suggesting that neither the current results, nor the reliability of future results from the SMARTEST study appear to be at risk of marked influence from systematic, non-random loss of real-world data. Taken together, the resulting reporting rate for each investigated variable in the NDR seems adequate, both for the primary purpose of monitoring the national quality of diabetes care as well as for the applicable use as source data in observational and interventional clinical studies.

Agreement of microvascular complications and events in the NDR versus EHR

Not surprisingly, the agreement between the NDR and EHRs was best for the more objective variables creatinine and retinopathy. For these variables, the minute discrepancies between the two sources were probably attributable to the human factor in EHR documentation or transfer to the NDR. Albuminuria is predominantly reported to the NDR as a grade based on the UACR value at one or several occasions. Thus, the quality of this variable is ultimately dependent on the clinicians’ judgement. Ignorance about the cut-offs between albuminuria grades may be one source of error. Clinicians may also see variant need for repeated measures of UACR before making the diagnosis of albuminuria. According to guidelines, albuminuria should be diagnosed if indicated on 2 out of 3 measurements within 3–6 months.²⁷ Most likely, departures from these guidelines are made on a regular basis in clinical practice. Nevertheless, more than half of all NDR entries as albuminuria of any stage and almost 80% of repeated NDR albuminuria entries with available preceding EHR information were based on 2/3 UACR measurements in the albuminuria range. The built-in requirement of confirmatory NDR entries in the preliminary definition of albuminuria events thus ensures that these events are well-grounded and the demonstrated excellent agreement between albuminuria progression based on NDR versus EHR data further corroborates the operability of this preliminary definition.

Regarding foot-at-risk grade, national guidelines from The Swedish Association of Local Authorities and Regions¹⁷ state that any pathologic finding with regards to tuning fork test, monofilament test or pulse palpation should prompt a foot-at-risk grade 2. In clinical practice however, clinicians may value findings differently and make an overall assessment of the foot-at-risk grade. In particular, negligence to report isolated abnormalities in tuning fork test were the cause of inconsistency between the sources in almost 80% of cases of false negative foot-at-risk grades. Abnormal findings on the monofilament test were less frequently disregarded. While the tuning fork test may be more sensitive than the monofilament test in detecting neuropathy,^28,29 results from the monofilament test have been indicated to better predict the risk to develop diabetic foot ulcers.^30,31 From this perspective, the supreme emphasis on the monofilament test in the clinical evaluation of the foot-at-risk grade appears well-grounded. Furthermore, when comparing documented risk foot grade in corresponding NDR and EHR entries, the agreement between these sources was close to perfect (Supplement S5). Thus, it is fair to state that the NDR appears to accurately reflect the clinician’s overall assessment of the foot-at-risk grade, while this assessment may not be in full agreement with the documented findings on clinical examination.

Comparison with other health-care registers

The SWEDEHEART³² register has been employed in numerous published and ongoing register-based randomized controlled trials.^33–35 According to annual internal quality control, SWEDEHEART variable agreement with EHR data has been reported to be in the range 92.4–97.4%,^32,36 which is in line with the agreement of NDR variables as presented in this study. The agreement is also similar to the Swedish Fracture Register,³⁷ which is being employed in several ongoing RRCTs.^38,39

Strengths and limitations

This is the first attempt to validate microvascular complication variables of the NDR. We have studied a large population with a long combined observational period. The extraction of EHR data has been systematic and has occurred independently of NDR data by independent health care staff with no self-interest in the outcome of the study.

There are however some important limitations that need to be addressed. First, many participants did not have unique NDR entries after randomization and accordingly, the number of events for validation were few. Following inclusion in the SMARTEST study, the study participant’s regular care provider is notified regarding study involvement and is actively encouraged to report to the NDR. Thus, there is no considerable risk that this sparsity of NDR entries after randomization would result in an overestimation of the agreement between corresponding data from the two sources. Regarding agreement of event fulfilment, we recognize the need for further validation at a later stage of the SMARTEST study. However, it seems likely that discrepancies between the two sources would be expected to diminish rather than enhance with prolonged follow-up time and accumulation of data. Second, some EHR data may have been overlooked due to complicated structure of the EHRs. There are also some private care providers whose EHRs are difficult to access for external healthcare personnel. Third, the cohort in this study was limited to participants residing in three out of twenty-one Swedish health care regions, amounting to approximately one third of the SMARTEST study population that had been recruited by the time the current EHR extraction was terminated. Although this selected cohort was highly representative of the nationwide SMARTEST population in terms of clinical characteristics,⁴⁰ neither the participants nor the primary care centers these participants attended are likely to fully reflect real-world diabetes care in Sweden. However, in the geographical area we studied there is a mixture of rural and urban residency, socioeconomic strata and private and public primary care centers, and this is similar to the whole country. Nevertheless, confirmatory validation in a less selected cohort of patients with type 2 diabetes and corresponding regular care providers may be warranted. Fourth, our analysis was limited to microvascular diabetes complications. The primary composite endpoint in the SMARTEST study also includes macrovascular complications and all-cause death, which are based on information from the National Patient Register and the Swedish Population Register, respectively. While validation of the implementation of these registers in the SMARTEST study is planned, it was not done at the time of the current analysis due to the scarcity of such events.

Regardless of the limitations it is likely that the validity of NDR’s microvascular complication data would be similar across Swedish primary care for type 2 diabetes. Importantly, a main aim of this study was to address whether the NDR would provide appropriate data to collect outcome events of microvascular complications in the specific context of a register-based randomized controlled trial. We find that the NDR is a robust and valid data source for this purpose. Nonetheless, further research is warranted, for example to address the validity of register data of long-term trajectories of diabetes complications. We hope that this study may serve as an example for validation of register data that may be extended to other local and national health registers that are employed both for monitoring of health care quality and for scientific purposes.

Conclusions

In summary, this is the first published study validating data on microvascular complications from the Swedish National Diabetes Register (NDR). We found that they largely display very good agreement with source data from EHRs. This is comparable to other Swedish registers used in RRCTs. RRCTs offer a cost-effective way to evaluate clinical interventions in a real world setting, and they are being conducted to an increasing extent in several disease areas.¹⁰ To our knowledge, the SMARTEST study is the first RRCT to be conducted within the field of diabetes. This validation study reinforces the use of the NDR in observational studies and also provides robust support for the NDR as source data in the ongoing RRCT SMARTEST as well as other future RRCTs.

Supplemental Material

Supplemental Material - Health care registers can be instrumental for endpoint capture in clinical diabetes trials: example of microvascular complications in Swedish patients with type 2 diabetes

Supplemental Material for Health care registers can be instrumental for endpoint capture in clinical diabetes trials: example of microvascular complications in Swedish patients with type 2 diabetes by Martin H Lundqvist, Vagia Patsoukaki, Stefan Jansson, Henrietta Norman, Elisabet Granstam, Maria K Svensson, Johan Sundström, Björn Eliasson and Jan W Eriksson in Diabetes & Vascular Disease Research

Footnotes

Acknowledgements

We are grateful to the scientifical advice provided by Peter Hallgren (consultant physician, PhD, Department of Internal Medicine, Falun Hospital, Sweden) and the meticulous extraction work conducted by Caroline Woxberg, Kerstin Marttala (Clinical Diabetes and Metabolism, Uppsala University), Anna Segerstedt, Christine Degner (Örebro Clinical Research Center) and Elisabeth Ekström (Mariefred Vårdcentral).

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: All authors have been involved in the conceptualization and/or conduct of the SMARTEST trial. BE and SJ are affiliated with the Swedish National Diabetes Register.

JS reports ownership in companies providing services to Itrim, Amgen, Janssen, Novo Nordisk, Eli Lilly, Boehringer Ingelheim, Bayer, Pfizer and AstraZeneca, outside the submitted work. BE reports personal fees (expert panels, lectures) from Amgen, AstraZeneca, Boehringer Ingelheim, Eli Lilly, Merck Sharp and Dohme, Mundipharma, Navamedic, Novo Nordisk, RLS Global, and grants and personal fees from Sanofi, all outside the submitted work. MES has received honoraria (expert panels, lectures) from Amgen, AstraZeneca, Boehringer Ingelheim and GSK, all outside the submitted work. EG has received honoraria (advisory boards, lectures) from Bayer, Novartis, Abbvie/Allergan and Novo Nordisk, all outside the submitted work. JWE has received honoraria and research support from AstraZeneca and honoraria from Novo Nordisk, Merck Sharp and Dohme and Ilya Pharma. HN owns shares in AstraZeneca, outside the submitted work.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the grants from Uppsala‐Örebro Regional Research Council (RFR-930976), the Swedish Research Council (project 2018-00904), Exodiab 2019-00978), VINNOVA (Swedish Innovation Agency), the Swedish Heart and Lung Foundation (project 20190403), the Swedish Eye Foundation, the Swedish Diabetes Foundation and ALF grants (governmental research support to university-affiliated health care organisations).

Data availability

Data will become available when the SMARTEST study is completed.

ORCID iD

Martin H Lundqvist

Supplemental Material

Supplemental material for this article is available online.

Appendix

References

Norhammar

Bodegard

Nystrom

, et al. Incidence, prevalence and mortality of type 2 diabetes requiring glucose-lowering treatment, and associated risks of cardiovascular complications: a nationwide study in Sweden, 2006-2013. Diabetologia 2016; 59: 1692–1701. DOI: 10.1007/s00125-016-3971-y 2016/05/18.

Cusick

Meleth

Agron

, et al. Associations of mortality and diabetes complications in patients with type 1 and type 2 diabetes: early treatment diabetic retinopathy study report no. 27. Diabetes Care 2005; 28: 617–625. DOI: 10.2337/diacare.28.3.617 2005/03/01.

Arnold

Khunti

Tang

, et al. Impact of micro-and macrovascular complications of type 2 diabetes on quality of life: Insights from the DISCOVER prospective cohort study. Endocrinol Diabetes Metab 2022; 5: e00321. DOI: 10.1002/edm2.321 e00321. 2022/01/15.

Wiviott

Raz

Bonaca

, et al. Dapagliflozin and cardiovascular outcomes in type 2 diabetes. N Engl J Med 2019; 380: 347–357. DOI: 10.1056/NEJMoa1812389 2018/11/13.

Zinman

Wanner

Lachin

, et al. Empagliflozin, cardiovascular outcomes, and mortality in type 2 diabetes. N Engl J Med 2015; 373: 2117–2128. DOI: 10.1056/NEJMoa1504720 2015/09/18.

Neal

Perkovic

Matthews

. Canagliflozin and cardiovascular and renal events in type 2 diabetes. N Engl J Med 2017; 377(2099): 2099. DOI: 10.1056/NEJMc1712572

Perkovic

Jardine

Neal

, et al. Canagliflozin and renal outcomes in type 2 diabetes and nephropathy. N Engl J Med 2019; 380: 2295–2306. DOI: 10.1056/NEJMoa1811744 2019/04/17.

Heerspink

HJL

Stefansson

Correa-Rotter

, et al. Dapagliflozin in patients with chronic kidney disease. N Engl J Med 2020; 383: 1436–1446. DOI: 10.1056/NEJMoa2024816 2020/09/25.

Sundstrom

Kristofi

Ostlund

, et al. A registry-based randomised trial comparing an SGLT2 inhibitor and metformin as standard treatment of early stage type 2 diabetes (SMARTEST): rationale, design and protocol. J Diabetes Complications 2021; 35(107996): 107996. DOI: 10.1016/j.jdiacomp.2021.107996

10.

Eriksson

Eliasson

Bennet

, et al.

Registry-based randomised clinical trials: a remedy for evidence-based diabetes care?

Diabetologia 2022; 65: 1575–1586. DOI: 10.1007/s00125-022-05762-x

11.

Helve

Sund

Arffman

, et al. Incidence of end-stage renal disease in patients with type 1 diabetes. Diabetes Care 2018; 41: 434–439. DOI: 10.2337/dc17-2364 2017/12/22.

12.

Knudsen

Bodegard

Birkeland

, et al. Risk factor management of type 2 diabetic patients in primary care in the Scandinavian countries between 2003 and 2015. Prim Care Diabetes 2021; 15: 262–268. DOI: 10.1016/j.pcd.2020.09.006 2020/10/10.

13.

Götaland. Nationwide results 1996-2020, Swedish national diabetes register NDR. Center of Registers Västra. 2021.

14.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–381. DOI: 10.1016/j.jbi.2008.08.010 2008/10/22.

15.

Levey

Coresh

Greene

, et al. Expressing the modification of diet in renal disease study equation for estimating glomerular filtration rate with standardized serum creatinine values. Clin Chem 2007; 53: 766–772. DOI: 10.1373/clinchem.2006.077180 2007/03/03.

16.

Levey

Stevens

Schmid

, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med 2009; 150: 604–612. DOI: 10.7326/0003-4819-150-9-200905050-00006 2009/05/06.

17.

Swedish National Task Force on Diabetes . Fotundersökning vid diabetes-nationellt vårdprogram för prevention av fotkomplikationer vid diabetes. Stockholm: The Swedish Association of Local and Regional Authorities, 2018.

18.

Meraz-Munoz

Weinstein

Wald

. eGFR decline after SGLT2 inhibitor initiation: the tortoise and the hare reimagined. Kidney360 2021; 2: 1042–1047. DOI: 10.34067/KID.0001172021 2022/04/05.

19.

Wanner

Heerspink

HJL

Zinman

, et al. Empagliflozin and kidney function decline in patients with type 2 diabetes: a slope analysis from the EMPA-REG outcome trial. J Am Soc Nephrol 2018; 29: 2755–2769. 2018/10/14. DOI: 10.1681/ASN.2018010103

20.

Shrout

Fleiss

. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420–428. DOI: 10.1037//0033-2909.86.2.420 1979/03/01.

21.

Cohen

. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960; 20: 37–46.

22.

Cohen

. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70: 213–220. DOI: 10.1037/h0026256 1968/10/01.

23.

Altman

. Practical statistics for medical research. Boca Raton, FL: CRC press [Diabetes, 2018.#699]1991.

24.

Gwet

. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 2008; 61: 29–48. DOI: 10.1348/000711006X126600 2008/05/17.

25.

Wongpakaran

Wedding

, et al. A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol 2013; 13: 61. DOI: 10.1186/1471-2288-13-61

26.

National Board of Health and Welfare . Nationella riktlinjer för diabetesvård - stöd för styrning och ledning. https://www.socialstyrelsen.se/globalassets/sharepoint-dokument/artikelkatalog/nationella-riktlinjer/2018-10-25.pdf (2018, accessed 17. November 2022).

27.

ElSayed

Aleppo

Aroda

, et al. 11. Chronic kidney disease and risk management: standards of care in diabetes-2023. Diabetes Care 2023; 46: S191–S202, doi:10.2337/dc23-S0112022/12/13

28.

Meijer

Smit

Lefrandt

, et al.

Back to basics in diagnosing diabetic polyneuropathy with the tuning fork!

Diabetes Care 2005; 28: 2201–2205. DOI: 10.2337/diacare.28.9.2201 2005/08/27.

29.

Gin

Rigalleau

Baillet

, et al. Comparison between monofilament, tuning fork and vibration perception tests for screening patients at risk of foot complication. Diabetes Metab 2002; 28: 457–461 2003/01/11.

30.

Boyko

Ahroni

Stensel

, et al. A prospective study of risk factors for diabetic foot ulcer. The Seattle Diabetic Foot Study. Diabetes Care 1999; 22: 1036–1042, doi:10.2337/diacare.22.7.10361999/07/02

31.

Rinkel

Castro Cabezas

Coert

. A new application of the rotterdam diabetic foot study test battery: grading pedal sensory loss to predict the risk of foot ulceration. Diabetes Res Clin Pract 2021; 175(108836): 108836. DOI: 10.1016/j.diabres.2021.108836

32.

Jernberg

Attebring

Hambraeus

, et al. The Swedish Web-system for enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies (SWEDEHEART). Heart 2010; 96: 1617–1621. DOI: 10.1136/hrt.2010.198804 2010/08/31.

33.

Lagerqvist

Frobert

Olivecrona

, et al. Outcomes 1 year after thrombus aspiration for myocardial infarction. N Engl J Med 2014; 371: 1111–1120. DOI: 10.1056/NEJMoa1405707 2014/09/02.

34.

Hofmann

James

Jernberg

, et al. Oxygen therapy in suspected acute myocardial infarction. N Engl J Med 2017; 377: 1240–1249. DOI: 10.1056/NEJMoa1706222 2017/08/29.

35.

Clinical Trials.gov . Dapagliflozin effects on cardiovascular events in patients with an acute heart attack (DAPA-MI), 2022, https://clinicaltrials.gov/ct2/show/NCT04564742 (accessed December 13).

36.

Desta

Jernberg

Lofman

, et al. Incidence, temporal trends, and prognostic impact of heart failure complicating acute myocardial infarction. The Swedeheart Registry (Swedish web-system for enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies): a study of 199,851 patients admitted with index acute myocardial infarctions, 1996 to 2008. JACC Heart Fail 2015; 3: 234–242. DOI: 10.1016/j.jchf.2014.10.007 2015/03/07.

37.

Knutsson

Wennergren

Bojan

, et al. Femoral fracture classification in the Swedish fracture register-a validity study. BMC Musculoskelet Disord 2019; 20(197): 197. DOI: 10.1186/s12891-019-2579-z

38.

Wolf

Sjoholm

Hailer

, et al. Study protocol: HipSTHeR - a register-based randomised controlled trial-hip screws or (total) hip replacement for undisplaced femoral neck fractures in older patients. BMC Geriatr 2020; 20(19): 19, doi:10.1186/s12877-020-1418-2

39.

Wolf

Mukka

Notini

, et al. Study protocol: the duality trial - a register-based, randomized controlled trial to investigate dual mobility cups in hip fracture patients. Acta Orthop 2020; 91: 506–513, doi:10.1080/17453674.2020.17800592020/06/23

40.

Eriksson

Bennet

Eliasson

, et al. IDF2022-0602 Metformin or SGLT2 inhibitor as 1st line treatment of type 2 diabetes? Design and interim results of the SMARTEST trial. Diabetes Research and Clinical Practice 2023; 197, doi:10.1016/j.diabres.2023.110382

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.63 MB