Validation of a multi-marker model for the prediction of incident type 2 diabetes mellitus: Combined results of the Inter99 and Botnia studies

Abstract

Purpose: To assess performance of a biomarker-based score that predicts the five-year risk of diabetes (Diabetes Risk Score, DRS) in an independent cohort that included 15-year follow-up.

Method: DRS was developed on the Inter99 cohort, and validated on the Botnia cohort. Performance was benchmarked against other risk-assessment tools comparing calibration, time to event analysis, and net reclassification.

Results: The area under the receiver-operating characteristic curve (AUC) was 0.84 for the Inter99 cohort and 0.78 for the Botnia cohort. In the Botnia cohort, DRS provided better discrimination than fasting plasma glucose (FPG), homeostasis model assessment of insulin resistance, oral glucose tolerance test or risk scores derived from Framingham or San Antonio Study cohorts. Overall reclassification with DRS was significantly better than using FPG and glucose tolerance status (p < 0.0001). In time to event analysis, rates of conversion to diabetes in low, moderate, and high DRS groups were significantly different (p < 0.001).

Conclusion: This study validates DRS performance in an independent population, and provides a more accurate assessment of T2DM risk than other methods.

Keywords

Biomarkers Diabetes Risk Score incident diabetes risk assessment type 2 diabetes mellitus

Introduction

The dramatic increase in the incidence of type 2 diabetes mellitus (T2DM) is a significant burden on global healthcare. Patients who develop T2DM experience multiple co-morbidities, increased mortality, and have significantly higher healthcare costs compared with non-diabetic patients.¹ As part of the effort to slow the expansion of this disease, several studies have demonstrated that both pharmacological and lifestyle interventions significantly decrease development of diabetes.^2–4 Given the estimated 80 million adults⁵ with ‘prediabetes’ in the United States alone, simple and accurate clinical tools are needed to focus limited healthcare resources on those patients with the highest risk for conversion to T2DM.

A variety of tests are available to assess diabetes risk. Glucose-based tests – fasting plasma glucose (FPG), oral glucose tolerance test (OGTT), and HbA1c – measure glycaemia in a fasting or postprandial state. These tests, though rapid and inexpensive, have well-known performance limitations.^6–8 Changes in glucose homeostasis occur relatively late in disease progression, and glucose alone does not reflect the complexity of metabolic disorders arising from a combination of unhealthy adiposity, reduced insulin sensitivity, impaired beta cell function, increased fatty acids, impaired incretin effects, abnormal glucagon, and inflammation.^9,10

Several clinical risk scores have been developed that use non-invasive clinical factors such as age, gender, weight, family history, and exercise habits.^11–13 Other risk indices use a combination of non-invasive clinical factors with fasting plasma glucose and other routine laboratory measures.^14–16 We hypothesise that a risk prediction tool relying primarily on measurements from fasting blood specimens, incorporating markers from multiple aetiopathological pathways, would combine accurate risk prediction with the convenience of a routine clinical blood test.

In a previous investigation of the Inter99 study, we identified biomarkers predictive of diabetes risk,^17,18 and developed a Diabetes Risk Score (DRS) that assesses five-year risk of incident T2DM. The current work was undertaken to validate the clinical performance of the DRS in an independent cohort – the Botnia study¹⁹ – in order to establish the DRS as a potential tool for identifying individuals at risk of T2DM development.

Research design and methods

Inter99 is a randomised non-pharmacological intervention trial for the reduction of cardiovascular disease in subjects, aged 30–60 years, from 11 municipalities in Copenhagen, Denmark. The study included lifestyle counselling on smoking cessation, physical activity, and diet. The five-year rate of progression to T2DM was 4.4% in the Inter99 cohort where five years of follow up was available.

The number of subjects participating in the baseline examination was 6784.²⁰ After five years, 4689 (69%) took part in a subsequent assessment. Fasting blood specimens, data on lifestyle factors, blood pressure, waist circumference, plasma lipids, fasting plasma glucose (FPG), and OGTT were available from baseline and five-year time points. Of 6380 subjects free of diabetes at baseline who participated in the Inter99 study, 215 subjects progressed to diabetes (converters) during five years of follow-up. A nested case-control study was performed using available serum samples from 202 converters and 597 randomly selected non-converters to develop and train the algorithm.¹⁸

The Botnia prospective study enrolled 2770 individuals, mostly from families with a history of T2DM, to identify genes that increase risk of developing T2DM.¹⁹ An OGTT was performed at baseline and every two to three years. Diabetes outcome with biomarkers measurements was available for 2350 subjects (1799 with at least five years of follow up). Among these, the rate of conversion to T2DM was 3.0% at five years (54 out of 1799), and 5.8% overall (136 out of 2350; median follow-up of 7.0 years).

For both the Inter99 and Botnia cohorts, the diagnosis of T2DM was based on the 1997 American Diabetes Association criteria utilising FPG ≥ 7.0 mmol/L or 2-h glucose ≥ 11.1 mmol/L during an OGTT. In addition, the designations of glucose status were based on American Diabetes Association criteria for normal fasting glucose (NFG, < 5.6 mmol/L), impaired fasting glucose (IFG, 5.6–6.9 mmol/L), normal glucose tolerance (NGT, 2-h glucose, < 7.8 mmol/L), and impaired glucose tolerance (IGT, 2-h glucose, 7.8–11.1 mmol/L).²¹ Demographic and clinical differences were compared in both cohorts for converters and non-converters using a Wilcoxon rank-sum test (Table 1).

Table 1.

Baseline clinical characteristics and biomarker levels of study populations

Variables^a	Inter99			Botnia
	Converters(n=171)	Non-Converters (n=4031)	p value^b	Converters(n=136)	Non-Converters (n=2214)	p value^b
	Median(IQR)	median(IQR)		Median (IQR)	Median(IQR)
Age, years	50.1 (45–55)	45.1 (40–50.2)	< 0.0001	52.2 (45–63)	45 (35.9–55.3)	< 0.0001
Gender, male n (%)	106 (62%)	1979 (49%)	p = 0.001	71 (52%)	1018 (46%)	p = 0.1838
BMI, kg/m²	28.6 (26–32.2)	25.3 (23–28)	< 0.0001	28 (25.7–30.8)	25.4 (23–27.9)	< 0.0001
Height, cm	172 (166–177.8)	172 (165–179)	p = 0.9028	168 (162–178)	169 (162.5–176)	p = 0.4863
Weight, kg	84.9 (75.3–97.5)	75.5 (66–86.2)	< 0.0001	81.3 (72–92)	73 (64–82.5)	< 0.0001
Waist, cm	95 (87–106.8)	85 (76–94)	< 0.0001	96 (88–104.2)	86 (78–95)	< 0.0001
Hip, cm	105 (99.2–111)	100 (94–105)	< 0.0001	103.5 (98.8–109.5)	98.5 (93–104)	< 0.0001
SBP, mmHg	140 (130–150)	128 (118–138)	< 0.0001	139 (128–153.5)	127 (116–139)	< 0.0001
DBP, mmHg	86 (80–95)	80 (75–90)	< 0.0001	86 (78–90)	79 (70–86)	< 0.0001
Family history of diabetes, n, (%)	56 (33%)	654 (16%)	< 0.0001	116 (89%)	1881 (85%)	p = 0.3116
Circulating biomarkers
HbA1c, %	6 (5.8–6.4)	5.8 (5.5–6)	< 0.0001	5.7 (5.4–6)	5.4 (5.1–5.7)	< 0.0001
Fasting plasma glucose, mmol/L	6.1 (5.6–6.5)	5.4 (5.1–5.7)	< 0.0001	5.9 (5.5–6.4)	5.5 (5.2–5.9)	< 0.0001
Fasting insulin, pmol/L	54 (35.5–77.5)	32 (22.8–47)	< 0.0001	42 (25.1–67.6)	26.3 (19.2–37)	< 0.0001
Adiponectin, mg/L	6.7 (5.4–8.5)	8.4 (6.6–10.8)	< 0.0001	6.3 (5.1–9.2)	7.3 (5.4–9.7)	p = 0.0138
CRP, mg/L	2.2 (1–4.1)	0.9 (0.4–2.4)	< 0.0001	1.6 (0.9–4)	0.9 (0.4–1.8)	< 0.0001
Ferritin, µg/L	150 (70.7–286.2)	90.7 (41.4–173)	< 0.0001	86 (37.8–152.2)	66 (30–127)	p = 0.0198
IL2Ra, U/mL	387.1 (311.3–483.4)	356.4 (285.7–446.9)	p = 0.0025	330 (275–411.8)	307 (239–393)	p = 0.0044

All circulating biomarkers were measured in serum, or converted to serum equivalents. Comparisons of HbA1c, glucose, and insulin should be interpreted cautiously as they were measured with different assays in Denmark and Sweden.

The p values for continuous variables are those associated with a Wilcoxon rank sum test for quantitative variables; the p values for males and family history are those associated with a Fisher’s exact test.

Laboratory measures

In both studies, anthropometric measurements, routine clinical measures, and the OGTT were performed as previously described.^19,20 Specimens were stored at -19°C. Protein biomarkers were measured in baseline specimens using assays developed for use in a clinical laboratory. Adiponectin (ADIPOQ) was measured with a sandwich enzyme-linked immunosorbent assay, C-reactive protein (CRP) was measured with an immuno-turbidometric assay, ferritin and interleukin-2 receptor alpha (IL2Ra) were measured with a solid-phase, two-site chemiluminescent immunometric assay.¹⁸

Model performance

The DRS was originally developed using one half of the available converters in a classic testing set – validation set study design.¹⁸ This score without any adjustment was calculated for the entire Inter99 cohort for which baseline samples were available and subjects from the Botnia cohort. A score of 5 corresponds to the mean population risk. Absolute five-year risk of conversion may be calculated from the DRS as described in the Supplemental Materials of Kolberg et al.¹⁷

Missing data ranged from 0% to 4.8% of assessments, with the exceptions of 2-h insulin and HbA1c, which were missing in 9% and 28% of Botnia subjects, respectively. Missing values were imputed using Harrell’s Additive Regression function (Hmisc: Harrell Miscellaneous. R package version 3.4-3). Calibration and discrimination metrics showed that the imputed data added little or no bias (data not shown). All circulating biomarker concentrations were log transformed to improve the distributional properties. Discrimination was estimated for the DRS and FPG in both the Botnia and Inter99 populations using the AUC (Figure 1(a)) and differences between the DRS and FPG were tested using the methods of DeLong et al.²³ Model calibration was evaluated both graphically and statistically for the DRS, Framingham diabetes risk score¹⁵ and San Antonio Heart risk score¹⁴ in the Botnia cohort. Graphical evaluation involved plotting the observed risk and mean predicted risk in quintiles of the data (Figure 1(b)). Statistical evaluation was accomplished using a Hosmer–Lemeshow goodness of fit test.²⁴

The Botnia data set was then used to compare the DRS with clinical models from the San Antonio Heart Study¹⁴ and the Framingham Offspring Study,¹⁵ homeostasis model assessment of insulin resistance (HOMA-IR) derived from fasting insulin and glucose (http://www.dtu.ox.ac.uk/homacalculator/index.php), 2-h glucose, peripheral biomarkers (cholesterol, HDL, and triglycerides) and non-invasive risk factors (BMI, waist, and blood pressure) in Table 2. Logistic regression models were developed for the DRS and each of these predictors alone or in combination with the DRS. All model inputs were converted to standard scores prior to analysis and standardised odds ratios were estimated for each predictor as either a univariate marker or after controlling for variation explained by the DRS. Model discrimination was estimated by the AUC of each predictor alone or in combination with the DRS. Incremental improvement of the DRS in combination with all the predictors was also estimated. These analyses were performed using five-year outcomes where converters had to convert within five years or were considered non-converters, and non-converters had to have at least five years of follow-up or they were removed from the data set (54 converters and 1745 non-converters).

In order to determine the relationship between diabetes risk and time to conversion, Botnia subjects were assigned to high (≥ 8.0), moderate (≥ 4.5 to < 8.0), or low (< 4.5) risk strata based on their DRS and diabetes risk was estimated from Kaplan–Meier survival curves. Differences among the curves were compared with a log-rank test and hazard ratios were estimated from a Cox proportional hazard model (Figure 2).

Clinical reclassification

In order to assess how this new risk assessment tool impacts clinical practice, we compared the classification of subjects by DRS to classification using standard fasting glucose and oral glucose tolerance categories in combination (Table 3). Because the number of five-year converters was small (n = 54), we considered all subjects who converted during the study (n = 136; median conversion time 5.5 years) to be converters and defined non-converters as those subjects who did not show evidence of conversion during the study and were followed for at least five years (n = 1663). Subjects with NFG were defined as low baseline risk; moderate and high-risk groups were defined as IFG/NGT and IFG/IGT respectively. The proportions of participants in the Botnia population who were reclassified into either higher or lower risk categories by the DRS were calculated. Net reclassification improvement (NRI) was estimated according the methods of Pencina et al.²⁵ The significance of the NRI estimate was calculated by permutation testing, counting the number of 10,000 permutations of model outputs that had greater NRI estimates than the comparison in question. All analyses were conducted using R V 2.12.1 (R Foundation for Statistical Computing, Vienna, Austria).

Results

The DRS was developed on the Inter99 cohort and validated in the Botnia cohort based on five years of follow-up data. For both cohorts, every variable except height is significantly different between converters and non-converters (Table 1), though gender and family history show differences in Inter99 only. Among the circulating biomarkers, all are significantly different in both cohorts between converters and non-converters. Median values and directions of change between converters and non-converters for each variable are largely consistent between the cohorts. The most striking difference between Inter99 and Botnia is the fraction of subjects with a family history of diabetes, which is expected given that Inter99 is a random population sample while Botnia is, in large part, a family-based study.

Figure 1(a) compares the ROC curves of the DRS for assessing risk of conversion to T2DM in the Inter99 cohort (AUC = 0.84) and in the Botnia cohort (AUC = 0.78). The AUC values for FPG are significantly less in both populations and are shown in both cohorts for comparison. This observation validates the performance of the DRS algorithm in a second population.

Figure 1.

Performance characteristics of the DRS. (a) Receiver operating characteristic (ROC) curves demonstrate the performance of the Diabetes Risk Score (DRS) to identify incident diabetes within five years in the Inter99 and Botnia cohorts. The area under the curve (AUC) for the DRS was 0.84 in the Inter99 cohort and 0.78 in the Botnia cohort. For comparison, the ROC curves of fasting plasma glucose (FPG) are plotted for both studies. The AUC for FPG in Inter99 is 0.78 and in Botnia the AUC for FPG is 0.67. (b) Calibration curves of the DRS, the San Antonio Heart risk score¹⁴ and the Framingham diabetes risk score.¹⁵ In this graph, the predicted risk of each quintile of the population is plotted versus the observed risk in each quintile. The dashed line represents perfect calibration.

Comparison of DRS performance with other risk assessment methods

Figure 1(b) compares the calibration of the DRS with two established clinical risk scores. The DRS slightly overestimates risk in the Botnia cohort by 10% (mean predicted risk = 3.4%, versus a population prior of 3.0%). As shown in Figure 1(b), overestimation of risk is larger with the San Antonio Heart risk score and the Framingham diabetes risk score, which have mean predicted risks of 19% and 36% in this population, respectively. Hosmer–Lemeshow is not significant (p = 0.23) for DRS, but was highly significant (p < 0.0001) for both Framingham and San Antonio Heart indices.

The DRS was compared with other predictors of diabetes risk, including multivariate risk models and scores, individual peripheral biomarkers, and non-invasive risk factors (Table 2). For each predictor, the standardised odds ratio (sOR) and AUC is calculated alone and with the DRS. By sOR, the DRS is superior to all the other predictors and none of the other predictors have a greater AUC than the DRS. Furthermore, adding individual predictors to the DRS to create a composite model reduces the predictability of these markers as evidenced by a reduced odds ratio when modelled with the DRS. In addition, a likelihood ratio test of the univariate marker model to a bivariate model with the marker and the DRS shows that the DRS adds significant value to each of the other predictors. In terms of AUC, these predictors generally fail to improve discrimination when added to DRS. The only exceptions are HOMA-IR (p = 0.035, ΔAUC = 0.024) and BMI (p = 0.02 ΔAUC = 0.026), although 2-h glucose (p = 0.054, ΔAUC = 0.046) and waist circumference (p = 0.059, ΔAUC = 0.023) were marginal and do significantly improve the fit of a bivariate model with DRS (likelihood ratio test, p < 0.0001) along with blood pressure (p < 0.05).

Table 2.

Comparison of risk factor performance

	Odds ratios		AUC
	Risk predictor	With DRS	Predictor alone	With DRS	Delta AUC^b
Multivariate and dynamic scores
DRS, units^a	2.84 (2.16–3.81)	−	0.784	−
Framingham, %	2.54 (1.92–3.43)	1.45 (1.00–2.14)*	0.757	0.788	0.004
San Antonio, %	2.26 (1.85–2.78)	1.51 (1.09–2.11)**	0.77	0.788	0.005
HOMA-IR, units	1.73 (1.47–2.05)	1.44 (1.23–1.70)****	0.751	0.808	0.024**
2-h glucose, mmol/L	2.77 (2.17–3.57)	2.21 (1.70–2.89)****	0.765	0.829	0.046*
Peripheral biomarkers
ADIPOQ, mg/L^a	0.69 (0.48–0.95)	−	0.592	−	−
CHOL, mmol/L	1.23 (0.95–1.57)	0.92 (0.69–1.20)	0.576	0.783	−0.001
CRP, mg/L^a	1.58 (1.31–1.88)	−	0.735	−	−
Fasting insulin, mU/L^a	1.69 (1.42–2.01)	−	0.753	−	−
Fasting plasma glucose, mmol/L^a	1.98 (1.49–2.65)	−	0.675	−	−
Ferritin, µg/L^a	1.07 (0.81–1.30)	−	0.552	−	−
HbA1c, %^a	1.59 (1.32–1.91)	−	0.735	−	−
HDL CHOL, mmol/L	0.67 (0.49–0.90)	0.86 (0.62–1.16)	0.61	0.782	−0.002
IL2Ra, U/mL^a	1.11 (0.85–1.40)	−	0.566	−	−
Triglycerides, mmol/L	1.28 (1.10–1.49)	1.17 (0.96–1.36)	0.675	0.787	0.004
Non-invasive risk factors
Age, years^a	1.87 (1.41–2.50)	−	0.677	−	−
Male^a	1.12 (0.85–1.47)	−	0.529	−	−
BMI, kg/m²	2.02 (1.62–2.52)	1.58 (1.22–2.03)***	0.755	0.809	0.026**
Waist, cm	2.31 (1.79–2.99)	1.68 (1.25–2.25)**	0.756	0.807	0.023*
DBP, mmHg	1.74 (1.33–2.27)	1.38 (1.03–1.85)**	0.661	0.789	0.005
SBP, mmHg	1.97 (1.56–2.48)	1.45 (1.11–1.88)***	0.702	0.787	0.004

Comparison of performance of the DRS against multiple predictors of diabetes risk. The standardized odds ratio (sOR) for each predictor is calculated with and without the DRS. A decrease in the sOR upon addition of DRS indicates that some of the risk for that predictor is being captured by the DRS. Similarly, the AUC for each predictor is calculated without and with the DRS. The delta AUC (DAUC) column shows the incremental information each risk predictor can add to the DRS. Superscripts in the standardized OR with DRS column correspond to a likelihood ratio test comparing it with a model with DRS alone. The superscripts correspond to p values of *< 0.1, **< 0.05, ***< 0.01, ****< 0.001. Superscripts in the delta AUC column compare the discrimination of the DRS with a model containing the DRS and the comparator. Symbols are the same as for the standardized ORs.

Component of the DRS, no correction possible.

p value compares the DRS alone with a model combining the comparator and the DRS.

Long-term risk prediction

Figure 2 presents a Kaplan-Meier plot for subjects classified as high (≥ 8), moderate (≥ 4.5 and < 8.0), or low (< 4.5) risk by DRS using previously defined thresholds.¹⁸ The five-year risk of diabetes for the low, moderate, and high risk groups is 1.1% (95% CI: 0.5–1.6%), 4.0% (2.3–5.7%), and 12.7% (7.0–18.1%), respectively. Although the DRS was designed on a five-year outcome study these risks continue to diverge and at 15 years the Kaplan–Meier estimates are 14% (0.1–26%), 23.3% (13.9–31.7%) and 59.8% (30.3–76.8%), respectively. The difference between the conversion rates among the groups is highly significant (p < 0.0001).

Figure 2.

Kaplan–Meier plot of the Botnia cohort based on DRS risk category. The Botnia cohort was categorised into high, moderate and low-risk groups using the DRS. The plot shows cumulative diabetic events over a period up to 15 years. The incidence for the high-risk group at five years is drawn as a horizontal line. The difference between the conversion rates of the groups is highly significant (χ²=115; 2 df; p < 0.0001).

Reclassification analysis

Reclassification analysis was used to compare the clinical utility of glucose status (using fasting glucose and OGTT) with the DRS (Table 3). Subjects were categorised into low (NFG), moderate (IFG/NGT), and high (IFG/IGT) risk groups based on glucose test results. DRS results were used to reclassify subjects into low, moderate, and high-risk categories using the previously defined thresholds. The overall NRI of 0.20 indicates that the DRS provides more accurate clinical classification than the combined performance of FPG and OGTT (p = 0.0001). This improvement in reclassification is due entirely to accurately reducing the perceived risk of true non-converters (NRI for the non-converters = 0.22, p = 0.0001), the two tests are equal in the ability to identify converters (NRI for the converters = −0.02, p > 0.9999). It should be noted that there is more than an eight-fold difference in risk between high-risk DRS and low-risk DRS (29.4% vs. 3.6%), while difference in risk between IFG/IGT and NFG is only 4.6-fold (22.6% vs. 4.9%).

Table 3.

Risk reclassification in the Botnia cohort comparing the DRS model with glucose measures

			Risk group by DRS			All DRS groups	Number reclassified	NRI (p value)^a
			Low(< 4.5)	Medium(≥ 4.5–8.0)	High(≥ 8.0)
Risk group by glucose measures	LowFPG < 5.6	No. of obs.	784	91	6	881	97(11%)	Non- converters 0.218(0.0001)
		% of pop.	43.6%	5.1%	0.3%	49.0%
		Conv. rate	3.6%	16.5%	0.0%	4.9%
	Medium(FPG 5.6–6.9 and OGTT < 7.8)	No. of obs.	393	271	64	728	457 (62.8%)	Converters −0.022(> 0.9999)
		% of pop.	21.8%	15.1%	3.6%	40.5%
		Conv. rate	3.1%	7.4%	28.1%	6.9%
	High(FPG 5.6–6.9and OGTT 7.8–11.1)	No. of obs.	37	97	56	190	134 (70.5%)
		% of pop.	2.1%	5.4%	3.1%	10.6%
		Conv. rate	10.8%	20.6%	33.9%	22.6%
	All glucose groups	No. of obs.	1214	459	126	1799	688 (38.2%)	Overall0.196(0.0001)
		% of pop.	67.5%	25.5%	7.0%	100%
		Conv. rate	3.6%	12.0%	29.4%	7.6%

DRS: Diabetes Risk Score, FPG: fasting plasma glucose (mmol/L), OGTT: 2-h glucose during oral glucose tolerance test (mmol/L), NRI: net reclassification index, No. of obs.: number of observations, % of pop.: percentage of population, Conv. rate: conversion rate.

Development of T2DM any time during follow-up is counted as an event in this table. Net reclassification index was 0.18 (p = 0.0006).

Conclusions

The major findings of this study are that the DRS assesses risk accurately in an independent population (Botnia; AUC = 0.78), and that it performs better than all standard clinical risk assessment tools, including OGTT. Despite differences in populations and subject-selection criteria in the two cohorts, this finding demonstrates that the methods used to select biomarkers and train the DRS model are not specific to the original population and are generally robust to the variances in sample collection, measurements, and storage. The discrimination in Botnia is somewhat less than Inter99 (AUCs of 0.78 and 0.84, respectively). However, this difference in performance is due almost entirely to the differences in glucose performance between these two populations. In Inter99 the discrimination due to fasting glucose is very good (AUC = 0.78) while the performance in Botnia is surprisingly poor (AUC = 0.67). Given that fasting glucose is part of the DRS, combined with the large amount of missing HbA1c data in the Botnia cohort, this suggests that the differences in marker measurements cause the difference in performance rather than the model being over fitted to the Inter99 cohort. Inter99 was a study in which all laboratory measurements were from a single laboratory. The Botnia study utilises data collected from several centres where glucose was mostly measured in serum but in other cases was measured in plasma and then corrected to serum equivalents. Recent analysis of samples from the Insulin Resistance Atherosclerosis Study (IRAS), a multi-ethnic US-based cohort, showed similar performance of the DRS with an AUC = 0.763.²⁶

It should be noted that most of the subjects in the Botnia cohort were recruited from families with a history of diabetes, while Inter99 is a true population-based study (age-stratified random sample). Given this difference in selection of subjects, it is somewhat surprising that the performance of DRS, which was initially developed on Inter99, is quite similar in the Botnia cohort. These results suggest that the contribution of family history to diabetes risk is being captured by components of DRS. In a previous study Lyssenko et al.²⁷ compared clinical models of diabetes prediction in the Botnia cohort and the Malmo Prospective Preventive Study, a population-based study. Similar to the results reported in this current study, there was very little difference in the predictive models in these two studies. The performance of the DRS, a model based on seven circulating biomarkers (ADIPOQ, CRP, Ferritin, IL2Ra, HbA1c, fasting glucose, and serum insulin) plus age and gender, was benchmarked against other routine laboratory measures, including BMI, waist circumference, blood pressure, total and HDL cholesterol, FPG, HOMA-IR, and OGTT, as well as the San Antonio and Framingham clinical risk models. DRS provided better discrimination between converters and non-converters than any of these other predictors (Table 2) and the calibration of DRS was also superior to that of either clinical risk model (Figure 1(b)). Furthermore, most of the risk contributed by these measures is already incorporated in the DRS since adding these predictors to DRS in most cases did not improve discrimination, with the notable exceptions of HOMA-IR, BMI, and perhaps OGTT and waist circumference. These observations suggest that the DRS could be improved with markers of β-cell function and obesity. While the DRS has similar performance in these two cohorts, additional studies are required to evaluate the performance of DRS in other populations.

The Botnia cohort had frequent testing for outcomes and a long period of follow-up. This allowed the DRS to be assessed using survival analysis over a longer time horizon than it had been trained for. The Kaplan–Meier (Figure 2) curve shows a strong difference in diabetes conversion between the three risk groups, with hazard ratios of 8.0 (95% CI 5.2–12.5) for the high and 3.2 (2.1–4.7) for the moderate risk groups relative to the low-risk group. The high-risk group has a cumulative risk of 12.7% at five years (see horizontal line, Figure 2), and 59.8% at 15 years; the moderate-risk and low-risk groups do not cross the 13% threshold until 11 years and > 15 years respectively. Thus it appears that the DRS provides sufficient intervention time for the highest-risk subjects, while giving advance warning for the moderate-risk subjects. The low-risk population appears to be truly low-risk; the conversion probability did not change significantly until > 10 years from baseline.

The clinical utility of the DRS was compared with glucose-based tests in a reclassification analysis. The DRS was able to significantly reclassify high, moderate, and low-risk subjects in a comparison with three strata of risk based on glucose measures – NFG, IFG/NGT, and IFG/IGT. A striking result is that subjects with a low DRS have a low risk of developing diabetes regardless of glucose status. In addition, 54% of IFG/NGT subjects have a lower risk than subjects with NFG. Similarly, the conversion rate of the high-risk DRS group across all glucose classes (29.4% incidence) is higher than IFG/IGT subjects (22.6% incidence). This indicates that prevention resources invested in high-risk DRS subjects may yield a greater health economic benefit than investing in all IFG/IGT subjects; preliminary results are encouraging.²⁸

The DRS was developed with the assumption that multiple biomarkers predictive of and associated with disease pathogenesis should provide more powerful tools to identify patients at highest risk. The seven circulating biomarkers used in the DRS are involved in diverse biological pathways associated with the pathogenesis of T2DM. Individually, no single marker improves upon current clinical tools, but the combination of biomarkers in a single score performs better than all the tests evaluated, in an independent population. The DRS identifies high-risk subjects for whom earlier and more aggressive interventions to delay or prevent the onset of diabetes could be considered.

Footnotes

Acknowledgements

The Inter99 study was initiated by T Jørgensen (PI), K Borch-Johnsen (PI for the diabetes part), T Thomsen, and H Ibsen; the present steering group comprises T Jørgensen (PI), K Borch-Johnsen (co-PI), and C Pisinger. The authors gratefully acknowledge the skilful technical assistance of Claudia Elkin, Amalia Ocampo, and Edward Moler of Tethys Bioscience, and would like to thank Laura Penny and Linda Wuestehube for assistance in preparing the manuscript.

This work was supported by the Sigrid Juselius Foundation, the Folkhälsan Foundation, the Finnish Diabetes Research Association, the Nordic Centre of Excellence in Disease Genetics and a Linne grant from the Swedish Research Council to the Lund University Diabetes Centre (Botnia study), the Danish Medical Research Council, the Danish Centre for Evaluation and Health Technology Assessment, Novo Nordisk, Copenhagen County, the Danish Heart Foundation, the Danish Pharmaceutical Association, Augustinus Foundation, Ib Henriksen Foundation and Becket Foundation (Inter99 study). All clinical activities of the Steno Diabetes Centre are paid by the public health care system, but Novo Nordisk strongly subsidises the research activities of the Steno Diabetes Centre. Tethys Bioscience funded the generation of data and part of the results presented in this manuscript.

LG has been a consultant for and served on advisory boards for Tethys Bioscience, Inc., Sanofi-Aventis, GSK, Lilly, Merck, and Novartis; also he has lectured at meetings organised by Novartis, GSK, and Sanofi-Aventis. Until 1 December 2010, KB-J was employed as a Director and Professor of the Steno Diabetes Centre, a hospital providing health service for the public health care system but owned by Novo Nordisk A/S, and thus has been an employee of Novo Nordisk A/S, a company that manufactures and markets pharmaceuticals related to the treatment of diabetes; he also owns shares in Novo Nordisk and has within the last five years received honorarium for invited lectures by Bristol-Myers Squibb and Lilly. OP is employed as a director of research at Hagedorn Research Institute, an academic research centre, which is owned by Novo Nordisk A/S; he owns employee shares in Novo Nordisk and has lectured at meetings organised by Novo Nordisk, Servier, Tethys, and GSK. JK, RG, MM, and MR are employees and shareholders of Tethys Bioscience, Inc. No other authors reported conflict of interest.

References

Hoerger

Hicks

Sorensen

. Cost-effectiveness of screening for pre-diabetes among overweight and obese U.S. adults. Diabetes Care 2007; 30: 2874–9.

Tuomilehto

Lindstrom

Eriksson

. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med 2001; 344: 1343–1350.

Knowler

Barrett-Connor

Fowler

. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 2002; 346: 393–403.

Buchanan

Xiang

Peters

. Preservation of pancreatic beta-cell function and prevention of type 2 diabetes by pharmacological treatment of insulin resistance in high-risk hispanic women. Diabetes 2002; 51: 2796–2803.

James

Bullard

Rolka

. Implications of alternative definitions of prediabetes for prevalence in U.S. adults. Diabetes Care 2011; 34: 387–391.

Edelstein

Knowler

Bain

. Predictors of progression from impaired glucose tolerance to NIDDM: an analysis of six prospective studies. Diabetes 1997; 46: 701–710.

Nichols

Hillier

Brown

. Progression from newly acquired impaired fasting glucose to type 2 diabetes. Diabetes Care 2007; 30: 228–233.

Selvin

Crainiceanu

Brancati

Coresh

. Short-term variability in measures of glycemia and implications for the classification of diabetes. Arch Intern Med 2007; 167: 1545–1551.

Bergman

. Orchestration of glucose homeostasis: from a small acorn to the California oak. Diabetes 2007; 56: 1489–1501.

10.

Defronzo

. Banting Lecture. From the triumvirate to the ominous octet: a new paradigm for the treatment of type 2 diabetes mellitus. Diabetes 2009; 58: 773–795.

11.

Lindstrom

Tuomilehto

. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 2003; 26: 725–731.

12.

Herman

Smith

Thompson

Engelgau

Aubert

. A new and simple questionnaire to identify people at increased risk for undiagnosed diabetes. Diabetes Care 1995; 18: 382–387.

13.

Schulze

Hoffmann

Boeing

. An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 2007; 30: 510–515.

14.

Stern

Williams

Haffner

. Identification of persons at high risk for type 2 diabetes mellitus: do we need the oral glucose tolerance test? Ann Intern Med 2002; 136: 575–581.

15.

Wilson

Meigs

Sullivan

Fox

Nathan

D’Agostino

Sr . Prediction of incident diabetes mellitus in middle-aged adults: the Framingham Offspring Study. Arch Intern Med 2007; 167: 1068–1074.

16.

Kahn

Cheng

Thompson

Imperatore

Gregg

. Two risk-scoring systems for predicting incident diabetes mellitus in U.S. adults age 45 to 64 years. Ann Intern Med 2009; 150: 741–751.

17.

Kolberg

Jørgensen

Gerwien

. Development of a type 2 diabetes risk model from a panel of serum biomarkers from the Inter99 cohort. Diabetes Care 2009; 32: 1207–1212.

18.

Urdea

Kolberg

Wilber

. Validation of a multimarker model for assessing risk of type 2 diabetes from a five-year prospective study of 6784 Danish people (Inter99). J Diabetes Sci Technol 2009; 3: 748–755.

19.

Lyssenko

Almgren

Anevski

. Predictors of and longitudinal changes in insulin sensitivity and secretion preceding onset of type 2 diabetes. Diabetes 2005; 54: 166–174.

20.

Jorgensen

Borch-Johnsen

Thomsen

Ibsen

Glumer

Pisinger

. A randomized non-pharmacological intervention study for prevention of ischaemic heart disease: baseline results Inter99. Eur J Cardiovasc Prev Rehabil 2003; 10: 377–386.

21.

Expert Committee on the Diagnosis and Classification of Diabetes Mellitus. Report of the Expert Committee on the Diagnosis and Classification of Diabetes Mellitus. Diabetes Care 1997; 20: 1183–1197.

22.

Hanley

McNeil

. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29–36.

23.

DeLong

Clarke-Pearson

. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837–845.

24.

Hosmer

Lemeshow

Applied logistic regression. New York: Wiley, 1989, p.xiii.

25.

Pencina

D’Agostino

Sr D’Agostino

Jr Vasan

. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27: 157–172; discussion 207–212.

26.

Kolberg

Wagenknecht

Rowe

Bergman

. Performance of a multi-marker diabetes risk score on the Insulin Resistance Atherosclerosis Study (IRAS), a multi-ethnic US cohort. In: 6th World Congress on Prevention of Diabetes and its Complications (WCPD 2010), 8–11 April 2010, Dresden, Germany.

27.

Lyssenko

Jonsson

Almgren

. Clinical risk factors, DNA variants, and the development of type 2 diabetes. N Engl J Med 2008; 359: 2220–2232.

28.

Sullivan

Pollock

Garrison

Rinde

Kolberg

. Long-term cost-effectiveness of a diabetes risk score. Diabetes 2009; 58: A44.