Sage Journals: Discover world-class research

Abstract

Background

Ionized hypocalcemia is common in critically ill patients with COVID-19 and is associated with adverse outcomes. We previously developed a linear model that estimates ionized calcium (I_Ca) by adjusting total calcium (T_Ca) for the three components of the anion gap and albumin. On internal validation, it outperformed the popular method that corrects T_Ca for albumin alone (cT_Ca) in diagnosing low I_Ca. In this study, we sought to externally validate our I_Ca model in hospitalized COVID-19 positive patients.

Methods

We retrospectively studied all 200 patients with COVID-19 who were admitted to the State University of New York Downstate Medical Center between March 11^th and April 30^th 2020 and referred to the nephrology service for renal failure, and who had I_Ca measured on a venous blood gas within 25 min of a comprehensive metabolic panel. We compared the performance of the I_Ca model and cT_Ca in diagnosing low I_Ca by ROC analysis, and also examined the accuracy of the absolute values predicted by the two methods relative to measured I_Ca.

Results

On ROC analysis, the I_Ca model was better than cT_Ca (area under ROC curve: 0.872 [0.025] vs. 0.835 [0.028]; p = 0.045). The I_Ca model estimated I_Ca accurately, but the cT_Ca method seemed to overcorrect T_Ca, as a substantial number of patients with clearly normal cT_Ca values had low I_Ca.

Conclusions

In an external validation cohort, the I_Ca model estimated I_Ca accurately and was better than cT_Ca in the diagnosis of low I_Ca. This finding can be useful in guiding direct I_Ca testing.

Keywords

Calcium electrolytes

Introduction

New York City was the epicenter of the COVID-19 pandemic, affecting more than 3400 persons per 100,000 in the borough of Brooklyn alone.¹ Many people were critically ill with multiorgan system involvement and required intensive care. Low ionized calcium (I_Ca) is a common abnormality among critically ill COVID-19 patients and has been found to be an important predictor of COVID-19 positivity at triage and of a poor outcome in infected patients, in analyses that accounted for pH and other covariates.^2-4 Although low I_Ca can be diagnosed with the ion-selective electrodes available on blood gas analyzers and other point-of-care analyzers, direct measurement is still not routine, because of its cost, labor and sample requirements,^5-8 and the ferocity of the pandemic has further strained hospital resources. As a result, it is still common clinical practice to infer the presence of an abnormal I_Ca indirectly, using an equation that “corrects” the concentration of total calcium (T_Ca) for that of albumin, with the resultant corrected value (cT_Ca) compared to the reference range for T_Ca.⁹ The cT_Ca equation was popularized in the 1970s, without validation against I_Ca. Subsequent external validation studies, which incorporated I_Ca, found the diagnostic performance of cT_Ca to be poor,^8,10-12 but by then its routine use in clinical practice had been cemented. Commonly suggested reasons for the poor performance of cT_Ca include its failure to account for alterations in the affinity of albumin for calcium caused by changes in pH and free fatty acids in critical care patients, or for the differences among centers in the albumin assay.^8,10-14 Another suggested reason is that cT_Ca does not account for variation in the level of small anions that can bind calcium.^13,15-17

To account for binding of calcium to small anions and albumin, we recently derived a novel linear regression equation (I_Ca model) that estimates I_Ca in critical care patients from routine measurements of T_Ca, serum albumin, and the three components of the anion gap—sodium (Na), chloride (Cl), and total carbon dioxide (tCO₂).¹³ In that study, the serum chemistry measurements were made using the Siemens ADVIA Chemistry System 1650 and 1800 analyzers, while I_Ca was measured using the Radiometer ABL800 FLEX analyzer. The model’s output consists of the most likely arterial I_Ca value for a given patient together with a 95% prediction interval (PI), the range that should include the patient’s true value 95% of the time. Upon internal validation, the model significantly outperformed cT_Ca in detecting low I_Ca. Since external validation is essential to demonstrate a model’s predictive value,¹⁸ in the present study, we tested the ability of the I_Ca model to diagnose low I_Ca and to estimate the concentration of I_Ca at a different clinical site using a different chemistry analyzer in patients who had closely timed chemistry and venous blood gas (VBG) panels during hospitalization for severe COVID-19 infection, and who, in the course of which, were referred to the nephrology service for renal failure.

Methods

Data collection and pairing

In March 2020, the State University of New York (SUNY) Downstate Medical Center was designated as COVID-19-only hospital by state directive.¹⁹ We retrospectively collected serum comprehensive metabolic panel and VBG panel results of every COVID-19 positive patient referred to the nephrology service at our center between March 11^th and April 30^th 2020, excluding metabolic panels missing T_Ca, albumin, Na, Cl, or tCO₂ and VBGs missing I_Ca. The diagnoses for which patients were seen by the nephrology service were either acute kidney injury or pre-existing end stage renal disease (ESRD), with the majority admitted to intensive care. Age and sex were recorded for each patient. In addition, we recorded and paired every metabolic panel with a VBG panel—if the panels were separated by no more than 25 min. Limiting each patient to the single most closely-timed pair of panels, and choosing the earliest pair in case of ties, left 200 pairs, with a mean intra-pair time gap of 0.87 min (range: 0–22 min). The protocol was approved by the institutional review board with a waiver of the need for informed consent (IRB no. 1584306-6).

Analytic methods

Serum metabolic panels were performed on the Beckman AU5800 analyzer (Beckman Coulter, Brea, CA, USA). The T_Ca assay uses the Arsenazo III method, with a linear range (LR) of 1.00–4.49 mmol/L (4.0–18.0 mg/dL) and an expected coefficient of variation (%CV) of 1.34% at a T_Ca of 2.03 mmol/L (8.12 mg/dL); the local reference range (LRR) is 2.05–2.50 mmol/L (8.2–10.0 mg/dL). Albumin was measured by the bromocresol green method (LR: 15–60 g/L; %CV: 1.5% at 29 g/L; LRR: 35–57 g/L). TCO₂ was measured by an enzymatic method (LR: 2.0–45.0 mmol/L; %CV: 4.0% at 31 mmol/L; LRR: 21–31 mmol/L). Indirect potentiometry was used to measure Na (LR: 50–200 mmol/L; %CV: 0.9% at 127 mmol/L; LRR: 136–145 mmol/L) and Cl (LR: 50–200 mmol/L; %CV: 0.9% at 83 mmol/L; LRR: 98–107 mmol/L). I_Ca was obtained from whole blood VBG panels analyzed on the ABL837 FLEX analyzer (Radiometer, Copenhagen), with I_Ca measured by calcium electrode (LR: 0.23–3.86 mmol/L; local %CV: 0.4% at 1.03 mmol/L; LRR: 1.15–1.29 mmol/L). VBGs were collected using the PULSATOR sampler (Smiths-Medical, Keene, NH, USA).

Data analysis

Continuous variables are summarized as the mean and SD and compared using the paired t-test and simple correlation. Hypocalcemia in VBGs was defined as I_Ca <1.11 mmol/L, as in our prior study.¹³ For each patient, the anion gap was calculated as: Na (mmol/L) – Cl (mmol/L) – tCO₂ (mmol/L), and cT_Ca (mmol/L) was calculated as: T_Ca (mmol/L) + 0.02 × [40 – albumin (g/L)].⁹ I_Ca (mmol/L) was estimated by the I_Ca model as: 0.219 +0.365 × T_Ca – 0.0034 × albumin – 0.0042 × Na + 0.0073 × Cl + 0.0047 × tCO₂; the average 95% PI of this estimate is: ± 0.115 mmol/L (i.e., an average range of 0.23 mmol/L).^13,20

The I_Ca and cT_Ca models were examined and compared in two ways. Calibration was assessed by examining their mean predictions relative to the observed mean I_Ca values in the entire cohort and in subgroups. Diagnostic discrimination for ionized hypocalcemia was assessed with ROC curves. Since the hypothesis being tested is that the I_Ca model would outperform cT_Ca in ranking patients for hypocalcemia, the areas under the curves (AUC) and SE were compared using a one-tailed test in accordance with the method of Hanley and McNeil.²¹ The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guideline was utilized,²² and the checklist is available in the Supplement. IBM SPSS Statistics for Windows, Version 24.0 (Armonk, NY: IBM Corp) was used.

Results

Baseline characteristics and simple comparisons

The characteristics of the cohort and their biochemical measurements are shown in Table 1. The mean value of T_Ca (2.06 mmol/L) was just above the lower limit of normal (LLN) for T_Ca (2.05 mmol/L), while the mean of I_Ca (1.10 mmol/L) was well below its LLN (1.15 mmol/L; p < 10⁻¹⁰). Ionized hypocalcemia, defined here as I_Ca < 1.11 mmol/L, was present in 96 of 200 patients (48%), and was more common in the patients with ESRD than in the remaining patients (57% versus 42%, p < 0.05). Albumin correlated with T_Ca (r = 0.50 p < 10⁻¹³), but not with either I_Ca (r = −0.07 p = 0.31) or the anion gap (r = 0.016 p = 0.82). I_Ca tended to decrease as the anion gap increased (r = −0.21 p = 0.003). The correlation of I_Ca with the three “indirect” measures of calcium status was strongest with the I_Ca model (r = 0.77 p < 10⁻³⁹), intermediate with cT_Ca (r = 0.71 p < 10⁻³¹), and weakest with T_Ca (r = 0.58 p < 10⁻¹⁸).

Table 1.

Demographic and laboratory data.

Age (years)	65.7 ± 15.4
Male: Female (n)	124:76
End stage renal disease (n) (%)	81 [40.5%]
Acute kidney injury	119 [59.5%]
T_Ca (mmol/L)	2.06 ± 0.20 [1.60–2.75]
I_Ca (mmol/L)	1.10 ± 0.10 [0.85–1.39]
Albumin (g/L)	31.6 ± 7.0 [15.2–47.7]
Sodium (mmol/L)	137.4 ± 9.2 [106–175]
Chloride (mmol/L)	100.7 ± 9.8 [76–139]
TCO₂ (mmol/L)	21.6 ± 5.8 [4.00–39.00]
Anion gap, mmol/L	15.0 ± 5.9 [4–44]

Data are presented as mean ± SD [minimum-maximum]. To convert total calcium to mg/dL, divide by 0.2495. To convert albumin to g/dL, divide by 10. Hypercalcemia was infrequent: 9 cases had I_Ca ≥1.3 mmol/L.

Diagnostic performance and calibration

The AUC of the I_Ca model was better than that of cT_Ca (0.872 [0.025] versus 0.835 [0.028]); p = 0.045; Figure 1) and much better than that of T_Ca (0.787 [0.032]). The AUC difference remained significant when we compared novel cT_Ca and I_Ca models derived and internally validated in the present cohort (see Supplemental Results).

Figure 1.

ROC curves illustrating the overall diagnostic performance for hypocalcemia of the I_Ca model (solid line) and cT_Ca (broken line). The AUCs are 0.872 (0.025) for the I_Ca model and 0.835 (0.028) for cT_Ca (p = 0.045). A diagonal line depicts the points of equal sensitivity and specificity. Going from upper left to lower right, it intersects the I_Ca curve at predicted I_Ca <1.120 mmol/L (sensitivity and specificity ∼81%) and the cT_Ca curve at cT_Ca < 2.230 mmol/L (sensitivity and specificity ∼77%).

There was reasonable agreement between mean I_Ca values predicted by the model and those observed across 6 subgroups (Table 2A), with the model having a slight overall positive bias of +0.022 mmol/L (1.124 [0.082] versus 1.102 [0.099] mmol/L, p < 10⁻⁵). The imprecision of the model’s predictions was at least as good as expected. For example, of the 37 patients whose predicted I_Ca values fell within the 0.03 mmol/L range from 1.08–1.11 mmol/L, the central 95% (n = 35) had observed I_Ca values that ranged from 0.98–1.16 mmol/L; the observed range (0.18 mmol/L) was better than the range of model’s expected 95% PI (0.23 mmol/L).

Table 2.

I_Ca values and hypocalcemia rates observed across six prediction categories of the I_Ca model (A) and of the cT_Ca equation (B).

(A) Patients Ranked by I_Ca Model
I_Ca model category^a	N	Mean predicted I_Ca^a	Mean I_Ca observed [range]^a	Rate of low I_Ca (%)
<1.0	11	0.95^b	0.96 [0.85–1.07]	100
1.00 to <1.10	64	1.06	1.04 [0.89–1.21]	83
1.10 to <1.15	53	1.12	1.10 [1.01–1.18]	43
1.15 to <1.25	60	1.19	1.16 [0.89–1.35]	13
1.25 to <1.30	6	1.27	1.24 [1.08–1.32]	17
≥1.30	6	1.33^b	1.30 [1.24–1.39]	0

(B) Patients Ranked by cT_Ca Model
cT_Ca category^a	N^c	Mean I_Ca observed [range]^a	Rate of low I_Ca (%)
1.55 to 1.93	11	0.94 [0.85–1.04]	100
1.94 to <2.19	64	1.05 [0.89–1.17]	77
2.19 to 2.28	53	1.11 [0.89–1.24]	45
2.29 to 2.52	60	1.17 [0.95–1.35]	17
2.53 to 2.56	6	1.24 [1.18–1.30]	0
2.57 to 2.72	6	1.20 [0.94–1.39]	33

^aUnits are mmol/L.

^bMinimum and maximum predicted I_ca were 0.91 and 0.998 mmol/L for first category (<1.0 mmol/L) and 1.30–1.34 mmol/L for the sixth category (≥1.30 mmol/L).

^cThe sizes of the cT_Ca categories were chosen to match those used for the I_Ca model in section A above.

Table 2B shows the performance of cT_Ca with the patients ranked into subgroups identical in size to those of Table 2A. The rates of hypocalcemia observed across the 6 subgroups show, as in the ROC analysis, that ranking by cT_Ca was worse than ranking by the I_Ca model. A more significant shortcoming of cT_Ca, as the overall mean cT_Ca of 2.23 (0.18) mmol/L suggests, was its misleading calibration relative to I_Ca. For example, the cT_Ca values in the third subgroup ranged from 2.19–2.28 mmol/L. Since each value was well above the LLN for T_Ca, a negligible risk of hypocalcemia would be expected. However, the mean I_Ca in this subgroup was only 1.11 mmol/L, and the rate of hypocalcemia was substantial (45%). This problem is evident in the fourth and sixth cT_Ca subgroups too.

Discussion

Abnormal I_Ca is detected poorly by cT_Ca, which adjusts T_Ca for albumin alone.^8,10-12 Since calcium binds to small anions as well as to albumin, the further adjustment of T_Ca for the anion gap has been suggested as a possible improvement.^13,15-17 We previously derived a model that estimates I_Ca by adjusting T_Ca for the components of the anion gap and albumin. On internal validation, it was better than cT_Ca in the diagnosis of ionized hypocalcemia.¹³ In the present study, we confirmed that I_Ca varies inversely with the anion gap, and externally validated our I_Ca model in a cohort of hospitalized patients with COVID-19 who developed concomitant renal injury or had pre-existing renal disease, conditions in which low I_Ca is prevalent and of clinical importance.^2-4,8 On ROC analysis, the I_Ca model was better than cT_Ca in the relative ranking of patients for the diagnosis of hypocalcemia (Figure 1). This was also evident by comparing the respective hypocalcemia rate trends observed across subgroups ranked by the I_Ca model and by the cT_Ca equation (Table 2).

In absolute terms, the cT_Ca equation appeared to “overcorrect” T_Ca by assigning values well above the LLN to several subgroups in Table 3 that proved to have significant rates of ionized hypocalcemia. Since cT_Ca values are intended to be judged by the reference range for T_Ca, reliance on cT_Ca will cause the diagnosis to be entirely overlooked in those patients. This poor alignment of cT_Ca with I_Ca could be the result of the absence of critically ill patients in the cT_Ca equation’s original derivation cohort, or calibration differences between the original and the local laboratories, and it could hypothetically be mitigated by introducing a variety of local center corrections, with the simplest being a fixed reduction to cT_Ca of, for example, 0.17 mmol/L (equivalent to changing the reference value of albumin used in the cT_Ca equation from 40 g/L to the current cohort’s mean of 31.6 g/L).^14,23 It is noteworthy that the I_Ca model required no such local correction in this external validation study. It provided accurate absolute estimates of mean I_Ca within the cohort overall and its subgroups, with only a small positive bias of approximately 0.02 mmol/L (Table 2). A bias of this size is clinically inconsequential, and especially so in respect to the size of the I_Ca model’s expected 95% PI (±0.115 mmol/L), a degree of imprecision similar to that of a recent canine I_Ca model.¹⁷ Such imprecision is inevitable when regression equations are applied to individual patients, with an extreme example being the 95% PI of glomerular filtration rate estimating (eGFR) equations.^24-26 Consequently, although the I_Ca model’s estimates are accurate on average, owing to their imprecision, they cannot replace direct I_Ca measurement, the gold standard for guiding treatment decisions at the individual patient level. Low I_Ca is associated with increased risk in COVID-19, but its measurement is relatively costly. Therefore, we propose that clinicians can more efficiently screen for low I_Ca and, in turn, decide when confirmatory I_Ca testing is needed by using the I_Ca model’s point prediction in one of two ways—either by simply comparing it to a cutoff (e.g., <1.15 mmol/L) or by combining it with its 95% PI to define a range (i.e., point prediction ±0.115 mmol/L) that can be used to intuitively assess the likelihood of low I_Ca.

Although the I_Ca model may not be easy to memorize, its output can be feasibly reported in metabolic panel reports, as is common practice for cT_Ca, anion gap, eGFR, and other forms of laboratory-based clinical decision support. Alternatively, the model output could be easily obtained with an internet-based calculator or smartphone app (one is available on the qxmd.com website).²⁰

The strength of our study is that it tested a prior hypothesis and that the patients studied had COVID-19 and renal failure, conditions in which identification of low I_Ca can have a clinical impact. Nonetheless, since most of these patients were critically ill, the application of the I_Ca model to less seriously ill patients, such as outpatients with ESRD, cannot be established from the present study, although the model has performed well in such patients in our informal experience. Further validation on other analytic platforms is also warranted. Other limitations of the study include its retrospective data collection, so the possibility that some VBG samples were processed with an excessive delay, potentially resulting in an artifactual fall in pH and reciprocal rise I_Ca,⁸ cannot be excluded, and that the cohort studied was relatively small and generated by referrals.

In summary, we externally validated our linear I_Ca model in a novel laboratory environment by showing that it accurately predicted I_Ca and was better than cT_Ca in diagnosing ionized hypocalcemia. The point predictions of the model, together with their 95% PI, can be used to help clinicians decide when direct I_Ca measurement is needed in individual patients.

Supplemental Material

sj-pdf-1-acb-10.1177_00045632211049983 – Supplemental Material for Predicting ionized hypocalcemia: External validation of an ionized calcium prediction model in patients with COVID-19 and renal failure

Supplemental Material, sj-pdf-1-acb-10.1177_00045632211049983 for Predicting ionized hypocalcemia: External validation of an ionized calcium prediction model in patients with COVID-19 and renal failure by Ernie Yap, Yohannes Melaku, Isha Puri, Jie Ouyang and Philip Goldwasser in Annals of Clinical Biochemistry

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

The protocol was approved by the SUNY Downstate Health Sciences University Institutional Review Board & Privacy Board with a waiver of the need for informed consent (IRB no. 1584306-6).

Guarantor

EY.

Contributorship

EY and PG contributed to the study design, statistical analysis, and results interpretation. YM, IP, and JO contributed to data acquisition. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved.

ORCID iD

Ernie Yap

Supplemental material

Supplemental material for this article is available online.

References

NYC Health . https://www1.nyc.gov/site/doh/covid/covid-19-main.page. https://www1.nyc.gov/site/doh/covid/covid-19-main.page (accessed April 2021)

Di Filippo

Formenti

Rovere-Querini

, et al. Hypocalcemia is highly prevalent and predicts hospitalization in patients with COVID-19. Endocrine 2020; 68: 475–478. DOI: 10.1007/s12020-020-02383-5.

Elezagic

Johannis

Burst

, et al. Venous blood gas analysis in patients with COVID-19 symptoms in the early assessment of virus positivity. J Lab Med 2021; 45: 27–30. DOI: 10.1515/labmed-2020-0126 .

Cappellini

Brivio

Casati

, et al. Low levels of total and ionized calcium in blood of COVID-19 patients. Clin Chem Lab Med (Cclm) 2020; 58: e171–e173. DOI: 10.1515/cclm-2020-0611.

Calvi

Bushinsky

. When is it appropriate to order an ionized calcium? J Am Soc Nephrol. 2008; 19: 1257–1260. DOI: 10.1681/asn.2007121327.

Anonymous. Reducing high cost ionized calcium testing. J Hosp Med 2014; 9 (suppl. 2): 254. https://shmabstracts.org/abstract/reducing-high-cost-ionized-calcium-testing/ (accessed April 2021).

Farmer

. Costs and stewardship of laboratory tests in the capital health district. Dalhousie Med J 2015; 42: 3–7.

Hamroun

Pekar

J-D

Lionet

, et al. Ionized calcium: analytical challenges and clinical relevance. J Lab Precision Med 2020; 5: 22. DOI: 10.21037/jlpm-20-60.

Anonymous . Correcting the calcium. Br Med J 1977; 1: 326.

10.

Dickerson

Alexander

Minard

, et al. Accuracy of methods to estimate ionized and “corrected” serum calcium concentrations in critically ill multiple trauma patients receiving specialized nutrition support. J Parenter Enteral Nutr 2004; 28: 133–141. DOI: 10.1177/0148607104028003133.

11.

Ladenson

Lewis

Boyd

. Failure of total calcium corrected for protein, albumin, and pH to correctly assess free calcium status. J Clin Endocrinol Metab 1978; 46: 986–993. DOI: 10.1210/jcem-46-6-986.

12.

Pekar

J-D

Grzych

Durand

, et al. Calcium state estimation by total calcium: the evidence to end the never-ending story. Clin Chem Lab Med 2020; 58: 222–231. DOI: 10.1515/cclm-2019-0568.

13.

Yap

Roche-Recinos

Goldwasser

. Predicting ionized hypocalcemia in critical care: an improved method based on the anion gap. J Appl Lab Med 2020; 5: 4–14. DOI: 10.1373/jalm.2019.029314.

14.

Jassam

Thomas

Hayden

, et al. The impact of the analytical performance specifications of calcium and albumin on adjusted calcium. Ann Clin Biochem 2020; 57: 382–388. DOI: 10.1177/0004563220944426.

15.

Nordin

Need

Hartley

, et al. Improved method for calculating calcium fractions in plasma: reference values and effect of menopause. Clin Chem 1989; 35: 14–17.

16.

Takano

Kaji

Hayashi

, et al. A calculation model for serum ionized calcium based on an equilibrium equation for complexation. Anal Chem Insights 2012; 7: ACI.S9681. DOI: 10.4137/aci.S9681.

17.

Danner

Ridgway

Rubin

, et al. Development of a multivariate predictive model to estimate ionized calcium concentration from serum biochemical profile results in dogs. J Vet Intern Med 2017; 31: 1392–1402. DOI: 10.1111/jvim.14800.

18.

Moons

KGM

Kengne

Grobbee

, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012; 98: 691–698. DOI: 10.1136/heartjnl-2011-301247.

19.

DeJesus

. Cuomo designates university hospital of Brooklyn as a COVID-19 only facility, 2020. https://brooklynreporter.com/2020/03/cuomo-designates-university-hospital/ (accessed April 2021)

20.

https://qxmd.com/calculate/calculator_704/predicting-ionized-hypocalcemia-in-critical-care.

21.

Hanley

McNeil

. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148: 839–843. DOI: 10.1148/radiology.148.3.6878708.

22.

Collins

Reitsma

Altman

, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med 2015; 13: 1. DOI: 10.1186/s12916-014-0241-z.

23.

Weaving

. Calcium — An adjustment too far or not far enough? Ann Clin Biochem 2020; 57: 401–403. DOI: 10.1177/0004563220961751.

24.

Altman

Gardner

. Calculating confidence intervals for regression and correlation. Br Med J (Clin Res Ed 1988; 296: 1238–1242. DOI: 10.1136/bmj.296.6631.1238.

25.

Roth

. Prediction interval analysis is underutilized and can be more helpful than just confidence interval analysis. J Clin Monit Comput 2009; 23: 181–183. DOI: 10.1007/s10877-009-9165-0.

26.

Sehgal

. Race and the false precision of glomerular filtration rate estimates. Ann Intern Med 2020; 173: 1008–1009. DOI: 10.7326/m20-4951.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB