Automated safety event monitoring using electronic medical records in a clinical trial setting: Validation study using the VA NEPHRON-D trial

Abstract

Background/Aims:

Electronic medical records are now frequently used for capturing patient-level data in clinical trials. Within the Veterans Affairs health care system, electronic medical record data have been widely used in clinical trials to assess eligibility, facilitate referrals for recruitment, and conduct follow-up and safety monitoring. Despite the potential for increased efficiency in using electronic medical records to capture safety data via a centralized algorithm, it is important to evaluate the integrity and accuracy of electronic medical record–captured data. To this end, this investigation assesses data collection, both for general and study-specific safety endpoints, by comparing electronic medical record–based safety monitoring versus safety data collected during the course of the Veterans Affairs Nephropathy in Diabetes (VA NEPHRON-D) clinical trial.

Methods:

The VA NEPHRON-D study was a multicenter, double-blind, randomized clinical trial designed to compare the effect of combination therapy (losartan plus lisinopril) versus monotherapy (losartan) on the progression of kidney disease in individuals with diabetes and proteinuria. The trial’s safety outcomes included serious adverse events, hyperkalemia, and acute kidney injury. A subset of the participants (~62%, n = 895) enrolled in the trial’s long-term follow-up sub-study and consented to electronic medical record data collection. We applied an automated algorithm to search and capture safety data using the VA Corporate Data Warehouse which houses electronic medical record data. Using study safety data reported during the trial as the gold standard, we evaluated the sensitivity and precision of electronic medical record–based safety data and related treatment effects.

Results:

The sensitivity of the electronic medical record–based safety for hospitalizations was 65.3% without non-VA hospitalization events and 92.3% with the non-VA hospitalization events included. The sensitivity was only 54.3% for acute kidney injury and 87.3% for hyperkalemia. The precision of electronic medical record–based safety data was 89.4%, 38%, and 63.2% for hospitalization, acute kidney injury, and hyperkalemia, respectively. Relative treatment differences under the study and electronic medical record settings were 15% and 3% for hospitalization, 123% and 29% for acute kidney injury, and 238% and 140% for hyperkalemia, respectively.

Conclusion:

The accuracy of using automated electronic medical record safety data depends on the events of interest. Identification of all-cause hospitalizations would be reliable if search methods could, in addition to VA hospitalizations, also capture non-VA hospitalizations. However, hospitalization is different from a cause-specific serious adverse event that could be more sensitive to treatment effects. In addition, some study-specific safety events were not easily identified using the electronic medical records. This limits the effectiveness of the automated central database search for purposes of safety monitoring. Hence, this data captured approach should be carefully considered when implementing endpoint data collection in future pragmatic trials.

Keywords

Electronic medical records centralized safety monitoring safety data validation clinical trial design automated search

Get full access to this article

View all access options for this article.

References

Vestbo

Leather

DiarBakerly

et al . Effectiveness of fluticasone furoate–vilanterol for COPD in clinical practice. N Engl J Med 2016; 375(13): 1253–1260.

U.S. Food and Drug Administration. Use of electronic health record data in clinical investigations (Draft guidance). Silver Spring, MD: U.S. Food and Drug Administration, 2016.

Fiore

Lavori

PW.

Integrating randomized comparative effectiveness research with patient care. N Engl J Med 2016; 374(22): 2152–2158.

Cowie

Blomster

Curtis

et al . Electronic health records to facilitate clinical research. Clin Res Cardiol 2017; 106(1): 1–9.

Rochefort

Buckeridge

Forster

AJ.

Accuracy of using automated methods for detecting adverse events from electronic health record data: a research protocol. Implement Sci 2015; 10: 5.

Rochefort

Buckeridge

Tanguay

et al . Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol. BMC Health Serv Res 2017; 17(1): 147.

FitzHenry

Murff

Matheny

et al . Exploring the frontier of electronic health record surveillance: the case of post-operative complications. Med Care 2013; 51: 509–516.

Fried

Emanuele

Zhang

et al . Combined angiotensin inhibition for the treatment of diabetic nephropathy. N Engl J Med 2013; 369(20): 1892–1903.

Brown

Lincoln

Groen

et al . VistA-U.S. Department of Veterans Affairs National-Scale HIS. Int J Med Inform 2003; 69(2–3): 135–156.

10.

Rajeevan

Niehoff

Charpentier

et al . Utilizing patient data from the veterans administration electronic health record to support web-based clinical decision support: informatics challenges and issues from three clinical domains. BMC Med Inform Decis Mak 2017; 17(1): 111.

11.

VA Corporate Data Warehouse documentation. https://www.virec.research.va.gov/

12.

Fried

Duckworth

Zhang

et al . Design of combination angiotensin receptor blocker and angiotensin-converting enzyme inhibitor for treatment of diabetic nephropathy (VA NEPHRON-D). Clin J Am Soc Nephrol 2009; 4(2): 361–368.

13.

Murff

FitzHenry

Matheny

et al . Automated identification of postoperative complications within an electronic medical record using natural language processing. JAMA 2011; 306(8): 848–855.

14.

Palevsky

Zhang

Seliger

et al . Incidence, severity, and outcomes of AKI associated with dual renin-angiotensin system blockade. Clin J Am Soc Nephrol 2016; 11(11): 1944–1953.

15.

Haerian

Varn

Vaidya

et al . Detection of pharmacovigilance related adverse events using electronic health records and automated methods. Clin Pharmacol Ther 2012; 92(2): 228–234.

16.

Price

Savitz

. Improving the measurement of surgical site infection risk stratification/outcome detection. Final Report (Prepared by Denver Health and its partners under Contract No. 290-2006-00-20), AHRQ Publication No. 12-0046-EF. Rockville, MD: Agency for Healthcare Research and Quality, 2012.