Electronic health records data are becoming a key data resource in clinical research. Owing to issues of data efficiency, electronic health records data are being used for clinical trials. This includes both large-scale pragmatic trails and smaller—more focused—point-of-care trials. While electronic health records data open up a number of scientific opportunities, they also present a number of analytic challenges. This article discusses five particular challenges related to organizing electronic health records data for analytic purposes. These are as follows: (1) data are not organized for research purposes, (2) data are both densely and irregularly observed, (3) we don’t have all data elements we may want or need, (4) data are both cross-sectional and longitudinal, and (5) data may be informatively observed. While laying out these challenges, the article notes how many of these challenges can be addressed by careful and thoughtful study design as well as by integration of clinicians and informaticians into the analytic team.
GoldsteinBANavarAMPencinaMJ, et al. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc2017; 24(1): 198–208.
2.
SetjiTLPageCPagidipatiNJ, et al. Differences in achieving HBA1C goals among patients seen by endocrinologists and primary care providers. Endocr Pract2019; 25(5): 461–469.
3.
CaseyJASchwartzBSStewartWF, et al. Using electronic health records for population health research: a review of methods and applications. Annu Rev Public Health2016; 37: 61–81.
4.
HooperMHWeavindLWheelerAP, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med2012; 40(7): 2096–2101.
5.
JohnstonAJonesWSHernandezAF. The ADAPTABLE trial and aspirin dosing in secondary prevention for patients with coronary artery disease. Curr Cardiol Rep2016; 18(8): 81.
6.
WengCLiYRyanP, et al. A distribution-based method for assessing the differences between clinical trial target populations and patient populations in electronic health records. Appl Clin Inform2014; 5(2): 463–479.
7.
WengC. Optimizing clinical research participant selection with informatics. Trends Pharmacol Sci2015; 36(11): 706–709.
8.
LaiYSAfsethJD. A review of the impact of utilising electronic medical records for clinical research recruitment. Clin Trials2019; 16(2): 194–203.
9.
ThorpeKEZwarensteinMOxmanAD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol2009; 62(5): 464–475.
10.
BrulandPMcGilchristMZapletalE, et al. Common data elements for secondary use of electronic health record data for clinical trial execution and serious adverse event reporting. BMC Med Res Methodol2016; 16(1): 159.
11.
GoldsteinBAPhelanMPagidipatiNJ, et al. An outcome model approach to transporting a randomized controlled trial results to a target population. J Am Med Inform Assoc2019; 26(5): 429–437.
12.
de HoonSEMHekKvan DijkL, et al. Adverse events recording in electronic health record systems in primary care. BMC Med Inform Decis Mak2017; 17(1): 163.
13.
Mc CordKAEwaldHLadanieA, et al. Current use and costs of electronic health records for clinical trial research: a descriptive study. CMAJ Open2019; 7(1): E23–E32.
WrightAMcCoyABHickmanTT, et al. Problem list completeness in electronic health records: a multi-site study and assessment of success factors. Int J Med Inform2015; 84(10): 784–790.
17.
RichessonRLRusincovitchSAWixtedD, et al. A comparison of phenotype definitions for diabetes mellitus. J Am Med Inform Assoc2013; 20(e2): e319–e326.
18.
SprattSEPereiraKGrangerBB, et al. Assessing electronic health record phenotypes against gold-standard diagnostic criteria for diabetes mellitus. J Am Med Inform Assoc2017; 24(e1): e121–e128.
19.
ChenYWangJChubakJ, et al. Inflation of type I error rates due to differential misclassification in EHR-derived outcomes: empirical illustration using breast cancer recurrence. Pharmacoepidemiol Drug Saf2019; 28(2): 264–268.
Observational Medical Outcomes Partnership (OMOP) Common Data Mode, http://omop.org (accessed 11 February 2020).
22.
WeiskopfNGRusanovAWengC. Sick patients have more data: the non-random completeness of electronic health records. AMIA Annu Symp Proc2013; 2013: 1472–1477.
23.
LittleRJARubinDB. Statistical analysis with missing data. 2nd ed. Hoboken, NJ: John Wiley & Sons, 2002.
24.
WellsBJChaginKMNowackiAS, et al. Strategies for handling missing data in electronic health record derived data. EGEMS Wash DC2013; 1(3): 1035.
25.
TherneauTMGrambschPM. Modeling survival data: extending the Cox model. New York: Springer, 2000.
26.
FitzmauriceGMLairdNMWareJH. Applied longitudinal analysis (Wiley Series in Probability and Statistics). 2nd ed. Hoboken, NJ: Wiley, 2011, p. 701.
27.
RizopoulosD. Joint models for longitudinal and time-to-event data: with applications in R (Chapman & Hall/CRC Biostatistics Series). Boca Raton: CRC Press, 2012, p. 261.
28.
GoldsteinBAPomannGMWinkelmayerWC, et al. A comparison of risk prediction methods using repeated observations: an application to electronic health records for hemodialysis. Stat Med2017; 36(17): 2750–2763.
29.
SweetingMJBarrettJKThompsonSG, et al. The use of repeated blood pressure measures for cardiovascular risk prediction: a comparison of statistical models in the ARIC study. Stat Med2017; 36(28): 4514–4528.
30.
CollinsGSOgundimuEOCookJA, et al. Quantifying the impact of different approaches for handling continuous predictors on the performance of a prognostic model. Stat Med2016; 35(23): 4124–4135.
31.
KöpckeFTrinczekBMajeedRW, et al. Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence. BMC Med Inform Decis Mak2013; 13: 37.
32.
PhelanMBhavsarNAGoldsteinBA. Illustrating informed presence bias in electronic health records data: how patient interactions with a health system can impact inference. EGEMS Wash DC2017; 5(1): 22.
33.
BhavsarNAGaoAPhelanM, et al. Value of neighborhood socioeconomic status in predicting risk of outcomes in studies that use electronic health record data. JAMA Netw Open2018; 1(5): e182716.
34.
XieSGreenblattRLevyMZ, et al. Enhancing electronic health record data with geospatial information. AMIA Jt Summits Transl Sci Proc2017; 2017: 123–132.
35.
TuKMitikuTFIversNM, et al. Evaluation of electronic medical record administrative data linked database (EMRALD). Am J Manag Care2014; 20(1): e15–e21.
36.
WeiskopfNGBakkenSHripcsakG, et al. A data quality assessment guideline for electronic health record data reuse. EGEMS Wash DC2017; 5(1): 14.
37.
GoldsteinBAAssimesTWinkelmayerWC, et al. Detecting clinically meaningful biomarkers with repeated measurements: an illustration with electronic health records. Biometrics2015; 71(2): 478–486.
38.
RayWA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol2003; 158(9): 915–920.
39.
SchuemieMJRyanPBManKKC, et al. A plea to stop using the case-control design in retrospective database studies. Stat Med2019; 38(22): 4199–4208.
40.
GoldsteinBABhavsarNAPhelanM, et al. Controlling for informed presence bias due to the number of health encounters in an electronic health record. Am J Epidemiol2016; 184(11): 847–855.
41.
GoldsteinBAPhelanMPagidipatiNJ, et al. How and when informative visit processes can bias inference when using electronic health records data for clinical research. J Am Med Inform Assoc2019; 26(12): 1609–1617.
42.
HaneuseSDanielsM. A general framework for considering selection bias in EHR-based studies: what data are observed and why?EGEMS Wash DC2016; 4(1): 1203.
43.
NeuhausJMMcCullochCEBoylanRD. Analysis of longitudinal data from outcome-dependent visit processes: failure of proposed methods in realistic settings and potential improvements. Stat Med2018; 37(29): 4457–4471.
44.
McCullochCENeuhausJMOlinRL. Biased and unbiased estimation in longitudinal studies with informative visit processes. Biometrics2016; 72(4): 1315–1324.
45.
CookRJLawlessJF. Independence conditions and the analysis of life history studies with intermittent observation. Biostatistics2019; 2019: kxz047.
46.
GaspariniAAbramsKRBarrettJK, et al. Mixed-effects models for health care longitudinal data with an informative visiting process: a Monte Carlo simulation study. Stat Neerl2019; 74(1): 5–23.
