Abstract
Limited work identifies best practices to assess functional electronic health record system performance when contracting for health information technology and information technology–related services. Without a set of best practices or specific contracting provisions to assess the performance of electronic health record systems, healthcare providers will not be able to fully leverage the performance of these systems to reduce the cost of care and improve patient outcomes. This work seeks to provide operational considerations and best practices when forming teams to negotiate health information technology system specifications in contracts. To better understand the contracting and performance assessment process, we conducted a cross-sectional survey of eligible healthcare personnel. Our study highlights a potential disconnect between respondents setting contract structure, knowledge of ongoing functional performance assessments in practice, and the relationship to those with direct system involvement to avoid potential legal risk.
Introduction
Extensive research exists detailing organizational best practices for implementing electronic health records (EHRs) in healthcare systems. However, there is little work in the current literature identifying organizational best practices for establishing contractual terms or evaluating the functional performance of EHRs. Ensuring accurate system performance is necessary to support a continuum in care, to report information to the general public and governmental agencies, and to communicate across health systems. Current research offers limited guidance on strategies to operationalize and improve contracts by evaluating system performance against agreed upon contract terms. Existing EHR system evaluations skew toward exploring organizational communication strategies,1,2 various technology acceptance models,3,4 or key elements of selecting and managing EHR technology.5–7 A contributing factor to this lack of research in establishing best practices is that contracted terms and agreements are rarely made public. That said, terms of the agreements should provide an enforceable tool to assess the performance of any EHR or broader health information technology (HIT) project against key clinical, financial, and operational benchmarks. Our prior work (A Framework for Contracted Health IT Performance Assessments, Hospital Topics, 97:1, 15–22) identifies a general approach to best practices that link performance in practice with contracting approaches when evaluating an EHR or other HIT-related services. Major approaches to assess system performance include internal review, external review, or joint (solution vendor and contracting healthcare organization) assessment of system performance. 8
The market for EHR systems and related HIT services in the United States remains dynamic. Industry literature suggests a trend toward replacement of recently deployed systems as opposed to initial installation of EHR systems. 9 Limited published and peer-reviewed work highlights potential causes of this trend to include an uptick in merger and acquisition activity in the United States of large healthcare systems, technological refinements, and data access or retention as key influencers of a decision to replace an EHR system. 10 A component of any EHR replacement strategy is the alignment of relevant vendor terms and conditions with purchaser expectations for reviewing and measuring system performance. The US Office of the National Coordinator (ONC) for Health Information Technology provides limited guidance in the document titled EHR Contracts Untangled: Selecting Wisely, Negotiating Terms, and Understanding the Fine Print (ONC Contracting Guidance) from 2016. 11 The ONC Contracting Guidance simply states “acceptance criteria should be specifically and concretely crafted with the assistance of technical personnel and documented to ensure that the EHR performs according to your expectations.” However, the document does not provide concrete examples of the types of terms regarding express warranties or system functionality that contracting entities should seek to maximize system performance and avoid the risk of liability under the False Claims Act. 11 In the United States, the False Claims Act provides legal recourse for the federal government to investigate the improper use of federal funds, including potential fraudulent payment under Medicare and Medicaid programs.
Findings associated with our study suggest that while parties contracting for EHR services regularly negotiate contracts, and with broad awareness of commonly used contract terms, the personnel engaged in contract negotiations lack the day-to-day involvement with HIT for which system performance must be assessed post-implementation. Our research seeks to understand and identify key areas of opportunity to facilitate stakeholder engagement between technical staff, clinical staff, and negotiating staff, to build an awareness of the need for ongoing assessments of system performance. A key focus of this work is defining areas where technical or clinical personnel are engaged in the formation or evaluation of functional performance requirements of EHR systems and where there are areas for improvement. We also seek to define how to more fully integrate key stakeholder input as a core element to optimize use in practice when navigating “rip and replace” decisions, given the potential increase in healthcare systems looking to replace HIT systems. 12 Our interest lies in understanding how levels of stakeholder engagement between contracted terms and use in practice affects the persistent challenge of effectively aggregating and reporting healthcare information and maximizing system value.
From an organizational and risk management perspective, linking contracting with value in use supports an integrated approach to structuring an agreement and defining metrics for evaluation. To maximize the benefits of EHR systems in the clinical context, contracts should include specific terms that reflect the performance requirements of clinicians and a means by which performance against these requirements can be evaluated. We expect that these terms may vary across care settings due to variations in both the structure of the care setting and organizational behavior. As such, we suggest that the utilization of specified system performance in contracting terms for HIT represents a major shift in the organizational acquisition of technology as value-based care becomes a major financial driver and policy objective influencing system purchase. These key organizational constructs in determining actual system performance in practice versus negotiated and contracted terms is an important next step. In addition, an added benefit from an operational perspective is the ability to more effectively manage risk associated with potential fraud and abuse claims. This research seeks to identify the extent to which key stakeholders engage in setting contractual language as well as establishing functional system requirements for EHR and HIT solutions. Furthermore, we evaluate ongoing assessment of system performance within an organization in conjunction with stakeholder awareness of commonly utilized terminology.
Methods
Study design and population
We conducted a cross-sectional study of practicing healthcare administration professionals and HIT vendors in the United States utilizing a novel survey instrument that was designed for this study. The team piloted this instrument with three industry professionals and revised survey elements in line with feedback to ensure instrument validity. Data collection occurred through communication of the survey instrument to multiple US-based healthcare administrative, financial, and IT professional societies and local chapters (Healthcare Information and Management Systems Society, American College of Healthcare Executives, Healthcare Financial Management Association) for distribution to members. In addition, the researchers undertook a direct email campaign using a database of known healthcare administration and IT professionals. Survey completion time was approximately 10 min. The primary investigator’s institutional review board (IRB) assessed the instrument prior to placement. Data collection commenced in January 2018 and concluded in November 2018. The completion rate upon opening the emailed survey was 19 percent, and a total of 130 responses were collected and analyzed. Respondents’ gender and age were not collected or deemed relevant to the study. However, details of tenure in the respondents’ current role, workplace setting, familiarity with common terms, and the respondent’s role in the contracting process were collected.
Statistical analysis and key variables
The assessment employs statistical testing for the comparison of groups across categorical responses in the survey data and testing of variable independence. 13 Contingency tables and Pearson’s chi-square tests are used to examine whether respondent’s primary function or job title, presence of a secondary role, setting of care, familiarity with widely used contract terms, and awareness of specified contracted terms within an organizational setting were independent of the respondent’s role in setting the structure of a contract. Logistic regression analysis is commonly used to evaluate relationships between studied observations. The dichotomization of variables is useful to establish model-specific hierarchies. We conducted logistic regression modeling on variables to further assess the relationship between work setting (dichotomized as clinical vs non-clinical) against respondent influence in setting contract structure, awareness of contracted terms, and seniority. 14
We identified and dichotomized the series of variables presented across Tables 1 and 2. Dichotomized variables included job titles classified as senior or executive in nature as presented in Table 1. Titles given a high level of seniority include respondents indicating roles at or above the level of director as well as those indicating executive leadership roles. Those given non-senior status include respondents indicating operational, consulting, or full-time status as outlined in Table 1.
General respondent demographics.
HIT: health information technology; HIE: health information exchange; PA: Physician Assistant; NP: Nurse Practitioner; ACO: Accountable Care Organization.
Role in contract setting and awareness of terms.
IT: information technology; ACO: Accountable Care Organization.
The research team then dichotomized work setting as clinical or non-clinical, also presented in Table 1, to better understand the role consultants or external vendors play in setting contract structure. In addition, we dichotomized the influence of a respondent’s role related to contract negotiation and relevant to establishing contract structure as reported in Table 2. Respondents with low influence responded with “no role,” “setting needs or requirements,” and “general evaluation of contracting.” Those with high influence on setting contract structure include respondents indicating “decision maker level status,” “responsibility for purchasing,” and “implementation of contracted terms.”
Finally, we dichotomized awareness of ongoing performance assessments as “aware or unaware” based on respondent’s indications presented in Table 2. Respondents answering “other” within the survey were assessed and categorized based on appropriate fit given the context of any collected free text response. A generalized logistic regression model was conducted using R (R statistics) version 3.5.1 to assess the relationship of seniority against clinical or non-clinical titles, influence, and awareness within an organizational structure. Significance level was determined using an alpha of 0.05.
Results
General demographics and work setting
Of the 130 total respondents, the mean tenure of the respondent in their current role was 8.1 years (range: 0–36 years; see Table 1). The survey instrument uses job titles and care setting reflecting delineations commonly used by the American Hospital Association (AHA) Hospital Statistics™. We expanded upon these categorizations for generalization and comparison between the collected data set and with other commonly used data resources. Additional job classifications were available in the survey, and the opportunity to indicate a joint appointment was provided to respondents. Respondents most commonly identified with clinical leadership roles for quality, HIT, and other service lines at the director level and above (22.8%, n = 29) (Table 1), followed by clinical providers such as physicians, nurses, physician assistants, or nurse practitioners (18.7%, n = 27). Most respondents did not hold a secondary appointment (73.08%, n = 95), while a smaller number of respondents indicated a dual clinical and administrative appointment (26.92%, n = 35) (Table 1). Respondents most frequently reported work setting as being “hospital settings” (academic medical center, suburban or urban, or rural) (42.11%, n = 48), followed by a vendor or consulting location (27.19%, n = 31) (Table 1).
Awareness and role in contracting
Contracts are an enforceable tool for establishing the exchange of goods, services, and intellectual property, 15 applicable in healthcare delivery systems around the world. A limitation of conducting research in this space is that contracts often lack “completeness” in terms of use both in practice and to establish reasonable conduct. 15 Gaining “visibility” into exact contracting terms for every HIT engagement is difficult to achieve from a study design perspective. A useful barometer of understanding the roles of various personnel involved in the contracting process is to compare job titles, roles, and awareness of contracting terms to better understand stakeholder needs. To understand both awareness of key contractual elements and role in the contracting process, survey respondents were presented questions assessing types of EHR and HIT systems used in practice and common contractual terms. By identifying a respondent’s capability for understanding commonly utilized contract terms, we establish both familiarity as well as the ability to provide input to support achieving operational performance. The intent is in assessing the extent of stakeholder involvement between personnel negotiating contracted language and individuals ensuring operational performance. A greater number of survey respondents (71.05%, n = 81) indicated that oversight or support of the contracting process was not within their purview or job responsibilities. Fewer respondents indicated that support or oversight of contracting requirements was within their purview or job description and primary system of responsibility (28.95%, n = 33) (Table 2). A total of 222 discrete responses were collected from the 130 respondents regarding roles in setting contract structure. The number of indicated responses is larger than the number of respondents, as a respondent could indicate more than one role in setting contract structure.
Respondents were also presented with commonly utilized support or business models within HIT settings together with commonly used contracted provisions. A total of 318 indicated responses were collected regarding familiarity with commonly utilized contract structures.
Participants were permitted to indicate multiple familiarities across business models and common contracting terms. Respondents indicated the greatest familiarity with Software as a Service (SaaS) type provisions (22.96%, n = 73), followed by Professional Service Fees (19.50%, n = 62). Other commonly indicated contracting approaches included Platform as a Service (PaaS) (15.41%, n = 49), Analytics as a Service (13.21%, n = 42), and general warranty coverage (11.95%, n = 38). Respondents less frequently identified guaranteed uptime (11.32%, n = 36), unfamiliarity with any proffered terms (4.72%, n = 15), or indication of “other” terms (0.94%, n = 3) (Table 2).
A cursory review of respondent’s indication of familiarity with “other” terms by text entry highlighted contractual needs around privacy and security measures, business associate agreements, or other security-related requirements not available in the survey instrument.
Finally, respondents were presented with a question surrounding awareness of specified contract terms or incentives within their respective organization. Specified contract approaches included language or terms around patient satisfaction, clinical outcomes for disease states, clinical services or financial data for bundled payments or episodes of care, and third-party data sharing agreements. The survey included an option for respondents to indicate “other” known terms with a follow-up text entry. Respondents most frequently identified patient satisfaction as the most common and known contracted term or incentive within an organization (24.44%, n = 35) followed by a lack of awareness of specified contract terms within an organization (18.59%, n = 29).
It is important to note the less commonly indicated responses for awareness of contract terms specific to the organization include terms for clinical outcomes for a specified disease or change in morbidity (16.67%, n = 26), clinical services and financial data for an episode or bundled payment (12.18%, n = 19), or transfer or sharing of proprietary data to a third party (12.18%, n = 19). The aforementioned areas represent key drivers whereby HIT solutions conceivably support transitions to value-based healthcare and improved outcomes. Other less frequently indicated contract terms include clinical outcomes for patients based on cohort performance (11.54%, n = 18) and clinical services and financial data for patients across a defined period for accountable care or capitation (4.49%, n = 7) (Table 2). These items also present major trends and business opportunities given an ongoing focus on value-based care. Lack of awareness regarding these terms may indicate a barrier to aligning contacting for EHR and HIT systems with requirements set forth in constructs associated with value-based care. Lower levels of awareness may also indicate potential impediments to deploying systems that optimize quality and outcomes in care.
Contingency tables and chi-square analysis results
Table 3 presents the crosstabulation of respondent’s primary function or job title, presence of secondary role, care setting, familiarity with widely used contract terms, awareness of specified contracted terms within an organizational setting, and their role in setting contract structure. No association was identified between indicated roles or job titles and role in setting contract structure (χ2 = 147.12, df = 126, p = 0.10), a secondary appointment (χ2 = 12.98, df = 7, p = 0.07), or general familiarity with contracted terms (χ2 = 57.10, df = 49, p = 0.20) (Table 3). The association between roles in setting contract structure and awareness of contracted terms within an organization specific to the performance of HIT was found to be significant (χ2 = 67.23, df = 49, p < 0.05) (Table 3).
Crosstab analysis on the independence of select characteristics.
HIT: health information technology.
Logistic regression analysis results
Within the logistic regression model, influence surrounding contracting inside an organizational structure (p = 0.052) and awareness of performance assessments (p = 0.825) were not found to have statistically significant association with the respondent’s seniority inside an organization. However, the dichotomization of clinical work setting versus non-clinical work setting was found to have statistically significant association with the respondent’s seniority within the organization (p = 0.029). Those with clinical roles had higher odds of being in a more senior position (OR = 3.315; 95% CI = 1.18; 10.51) with interpretation of the finding below. McFadden’s pseudo R-squared value was also calculated to be 0.27 for the logistic regression model (Table 4).
Logistic regression model results.
OR: odds ratio; CI: confidence interval.
Dependent variable: respondent’s seniority; McFadden’s pseudo-R2 = 0.27.
Discussion and conclusion
The role and extent that key functional stakeholders inform the contracting process for EHR and HIT is sparsely researched. Oversight and awareness of contracting provisions remains outside the purview of many key clinical and operational personnel surveyed within our study yet remains an important constituency to provide feedback on system performance. This signals potential and important divergence among stakeholders between those most familiar with functional requirements for system performance who are not aware of contractual obligations or extant contract language. An important and relevant example of this disconnect includes recent False Claims Act litigation in the United States for the failure of HIT systems to perform at specified levels associated with deploying Certified EHR Technology. 16 The Meaningful Use regulations adopted by eligible hospitals and providers created financial incentives for healthcare providers to adopt EHRs by providing reimbursement when healthcare providers adopted EHR systems that met certain regulatorily established benchmarks. There have been several cases in which care providers attested to having systems meeting these requirements to obtain federal reimbursement but had failed to require such performance in the underlying service contracts with their EHR vendors. When these systems fail to perform to the standards required for reimbursement, the US government takes the position that this amounts to a false claim by the care provider seeking reimbursement, which is subject to enforcement under the False Claims Act. In essence, healthcare organizations in the United States adopted technology and attested to meeting certain criteria which qualify the participant for monetary reimbursement from the federal government. The Meaningful Use program incentivized healthcare organizations by recouping some of monetary outlay for adopting EHRs in practice, while creating infrastructure necessary to achieve policy goals of transitioning to value-based care. Since the acquired technology is not “performing to established benchmarks,” the claim to payment is thus “fraudulent” in the eyes of the government. The overall impact is both “upstream” and “downstream” from a contracting perspective. Downstream healthcare organizations discover that system performance is not in line with contracted terms only by judicial oversight and “whistleblower” testimony associated with fraudulent behavior by the software vendors or healthcare organizations receiving payment, not by internal review process. Upstream impacts include the inability for governments to reach policy objectives by incentivizing or mandating the use of technology to further the notion of value-based care. Another concern is that much of the software in the United States undergoes “certification” as well as testing and validation during system implementation in clinical environments. Certification of technology is required for Meaningful Use incentives (payments made to healthcare organizations adopting technology), but recent litigation reveals flaws associated with the review and implementation process. There is a need for organizations to further assess and address risk of litigation or loss of operational benefit arising from system underperformance against contracted benchmarks when considering “rip and replacement” of EHR systems. Furthermore, we see recent US federal rulemakings embracing the HL7 FHIR (Fast Healthcare Interoperability Resources)–based application programming interfaces (APIs) as another area to consider when defining and assessing contracted functional performance of systems. For example, the need for testing and validation of various “put” or “get” commands to facilitate system interoperability and associated with query or send requests should be contractually specified and benchmarked when purchasing systems utilizing open APIs.
In summary, this study provides insights into how organizations currently approach integrating key stakeholders into creating, reviewing, and managing contract negotiations specific to EHR and HIT deployments. The impact of serving in a non-clinical capacity or as a vendor of a software product and for whom contractual performance is important and noteworthy. Counterpoint is the omission of personnel with clinical backgrounds and routine experience with system performance. A lack of engagement by these individuals compromises the ability to structure agreements that link common contractual language to system performance at a functional level.
Roles in setting contracting terms are significantly associated with awareness of existing contracted performance metrics within an organization. This underscores a need for senior managers and executives associated with ongoing HIT deployments to continually engage all relevant stakeholders. Non-significant yet noteworthy findings include high awareness of commonly utilized terms associated with establishing HIT contract structures. The indication of patient satisfaction scores is also noteworthy from an awareness perspective—likely a result of focus on performing well on the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey required by the US Centers for Medicare and Medicaid Services (CMS).
We note the following limitations of the methods utilized for exploratory data analysis. In this study, there are challenges in defining an appropriate sample population, given the diffuse way in which contracts may be negotiated and inconsistencies across the roles. For example, vendors or non-clinical participants in the contracting process, by nature of roles and responsibilities, exhibit a greater propensity and involvement in the contracting process. Furthermore, the use of “gag clauses” and hold harmless language in existing contract structures curtails appropriate study and dissemination of best practices or potential system flaws. 17 Identification or stratification of sample populations or expanded sampling techniques proves challenging, as setting contract structures at some level is indicated across many stakeholders. We also note the effect of limited response rates across outlier variables exerts on the sensitivity of the chi-square test (typically stated as fewer than 5 cases in a category or table) impacting the robustness of the contingency table measurement. 18 An additional point is the impact of dichotomizing variables on statistical significance from a study design perspective. 19 The evaluation of the relationships as indicated by respondents is useful for furthering discussion and exploring approaches to managing organizational processes that are difficult to study. Evaluating these relationships illustrates the extent to which key stakeholders, namely clinical personnel, should play a role in assessing ongoing system performance. We also note the McFadden’s pseudo R-squared value of 0.27 and that the model captures a portion of important factors that seniority imparts on understanding and evaluating contracted performance within an organizational structure.
In summary, ensuring inclusive stakeholder engagement at both the negotiation phase and monitoring of the performance phase is essential to achieve functional performance requirements when contracting for EHR and HIT systems. The process for delivering value through clinical use in practice requires including individuals not directly involved in the of negotiation contract terms. While awareness of contract performance assessments in an organization was not statistically significant when regressed against seniority, there is managerial and operational benefit to avoid omitting key stakeholders when defining contract terms and functional metrics in practice. Such inclusion supports engagement and continuous monitoring to optimize use in practice vis-à-vis contractual language. This alignment is foundational to support implementing quality payment programs and the movement toward value-based care.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
