Revisions to the Common Rule: A proposal in search of evidence

Abstract

Proposed changes to the Common Rule are proffered to save almost 7,000 reviews annually and consequently vast amounts of investigator and IRB-member time. However, the proposed changes have been subject to criticism. While some have lauded the changes as being imperfect, but nevertheless as improvements, others have contended that ‘neither the scientific community nor the public can be confident that improved practices will emerge from the regulatory changes mandated by the NPRM.’

In the present article, I discuss an important aspect that has been overlooked: the question of whether benefits will emerge is demonstrably empirical, yet data upon which to draw conclusions are conspicuous by their absence.

This is thrown into sharp relief when we consider the current environment in which health research is increasingly focused on providing evidence of need or benefit, where there is greater emphasis on evidence-based practice, and when we have the nascent field of implementation science.

On esentence summary

Ethics review of clinical research remains a much-discussed topic, often focused on concerns regarding the efficiency and effectiveness of ethics review processes. Criticisms range from the amount of paperwork required (Jamrozik, 2004) or inconsistency of decisions (Dziak et al., 2005; Glasziou and Chalmers, 2004; Greene and Geiger, 2006; Silverman et al., 2001), to suggestions that ethics review systems are not equipped to review specific types of research (Schrag, 2011).

Subsequently, there have been calls for revising ethics review processes, and specifically the regulations included within 45 Code of Federal Regulations 46 (45 CFR 46) – otherwise known as the ‘Common Rule’. These regulations cover the requirement for ethics review of human-subjects research. In 2011, the Department of Health and Human Services and a number of other Departments released an Advanced Notice of Proposed Rulemaking (ANPRM) in relation to revisions to the Common Rule. In September 2015, a formal Notice of Proposed Rule Making (NPRM) was released (Federal Register, 2015), with opportunities to provide comment. The two major categories of change within the proposal relate to consent requirements and oversight required for different study types. Motivation for these proposed changes has been a desire to improve human subject protections, while at the same time increasing efficiencies. We are now likely close to a final conclusion – although this has been said before (Emanuel, 2015) – and a number of modifications to a set of major regulations that govern health research.

Although some have lauded the proposed changes as being imperfect, but nevertheless as improvements (Emanuel, 2015), others have contended that ‘neither the scientific community nor the public can be confident that improved practices will emerge from the regulatory changes mandated by the NPRM’ (Strauss et al., 2016). While both sets of authors assert their positions, an important aspect has been overlooked: whether the changes lead to improvements is a demonstrably empirical question that one can subject to analysis.

Surprisingly, data upon which to draw conclusions about the benefit (or likely benefit) of the changes are conspicuous by their absence, be it evidence used to indicate a need for changes, or discussion of evidence that can or should be collected to illustrate the impact of the changes.

Evidence of likely benefit from proposed changes

Although the NPRM runs to 131 pages, there is a lack of evidence cited to justify how the proposed changes address current deficiencies (Strauss et al., 2016). Furthermore, although the NPRN does include a regulatory impact analysis (RIA), many of the estimates are derived from a 1998 NIH-sponsored study (Bell et al., 1998) or anecdotal evidence ‘[b]ecause of the lack of available data about IRB effectiveness and how IRBs function operationally’ (Federal Register, 2015: 53996).

Given that a motivation for change is the belief that ‘the volume and landscape of research involving human subjects have changed considerably [since the Common Rule was introduced]’ (Federal Register, 2015: 53935), it is unclear how estimates from 1998 represent the current landscape of research, and thus provide realistic assessments of impact. In particular, the use of 1998 data regarding the percentage of full and expedited reviews seems highly problematic given the reported changes in volume and type of reviews (Abbott and Grady, 2011; Wolzt et al., 2009).

Irrespective, the excuse of a paucity of data is indefensible, for two reasons. First, there are numerous studies that have sought to quantify the impact of ethics review with respect to investigator time, costs, or IRB workload (Abbott and Grady, 2011). A recent scoping review identified 198 studies that had sought in one way or another to evaluate ethics review (Nicholls et al., 2015). Second, the use of the prior analysis conducted by Bell and colleagues for the NIH suggests that data could have been collected to inform the proposed changes. For the 1998 study to have been conducted, relevant outcomes or metrics for evaluation must have been developed. Consequently, a preliminary step for the current NPRM should have been to update the data to inform the RIA and thus base the analyses on contemporary data.

Evidence of impact: A burden of proof

Moreover, although the proposals are made on the basis of the RIA, the NPRM lacks any description of activities to collate evidence with which one can assess whether the RIA was realistic. This seems a huge oversight given the evidence-based medicine world in which we now live.

Again, this should not be dismissed because of data collection being onerous. Many of the assumptions built into the NPRM RIA could easily be evaluated. For example, within the RIA it is estimated that institutional review board (IRB) chairs and IRB voting members, as well as investigators, will each spend five hours to learn the proposed changes to the Common Rule. It is also estimated that institutional officials would spend two hours to learn new procedures (at 53999). A survey to IRBs would be relatively straightforward to capture whether these estimates were realistic. Equally, it is estimated that proposals to exclude certain forms of research from the purview of the IRB will lead to a reduction of 6,754 reviews. It would seem pertinent that data are collected on whether changes to review oversight do actually result in efficiencies or decrease the number of reviews. Yet such – relatively – simple data collection proposals are absent from the NPRM.

Evidence and ethics review: A perpetual discussion

Such failures to embed evidence-based approaches are symptomatic of the literature and practice of ethics review, where there has been much talk about the need for data but little action (Coleman and Bouësseau, 2008). Gray (1975), for example, laments that, despite the introduction in 1966 of regulations requiring institutional review of projects funded by the US Public Health Service, ‘its adequacy and efficiency have not been sufficiently evaluated. Nor have the various revisions and modifications to the policy been based on systematic empirical knowledge.’ Over 40 years later, similar charges can be laid against the Common Rule.

It seems bizarre that in an environment where health research is increasingly focused on providing evidence of need or benefit, where there is greater emphasis on evidence-based practice, and when we have the nascent field of implementation science, health policy to improve human subject protections and to increase efficiency of review is devoid of evidence. The result is that, even when the long-drawn-out NPRM process is complete, we will be no nearer knowing whether the implemented changes actually achieve their goal of enhancing human subject protections and increasing efficiency.