Science review in research ethics committees: Double jeopardy?

Abstract

Research ethics committees ‘(RECs) members’ perceptions of their role in regard to the science of research proposals are discussed. Our study, which involved the interviewing of 20 participants from amongst the UK’s independent (Phase I) ethics committees, revealed that the members consider that it is the role of the REC to examine and approve the scientific adequacy of the research – and this notwithstanding the fact that a more competent body will already have done this and even when that other body has the legal responsibility for this function. The problematic nature of this situation, tantamount to double jeopardy, is considered: it can delay research and so add to costs whilst offering no countervailing benefits, or the double jeopardy may be just the cost society imposes, through its RECs, on researchers as the price for research on human subjects.

Original proposals in the summer of 2012 to replace the European directives which govern both clinical trials and medical devices with new European regulations envisaged separating ethics review from the review of the science (Europa, 2012). However, following responses from the Research Ethics Committee (REC) community, such a separation is no longer to be a requirement of the new regulation (Europa, 2014). Here, findings of REC members on their attitudes towards this issue of responsibility for review of the science of research proposals are reported.

When RECs were first established in the 1960s and 1970s it was at a time when many scientists realized that it was necessary and appropriate for outsiders to help them understand what limits should be placed on their developments, because ‘whenever we develop a technology we inevitably use it’ (Packard, 1978: 329). Scientists thus invited ‘the views of ethicists, philosophers, theologians, public servants. Some even accept[ed] … that the public deserves to have a say …’ (Packard, 1978: 329). This was the justification for including a lay (or similar) membership of RECs, but there was no expectation that these ‘outsiders’ would critique the appropriateness of the chosen methodologies selected by the researchers to achieve their goals.

In a recent UK legal case, the judge felt it appropriate to distinguish (obiter dicta) between the science and the ethics of applications, noting that the REC’s role was to concentrate on the latter:

In order to carry out research on patients, those working in the National Health Service are required not only to demonstrate the safety of the proposed research programme, but also to satisfy a Research Ethics Committee that the research is ethical. That is a separate matter from the safety of the research and involves a variety of considerations, including such matters as whether the research is likely to yield useful new information, whether the risks are explained to prospective participants with sufficient clarity, whether appropriate procedures are in place to ensure that patients are not pressurised into agreeing to take part, and whether the researcher has or may have any personal financial interest in the outcome of the research project. (Noordeen v Hill [2012] EWHC 2847 (QB) at 7, emphasis added)

The distinction between a scientific and an ethics review is emphasized in the standard operating procedures that govern NHS RECs (Department of Health, 2011, esp. para 5.4.2). RECs are not expected to routinely consider the scientific design of a study and they are not constituted to do this. The science/ethics distinction, however, remains a highly contested one. If science and ethics in an ethics review were to be separated, however, as the High Court expects, this should address most of the criticisms that have been levelled at RECs and could have significant time and cost benefits to industry, academia and other research, and all without compromising the protection mechanisms RECs offer, but it is not a view shared by all REC members and remains probably the most contentious issue in ethics review.

Those who argue that RECs have an obligation to approve or disapprove the scientific design of the research proposal note that ethics codes require good scientific design and that the risk to subjects necessarily relies on a prior determination about that design (Dawson and Yentis, 2007). Opponents, while conceding these two points, argue that RECs are not themselves necessarily constituted to make expert judgements about the adequacy of the scientific design at issue. Instead, RECs should form an opinion about the value of the science (‘the humanitarian importance of the problem to be solved’, as para. 6 of the Nuremberg code, 1947, put it) and leave the scientific issues (study design/methodology) to be assessed by others. Thus RECs must ensure that the science has been appropriately reviewed – but not necessarily review it themselves.

The study

The research to be discussed here takes the case of the ethics review of the testing of drugs (more formally, ‘investigational medicinal products’, IMPs). No drug can be tested on volunteers without the sanction of both an authorized REC and the approval of a ‘competent authority’. In the UK the competent authority is the Medicines and Healthcare Products Regulatory Agency (MHRA). This organization was established in the light of the European Union Clinical Trials Directive and has responsibility for approving the safety of all clinical trials. The MHRA’s experts approve the safety and scientific design of the trial, and the Phase I RECs were requested to relinquish their roles in adjudicating either on matters concerning the adequacy of the science involved or upon the safety of the study so far as the chemical entity was concerned. Whilst safety is a concern for RECs, that concern was expected to acknowledge and recognize that the MHRA already utilized independent persons capable of giving informed expert advice on the safety and study design proposed. RECs are thus not expected to have cause, or the expertise, to doubt or challenge this independent expertise, and a series of ‘memoranda of understanding’ were circulated by the MHRA and the relevant ethics review bodies to clarify this allocation of responsibilities (NRES, 2010). As the science underpinning the protocol was to be approved by the MHRA, the research reported here explored how influential the distinction between science (scientific design/methodology of the study) and ethics had become in the relevant RECs.

Although independent Phase 1 RECs no longer exist, this does not negate the relevance that their approach to the science and ethics distinction has to the understanding of the operation of other, currently existing RECs, as, where such committees are comprised of experts and lay members, the expert members feel obliged to express their view on those matters most closely related to their expertise, even where that expertise is only tangential to the study (Humphreys et al., 2014).

Of course, not all research involves IMPs, and thus the MHRA will not be involved in all research, but prior peer review of the science is always expected for all research submitted for NHS REC approval. Thus, whilst ‘ethics committees are not constituted to review the science of a project they … have an important role in checking that the science has been peer-reviewed’ (emphases added) (Edwards, 2010a: 58). Where it appears to a REC that an adequate independent peer review of the science or safety have not been conducted, they can refer the research proposal for independent review. Thus, student research is subject to academic critique by the supervisory team in the university (Humphreys, 2008), any statistical arrangements required in the research need to be verified by an independent statistician before submission of the protocol can be made to the REC (Williamson et al., 2000), and use of radiation has to be approved by a medical physicist. Staley (2012) has also pointed out that where there is evidence of adequate prior patient involvement in the planning of research, the RECs might also accept this evidence as demonstrative of patients being broadly content with the research proposed. RECs receive confirmation that such scrutiny has taken place.

The distinction between re-reviewing and accepting that something has already been properly reviewed must be understood if the situation of, in effect, a second trial for the same ‘offence’ (i.e. ‘double jeopardy’) is to be avoided. The REC must see evidence that a proper review has taken place, and where it has evidence that such a review has taken place it should not take it upon itself to ignore that fact. The science and the safety issues must be satisfactory because, if they are not, then the research may be unethical (either dangerous or wasteful of resources) – the issue is who should be responsible for the scientific review.

UK RECs are not constituted in such a way as to guarantee them a legitimate claim to scrutinize the methodology involved in a particular study. The ‘experts’ on a committee (who are defined by sch 2, section 1 of the Medicine for Human Use (Clinical Trials) Regulations 2004, SI 2004/1031 as current or former health care professionals or persons with statistical qualifications or experience in clinical research), for instance, may have had no relevant experience and could be either non-medical (e.g. therapists, podiatrists etc.) or long into their retirement.

REC review is supposedly an ethics and not a scientific review, yet the literature (which has not heretofore had access to the UK’s IECs) describes REC debates as typically embedded in a scientific rationality (Dyer, 2004). Rothstein and Phuong have noted how several official reports in the US have ‘indicated concern that scientific rather than ethical perspectives tended to predominate in IRB deliberations’ (Rothstein and Phuong, 2007: 76). This has also been reported in relation to the UK (Cheung, 2007; Collier, 1997; Nicholson, 1986), where Jennings has recognized that ‘[research] ethics committees lack expertise, not just on the research they review, but also on research ethics’ (Jennings, 2012: 95), and has called for a national, consensually agreed, REC training syllabus. If members followed a training syllabus they would appreciate that an ethics review differs from a re-review of the science.

Method

A qualitative interview study was designed to capture perceptions of members of Independent Ethics Committees (IECs) about their roles. Several features of the role were focused on in that study, but in this article concentration is given to the findings relating to the scientific critique in the ethics review. The study’s methodology has been described in detail elsewhere (Humphreys et al., 2014); briefly, it involved interviewing expert and lay members from amongst all the UK IECs in 2010 about the roles of members. The research was the first to study the workings of the UK’s IECs. Now disbanded, with their functions absorbed by the NHS’s National Research Ethics Service, IECs considered only research proposals involving Phase I drug trials. Such trials typically involved the first-in-human experience of a new chemical entity, but are more properly characterized as instances of a potential medication being given to healthy volunteers – individuals who do not have the disease that the drug targets and so who cannot benefit from taking the entity. The volunteers could only expose themselves to risk whilst their involvement helped the researchers to establish the potential medication’s safety and tolerability.

RECs are often noted as difficult to access for those wishing to subject them to research (De Vries, 2004), but IECs have been particularly impenetrable because of a combination of their low profile (McHale, 2004) and the commercial sensitivity that surrounds the studies they review (Beyleveld and Sethe, 2008). The study reported here was facilitated through the lead author’s lay membership of an IEC which provided the necessary contacts to overcome gatekeeper hurdles.

Interviewees were approached via their committee co-ordinators, who were provided with information about the study and asked to forward it to their members by way of an invitation to participate. Twenty interviewees representing all IECs, apart from the committee to which the researcher was attached, were interviewed. Equal numbers of experts and lay members participated, of whom nine were women and 11 men.

Interviews were all audio-recorded and conducted by the first author in both England and Scotland, in meeting rooms and quiet areas of restaurants, following negotiation with the participants concerned. Six interviews were conducted by telephone, where schedules combined with distance to make a face-to-face meeting impracticable. There did not appear to be any differences between face-to-face and telephone interviews.

Findings

Interviewees suggested that they expected the ‘experts’ on a committee to comment on the science involved, whereas the lay members were to put themselves in the position of the volunteer, to consider if the information to be given to the participants in the information sheet and in consent documents fairly represented the reality of the situation, so that participants’ consent could be properly informed. However, despite a generally held view of there being at least theoretical roles for both the ‘expert’ and ‘lay’ members, in practice the distinction between expert and lay was not clear, and any such distinction broke down in several ways. Firstly, although an expert member could include virtually all clinicians, interviewees tended to conflate ‘expert’ with ‘medical member’, thus suggesting that the latter group held a privileged position in the understanding of the science of the project under review.

The interview sample was clear that the experts who mattered were those who could say something about the investigational medicinal product (the test drug) – this might be from a medical, pharmacological or toxicological point of view. Typically, this was in terms of either experience with the specific drug or with the class of drugs to which it belonged. No other expertise gained recognition in practice, and this despite a ubiquitous presence of nurse members who are potentially recognized as belonging to the ‘expert’ group in the governing legislation. It was thus clear to the members what the expert role was – even if it was less clear to many of them who the experts were. Several members were uncertain who were in the expert or lay category on their committee:

I’m not the Co-ordinator, I don’t know who is an expert and who is a lay person but I would count [the retired nurse] as an expert. (E20)

I:

What about [your committee] now, is it chaired by a lay person?

R:

Let me just think … [long pause] No. (E18)

I would have to look through our list of attendees to see who is down as ‘lay’. We’ve got quite a few nurses …? (L11)

This uncertainty extended even to the members’ own status:

[W]hen I am in the lead role I am usually under the heading ‘expert’ because I understand the science […] I don’t really care what you call me [‘expert’ or ‘lay’]. (L11)

[T]he terms are a bit elastic. (E20)

The experts also took an interest in the information sheets, and the lay members might involve themselves in the drug’s mechanism of action:

I’m also interested because of my background in how the drug is working and what the outcome is going to be and I find that very interesting and, but I sometimes have to struggle to keep up with some of the toxicological and all the rest of it as I suspect most of the committee does … I want guidance from the experts on the committee. (Lay member, former nurse)

I’ve occasionally looked up the drug for myself and raised questions about dosing levels, side effects, whatever, purely because I couldn’t understand the logic behind the plan. (L01)

This latter quotation, whilst not the typical perspective of a lay member on an IEC, nevertheless demonstrates that the issues raised in committees tend to be standardized ones addressing such matters as the drug’s class-effects, appropriate starting doses, dose-escalation rates, stopping rules and so forth. These issues were always presented in either the protocol or the investigator’s brochure, both of which accompany an application so, given time, lay members could learn the right scientific questions that had to be addressed in determining whether a drug protocol would be likely to be ‘approvable’ by their IEC. These questions were also ones the MHRA would be considering. This quotation of L01 reveals how central science was to the ethics review.

Safety was also very much a concern for the whole committee:

R:

Basically the safety is huge … [You need to] know that what they [the sponsor, the personal investigator] are saying has been researched and they’re not just saying [so] … we have to … [ensure] just as far as we can see everything has been covered from the safety point. (L09)

[T]he main concern [for the IEC] is the patient or the subject. The subject’s safety is the number one priority. (E12)

The scientific rationale was always coupled with the safety aspects:

We have a strong consensus for if the science isn’t valid it isn’t an ethical trial so we have to find at least a reasonable scientific justification before we’d approve a trial. (E20)

Medicines and Healthcare Products Regulatory Agency

For 18 out of the 20 interviewees, issues of safety were always the main concern, and thus, they felt, a distinct role for their IEC to police. They reported that their committees addressed safety very carefully: ensuring that the trial was scientifically sound; that the drugs involved were administered at a dosage level commensurate with either what the toxicological data or previous human studies had indicated was a likely safe initial (often sub-pharmacological) dosage level; that dose escalation levels were appropriate for the type of molecule in question; that procedures were in place for monitoring any adverse side-effects; and that appropriate stopping rules had been established. All interviewees were clear that no protocol would be approved by the IEC unless the members were satisfied about the trinity of the science, safety and methodology. All of these specific matters, though, were considerations that the MHRA have had within its established remit to consider when it has looked at the proposed identical protocol that the IEC was considering.

There was universal uncertainty about the role of the MHRA amongst the interviewees, which raised the question of whether there was potential for ‘double jeopardy’, with the ethics committee essentially replicating a role that the MHRA was specifically constituted to address. Interviewees however, emphasizing the need to ensure safety, had no concerns about ‘double-checking’.

One lay member, whilst acknowledging that the MHRA was officially supposed to look at the science and safety behind the IMP, argued that the MHRA were ‘so new to it’ (L11) after the Clinical Trial Directive that they could not be relied upon. When pressed, though, respondents had difficulty in providing credible justification for their committee’s involvement in considering the science of the protocols. One toxicologist agreed that ‘we can leave it to the MHRA to make appropriate decisions on the science and safety’ but elsewhere in the interview acknowledged personally looking at these very issues in some depth on their IEC. Indeed, the consensus view was that even if there was ambiguity about whether the MHRA should have the role of being responsible for the science, safety and methodology, it was still the role for the IEC too.

Discussion

Previous studies have demonstrated that REC members are generally unsure about their role (Dyer, 2004; Nicholson, 1986). They have tended to circumvent their uncertainty by addressing scientific and technical matters more than wider concerns (Cheung, 2007). This situation of privileging scientific or technical matters was seen in the IECs under study, despite legislation and specific guidance which made it clear that this was inappropriate as the science and safety involved were the responsibility of a separate and specifically ‘competent authority’ (Edwards, 2010a, 2010b).

When interviewees were asked to relate the role of their ethics committee to the role of the MHRA, they universally acknowledged that the latter body had the legal responsibility, and the greater technical competence, to adjudicate on the science involved in the drug trial. Members, however, then struggled to explain the specific role of the ethics committee, or to justify the dominance of a scientific review within a committee established to address the ethics concerns that might underpin the proposed study, given that the MHRA was to concentrate on the scientific (methods, methodology, safety) aspects of the study. Although some commentators have recognized that the science of studies has often been the main emphasis in ethics review, such findings (Cheung, 2007; EULABOR, 2005) were made in situations where there was no specific body (such as the MHRA) having a separate but specific involvement, or where the science had not already been subject to a competent independent review. The uncertainty of interviewees regarding the role of the ethics committee vis-à-vis that of the MHRA (several years after its establishment with that role) is an important finding.

RECs insist that science and ethics cannot be separated because bad science equates to bad ethics (Angell et al., 2008; Hunter, 2007). Thus, over the years, an accepted review practice arose in RECs based on the principle that science and ethics could not be considered separately. This practice, it is contended, rests either on a misunderstanding of ethics review and/or a continuation of medical dominance. The theory of medical dominance (Freidson, 1970) in RECs is one we explore more fully in a separate article (Humphreys et al., 2014); suffice it here to state that the ‘expert’ (in practice read clinician, with the medical member being the paramount clinician) role has been operationalized in RECs in such a way that the basis of the expertise – science – has become the focus of the ‘ethics’ review.

The IECs could legitimately rely on the MHRA to adjudicate on the scientific design and the safety of the trial and turn their attention to ethics issues such as how the participants are to be treated, whether the insurance is at an appropriate level, whether certain groups of people are being unfairly prevented from participating in lucrative trials, whether the money on offer is too much (or inadequate), whether the information sheet is sufficiently comprehensive and clear, how incidental findings are to be dealt with, and so on. These are all more clearly ethical concerns yet have been relatively under-emphasized in a research ethics review which, as the research reported here again demonstrates, instead continues to concentrate on the science and safety of the trial – a role which merely, and inadequately, imitates that of the more expert teams utilized by the MHRA. It is not that science or safety are not matters for the ethics committee, but REC involvement can reflect and be respectful of any prior involvement of other independent experts who have responsibility for such matters. The ethics committee need only engage where they have cause to doubt the ability or independence of the expertise employed, and this should surely not be routine as it introduces double jeopardy for the researchers, requiring that two bodies be hurdled – the MHRA and the phase I REC – on the same issue of the scientific suitability of the study. The REC will not have been constituted to address the scientific appropriateness of the protocol, yet by raising questions about the science involved they delay the start of any trial as the MHRA and/or the pharmaceutical company has to respond to the REC’s comments. Even if this resultant delay is only of a few days, in the pharmaceutical industry a delay of even a day, even at this early stage in the research, can cost the company hundreds of thousands of pounds (Pope, 2012). Requiring researchers to undergo such double jeopardy can be regarded as inappropriate in ethics review given that the REC is specifically not constituted to engage in such a critique of the scientific design or, in the case of IMPs, in the safety of the novel compounds. An alternative understanding is that there is double jeopardy but that this is regarded by society as the cost it imposes on researchers wishing to engage in human subjects’ research.

The findings of our study suggest that debate amongst the interested parties about what is expected of an ethics review is appropriate, given that researchers can reasonably perceive themselves as being invariably faced with a re-review of their science when they attend an ethics committee.