Abstract
Background:
Medical research involving human subjects must be evaluated by a research ethics committee (REC) before a study is initiated. However, knowledge of REC decision processes, particularly in relation to evaluating the risk–benefit balance in various study types, appears scant.
Methods:
The study protocols and records of a Finnish university hospital REC were surveyed for a time span of 5 years. All study procedures in each study plan were examined, and the studies were divided into clinical drug trials, clinical trials with medical devices, studies with other invasive interventions, studies with non-invasive physical procedures, and non-physical procedures only.
Results:
In clinical drug trials, the REC posed queries more frequently about the risk–benefit ratio and less frequently about study methods and participant-related issues in comparison with other study types. Relative to other studies, those with non-physical procedures were subject to more frequent queries related to reliability of the study methodology and to data protection and confidentiality. Most of the queries focused on the recruitment process.
Conclusions:
The emphasis on risk–benefit evaluation in clinical drug trials may arise from RECs’ concern that clinical drug trials are most often conducted on patients and may involve new chemical entities with limited safety data. The relative scarcity of method queries in relation to clinical drug trials may indicate high scientific quality of industry-sponsored research, whereas the elevated percentage of these queries in investigator-initiated studies with non-physical procedures may suggest a need for better scientific education of independent clinical investigators.
Background
All medical research involving human subjects has to be evaluated by a research ethics committee (REC) before being undertaken. Principles of ethical assessment have been described in the World Health Organization (WHO) Guidance for Ethics Review of Health-related Research with Human Participants (WHO, 2011) and, particularly, in the World Medical Association Declaration of Helsinki (WMA, 2013). In general, the main task of a REC is to ensure the scientific and ethical validity of the research, especially through a risk–benefit evaluation for the proposed study’s protocol. In addition, RECs are required for ensuring that the proposed research is to be conducted in accordance with the relevant national legislation and regulations.
According to the WHO guidelines and the Declaration of Helsinki, a REC should be transparent in its function. Evaluation processes of RECs can be analysed through assessment of their decision documents. In this connection, study methods, such as selection and recruitment of subjects, along with statistics, have been reported to be the key subjects of queries made or amendments requested by RECs (Adams et al., 2013; Angell et al., 2008; Boyce et al., 2002; Bueno et al., 2009; Dal-Re et al., 1999; Kent, 1999; Lutz et al., 2012; Mansbach et al., 2007; Martín-Arribas et al., 2012; van Lent et al., 2014). Studies involving vulnerable groups, such as minors, often prompt certain queries (Adams et al., 2013; Lidz et al., 2012; Martín-Arribas et al., 2012). In addition, issues related to the autonomy of the study subjects and to recruitment procedures have been important among the general spheres of queries from RECs (Adams et al., 2013; Angell et al., 2010; Boyce et al., 2002; Bueno et al., 2009; Dal-Re et al., 1999; Decullier et al., 2005; Kent, 1999; Lidz et al., 2012; Lutz et al., 2012; Mansbach et al., 2007; Martín-Arribas et al., 2012; van Lent et al., 2014).
One of the main tasks of the REC is to ensure the safety and well-being of study participants (WMA, 2013). However, previous work suggests that only 8–20 percent of REC queries directly address risks and benefits associated with participation (Boyce et al., 2002; Bueno et al., 2009; Martín-Arribas et al., 2012; van Lent et al., 2014). There also seems to be variability between RECs in risk–benefit evaluation. In an analysis of 20 protocol reviews by one REC, the committee did not consider risk–benefit comparison in 60 percent of cases (Lidz et al., 2012). In another study, the total number of queries about risks and benefits was quite high, and these were raised for as many as 37 percent of the protocols (Adams et al., 2013). In addition, uncertainty about a study’s risks and benefits might be the key issue for some REC members when they decide on approval of clinical trials (Wao et al., 2014).
Rid and Wendler (2011) have proposed a specific framework for evaluating the risks and benefits associated with clinical research. In line with the first step in their framework for risk–benefit evaluation in biomedical research, several studies have addressed the RECs’ role in ensuring a study’s social value, including the appropriateness of the data collection and analysis (Angell et al., 2008; Boyce et al., 2002; Bueno et al., 2009; Kent, 1999; Lutz et al., 2012; Mansbach et al., 2007; van Lent et al., 2014). The next step with the guidance framework focuses on evaluating the study interventions. However, previous studies have not examined how the level of possible intervention risks affects the REC evaluation process.
In our study, we surveyed the records of a university hospital REC that has a duty to review all medical research carried out in humans in its region (population: 565,000). Our main aim was quantitative evaluation of the types of queries pertaining to the study protocols, including ethics statements by principal investigators (PIs) – an official REC requirement in Finland. We also sought to determine whether studies with invasive, and hence more risky, procedures were more frequently subject to queries by the REC.
Material and methods
The study sample
We surveyed the records of the REC of the North Savo Hospital District for 1 January 2009 to 31 December 2013. The study material consisted of all study protocols of medical research involving humans submitted for REC review. Data were manually extracted by the first author and categorized by the first and second author. The procedure for selection of studies for inclusion in analysis in the study is shown in Figure 1. The study material comprised, in total, 349 protocols, their attachments (such as the ethics statement provided by the PI), and the related REC decision letters, including all the queries directed to a study protocol. The REC had conducted the evaluation process according to pre-specified, standardized protocol. The evaluation protocol included the following issues: (i) scientific value and validity of the study; (ii) subject selection criteria and recruitment protocol; (iii) benefit–risk evaluation; (iv) research process and quality assurance; (v) personal data security; (vi) publication of results; and (vii) financial and insurance issues. All queries are directed to the principal investigator in a decision letter.
Selection of the study protocols for inclusion in the study.
Ethics and confidentiality
As this study did not involve human subjects or confidential health records, neither REC approval nor informed consent was required under Finnish legislation (Finnish Medical Research Act, 1999; Finnish Personal Data Act, 1999). For ensuring the confidentiality of the study protocols and REC decision letters and gaining access to the research data, an organizational permit was applied for from Kuopio University Hospital. The study results are presented such that the original study protocols cannot be identified from the details provided.
Data extraction and classification
All study procedures in each study plan were examined and categorized into the following: (i) invasive physical study procedures; (ii) non-invasive physical procedures; and (iii) non-physical (i.e. psychological) procedures. In the studies involving invasive physical procedures, a participant was an object of methods involving the use of instruments, objects, or agents on the body, as in taking of blood samples and biopsies, surgical experiments, and introduction of investigational drugs or nutritional agents. In addition, exercise stress testing and imaging methods causing radiation exposure to participants were categorized as invasive physical study procedures. Non-invasive physical study procedures were defined as research methods wherein there was physical contact, measurement, or physical exercise but the procedure did not involve use of invasive instruments or other objects/agents on the body. The non-invasive study procedures comprised, for example: measuring weight, height, vital signs, or physical activity; collecting urine or stool samples; and performing imaging that did not cause radiation exposure. The non-physical study procedures were employed in studies in which a participant’s physical integrity was not interfered with; that is, there was no physical contact with the participant, no physical exercise interventions were applied, no samples were collected, and no physiological measurements were performed. Among these study procedures were questionnaires, interviews, and psychotherapy interventions.
After categorization of the study procedures, studies entailing invasive physical interventions were divided among the following: (i) clinical drug trials; (ii) clinical trials with medical devices; and (iii) studies with other invasive interventions. The remaining – non-invasive – studies consisted of studies with (iv) non-invasive physical procedures and (v) non-physical procedures only.
The characteristics of the final sample of studies are presented in Table 1. Vulnerable groups were defined in the manner presented in the Finnish Medical Research Act (1999) as: (i) persons who, for mental or psychiatric reasons, are not able to give consent; (ii) minors; (iii) pregnant or nursing mothers; and (iv) prisoners and forensic patients.
Characteristics of the final sample of studies analysed in the study, n (%).
The summed percentages exceed 100 because studies may be included in multiple categories.
More than one group of human subjects may participate in a given study.
In addition, we recorded the number of clinical drug trials investigating new chemical entities, and the study phases of the drug trials. Altogether, there were 37 (51%) phase 3 clinical drug trials, 25 (34%) trials were in phase 4, 8 (11%) trials in phase 2 and 3 trials (4%) in phase 1, respectively. New chemical entity was investigated in 31 (42%) trials. For clinical drug trials, we also studied whether an investigation of a new chemical entity or a study phase of a trial had any effect on REC risk evaluation.
The focal areas of issues raised in the REC queries were classified into five main categories: (i) the rationale and purpose of the study; (ii) study methods; (iii) participant-related issues; (iv) risks and benefits; and (v) administrative issues such as amendments related to investigator information or research sites.
Statistical analysis
Differences between the study types examined, shown in Table 2, were tested with chi-squared and Fisher’s exact tests. A p-value below 0.05 was considered to be statistically significant. Statistical analyses were performed with the software R (version 3.2.0, from the R Foundation for Statistical Computing, 2015).
Statements of the research ethics committee by study type, n (%).
Results
In total, 14 of the study protocols (4%) were given a favourable statement in their first REC review. A provisional favourable statement was issued for 272 protocols (78%). The remaining 63 protocols were given an unfavourable statement on first review. Of these, 58 (17%) received a favourable statement after amendments. In the case of five study protocols (1%), the final decision was unfavourable. The distribution of statement types was comparable across all study classes (see Table 2).
The main issues raised in the queries from the REC included matters associated with the recruitment process, various aspects of methodology, and patient-care-related issues (see Table 3). Of all the categories of study investigated, clinical drug trials showed the highest percentage of queries related to risks and possible health benefits, and they showed a lower percentage of queries related to study methods and participant-related issues than did protocols for other study types. In addition, investigation of a new chemical entity or a study phase of a clinical drug trial did not significantly affect REC risk evaluations (p > 0.05; data not shown). In studies involving medical devices, the queries more often pertained to participant-related issues such as the recruitment process and taking participants’ points of view into account in the study design. When a study utilized only non-physical study procedures, the REC’s queries more frequently had to do with issues of the reliability of the study methods, privacy and confidentiality. All told, 23 of the studies utilizing vulnerable groups (30%) were directed queries aimed at further clarification of the justification for inclusion of these groups.
Topics of the queries by the research ethics committee, n (%).
p-value < 0.05; ** p-value < 0.01.
Discussion
In the present study, characteristics of certain study types were examined in relation to frequency of REC queries. Queries about risk and harm and on possible health benefits for the participants were directed more frequently to those submitting protocols for clinical drug trials (32%) than those with other study types (3–17%). Furthermore, we observed that the REC devoted queries to study methods and participant-related issues less frequently when evaluating clinical drug trials, relative to other study types. In studies with non-physical procedures, the queries more frequently focused on reliability of the study methods and matters of data protection and confidentiality. Overall, most of the queries were focused on the recruitment process.
In our research, study methods and participant-related issues were found to be the most common query themes, a finding consistent with previous studies (Adams et al., 2013; Angell et al., 2010; Boyce et al., 2002; Bueno et al., 2009; Dal-Re et al., 1999; Kent; 1999; Lutz et al., 2012; Mansbach et al., 2007; Martín-Arribas et al., 2012; van Lent et al., 2014). The RECs’ role in assessing scientific issues of medical protocols has been seen to be an inevitable one, related to ensuring appropriate study ethics (Angell et al., 2008; Dawson and Yentis, 2007; Humpreys et al., 2014). In an interview-based study by Humphreys et al. (2014), REC members concluded that participant safety and ethics factors could be deemed appropriately addressed only if the study methods are scientifically sound. These factors were seen as necessary for approval of a medical protocol. In our study, the REC made queries about study methods just as often as about participant-related issues. Our observations as to the scientific concerns expressed in REC decisions are in line with previous studies of RECs’ evaluation processes (Adams et al., 2013; Angell et al., 2008; Boyce et al., 2002; Bueno et al., 2009; Dal-Re et al., 1999; Kent, 1999; Lutz et al., 2012; Mansbach et al., 2007; van Lent et al., 2014).
As we observed, clinical drug trials were associated with lower frequency of queries related to study methods and participant-related issues than were other study types. This is in contrast to observations in a previous study (van Lent et al., 2014), in which study methods and statistics-related issues were addressed in 71 percent of clinical drug trial queries. In that research, the proportion of industry-sponsored trials was significantly lower (46%) than in our study (73%). Furthermore, the design and conduct of clinical drug trials are strictly regulated by drug safety authorities; this may affect the trials’ scientific quality.
In parallel with some previous reports, most of the protocols submitted for review to the North Savo Hospital District REC received a provisional favourable statement (Adams et al., 2013; Angell et al., 2008, 2010; Boyce et al., 2002; Bueno et al., 2009; Decullier et al., 2005; Kent, 1999; Lutz et al., 2012; Mansbach et al., 2007). However, there seems to be great variation between RECs in terms of decision procedure (Dal-Re et al., 1999; Edwards et al., 2007; Kent, 1999; Lidz et al., 2012; Mansbach et al., 2007). In a study that surveyed responses from several RECs evaluating the same multi-centre protocol in the UK, the proportion of protocols approved without any queries ranged from 6 to 32 percent between individual RECs (Kent, 1999). In Spain, one REC approved 61 percent of applications without any queries (Dal-Re et al., 1999). In our survey, the proportion of negative final decisions (1%) was very low and within a similar range as in some previous reports: 1.5–3 percent (Adams et al., 2013; Bueno et al., 2009; Dal-Re et al., 1999).
In the Rid and Wendler (2011) framework the risks of the study procedures should be evaluated. These authors suggest evaluating the possible benefits of each study procedure. In our study, queries directly related to risks and benefits were raised for 26 percent of the protocols – a figure quite comparable to the levels presented in previous surveys (8–20%) (Boyce et al., 2002; Bueno et al., 2009; Martín-Arribas et al., 2012; van Lent et al., 2014). As risk–benefit assessment for the protocols is judged to be among a REC’s main tasks (WMA, 2013), this figure seems fairly low. However, some of the concerns raised about protocols’ methodological issues may be associated with safety and risks, correspondingly reducing the percentages of queries assigned to the risk–benefit category. Furthermore, in our study, possible risks were addressed for only a minority of non-invasive study protocols. This may indicate that, in terms of risk evaluation, RECs may focus more actively on studies applying invasive procedures. As we observed, the phase of a clinical drug trial or investigation of a new chemical entity did not have a significant effect on the REC evaluation process regarding risk and harm. This may indicate that the REC focuses on the risks arising from the protocol as whole, and not only on those related to a specific study phase or a new chemical entity. Except for clinical drug trials, the proportion of queries to do with benefits of the research remained rather low. Either study protocols address possible benefits better than they do risks or it is understood that research cannot always guarantee direct health benefits to the participants. On the other hand, even in studies with no therapeutic intention, participants may experience psychological health benefits (Decker et al., 2011; Lowes and Gill, 2006).
Strengths and limitations
Some limitations need to be taken into account in interpretation of our findings. The dataset utilized represents the decisions of one REC. However, the chair and the members of the REC have changed during this 5-year period, and thus the decision-making is not done by the same individuals. Although several aspects of the queries made by the REC considered here are consistent with findings from other reports (Adams et al., 2013; Angell et al., 2008, 2010; Boyce et al., 2002; Bueno et al., 2009; Dal-Re et al., 1999; Decullier et al., 2005; Kent, 1999; Lutz et al., 2012; Mansbach et al., 2007; van Lent et al., 2014), RECs do show variations in their decisions (Dal-Re et al., 1999; Edwards et al., 2007; Kent, 1999; Lidz et al., 2012; Mansbach et al., 2007). We focused on issues related to study protocols rather than participant materials such as the information sheets. Previous studies have shown that often the main elements prompting queries or amendment requests are inadequate participant information sheets and informed consent documents (Adams et al., 2013; Angell et al., 2010; Boyce et al., 2002; Dal-Re et al., 1999; Decullier et al., 2005; Kent, 1999; Lidz et al., 2012; Mansbach et al., 2007; Martín-Arribas et al., 2012; van Lent et al., 2014). This can be considered to match our finding that most of the protocol-related queries pertained to the recruitment process. One of the strengths of our study was that we performed comprehensive evaluation of the study protocols of one REC and assessed the outcomes for various study types, indications and interventions.
Conclusion
We found that characteristics of certain study types were correlated with the frequency of queries by the REC. Queries related to risk and harm and to possible health benefits for the participants were more commonplace with clinical drug trials than with other study types. This, in turn, may be linked to the fact that clinical drug trials are most often conducted in patients and also involve new chemical entities with limited availability of safety data. Additionally, placebo exposure may increase the risks of participation for patient populations. That fewer queries addressed study methods and participant-related issues with industry-sponsored clinical trial protocols relative to other study types may indicate a high scientific quality of the sponsored research. In contrast, queries directed to studies with non-physical procedures more often focused on reliability of the methods and on data protection and confidentiality. Because most studies involving only non-physical procedures were investigator-initiated, this finding may suggest a need for better scientific education of clinical investigators.
Footnotes
Declaration of conflicting interests
SML and TK are former chairpersons of the North Savo Hospital District REC. AH is the scientific secretary of the North Savo Hospital District REC.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.