Abstract
In his article ‘The case against ethics review in the social sciences’, Schrag asserts that the social sciences should not be subject to ethical review. He recounts a number of examples where ethical review has seemingly failed. He further suggests some alternative models for dealing with ethical review in the social sciences. Finally, he concludes, and we concur, that there is a lack of empirical evidence as to the benefit of research ethics review.
Introduction
We contend that the logic behind Schrag (2011)’s argument that social sciences should not undergo ethical review is unclear. We interpret his overall argument as follows:
Social science research is subject to ethical review.
Many social sciences researchers have had bad experiences with ethical review, or complain about ethical review.
Social science research imposes few risks that are rare.
Therefore, social science research should not be subject to ethical review.
We suggest that his conclusion does not follow from his premises, and that whilst social scientists may complain that their research poses rare and limited harms, and is sometimes incorrectly reviewed, this does not preclude social science research from ethical review. Rather than a case against ethics review in the social sciences, we would offer that what Schrag does demonstrate are failures of practice that point to a need for improved standards and guidance that can inform the education of research ethics committees.
Rather than throwing the baby out with the bathwater, and doing away with ethical review of social science research, we suggest that standards and guidelines are needed to inform better education of ethics committees on the specific ethical issues that social sciences raise. We suggest that models of developing standards for ethics committees about challenging designs exist, and highlight recent work to elucidate challenges around, and develop guidance for, ethical review of cluster randomized trials (CRTs) Weijer C, Grimshaw JM, Taljaard M et al. 2011:100; Taljaard M, Weijer C, Grimshaw JM et al., 2009; McRae AD, Weijer C, Binik A et al. 2011:202), which could more effectively address the problems Schrag identifies.
Social science and ethics review
Ethical review of social science research is problematic and, as Schrag himself notes, not a new issue (Oakes JM, 2002; De Vries R, DeBruin DA, Goodgame A, 2004). Dingwall, for example, in his 2006 Plenary Address to the Annual British Sociological Association (BSA) Medical Sociology Group Conference, suggested that ethical review is problematic in part because the harms from social science are not directly comparable with those of traditional biomedical research (Dingwall R, 2006: 51–8). This is echoed by Schrag when he contends that there is a lack of proven abuses within the social sciences and humanities. Furthermore, he argues that:
[…] compared to the problems of medical research, serious social-science abuses are quite rare. (Schrag, 2011)
We would counter that if ethical review can minimize the potential for harm, or even the probability, then this would be beneficial. Moreover, whilst there may be no comparable atrocities to the nazi medical experiments (which are generally given as the motivation behind ethical review), the fact that there is the potential for abuse in itself justifies ethical review. Certainly social science poses risks to the invasion of privacy, psychological trauma, and stigmatization (Oakes JM, 2002), all of which may have serious and long lasting implications. There are also examples of social science research that have serious adverse effects. In the infamous sociopsychological studies of obedience conducted by Milgram, he notes that:
Many subjects showed signs of nervousness in the experimental situation, and especially upon administering the more powerful shocks. In a large number of cases the degree of tension reached extremes that are rarely seen in sociopsychological laboratory studies. Subjects were observed to sweat, tremble, stutter, bite their lips, groan, and dig their finger- nails into their flesh. These were characteristic rather than exceptional responses to the experiment […]. On one occasion we observed a seizure so violently convulsive that it was necessary to call a halt to the experiment. (Milgram, 1963)
In his summary, Milgram concedes that this tension was ‘unanticipated’. The potential for such unanticipated psychological harms would, one would imagine, warrant review. Yet even if one views psychology as not falling within social science, as Schrag appears to, then one can point to serious unintended consequences of social science research. For example, Israel and Hay (2006) note examples where surveys of flood hazard perception and concomitant measurements of street elevations generated rumours that homes would be removed for a new road. This caused anxiety and concern amongst respondents. In other research in a closely connected community it was noted that, despite steps to anonymize information, locals who read the final project report were able to identify participants and provide additional information, thereby removing any semblance of confidentiality.
Schrag’s position is also open to empirical assessment given that we know very little about the non-physical risks posed by social science research; how do we measure such harms? How rare do such harms have to be to remove the requirement for ethical review? What counts as serious? A lack of information means that review committees must speculate in the absence of information (Oakes JM, 2002).
Even when the researcher has made conscientious attempts to minimize the potential for harm, they inevitably have an investment in the study. Independent review may identify potential harms that the researcher has missed. Indeed, a further benefit of ethical review is the opportunity for review of the scientific endeavour and potential for improvement. Whilst some may argue that scientific review should be considered independently of the ethical issues, we contend that the science/ethics distinction is intractable for several reasons, each of which points to the need for review.
Firstly, there are opportunity costs to the funding of any research in the sense that other research will not be funded. A funded but poorly conducted study means that another potentially well conducted study is not (Hunter D, 2007; Dawson A, Yentis SM, 2007). Ethical review may, therefore, be supported from a resource allocation perspective. Furthermore, if conducted studies are of poor quality – in the sense that they are unable to answer the research question they set out to address (Hunter D, 2007) − then society may be harmed through the failure to identify poor, socially harmful, practices that could be attenuated. Alternatively, the converse may be true and society is deprived of potentially useful information Dawson A, Yentis SM, 2007). For example, if a poorly conducted study led one to draw erroneous conclusions that then influenced practice (whatever this may be), then individuals may be misled but also potentially harmed. Such harms may be considered indirect harms of poor research (Hunter D, 2007).
Moreover, if some social science research is poorly conducted this may have a negative impact on the discipline; research that does not achieve its objectives, or that has unintended negative consequences, can lead to bad feeling that may have a detrimental effect on future research (Bryman A, 2004; Dawson A, 2006; Dawson A, Yentis SM, 2007). This may deter individuals from participating, or lead to poor uptake of results generated from social science research.
Furthermore, ethical review may be the only source of review that some research undergoes. This in itself may be beneficial; as van Teijlingen has argued, whilst there is considerable work to be put into the ethics review process:
it should be remembered that quality of the research proposal will be improved by early consideration of these details. (van Teijlingen, 2006)
Hence, by going through the ethics review process the researcher is forced to think about the ethical issues that their study may confront. This process may, in itself, be beneficial.
Process not principle
Despite the examples presented, Schrag fails to provide a principled argument against ethical review. Rather, by citing examples of poor ethical review, we believe that Schrag is trying to turn an issue of process into one of principle. Some of Schrag’s own comments point to this when he concedes that the revised Tri-Council Policy Statement addresses some of his concerns. That the issues have been addressed points to the fact that it is not a principle that social science research should not undergo ethical review, but rather that the current process of ethical review can be inappropriate for some social science studies.
The issue then becomes, in essence, one of knowledge translation (KT). In the UK, for example, the National Research Ethics Service (NRES), a centralized body with oversight of ethical approval processes in the UK, offers training on the research ethics process and specific issues related to qualitative research. This is offered as an educational supplement to both ethics committee members and researchers. The development of tools related to social science methodology could be incorporated into similar existing mechanisms elsewhere, thereby facilitating the uptake of this knowledge into practice.
Therefore, in contrast to Schrag, we posit that there is a need for independent oversight of social science studies. We accept that in studies where there are minimal well-defined risks, for example with anonymous survey research where only aggregate responses are reported, simple procedures can be highly effective. Furthermore, we agree that when there is great inconsistency between the responses of ethics committees, that there is a problem – more so if there is a lack of a commonly held frameworks or rationale for decision-making. As such, we believe that there should be explicit standards and that a collaborative process, involving social science investigators and ethicists, would be desirable.
The ethics of cluster randomized trials as a practical example
We propose that recent work around the ethics of cluster randomized trials (CRTs) may serve as a useful approach to addressing challenges posed by ethical review of less understood research designs.
CRTs are an increasingly popular methodological design in which the units of randomization are social units or clusters of individuals rather than individuals themselves (Taljaard M, Weijer C, Grimshaw JM et al., 2009). CRTs can involve different units of randomization, intervention and analysis, in contrast with more standard randomized control trials (RCTs), where all three are targeted at the individual participant. This aspect of CRTs presents particular ethical challenges. For example, in studies where the intervention targets clusters such as cities or villages, it can be unclear whether all community members must be considered research subjects, and must therefore be afforded ethical protections such as informed consent (McRae AD, Weijer C, Binik A et al., 2011). Problems such as these have led to notable variation in the experiences of ethics review by researchers conducting CRTs (Chaney E, Rabuck LG, Uman J et al., 2008; Sabin JE, Mazor K, Meterko V et al., 2008; Dziak K, Anderson R, Sevik MA et al., 2005) in much the same way that there has been with qualitative social science review.
In order to address this, a mixed methods research program was undertaken to develop guidance for the ethical review of CRTs. This process involved a series of normative analyses identifying key ethical issues presented by CRTs, and reasoned arguments resulting in suggestions about how ethical review should proceed in a way that maintains protections without unnecessarily hampering research. Empirical work sought input from trialists, participants and REB chairs, to ensure that all relevant perspectives were incorporated into the guidance. Based on input from a consensus conference of interested stakeholders, an expert panel convened to draft the guidelines, and feedback has been sought from a wider array of interested parties through publication, on online wiki, conference presentations, and other mechanisms of knowledge translation. 1
This collaborative approach between stakeholders, including researchers and ethics committee members, together with other stakeholders such as research funders and policy-makers, has promoted discussion regarding the salient issues and best practice for ethical review. By using such an inclusive approach, one not only identifies relevant issues from the differing perspectives but also facilitates reasonable solutions. Furthermore, the ultimate audiences are likely to be receptive to the changes given their involvement in their development. In this way the process may overcome some of the dissatisfaction experienced by social scientist researchers; historically they have been excluded from developing review guidelines yet forced to comply with them (Oakes JM, 2002). In the same way, it may be that such an approach could facilitate discussion and best practice regarding the appropriate review of social science research.
Evidence of benefit and quality of ethics review
Schrag concludes his article by arguing that his criticisms would be more muted if one could demonstrate the effect and benefit of research ethics review. This he argues is not the case and there is no record demonstrating their effectiveness at protecting research participants.
We suggest that in this regard Schrag has touched on a pressing issue for research ethics review: the need to evaluate the ethics review process and the current paucity of data as to the ‘quality’ of research ethics review. Whilst bodies such as the Food and Drug Administration (FDA) and the Office for Human Research Protections (OHRP) provide oversight of research ethics boards, existing programs of evaluation or accreditation tend to focus on meeting administrative requirements as opposed to assessing the quality of decisions made or improvements engendered by review. This is reflected in the literature, which has centred on process issues such as the correct application of administrative rules (Burris S, and Welsh J, 2007), length of time taken to process applications (Dziak K, Anderson R, Sevick MA et al., 2005; Greene SM, Geiger AM, 2006; Larson E, Bratts T, Zwanziger J et al., 2004), composition of ethics committees (Fauriel I, Moutel G, Duchange N et al., 2004), and the content of letters from committees to reviewers (Angell EL, Bryman A, Ashcroft RE et al., 2008).
And herein lies the rub; these evaluations ‘[do] not make any statement on the outcome of REC’s deliberations in terms of research safety or acceptability.’ Furthermore, as Coleman and Bouësseau (2008) argue:
[…] the fact that an REC has documented that it has considered a protocol's risks and potential benefits does not mean that it has done a good job identifying or weighing these factors. […] In short, all that auditing and accreditation programs tell us is whether RECs are carrying out the specific tasks that have been assigned to them.
To date, despite numerous calls for the development of quality indicators that stretch back over decades, there has been little movement on this front (Coleman CH, Bouësseau M-C, 2008; Taylor HA, 2007; Pritchard IA, 2011). Furthermore, if one wishes to engage research ethics committees in a program of education about social science research then this education must be empirically sound. As Oakes (2002) notes: ‘Speculation and anecdotes, although perhaps part of the process, will not be sufficient.’
We agree, therefore, that there is a need for research and data regarding the impact of research ethics review and a desirability of quality indicators and proof of concept in terms of the benefits of research ethics review.