Asking the unanswerable: Stymied again by the impossibility of sensible controls

Professor David Fergusson and his team, internationally known and respected for their longitudinal research, have added a new chapter in the fire dance of the abortion research debate with their recent article in ANZJP (Fergusson et al., 2013). They have been working in this area for some time (Boden et al., 2009; Dwyer and Jackson, 2008; Fergusson et al., 1979, 2006, 2007, 2008, 2009a, 2009b, 2013). Abortion, or termination of pregnancy (TOP), as many prefer to call the medically managed deliberate cessation of an unwanted pregnancy by drug or surgical techniques, is, as we all know, a highly contentious matter – at times even leading to the murder of people undertaking TOPs. TOP and infanticide has been present throughout history and in all cultures, so this complex issue cannot be seen as the consequence of modern day vagaries. TOP, as part of reproductive rights, continues to be a flash point for feminists who work to ensure equal opportunity for men and women, despite innate biological differences. For women, the capacity to conceive and bear children is both a unique strength and an exquisite vulnerability; managing this heavy burden is a responsibility which women take very seriously. We can debate how much influence societal institutions should have over decisions each woman makes, as she manages her reproductive potential; as we do so, we should remember that a woman’s willingness to surrender her own interests for the benefit of children she carries and cares for has been exploited by men and societal institutions over the ages.

This article is a partial review of the literature on TOP and mental health. It addresses a new question, a twist on the old debate about whether TOPs cause mental disorder in women who have them. The revised question is whether TOPs reduce the subsequent risk of mental disorder (i.e. whether a TOP is ‘good for the woman’s mental health’). Fergusson and colleagues suggest this is a valid question because most TOPs in New Zealand and similarly legally conservative jurisdictions are carried out under a Mental Health clause. The authors’ argument is that if continuation of an unwanted pregnancy has adverse mental health consequences for women, then research data should be able to show that provision of TOPs leads to improved mental health outcomes for this group of women.

In their review, the authors cite two recent reviews in their introduction (Coleman, 2011; National Collaborating Centre for Mental Health (NCCMH), 2011). They do not discuss the quality of the two reviews, although they provided the references for the interested reader to pursue if he or she has the stamina to assess each review for themselves. Surprisingly, they treat them as of equal scientific value. It quickly becomes clear that one is of excellent quality and a major contribution (NCCMH, 2011). This work was undertaken by the NCCMH, part of the UK’s National Institute for Health and Care Excellence (NICE) enterprise, responsible for the development of mental health guidelines. This systematic review aimed to synthesize a new narrative review and undertake a limited quantitative meta-analysis. Studies with a follow-up period of at least 90 days comparing abortion with delivery of an unwanted pregnancy were given multiple quality assessments. In addition to this review, they also published a summary in excess of 300 pages of an extensive collaboration process they devised, sending a draft to a wide range of interested organizations for comment.

The second, by contrast, has been roundly trounced in the journal which published it by seven immediate letters to the editor from senior and well-respected researchers. These trenchantly described many deficiencies in that review. These included:

• poor methodology (no description given of the search strategy, only two data bases used), only one rater (who had authored over half the articles, giving a significant conflict of interest), inclusion of more than one effect size from some studies and incorrect application of the population attributable risk metric (PAR) because causality and independence had not been established (Howard et al., 2012)

• incorrect use of PAR as there was not a single cause postulated (Abel et al., 2012)

• failure on all Assessment of Multiple Systematic Reviews items (AMSTAR) criteria, a checklist for the quality of systematic reviews (Littell and Coyne, 2012; Shea et al., 2007, 2009)

• failure to assess the validity of studies, examine statistical heterogeneity and an unjustifiable combination of estimates for diverse outcome (Polis et al., 2012)

• non-disclosure of conflict of interest, no assessment of study quality or publication bias (Goldacre and Lee, 2012)

• confusion of association with causality (Robinson and Stotland, 2011; Robinson et al., 2012).

It is an entertaining and informative discourse to witness!

One quick but robust way to assess the quality of a systematic review is by using the AMSTAR criteria (Shea et al., 2007, 2009). This can be seen as a simpler alternative to the rigorous Cochrane approach. The three reviews are contrasted in Table 1. Fergusson and colleagues (2013) did not search for relevant articles or judge study quality themselves; they reused the quality ratings from the NCCMH review. They described all the studies evaluated by both reviews, including those deemed poor by the NCCMH authors and not included. Analyses included adjusted odds ratios of the combined figures for a subsequent disorder after TOP or no TOP. They sequentially excluded the poorer studies in two steps: included by NCCMH or rated highly by NCCMH. To be fair, they did not claim to be undertaking a meta-analytic systematic review; however, what we have is a dubious hybrid where they have adopted the criteria for selection of articles and quality assessment from another group and then processed the figures. It would have been better if Fergusson and colleagues had undertaken their own literature search and quality assessment and let the reader see their reasoning.

Table 1.

AMSTAR criteria: a tool to assess the methodological quality of systematic reviews.

Item	Coleman (2011)	NCCMH (AMRC) (2011)	Fergusson et al. (2013)
1. Was an ‘a priori’ design provided?	N	Y	Y
2. Was there duplicate study selection and data extraction?	N	Y	N
3. Was a comprehensive literature search performed?	N	Y	N a
4. Was the status of publication (i.e. grey literature) used as an inclusion criterion?	N	Y	N a
5. Was a list of studies (included and excluded) provided?	N	Y	Y
6. Were the characteristics of the included studies provided?	N	Y	Reused NCCMH criteria
7. Was the scientific quality of the included studies assessed and documented?	N	Y	N, reused NCCMH quality assessments
8. Was the scientific quality of the included studies used appropriately in formulating conclusions?	N	Y	Y
9. Were the methods used to combine the findings of studies appropriate?	N	Y	NA
10. Was the likelihood of publication bias assessed?	N	Y	NA
11. Was the conflict of interest stated?	N	Y	CA

N: no; Y: yes; NA: not applicable; CA: can’t answer.

a

Fergusson et al. take articles identified by the two earlier reviews only.

b

Conclusions come from the inclusion of all studies cited in either the Coleman or NCCMH reviews.

However, the key problem running through these studies and related discussions is that there is not and cannot ever be an adequate control group. This is a ‘fatal flaw’. The complex social context in which an unwanted pregnancy arises cannot be standardized prior to randomization. Women who, on balance, decide not to proceed with an unwanted pregnancy who then seek and obtain a TOP are not the same as those who express the view that their pregnancy is unwanted but do not obtain a TOP. The second group may have a less intensely negative view of their pregnancy being unwanted (after all, a degree of ambivalence is common in pregnancy – especially in the early weeks), their social support for proceeding with the pregnancy may have been stronger, their resources (knowledge, finances, etc.) to overcome the TOP access barriers poorer, and so on. These are key variables which directly confound all results generated.

In general, the main shortcoming of evidence-based medicine is that it often cannot provide answers to the most pressing and arguably most important medical and social questions of our time. Predominantly this is because we cannot use random assignments to the intervention and non-intervention conditions (as is certainly the case with TOP research). The Plan B then is to do an epidemiologically sound cohort study with well-planned control groups to provide a base condition against which the cohort of interest can be assessed; such Plan B research may provide answers that are useful and clinically informative. However, this will never be the situation for TOP as it will not ever be possible to create sensible control groups.

What to do, in cases where medical science, particularly evidence-based medicine, cannot guide us? There are two more options to Plan B studies which may enhance thinking and debate.

1. Animal models. There are models for depression and anxiety in rats, guinea pigs and related mammals. Perhaps they could be given TOPs and not, in a random manner to see the relationship between pregnancy and depression equivalents. It will be impossible to assess unwantedness but still could provide a background of physiological data which may help.

2. ‘Thought experiments’, a known strategy used by our colleagues in philosophy. The best abortion example is the famous violinist scenario which highlights for men and others the issue of having one’s body taken over by a separate, frail, human entity (Thomson, 1971). Thomson uses this experiment to focus attention on the moral questions around TOP and concludes, by extrapolation from her experiment, that there are cogent moral arguments at times to support TOP.

Perhaps future technological developments, as yet science fiction, may ease our social discomfort. Artificial wombs may be ‘game-changers’. Unwanted foetuses could be transferred to a ‘Rent a WOMB-ROOM’, managed by an interested party’s financial-social-parental interest. Meanwhile, despite the industrious work of Fergusson and colleagues, we must proceed forward, not knowing and knowing that perhaps we will never know, what the impact of TOP is on the future mental health of the woman unfortunate enough to be pregnant at a time and place when she does not want to be.