Abstract
U.K. national guidelines for the treatment of schizophrenia recommend art therapy among other approaches. However, a recent major trial called MATISSE (Multicenter evaluation of Art Therapy in Schizophrenia: Systematic Evaluation) suggests that art therapy may not be helpful. The purpose of the present study was to explore reasons for the MATISSE trial findings. A critical review of the MATISSE trial drawing on six papers reporting on the trial and its processes was performed. The MATISSE trial appeared to have weak conceptualization of the mechanisms for change, lack of piloting, incomplete process and subgroup analyses, and inappropriate assumptions about the generalizability of findings. The MATISSE trial’s conclusion that art therapy is of no value to people with a diagnosis of schizophrenia is unwarranted. More account should be taken of extant quality guidelines for complex interventions, including proposed change mechanisms, piloting, process analyses, variations in practice and contexts, and the effect of randomization on generalizability.
Background
Schizophrenia and Its Treatment
The United Kingdom’s National Institute for Health and Clinical Excellence (NICE; 2009) guidelines for the treatment of schizophrenia describe it broadly in accordance with the International Classification of Diseases (ICD) version 10 (World Health Organization, 1992). They describe “positive” and “negative” symptoms, the positive being “hallucinations, delusions and behavioural disturbances” (NICE, 2009, p. 16), while the negative include social withdrawal, poor memory and concentration, and reduced interest and motivation in daily activities. The document refers to severe stigma and discrimination affecting those with the diagnosis, and recognizes these, as well as medication “side effects,” as complicating factors for recovery of a “normal” life. The effect of stigma has been demonstrated in a 2-year follow-up of discharged mental health patients, most with a diagnosis of schizophrenia, in the United States (Wright, Gronfein, & Owens, 2000), and by a review of research on attitudes toward mental distress by Angermeyer, Holzinger, Carta, and Schomerus (2011), in which beliefs in biological causes of schizophrenia were associated with increased stigma.
Unfortunately, negative views of people diagnosed with schizophrenia can be similar or worse among mental health professionals than other members of society (Caldwell & Jorm, 2001; Wahl & Aroesty-Cohen, 2009). Service users and carers have reported being told, along with the diagnosis of schizophrenia, that the person with this diagnosis will never get better (Chandler, Bradstreet, & Hayward, 2013). Yet reviews of the evidence by Boyle (2002), Moncrieff (2013) and Moncrieff and Leo (2010) highlight a lack of hard evidence of an identifiable pattern of brain degeneration independent of the effects of long-term medication. Indeed, as Moncrieff’s (2013) detailed review highlights, long-term antipsychotic medication, the main treatment for those given the schizophrenia diagnosis in Western countries, appears to produce cognitive impairment, weight gain, and cardiovascular effects, and in young people many of these effects appear within a few weeks of starting medication (National Institute of Health and Care Excellence [NICE], 2013). While the general dampening of mental and physical activity may produce some relief from intrusive thoughts and voices in the short term, common descriptions by adult long-term users convey a subjective experience of general demotivation and emotional and cognitive deadening (Moncrieff, Cohen, & Mason, 2009).
Moncrieff’s (2013) review of the evidence on “antipsychotic medications” illustrates serious weaknesses in many of the trials that purport to demonstrate their efficacy, and documents cases of suppression of some studies by pharmaceutical companies. Bentall and Morrison (2002) drew attention to the harmful effects of antipsychotic medication given to young people believed to be at risk of developing psychosis. The NICE (2013) guidelines for the treatment of “schizophrenia” in young people do recognize negative effects of medication, cautioning against prescribing antipsychotics to young people who have not been diagnosed with schizophrenia or psychosis but are considered likely to develop it. Instead, cognitive behavioral therapy and treatments in accordance with the identified difficulties (such as anxiety and depression) are suggested. For adults already on antipsychotic drugs, however, it has been made harder for them to opt out of continuing, as compulsory community treatment may be instituted (Moncrieff, 2013).
Setting aside Boyle’s (2002), Bentall’s (2009), and Moncrieff’s (2013) rigorous scientific critiques of the notion of schizophrenia as a brain disease and of the flawed assumptions about the specificity and efficacy of drug treatment, the NICE (2009) document describes schizophrenia’s causes in terms of a combination of genetic and environmental factors, the suggestion being that genetic differences may make some people more easily develop this contested condition under stressful circumstances. However, a review by Shah, Mizrahi, and McKenzie (2011) suggests that multiple genes are likely to be involved and the same genetic make-up can confer vulnerability to mental distress in one set of circumstances and be protective in another. Thus, efforts to “fix” people’s genetic make-up may be misguided. Shah et al. (2011) also point out the growing body of evidence that severe or prolonged stresses and abuse, including marginalization, discrimination and isolation in adulthood, and abuse and neglect during childhood, are more important than previously thought in the causation of disorders that come to be labeled as schizophrenia. Shah et al. (2011) suggest the need for more attention to fostering well-being through psychologically healthy communities.
Bond and Drake (2008) reviewed studies on employment and education support for adults with a long-standing diagnosis of schizophrenia and reported that such support was the best predictor of their entering work or education, rather than the presence or absence of “symptoms” or their severity. Other research, reviewed by Rinaldi et al. (2010), suggested that more than two thirds of young people diagnosed with schizophrenia entered employment or education when specialist support was included in their care, compared with only one third when it was not. The NICE (2009, 2013) guidelines for the treatment of schizophrenia in adults and young people, respectively, recommend that mental health services work with local education and employment support agencies.
Art Therapy
In the United Kingdom and internationally, art therapists have worked with people with a diagnosis of schizophrenia in both inpatient and community settings for many decades (Wood, 1997, 2013). Ever since its early days, there have been different strands of art therapy, with initial focus on supporting spontaneous artistic expression within hospitals in the first half of the 20th century (McNiff, 2004; Wood, 1997), and later the increasing adoption of different strands of psychoanalytic thinking (Wood, 1997). Waller (1993) identified two major conceptions of art therapy: “art as therapy and art psychotherapy” (p. 8). The former appears consistent with the historically earlier approach, in which artistic activity is viewed as healing by its very nature and is often instigated by the person suffering mental distress (McNiff, 2004; Wood, 1997). Here, there can also be a focus on developing artistic skills and celebrating creativity. Approaches in the “art psychotherapy” mode entail more emphasis on understanding clients and their artwork in psychodynamic terms and making interpretations, which may or may not be communicated to clients at different points in therapy. A range of psychodynamic theories may be drawn upon (Wood, 1997).
Wood (1997) offers a possible explanation of why such a split may have been perpetuated in the 1970s and beyond, suggesting that tensions with institutional psychiatry led to art therapists choosing one of two routes: either emphasizing their use of psychodynamic theory to demonstrate clinical credentials, or their art expertise and focus to avoid conflict with psychiatry by offering something different. Yet, there could also be a values-driven subversive feel to this more art-based focus, with the asylum-based art studio as “an asylum within an asylum” (Wood, 1997, p. 172). McNiff (2004) also writes of his work in the 1970s as constituting a life-affirming haven from the institutionalization of a large mental hospital in the United States.
Some art therapists increasingly work within a model of participatory or community arts (e.g., Lu & Yuen, 2012; Margrove, South Essex Service User Research Group [SE-SURG], Heydinrich, & Secker, 2013). This appears to be within the “art as therapy” model where mental distress as such may not be an explicit focus even where people are referred because of it. The goals of the work can encompass developing artistic skills including qualifications (Margrove et al., 2013), and celebrating and communicating about the life and values of an ethnic group or local community (Lu & Yuen, 2012; Moxley, Feen-Calligan, Washington, & Garriott, 2011). The work may also have a flavor of “witnessing” or “telling our story” and of social action (Lu & Yuen, 2012; Shaer et al., 2008; Slayton, 2012). Exhibitions of the artwork are common in this approach.
While it is difficult to locate controlled studies of any kind of art therapy specifically with people with a diagnosis of schizophrenia (Ruddy & Milnes, 2005), one such study was conducted by Richardson, Jones, Evans, Stevens, and Rowe (2007). It was an exploratory randomized controlled trial (RCT) of art therapy based in the community, and reported a significant reduction in “negative symptoms” after only 12 sessions, which these researchers considered a bare minimum necessary for therapeutic impact. The therapy model was group interactive art therapy (Waller, 1993). While this approach was initially modeled on Yalom’s (1975) group interactive psychotherapy, with its focus on “here-and-now” interactions, Waller (1993) and Wood (1997) have pointed out that group interactive art psychotherapy in practice can incorporate a range of psychodynamic theories, with differing degrees of focus on transference and countertransference, the exploration of group members’ past experiences, and making psychoanalytic interpretations of the unconscious meaning of their utterances, artworks, and behavior. To date, it has not been standardized, and nor did the chapter in Waller (1993) referred to by Richardson et al. (2007) make specific reference to adaptations of the approach for severe mental distress or psychotic states.
Taking account of Richardson et al. (2007) and two other trials (Green, 1987; Meng et al., 2005), NICE guidelines for the treatment of schizophrenia (NICE, 2009) recommended that art therapy (as well as other arts therapies) be considered for all people with a diagnosis of schizophrenia, especially in view of the effects on “negative symptoms.” However, in recognition of the complexity of issues facing people given the schizophrenia diagnosis, NICE (2009) recommended a comprehensive package of care rather than any single approach. The range of provisions should include attention to people’s social networks, working in partnership with carers, and with agencies that help with access to employment or education. As a result of reported deficits in service provision by service users and carers as described in the NICE (2009) document, the first main recommendation was to “take time to build supportive empathic relationships as an essential part of care” (NICE, 2009, p. 362). Wood (2013), in documenting the work of art therapists over many years with this client group, has summarized a number of adaptations they have made, which are consistent with the NICE (2009) guidelines relating to arts therapies in that they highlight “expression, communication, social connection and self-awareness, through supportive and interactive experiences, with less emphasis on the use of ‘uncovering’ psychoanalytic approaches” (NICE, 2009, p. 252). This seems more consistent with an “art as therapy” than an “art psychotherapy” approach if the latter is interpreted as “psychoanalytic uncovering.”
In theoretical terms, art as therapy would allow people who have had relatively little opportunity to talk about and try to come to terms with early life trauma and prolonged stresses in their past and present life to begin to express their unarticulated and complicated emotions through the relatively safe medium of art, as has been described vividly by service users with a diagnosis of borderline personality disorder (Morgan, Knight, Bagwash, & Thompson, 2012; Springham, Findlay, Woods, & Harris, 2012). A supportive group experience may enable group members to identify with each other’s visual portrayals of difficult emotions without immediately discussing them in-depth, as well as simply enjoy creativity and the sensory stimulation of the art materials and colors, and through this shared activity gradually build mutual support so that group members can pace their self-exploration and sharing (Wood, 1992, 2013). For people who have been marginalized and excluded from mainstream society, on top of damaging early experiences, this process may take time (Wood, 1992). To the extent that mutual appreciation and enjoyment is expressed about the artistic creations, stigma can begin to be counteracted (Wood, 1997).
The Multicenter evaluation of Art Therapy in Schizophrenia: Systematic Evaluation (MATISSE) Trial
The findings of the MATISSE trial appeared in two publications by Crawford et al. (2012a, 2012b). It was a “pragmatic” RCT (Crawford et al., 2012a, p. 1), designed to evaluate effectiveness (how well a treatment works in routine practice) as opposed to the narrower conditions of an efficacy trial. Two of the trial’s main researchers, Crawford and Patterson (2007, p. 70), had described arts therapies as “complex interventions” and referred to the Medical Research Council’s (MRC; 2000) guidelines for RCTs of such interventions (summarized in Campbell et al., 2000).
MATISSE had three arms (Crawford et al., 2010; Crawford et al., 2012a, 2012b): standard care, standard care plus a choice of non-art-based weekly activities, and standard care plus weekly art therapy in the community lasting from 9 to 12 months. Clients were selected on the basis of a diagnosis of schizophrenia and did not need to have an interest in art therapy. Main outcomes were measured two years from participant entry, and participants were discouraged from engaging in art activities during the second year. In their conclusions, Crawford et al. (2012a) wrote that art therapy “does not lead to improved patient outcomes when offered to most people with [schizophrenia]” (p. 4), and called the NICE (2009) guidelines into question. As appeared to be in keeping with the MRC (2000) guidelines and the updated MRC (2008) guidelines for trials of complex interventions, Crawford et al. (2012a) stated that a “process evaluation” had been conducted to study “the organization and delivery of treatment in the study and the relation between treatment process and treatment outcomes” (p. 2). The present authors therefore sought out the additional papers reporting these process analyses (see below).
Rationale and Aims of the Critical Review
The MATISSE trial conclusion had potentially serious implications for art therapists, services, and people who have received a diagnosis of schizophrenia. The present authors studied the MATISSE reports to determine whether radical changes were needed in art therapy or whether the trial’s conclusion should be challenged. Through examining six key papers relating to the MATISSE trial, the aim was to review the trial and the validity of its conclusions from its reported findings. Three of the papers were those that appeared to be process analyses relating to the trial, and for this reason the section on process analysis is longer than others, although all papers had information relevant to a number of issues presented in the Results section of this paper. All six papers are listed in the Methods section.
Method
Selection of Papers
Six papers were selected because they reported on key aspects of MATISSE: one early paper describing the methodology (Crawford et al., 2010), two reports of the findings (Crawford et al., 2012a, 2012b), and three that were presented as process analyses (Patterson, Borschmann, & Waller, 2013; Patterson, Crawford, Ainsworth, & Waller, 2011; Patterson, Kramo, Soteriou, & Crawford, 2010).
Criteria for Review
The present authors assessed the trial against criteria derived from the updated MRC guidelines for trials of complex interventions (MRC, 2008). Areas of good and poor compliance by MATISSE were noted, with the evidence for conclusions about these, and the implications for future research and practice.
Procedure
The six papers were trawled for material relating to each MRC criterion (Table 1). All information relevant to a criterion was studied carefully to reach a qualitative judgment with justification from the papers.
Elements of MATISSE Trial Assessed Against Criteria for Complex Interventions.
Results
Trial Development and Preparation
Use of recognized development/testing framework
There was no mention in the papers of use of a recommended framework for developing and testing complex interventions. This was despite the recognition by NICE (2009) and by two key MATISSE researchers that arts therapies are complex interventions (Crawford & Patterson, 2007). This framework had been presented in the earlier MRC guidelines (Campbell et al., 2000) as well as the updated version (MRC, 2008).
Theory of change
Reiterating their earlier recommendations, the updated MRC guidance (2008) specifies that “a good theoretical understanding is needed of how the intervention causes change, so that weak links in the causal chain can be identified and strengthened” (p. 7). Crawford et al. (2010) and Crawford et al. (2012b) referred to enabling self-expression while containing powerful emotions (Killick, 1997), with the key ingredients being art activity and therapeutic relationship (Edwards, 2004), and in group art therapy the relationships with group members, citing “Waller & Gilroy, 2000” ([
None of the papers attempted any theoretical explication of the psychosocial difficulties of people with a diagnosis of schizophrenia who had received standard mental health services for several years, as had MATISSE participants (Crawford et al., 2012b). Nor was there a rationale for the proposed key ingredients of art therapy for this client group. Regarding the conceptualization of schizophrenia itself in Crawford et al. (2010) and Crawford et al. (2012a, 2012b), it appears to be little more than a listing of main “symptoms.” There is no unpacking of the factors that cause or maintain disability, many of which, including stigma and discrimination, are recognized in the NICE (2009) guidelines and have been known about for some years (Wright et al., 2000). The disabling effects of long-term antipsychotic medication (Moncrieff, 2013; Moncrieff & Leo, 2010) are not mentioned at all.
Review of evidence
All the papers cited a lack of well-conducted RCTs and flaws in previous trials (Ruddy & Milnes, 2005). Patterson et al. (2013) critiqued the MATISSE trial itself, as possibly not employing the design or measures most likely to highlight positive effects of art therapy, and their own study suggested that such effects were demonstrable in people who engaged with it. Despite the MRC (2000, 2008) guidelines recommending considering a range of evidence including qualitative and case studies, only randomized trials were reviewed by the MATISSE trial authors, thus possibly missing information that might have illuminated processes and mechanisms.
Piloting
There was no mention of piloting in the papers, although Crawford et al. (2012a) referred to a previous study by Richardson et al. (2007) as a “pilot trial” (p. 2). However, there is no evidence of interchange between the two teams except after the start of MATISSE when its team wished to compare participants’ attendance rates with those in the previous trial. Crawford et al. (2012b) reported that after the trial started, recruitment was increased from 9 to 20 months due to slow uptake, and the target number of participants was increased by 10% due to low attendance. It seems possible that piloting might have alleviated these problems.
Conduct of the Trial
Stakeholder involvement
Service user involvement was not mentioned in Crawford et al. (2010) or Crawford et al. (2012a), but Crawford et al. (2012b) included the original protocol for the study, which named two service users who would assist in producing the final report and one for a “service user journal” (p. 67). However, their names are not on the study reports (Crawford et al., 2012a, 2012b) and searches failed to locate any publication by them. This raises the question of how real service user participation was in the trial. There were 15 academic and clinical authors, and Rose (2009) has highlighted how even as an experienced service user consultant it is possible to be positioned as having little legitimacy to voice one’s opinions.
Adaptation to settings
Complex interventions can appropriately vary by locality, and therefore it is important to specify the variation permitted (MRC, 2008). Crawford et al. (2010) and Crawford et al. (2012a, 2012b) mentioned therapists tailoring interventions according to contingencies such as client lateness and group facilitators’ leave, and in handling psychological material. There were specified procedures on these for activity groups, but in art therapy it was left to clinical judgment. However, despite the well-known difference in emphasis among practitioners on “art as therapy” versus “art psychotherapy” (Waller, 1993), there was no recording of individual therapists’ use of psychodynamic, integrative, or other therapy model or how much emphasis they placed on creativity, learning artistic skills, or social action. Nor was degree of integration of art therapy with local mental health or other relevant services addressed. Studies by Patterson et al. (2010), Patterson, Crawford et al., (2011) and Patterson, Debate, Anju, Waller, and Crawford (2011) indicate that these things can vary.
Outcome measures and their timing
In Crawford et al. (2010) and Crawford et al. (2012a, 2012b), the hypotheses about what would change (and thus what was measured) did not lead from the brief mention of art therapy’s supposed active ingredients: There were no measures of self-understanding, emotional expression, or integration of thoughts and feelings. Nothing in the brief coverage of art therapy mechanisms suggested that “symptoms” or medication use should change, which were measured, and no account was taken of the possibility that people might in any case be either strongly discouraged or actively prevented by mainstream mental health services, from which participants continued to receive services, from reducing or discontinuing medication (Moncrieff, 2013).
Subgroup analyses
The MRC (2008) guidelines suggest that where one would expect local variation, it is worth carrying out subgroup analyses. However, it appears that there was no plan for this (Crawford et al., 2010) and except for qualitative examination of recruitment issues (Patterson et al., 2010) none were reported (Crawford et al., 2012a, 2012b). Patterson et al. (2010) reported variations between sites and over time in willingness of clinicians and/or their managers to refer clients to the trial, but did not investigate any possible links with therapy implementation and outcome variations. Geographical location was included in the statistical analyses but not therapy variations.
Process evaluation
The MRC (2008) guidelines suggest that process evaluation may explain “discrepancies between expected and observed outcomes” (p. 4). Initially, it appeared that there had been full process analyses, as mentioned earlier (Crawford et al., 2012a). Three papers appeared to report these analyses: One examined the referral process (Patterson et al., 2010). A second (Patterson, Crawford, et al., 2011) involved interviews with MATISSE therapists and other art therapists, but was not a process analysis because it was not concerned with how therapists actually worked in MATISSE.
The third (Patterson et al., 2013) was a process analysis of engagement and treatment in the trial, reporting on interviews with MATISSE service user participants, with triangulation data from therapists. The authors built a grounded theory of clients’ engagement and therapy experiences, linking these with the outcomes reported by those 23 participants. Six participants engaged fully with art therapy, two despite significant travel difficulties, and it seems to have related to the art therapist fostering a sense of joy and achievement in creating images, as well as positive interactions between participants. There were reports of “feeling accepted and comprehended in a way that was exceptional in participants’ experience” (p. 6). This seems entirely consistent with the theoretical understanding presented in the NICE (2009) guidelines of the mechanisms of appropriate arts therapies for people with the schizophrenia diagnosis, and also with Wood’s (1992, 2013) suggestions.
These experiences seem at odds with the trial as a whole, however, in which less than one third of participants of either activity control or art therapy groups attended more than nine sessions out of weekly yearlong treatment offered. This low attendance, as well as the poor overall outcome, might have prompted a search for explanations in the process analyses: “Lack of impact may reflect implementation failure (or teething problems) rather than genuine ineffectiveness” (MRC, 2008, p. 7). Yet, these analyses were reported as separate papers and not drawn upon in the main outcome reports (Crawford et al., 2012a, 2012b).
Crawford et al. (2012b) stated that they “deliberately set out to recruit a representative sample of those with schizophrenia rather than a self-selected subgroup of those who wanted art therapy” (p. 37). The authors concede that clients may be more motivated when entering art therapy in routine clinical practice as opposed to a randomized trial. This does seem likely as Patterson et al. (2013) found that service users had poor understanding of the trial, some seeing it as “an opportunity to access
Ulmer, Robinaugh, Friedberg, Lipsitz, and Natarajan (2008) have suggested that a run-in period is advisable with behavioral intervention trials requiring high commitment, as perhaps randomization for a whole year followed by no art therapy for a year would do, even for people without the myriad disadvantages of those diagnosed with schizophrenia. After initial screening for suitability, potential participants spend a few weeks carrying out initial tasks or assessments before randomization, so that only clearly committed participants are randomized. Those who do not commit can be compared with those who do to check for threats to generalizability of subsequent findings. Perhaps rather than weeding out the uncommitted, however, such a process might alternatively serve to engage more potential participants and, in involving preliminary meetings, assessment, and information-giving, would be in keeping with usual art therapy practice (Wood, 2013).
Trial Reporting
Reporting framework
Crawford et al. (2012a, 2012b) used elements of the Consolidated Standards of Reporting Trials (CONSORT; Zwarenstein et al., 2008), for example in presenting the recruitment pathways diagrammatically. However, Patterson et al. (2010) propose that reporting should encompass earlier stages because of the problem of staff pre-selecting clients to refer to trials, with idiosyncratic and undocumented decision-making. The number of people excluded at assessment for the 2-yearlong MATISSE trial seemed anomalous at 36% (Crawford et al., 2012b). Richardson et al. (2007) had to exclude 80% of those referred to their much shorter randomized trial of art therapy for people with a diagnosis of schizophrenia in a community setting. Crawford et al. (2012b) reported that staff members were not referring at the expected rates and they had to add new sites, raising questions about the accuracy of data about excluded clients. Patterson et al. (2010) suggested a refinement to the CONSORT diagram for future studies, such that it included stages earlier than currently accounted for.
Description of the intervention
Although there was a description of both the activity control and art therapy intervention, these did not take into account possible variations in practice such as the type of psychodynamic or other models drawn upon, the degree to which practice was seen as “art psychotherapy” or “art as therapy” (Waller, 1993, p. 8), or the level of integration with local mental health services, and art therapy facilities available (Patterson, Debate, Anju, Waller, & Crawford, 2011). There was allowance for therapists to adapt to client needs, but no documentation of differences in interaction style. Weekly monitoring was only for “treatment fidelity” (Crawford et al., 2012b, p. 7), defined as checking that activity groups did not involve discussion of clients’ difficulties, and that lateness and therapist leave were handled idiosyncratically by art therapists and according to prescribed practice by activity group facilitators. These checks could not have established whether specific therapeutic ingredients were present in the art therapy groups, which is consistent with the lack of a clear model of how art therapy should work for the client group. Furthermore, none of the trial reports indicated a minimum length of experience therapists had of working with the client group or number of clients they had seen, raising the possibility that some therapists may have been inexperienced and relatively unprepared for the difficulties of the clients who would be referred to them, while others may have been well used to adapting and containing clients’ difficulties.
Description of contexts and changes
The study by Patterson et al. (2010) comes closest to providing insights into variations between local contexts and the effect of policy changes during the trial. There was reference to some trusts having to make “efficiency savings” (p. 538) and “pressure to achieve service targets” (p. 539). Referring people to the trial would mean more “filling out forms” (p. 538) and clinicians felt too busy to do this. The main concern of the paper, however, seemed to be the difficulty of recruiting participants to large trials rather than how variations by site might have affected therapy implementation and its outcome for the MATISSE trial itself. Thus, another potentially illuminating source of insight into variation between sites and subsequent outcomes appears to have been lost. In light of the NICE (2009) guidelines’ and Wood’s (2013) recognition of the need for integrated services, based on the recognized complexity of difficulties experienced by those with a schizophrenia diagnosis, the fact that integration varied between settings must have been a factor in successful referral, implementation, and subsequent outcome.
Statistical Analysis
Although the main statistical and economic analyses appear to comply well with the MRC criteria, other issues need to be discussed. As MATISSE was a “pragmatic” trial, the authors carried out an “intention to treat” (ITT) analysis (Hollis & Campbell, 1999). Irrespective of actual attendance at treatment, follow-up data are gathered on as many participants as possible, with the assumption that their level of attendance reflects routine practice. Thus, more than 80% of participants were assessed at follow-up, and the main reason for lack of reported effectiveness appears to have been low attendance. Crawford et al. (2012b) stated that “the proportion of participants who dropped out was less than predicted” (p. 17), referring to the high level of those providing follow-up data. However, one could suggest that dropout was in fact more like two thirds in both intervention arms, as researchers pursued participants for follow-up assessments with the offer of £15 for their time and “repeated offers to be assessed” (Crawford et al., 2012b, p. 17). Participants would have attended no sessions within the previous 12 months because the outcomes concerned were assessed at 2-year follow-up. As Patterson et al. (2013) indicated, some “participants,” even at 12 months, had forgotten they were in the trial.
It is entirely possible that art therapy was effective for the small proportion of clients who attended a substantial number of sessions (“efficacy”). Dunn et al. (2003) have suggested additional analyses to ITT to evaluate efficacy, and Hernan and Hernandez-Diaz (2012) emphasize that treatment non-compliance biases the result of ITT analysis against detecting effects of treatment. They recommend that where there is “substantial lack of adherence or loss to follow-up,” additional analyses are needed, including “appropriately adjusted ‘per protocol’ and ‘as treated’ analyses” (p. 54).
Although Crawford et al. (2012a, 2012b) reported an additional instrumental variable analysis, Hernan and Hernandez-Diaz (2012) state that this requires a model of the dose–response relationship. Crawford et al. (2012b) admitted that their assumption of a linear relationship between number of sessions and response was “naïve” (p. 37). Dunn suggests that the dose–response modeling (such as it was) could never have provided a point estimate for efficacy, and that “the data do not rule out ‘efficacy’ for those who attended a significant number of art therapy sessions” (personal communication, July 2013).
Finally, in their secondary analysis of primary outcomes at 24 months Crawford et al. (2012a) stated that there were no data on the minimum number of art therapy sessions needed to derive benefit, ignoring Richardson et al. (2007) who stated that “12 sessions was regarded as an absolute minimum clinical requirement, though clearly this would be regarded as sub optimal by many art therapists” (p. 484). In the absence of hard data, most researchers would listen to practitioners with relevant experience as a minimum in the hierarchy of evidence (Castillo, Scharfstein, & MacKenzie, 2012).
Discussion
Rigorous review of the MATISSE trial has suggested that widely recognized MRC recommendations (2000, 2008) for trials of complex interventions were not followed, despite having been acknowledged earlier by key members of the trial’s team (Crawford & Patterson, 2007). The lack of piloting, clear theory of mechanisms and clear description of practice and its variations arguably made it likely from the start that the trial would yield weak or unexplained outcomes. Compounding this was the lack of integrated process analyses. Notes by trial therapists of their weekly practice were neither intended nor used for this. Nor were subgroup analyses conducted according to practice variations or local trust context, even though cultures varied between Trusts (Patterson et al., 2010), and there are known differences of practice within art therapy, at minimum relating to more emphasis on “art as therapy” versus “art psychotherapy” (Waller, 1993; Wood, 1997).
Notwithstanding the difficulty of mounting this kind of trial, the MRC (2008) guidelines remind us that process evaluations can help explain “why an intervention fails unexpectedly” (p. 7). If MATISSE had shown negative results despite good client attendance, its authors might have a case for challenging the existing NICE (2009) guidelines on arts therapies. An alternative reading of their findings is that trial implementation failed despite the researchers’ best efforts.
There was no attempt to measure the kinds of things that, despite the lack of a well-elucidated model, art therapists have consistently suggested should change (Patterson, Crawford, et al., 2011; Wood, 1997), such as integration of thoughts and feelings, enhanced sense of self, and emotional expression. Nor was attention paid to personal recovery (Leamy, Bird, Le Boutillier, & Slade, 2011; Slade, 2009). The low attendance at therapy sessions meant that ITT analysis greatly underestimated any effects of the interventions (Hernan & Hernandez-Diaz, 2012). Patterson et al. (2013) found that service users who engaged with art therapy appeared to benefit, and these findings appear to be in keeping with the NICE (2009) guidelines for arts therapies, such that greater emphasis be placed upon emotional expression and connection in a supportive atmosphere rather than psychoanalytic “uncovering.”
Limitations of the Review
This review is limited by incomplete access to the details of what happened during MATISSE as reported in the six papers. However, there was enough information to assess with a fair degree of detail the issues in the MRC (2000, 2008) guidelines. These guidelines appear to be widely recognized as a framework for the development and conduct of trials of complex interventions, as attested to by the number of papers that now draw upon them in developing and testing a variety of health interventions. This framework is also one that key MATISSE researchers have themselves recognized (Crawford & Patterson, 2007). Other literature has been drawn upon where appropriate.
Implications for Art Therapy and Other Large Trials
Because of the issues highlighted here, the conclusion of the MATISSE authors that art therapy has no value for most people with a diagnosis of schizophrenia is unwarranted. All that can be concluded is that for large “pragmatic” trials of complex interventions, scarce resources should only be allocated if they have well-founded theoretical formulations, have been piloted, recognize significant variations in practice and by locality, plan full process analyses and are prepared to examine these in detail and learn from them if outcomes are not as might be expected, and include supplementary “as treated” analyses. We would urge all who are considering mounting large trials of complex interventions, and those reviewing them, to consider the MRC (2008) guidelines when planning or assessing the potential of such trials. Rather than needing to disseminate lessons for art therapy practice, it appears that we are no further forward as a result of MATISSE. A run-in period as recommended by Ulmer et al. (2008) might help ensure that practice really does resemble routine art therapy, where potential participants are assessed and have the opportunity to meet the therapist and try it out first. If a control condition is used, then potential participants also experience the control condition. However, it seems arguable that treating art therapy or any other additional intervention, as something that is simply added on, overlooks the true nature of complex interventions, which tend not to be simply an additive process but to involve multidirectional and socially and individually motivated human interactions (Marchal et al., 2013), both within them and between them and their context. Therefore, context must be engaged with from an early stage rather than viewed as just a nuisance variable.
Areas for Future Research
Given that most of the limited theoretical basis for art therapy cited by Crawford et al. (2010) and Crawford et al. (2012a, 2012b) was from books published originally in 1992-1993 and reprinted several times (Waller, 1993; Waller & Gilroy, 1992), one has to ask what new theoretical developments have been made in art therapy since 1993. One other text is mentioned by Crawford et al. (2012b), that of Edwards (2004), but like the previous ones it was not geared specifically to psychosis. A volume edited by Killick and Schaverien (1997) is directly aimed at work with psychosis, and a chapter by Killick (1997) is cited by Crawford et al. (2012a, 2012b).
However, it seems necessary not only to have a clear understanding of art therapy’s mechanisms but also of the mechanisms occurring for people given a diagnosis of schizophrenia: the mechanisms maintaining their social exclusion and disability, such as discrimination (NICE, 2009) and medication (Moncrieff, 2013). There is a need for studies that build on that of Patterson, Crawford, et al. (2011) in ascertaining therapists’ understanding of their practice, on Patterson et al. (2013) about service users’ experiences, and that involve collaborative approaches between staff and service users such as that of Springham, Findlay et al. (2012). This could build on the currently somewhat thin theoretical framework outlined in the introduction to the present paper. The distinction between “art as therapy” and “art psychotherapy” also needs to be fully acknowledged, as this is not only a long-established distinction in art therapy literature (Waller, 1993), but has also been alluded to by service users as an important one (Morgan et al., 2012). In relation to guidelines by art therapists and for art therapists, Springham, Dunne, Noyse, and Swearingen (2012) have begun developing guidelines for art therapy with specific diagnoses, the first being borderline personality disorder.
Conclusion
As a result of the present review, the MATISSE trial authors’ conclusion that art therapy is of no value for most people given the diagnosis of schizophrenia is incorrect. No doubt, the MATISSE team intended their research to be helpful to service users and services. The trial conclusions, published in a relatively prestigious journal, may have a damaging influence, such that vulnerable people who rarely receive much more than maintenance on medication will be deprived of a therapy that could change the lives of at least some, if not many. Yet it should not. The NICE guidelines (NICE, 2009), that arts therapies should be offered, still stand. Future trial planners, reviewers, and potential funders should pay more attention to whether there has been and is real service user involvement, clear theoretical underpinnings guiding the choice and timing of measures, pilot work by the team planning the trial and evidence of learning from it, real consideration given to whether expecting service users to accept randomization for 2 years can reflect usual care, and good plans for process and subgroup analyses where practice or its context varies.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Both authors work for the British Association of Art Therapists (Dr Sue Holttum as part-time research officer and Val Huet as Chief Executive Officer).
Funding
The author(s) received no financial support for the research and/or authorship of this article.