Abstract
In recent years, there has been a significant increase in the use of overviews, making it necessary to establish methodological guidelines for their planning and elaboration. Most existing guides are not always suited to qualitative overviews or present specific examples that aid understanding of what this research method requires. This study aims to offer methodological guidelines for the design, elaboration and reporting of qualitative overviews following an example of art therapy interventions involving children with trauma. The different phases of the overview are developed on the basis of the research question that served as an example: What are the main findings of systematic reviews investigating the effects of art therapy on children exposed to traumatic situations? Each phase is accompanied by a brief definition intended to provide guidance on what is expected in that phase; frequently asked questions that might arise during the process; the procedure that has been followed for the design, development and analysis of the overview itself; recommendations for future reviews; difficulties, possible threats and common errors; and how to conduct a qualitative overview with an example of art therapy. The methodological challenges of planning and conducting a qualitative overview are presented in terms of the difficulty of framing the research question, the overlap of primary studies and data, the reliability of the findings obtained, and the assessment of the risk of bias. This study intends to provide the necessary support to doctoral students, academics and practitioners who wish to work with this research method.
Introduction
The growth of systematic reviews (hereafter SRs), together with the increasing demand from policymakers for rapid reviews of research, has prompted a recent new form of synthesis: the overview of reviews or overview (Gates et al., 2020; Liu et al., 2020; López-López et al., 2022; Pollock et al., 2022).
An overview is a research method that involves identifying, retrieving and summarizing SRs on a topic. It often addresses broader questions than those raised in the included SRs and requires assessment of the bias and the quality of the report, and integration of findings (Pollock et al., 2022). Its purposes include mapping the available evidence, summarizing the effects of an intervention for different conditions or populations, assessing the effects of different interventions with similar conditions (Pollock et al., 2022), examining reasons for discrepancy between findings, and assessing the adverse effects of multiple interventions under a specific condition (Papageorgiou & Biondi-Zoccai, 2016), as well as drawing conclusions from the SRs involved (Lunny et al., 2018). A potential benefit is that overviews can leverage the efforts of previous SRs in a shorter time frame (Ballard & Montgomery, 2017).
Overviews aim to synthesize evidence derived from existing systematic reviews (SRs) and their underlying primary studies. These reviews often address broader or more integrative research questions than those examined in individual SRs (Lopez-Lopez et al., 2022). By consolidating findings across multiple systematic reviews, overviews offer a comprehensive and methodologically robust perspective on a given topic, thereby supporting evidence-informed policy and practice. Furthermore, they enable researchers to identify overarching patterns, inconsistencies, or gaps within the literature (Bonczar et al., 2023). Because overviews encompass systematic reviews conducted across diverse contexts and populations, their conclusions tend to be more generalizable than those of any single study or review.
It is therefore not uncommon to assume that an overview is considered as a direct extension of its precursor, the SR of primary studies. Many of the steps and methods used in the implementation of SRs are directly transferable to overviews: (a) definition of the research question; (b) inclusion and exclusion criteria; (c) literature search; (d) data extraction; (e) assessment of the risk of bias and the quality of the report; (f) presentation of results; and (g) meta-review reporting (López-López et al., 2022), although unique methodological issues can be identified (Gates et al., 2020). This presents well-known challenges for those experienced in SRs of the literature, but also involves problems specific to overviews that require methodological solutions for which there is no parallel with primary document reviews.
Overviews involve methodological challenges of their own that mainly arise from the overlap of the same primary studies included among the selected SRs, which contributes to data duplication (Aromataris et al., 2015; McKenzie & Brennan, 2017). Another challenge is the lack of alignment between the overall question guiding the overview and the SR questions. Rarely are SR questions identical, so choosing one review among several may result in the loss of important data or complete studies. Likewise, efforts may be hampered if data are missing from primary studies or information is not consistently provided in the included SRs (Bonczar et al., 2023; McKenzie & Brennan, 2017). A similar situation occurs when assessing the quality and risk of bias of SRs and primary studies, which is one of the biggest methodological challenges in conducting an overview of SRs (Gates et al., 2020). Therefore, this and other scenarios require the application of different or additional strategies and methods to those used in SRs (Lunny et al., 2018; McKenzie & Brennan, 2017).
On the other hand, the development of methodological guides for students, practitioners and researchers for the planning and execution of overviews remains another challenge. Due to their long tradition in the scientific literature, most of the available manuals for overviews have been created in and for health sciences, while this has not been the case in other disciplines (McKenzie & Brennan, 2017). For example, they concentrate on the effectiveness of clinical interventions; provide a theoretical explanation of the phases (Liu et al., 2020); focus on a particular phase of the overview (Pollock et al., 2019); compare exemplary overviews to assess the impact of certain methodological decisions (Pollock et al., 2017); or include experience in terms of lessons learned (Pantoja et al., 2015).
However, these guidelines are not always directly applicable to qualitative overviews (Gates et al., 2020); none of them provides a summary of the process involved in their elaboration, describing the phases, possible errors, as well as recommendations based on a specific example, or they make materials available that are necessary to prepare overviews such as the coding manual for the SRs and the primary studies, or the list of the primary studies.
Therefore, once the gap in the literature has been identified and due to the researchers’ involvement in a national project funded by the Ministry of Science and Innovation (Spain) on the impact of art therapy on children with trauma, it was decided to use this topic as an example of the guide addressed in this paper. Those who train to be art therapists come from different disciplines, such as art, psychology and education, where the route to and training in the overview is different, thereby providing an opportunity for learning and scientific progress.
To date, the limited number of overviews found in this context examine art therapy for people with learning disabilities (Dee, 2012) or with cancer (Geue et al., 2010), or focus on evaluation tools (Betts, 2015). However, only two studies address the issue: van Westrhenen (2014) who analyzes creative art therapies used in children with trauma up to the age of 18 and Diseth and Christie (2005) who focus on the available tools for evaluating therapeutic interventions used for children and adolescents in the Nordic countries. The first study includes adolescents and the second analyzes other therapies apart from art therapy. In any case, both have the term ‘overview’ in their titles, but neither of them is a review of SRs.
Consequently, the aim of this study is to provide methodological guidance for the design, development and reporting of qualitative overviews following an example of art therapy interventions in children with trauma.
This paper aims to contribute, on the one hand, by presenting general guidelines for the design and development of qualitative overviews, but also methodological guidelines that lead to a better understanding of this method in disciplines related to art and therapy; and, on the other hand, by presenting the specific characteristics of a process carried out in art therapy interventions with children exposed to traumatic situations. The methodological guide is expected to further strengthen evidence-based research and to provide the necessary support to doctoral students, academics and practitioners who wish to work in this field.
The article presents a structured methodological framework for the phases of the overview accompanied by a brief definition of what is expected in each phase; frequently asked questions that might arise during the process; the procedure that has been followed for the design, elaboration and analysis of the UR itself; recommendations for future reviews; difficulties, possible threats and common errors; and how to conduct a qualitative overview with an example of art therapy. The materials used in the overview are shared as part of the teaching-learning process and with the aim of providing information for future overviews.
Phases of the Overview Process Through an Illustrative Example
Phase 1. Research Question
Definition
The research question is a question that guides and drives the entire process of searching, selecting, analyzing and reporting results (Mohamed Shaffril et al., 2021). It has the advantage of presenting the research problem in a delimited and straightforward manner, minimizing distortion.
Frequently Asked Questions
How does the research question arise? How does the research question differ from the objective of the study? How is it formulated? How broad or specific should it be? How do we know if it can be answered?
Procedure
A first search of overviews on art therapy was conducted but none were found that focused on children with trauma, a finding that laid the foundations for this overview. Following this, we conducted a second search of SRs related to the topic of interest in order to obtain an initial evaluation of the literature available in this field (Annous et al., 2022; Morison et al., 2022; van Westrhenen, 2014). Using this primary investigation, we formulated the research question that will guide the overview. It was decided that the question should be broad because research on art therapy and trauma has a short history in the scientific literature (Annous et al., 2022; Damianakis, 2007; Register & Hilliard, 2008; Schouten et al., 2015).
Recommendations
Conduct an overview of previous SRs and, if one is found related to the topic of interest, it could be decided to continue with it or to pose a new question that responds to the gap found in the literature. For the formulation of the research question that will guide the overview, it is advisable to follow the PICO methodology, although other formats (e.g. CEEC, SPIDER, FINER) could also be used. For example, when formulating the research question, the Cochrane Handbook by Pollock et al. (2022) -although it is not the only source-can be consulted, as it describes the use of PICO as a central framework for formulating clear and answerable research questions; defining inclusion and exclusion criteria in systematic reviews and overviews; guiding the literature search and study selection; and increasing the transparency, coherence, and replicability of the research process. The beginning of an overview is uncertain because until progress is made in the search for documents, there is no guarantee that it can be conducted.
Difficulties, Threats and Possible Errors
Posing a research question without having identified gaps in the scientific literature or without having verified the existence of a sufficient body of primary research. Starting the overview without doing a previous review of overviews or SRs, which will require readjusting or posing a new question.
Illustrative Example
Drafting of the Research Question According to the PICO Methodology
Phase 2. Inclusion and Exclusion Criteria
Definition
Inclusion and exclusion criteria are rules that indicate which studies can be selected for inclusion in the review. These selection criteria, called inclusion and exclusion criteria, impose restrictions on the overview and are determined by the research question and the conceptual framework (Patino & Ferreira, 2018).
Frequently Asked Questions
What are the principles that guide the inclusion and exclusion criteria? How are they decided? Can documents from conference proceedings, reports, doctoral theses, secondary sources or books be included? Is limiting the temporal search for documents always an inclusion criterion?
Procedure
The inclusion and exclusion criteria were established on the basis of the research question and the SRs found in the previous phase. Criteria were organized according to contextual aspects referring to publication information (e.g. year of publication, full text, language); methodological aspects (e.g. age of children, SRs as a research method) and substantive aspects related to the object of the study (e.g. art therapy as a method of intervention for children exposed to traumatic situations). To define the exclusion criteria, we took some of the inclusion criteria and looked for their opposite (e.g. adolescents, young people and adults versus children) or SRs that considered art therapy to be an artistic activity or undifferentiated results of reviews involving art therapy interventions with children and adolescents.
Recommendations
Review SRs related to the topic to identify inclusion and exclusion criteria. Adjust inclusion and exclusion criteria according to the research question of the overview. If necessary, once the document search has commenced, the inclusion and exclusion criteria could be readjusted, requiring a re-examination and re-registration of the process from the beginning. This flexibility should not entail “adapting” the criteria to what is being found. In fact, it is common practice to publish the review protocol before conducting the search.
Difficulties, Threats and Frequent Errors
Lack of theoretical justification to support the inclusion criteria. Use as exclusion criteria only issues related to the information in the publications (e.g. type of document sources, language, etc.), without considering methodological and substantive aspects. Significantly changing the inclusion and exclusion criteria once the search has started. Establishing conflicting or non-exclusive inclusion and exclusion criteria.
Illustrative Example
Inclusion criteria were SRs, published in scientific journals up to 2023, written in English or Spanish, and in full text, aimed at evaluating art therapy interventions in children with trauma up to the age of 12 years. SRs were also included if at least one of their primary studies met this objective. In the SRs, art therapy is either the only intervention method used or it can be combined with other methods.
Hence, art therapy is understood as a form of psychotherapy that uses visual and haptic media as a means of self-expression and communication. The aim is to help people of all ages and abilities, and at all stages of life, to discover an escape from often complex and confusing feelings, and to encourage self-awareness and growth (British Association of Art Therapists, 2023). We also assumed the definition of childhood trauma proposed by Terr (1991), which includes three broad categories as a result of a single event, multiple events or cross-trauma that can occur when a child faces ongoing stress after a single traumatic event. SRs were included that evaluated or did not evaluate the methodological quality of their primary studies, and also primary studies with greater or lesser methodological quality.
The inclusion criteria were literature reviews and narrative reviews. Additionally, we excluded SRs in which none of the primary studies explicitly mentioned the terms “trauma” and “art therapy”, or that involved working with adolescents, youth and adults. If results from children and adolescents were presented in the same study, they should be separated by developmental stages (sub-samples); otherwise, the study was to be rejected. Reviews in which artistic activities were conducted without therapy and those that did not present results obtained from the analysis of the selected primary studies were also excluded.
Phase 3. Literature Review: Document Search
Definition
This is the process that describes how the documents to be included in the overview have been identified. In order to guarantee the possible replicability of the study, it must be carried out with maximum transparency, indicating the selected databases, descriptors and Boolean connectors used.
Frequently Asked Questions
Which databases should be consulted? Is there a minimum or maximum number of databases? Which descriptors should be chosen and using which criteria? While reviewing the literature, how are the documents organized? Once the literature has been reviewed, how can subjectivity during the process be minimized? How should information on papers that have or have not met the inclusion criteria be presented? How many final documents need to be found?
Procedure
The databases to be consulted were decided on the basis of previous SRs related to the subject. The most widely used in this field are SCOPUS, WOS, PubMed and PsycInfo. Journals specializing in art therapy were also consulted. As for the descriptors, a list of terms and their synonyms was drawn up. To this end, research articles on the subject were reviewed to identify key words and the UNESCO Thesaurus was consulted, as well as others specializing in Health Sciences (Descriptors in Health Sciences: DeCS), Education (ERIC Institute of Education Science) and Psychology (APA Thesaurus, ISOC Psychology Thesaurus).
The search was conducted through the library services at the university. As a test, a first informal search was conducted across the databases to assess the performance of the descriptors and begin the literature review in a systematic way. The database information was downloaded in Excel format (e.g. name of authors, year of publication, title of article, name of journal, type of source and abstract). Then we conducted a search of specialized journals. The Excel files that came from the databases were combined in a single Excel document to which the information from the journals was added and at the same time new columns were included for rapid discarding of documents: whether the full text version was available or not, and language.
Next, a further reading of the papers required more columns to be added to the Excel sheet, but this time with much more detailed information on the primary studies present in the SRs: age of the sample, whether the children had been exposed to traumatic situations and whether they used art therapy as an intervention method. Finally, two new columns were added: final decision (yes or no) and reasons for the decision (brief explanation of why it was decided to include the papers or not).
This process was iterative and cyclical: enumerate the document, fill in the Excel sheet, re-read the summary or the whole text, adjust the Excel sheet, if necessary, go back to the document to confirm and check the information, and begin with a new document. SRs that met the inclusion criteria were downloaded and organized into folders according to the databases consulted. Each SR was assigned an identification number. This number was also recorded on the Excel sheet to keep track of the documents.
Once the document screening was completed, a flow chart was drawn up specifying the number of initial and final documents, as well as the reasons why some SRs were discarded. The researchers involved in the study participated in the screening of the documents.
Recommendations
It is useful to become familiar with databases and their specificities. There is no minimum or maximum number of descriptors, databases or final documents. Use bibliographic management software to manage bibliographic lists (e.g. Endnote). Record all documents generated by the databases. Prepare the Excel file with criteria that facilitate decision-making. In some cases, just reading the abstract is enough to discard the study, while in other cases a full reading of the text is necessary. All decisions should be recorded because the review is systematic. The rationale behind the exclusion of a study should not be too general (e.g. not related to the topic). Screening should preferably be done by several researchers for inter-observer evaluation.
Difficulties, Threats and Frequent Errors
Lack of knowledge of how databases work. Incorrect use of Boolean operators. Selection of very broad descriptors or descriptors that do not relate to the research question. Initiating the selection and exclusion of documents without applying a systematic approach. Assuming that some studies are recorded and others are not. Lack of clear criteria for discarding documents. Not making efforts to find the full text version of selected documents (e.g. search on Google, use of interlibrary loan or even write to the author). Not doing an interobserver evaluation.
Illustrative Example
The databases consulted were SCOPUS, WOS, MedLine and PsycInfo. Three specialized art therapy journals were also consulted: the International Journal of Art Therapy, The Arts in Psychotherapy and Art Therapy: Journal of the American Art Therapy Association. The descriptors used were “art therapy” AND (“systematic review”, “meta-analysis”, “review”, OR “literature”), AND “trauma” AND “children” in the title and/or summary of the document, depending on the specific characteristics of each journal and database.
A total of 36 papers were found in databases and specialized journals, but only 12 SRs met the inclusion criteria (see the references of these 12 documents in supplemental material 1). Figure 1 shows the review process and its phases according to the PRISMA criteria (Page et al., 2021). It was decided to include the overviews by Diseth and Christie (2005) and van Westrhenen (2014) because although their titles include the word overview, they are SRs. The paper search process involved all the researchers in the study in order to reduce subjectivity and ensure the quality of the selected studies. The databases and reviews to be analyzed were distributed among them. Once the information had been entered into Excel, the authors expressed their doubts and disagreements until a consensus was reached (interobserver evaluation). Flow diagram
Phase 4. Coding of Studies
Definition
Refers to the careful and detailed process of extracting relevant information from the selected papers. What is considered relevant depends primarily on the research question and the previously described inclusion criteria (Siddaway et al., 2019).
Frequently Asked Questions
What information is relevant? How should information from studies that meet the inclusion criteria be organized?
Procedure
To expand the information in the selected SRs, new columns were added to the Excel file based on extrinsic, methodological and substantive aspects or variables. Another Excel file was also created for the coding of the primary studies present in these reviews. Some variables were coded numerically (e.g. 1 = case studies, 2 = quasi-experimental designs), while others were coded qualitatively (e.g. study objectives).
Recommendations
The information presented in the Excel should be summarized as concisely as possible, and the detailed information for each document should be saved in a Word document (or similar) to be used in the analysis of the results. Visualize which contextual, methodological and/or substantive variables help to better understand the topic of study in order to answer the research question.
Difficulties, Threats and Potential Pitfalls
Include characteristics of the studies that do not provide information related to the research question. While rare, it may happen that during this phase of the process some documents are identified as included by mistake, which implies reviewing the previous phases and making the necessary adjustments.
Illustrative Example
Two coding manuals were developed. The first manual (supplemental material 2) summarizes the main characteristics of the 12 SRs that met the inclusion criteria according to extrinsic variables (authors, year of publication, institutions, countries, journals), methodological variables (descriptors, databases, inclusion-exclusion criteria, initial and final documents), and substantive variables (research objectives and results). The second manual (supplemental material 3) presents information about the primary studies). Of the 23 primary studies identified in the 12 SRs included, we only found 17 full-text primary studies (see the references of these 17 documents in supplemental material 4).
Phase 5. Assessment of the Risk of Bias and the Quality of the Report
Definition
This phase is part of the protocols for the evaluation of SRs (e.g. PRISMA), and, by extension, of overviews: this process involves critical appraisal and judgment relating to whether there were any potential risks of bias within the studies (Pollock & Berge, 2018). Therefore, if the SRs have conducted an evaluation of the methodological quality of their primary studies, this may or may not be a criterion for inclusion or exclusion of an overview.
Frequently Asked Questions
Why should the methodological quality of studies be assessed? In an overview, is methodological quality a criterion for inclusion or exclusion? Is it expected that there will be an assessment of the methodological quality of the primary studies in the SRs included? Do we have tools to conduct this assessment?
Procedure
It was decided to analyze the quality of the primary studies and not the SRs because not all the primary studies in the SRs selected study children. Some of these studies also include adolescents or work with other types of therapy.
There is no single approach to assessing the quality of studies, so the criteria for selecting the tool depend on the methodological design of the primary studies. Therefore, it was decided to work with two tools: RoB2 for experimental and quasi-experimental designs (Sterne et al., 2019), and Schuermans’ proposal for descriptive and case studies (Schuermans, 2013). The evaluation of each primary study was discussed among the researchers until a final agreement was reached. Both assessments were added to the coding manual for the primary studies, which required new columns to be added to the Excel file. As can be seen in the coding phase (phase 4), methodological quality was not an exclusion criterion, therefore analysis of the primary studies was undertaken, irrespective of their risk of bias level.
Recommendations
Study the research designs. Select the evaluation tool that is best adapted to the methodological designs of the SRs selected or of their primary studies. Decide whether to include SRs that have evaluated or have not evaluated the methodological quality of their primary studies. In view of the overview question, decide whether it is necessary to evaluate the methodological quality of the primary studies of the SR included.
Difficulties, Threats and Possible Errors
Failure to understand what is meant by methodological bias and its implications in the quality of results and conclusions of the overview. Lack of knowledge about research designs and differences between quantitative and qualitative designs. Inadequate use of the tool to assess methodological quality.
Illustrative Example
Out of the 17 primary studies, six are experimental or quasi-experimental designs (PS1 1 , PS3, PS4, PS5, PS11), six are case studies (PS2, PS7, PS8, PS9, PS10, PS14), and five did not provide information (PS6, PS12, PS15, PS16, PS17).
Figure 2 summarizes the quality assessment of the controlled trial designs using the tool RoB2. PS11 and PS13 presented a low-level risk of bias: in PS11, an independent psychologist with no knowledge of the groups scored all tests, and PS13 conducted multiple measurements and data analyses. In contrast, PS5 did not work with a random sampling and did not ensure balance between the experimental and the control groups (baseline). On the other hand, PS4 used matching to form the experimental and control groups, so the risk level was high in both cases. Finally, PS1 and PS3 obtained the same low and high-risk levels. PS1 randomly selected schools but lost samples in the follow-up assessment, while PS3 randomly assigned five classes to the intervention group and another five to the waiting list but failed to obtain the initial scores for the control group. Assessment of methodological quality of primary studies according to RoB2
Assessment of Methodological Quality of Primary Studies According to Schuermans (2013)
Key: -poor; +fair; ++good.
Phase 6. Analysis of Results
Definition
Involves establishing a synthesis of the evidence found. This process involves summarizing results in tables and in narrative form. Frequencies, percentages and graphs are presented, as well as a qualitative analysis to discover commonalities and identify patterns among the data, which does not mean looking for emergent exceptions to common patterns (see Booth et al., 2016 for a description of the 12 approaches to qualitative synthesis).
Frequently Asked Questions
How are the results of the overview analyzed and presented? How are data analyzed? How is the information synthesized? Where to start?
Procedure
We first analyzed the results derived from the 12 SRs and, in a second step, the 17 primary studies found in full text. This approach allowed us to provide an overview of the evidence and then go into detail about the impact of art therapy on children with trauma.
We used Atlas.ti to analyze qualitative data such as the abstract, objectives, inclusion-exclusion criteria, intervention characteristics and outcomes, whereas in the Excel file, we collected quantitative data such as years of publication, number of initial and final papers, methodological quality assessment or qualitative data that could be transformed into numerical labels, for example for countries (e.g. 1 = United States, 2 = Canada), nationality or research designs, among others.
The results of the overview were presented in a descriptive and narrative manner accompanied by frequencies, percentages, tables and graphs both for SRs and the primary studies. The report synthesized the commonalities of the SRs and the primary studies share with each other, as well as their differences.
Recommendations
Before starting the analysis, read the coding manual (SRs and primary studies) to get both a general and specific overview of the papers. Note the strengths, weaknesses and interesting aspects of each paper. Do not lose sight of the identification number of each paper during data analysis. Find a balance between generalization and specificity of the results. The results obtained should be analyzed according to the research question.
Difficulties, Threats and Possible Errors
Including information that does not answer the research question. Predominance of individual analysis to the detriment of comparative analysis. Losing the sense of why and for what purpose the data are being analyzed. Creating tables, graphs or figures that do not provide information.
Illustrative Example
Below, we present the results of the SRs that have met the inclusion criteria and of their primary studies.
The SRs were published between 2005 and 2022, with five out of the 12 more recently, between 2019 and 2022 (SR1 2 , SR7, SR8, SR9, SR11; 41.67%). The studies involved departments of psychiatry, psychology, social work and medicine, and in two studies art therapy schools and departments such as the Art Therapy Counseling Department of Marylhurst University in the United States (SR6) and The School of Creative Arts Therapies at the University of Haifa in Israel (SR8). The authors come from universities and institutions in Canada (SR3, SR4), United Arab Emirates (SR1), United States (SR6), Ireland (SR7), Israel (SR8), Lebanon (SR1), Norway (SR2, SR10), Netherlands (SR5), United Kingdom (SR7, SR11), South Africa (SR12) and Turkey (SR7, SR9), but none are from Oceania. More than half, seven out of the 12 reviews, have been published in health journals (SR1, SR2, SR3, SR5, SR7, SR10, SR12; 58.33%), and the remaining five in art therapy (SR6, SR8; 16.67%), social (SR4,SR9; 16.67%), art and health (SR11; 8.33%) journals.
However, only three reviews focus on evaluating empirical evidence on the efficacy of art therapy in children with trauma, thus coinciding with the objectives of the present overview: SR1, SR11 and SR12. In contrast, the focus of reviews SR2, SR4, SR5, SR9 and SR10 is on children with trauma, with art therapy being one of several interventions. Also, in reviews SR7 and SR8, the focus is on art therapy interventions with children where some of the primary studies indicate that they have experienced traumatic situations. In this sense, the efficacy of art therapy, among the creative expression therapies, is one of the results of SR3. Finally, SR6 focused only on the impact of art therapy as an intervention method in clinical and non-clinical populations of all ages, identifying primary studies with children exposed to traumatic conditions.
Therefore, the aims of a set of SRs were: to analyze current and emerging treatments and their effects on children and young people exposed to traumatic events (SR4); to evaluate psychotherapeutic treatments for children who have experienced child maltreatment (SR5); and to examine whether art psychotherapy is effective in children with mental health disorders (SR7). Another group specifically investigated the use of art therapy to reduce levels of post-traumatic stress disorder in refugees (SR1) or the efficacy of treatment modalities in children and adolescents with trauma (SR9). Also, the context of several reviews was the school: SR3 identified creative expression programmes in the classroom and their effects on children’s mental health, while SR10 evaluated the effectiveness of school-based intervention programmes aimed at reducing the symptoms of post-traumatic stress disorder. SR6 is a continuation of the SR by Reynolds et al. (2000) of the impact of art therapy on clinical and non-clinical populations. Finally, SR2 differs from the others in that it analyzes the screening and diagnostic instruments available in the Nordic countries for the treatment of dissociative disorder symptomatology in children and adolescents.
Number of Primary Studies per Systematic Review Analyzed
aDocuments that were not found in full text.
The analysis of the primary studies in terms of contextual, methodological and substantive variables is presented below.
Contextual Variables
The primary studies were published between 2001 and 2016, which is almost one study per year, with no recent research found (Figure 3). Number of primary studies per year of publication
Out of a total of 44 authors, 22.73% participated in more than one study, including Gauthier (PS6, PS13), Lee (PS7, PS8), Lacroix (PS6, PS12, PS13) and Rousseau (PS6, PS12, PS13), the latter two with the highest number of publications each.
A consolidated research group can be observed in Canada, with Gauthier, Lacroix and Rousseau from McGill University (Department of Psychiatry), Concordia University (Department of Creative Arts Therapy) and the Montreal Children’s Hospital (Department of Transcultural Psychiatry), who published three studies between 2003 and 2009 (PS6, PS12, PS13).
Three journals account for 70.59% of the studies: Art Therapy: Journal of the American Art Therapy Association (PS7, PS8, PS10, PS12, PS14, PS16, PS17), Journal of Child Psychology and Psychiatry (PS1, PS13) and The Arts in Psychotherapy (PS2, PS6, PS9). Two of the journals specialize in health sciences (psychology and medicine), and the other, Child and Adolescent Social Work Journal (PS4), in the social field.
The United States (PS1, PS2, PS7, PS8, PS10, PS14, PS17; 36.84%) and Canada (PS4, PS6, PS9, PS12, PS13; 26.32%) account for 63.16% of the countries responsible for the research, with the rest having only one study each, such as Australia (PS15), Italy (PS5), Israel (PS3), United Kingdom (PS1), Russia (PS16), South Africa (PS11) and Uganda (PS1). Only study 1 was conducted jointly by several countries (USA, UK and Uganda).
Universities and institutions collaborated in the primary studies, such as Care and Protection of Children in Crisis-Affected Countries Initiative, Save The Children, The New Children’s Hospital Sydney, Putnam/Northern Westchester Women’s Resource Center in Mahopac, Jewish Board of Family and Children’s Services in Brooklyn and Queens Children’s Psychiatric Center. PS9 is the result of the experience of a self-employed art therapist.
Methodological Variables
As for the sample, 58.82% of the primary studies (PS2, PS4, PS5, PS6, PS9, PS10, PS12, PS14, PS15, PS17) do not specify the nationality of the participants, but some of them reveal that they are children from a French multi-ethnic school in Montreal (PS6, PS12) or attending a community programme in the South Bronx in New York (PS10). The studies also involved orphaned children from Russia (PS16), children exposed to terrorism in Israel (PS3), children exposed to the territorial conflict in Uganda (PS1) or the tsunami in Sri Lanka (PS13), sexually abused girls from South Africa (PS11) or South Korean migrants in the United States (PS7, PS8).
The age range is from 2 to 12 years old. Studies PS2, PS6, PS13 and PS14 include ages specific to preschool education (4 to 6 years old); studies PS1, PS3, PS4, PS7, PS8, PS9, PS10, PS11, PS16 and PS17 relate to primary education ages, while PS5, PS12 and PS15 included ages specific of both, preschool and primary education.
Among the research designs analyzed, single case studies (PS2, PS9, PS10, PS14) and multiple case studies (PS2, PS7) stood out with a percentage of 20%; experimental and control group designs (PS1, PS3, PS5, PS13) accounted for 16.67% of the designs; pre- and post-test studies (PS1, PS4, PS13) accounted for 10% of the designs, while quasi-randomized controlled trials accounted for 13.33% of the designs (PS3, PS4, PS5, PS11). PS7 and PS8 combined case studies with grounded theory and conducted a cross-case analysis, whereas PS2 opted for an observational design and PS11 for Solomon’s four-group design.
Substantive Variables
47.06% of the studies focused on assessing the impact and effectiveness of art therapy interventions for land conflict (PS1); behaviors and coping skills (or lack thereof) related to experiences of abuse, loss, trauma and anxiety (PS4); adapting to the circumstances of a new home in another country (PS7, PS8); post-traumatic stress from acquired brain injury due to car accident (PS9); depression, anxiety, sexual trauma and low self-esteem in sexually abused girls (PS11); use of myths to express and share immigrants and refugees experiences (PS12); neurological basis and its role in the integration of brain structures due to early relational trauma and attachment disruption (PS14).
On the other hand, 17.65% are prevention programmes aimed at reducing post-traumatic stress due to exposure to terrorism (PS3); reducing anxiety during and after painful medical procedures (PS5); and reducing emotional and behavioral problems to improve self-esteem in immigrant and refugee children (PS13).
Finally, 35.29% are studies that explore, describe, observe and examine exchanges between art therapist, client and artistic creations (PS2); spontaneous representations after the tsunami (PS6); the use of transitional objects to cope with parental separation and divorce (PS10); parental separation and family violence (PS15); engagement in art therapy by a group of orphaned children (PS16); and the collaborative creation of a mural commemorating the 9/11 attack in the United States.
Also, in 52.94% of the primary studies, art therapy was the only intervention approach (PS2, PS3, PS5, PS9, PS10, PS14, PS15, PS16, PS17), while in 47.06% art therapy was combined with sandplay therapy (PS1, PS6, PS13), mindfulness (PS4), flow (PS7, PS8) and creative expression with myths and metaphors (PS12). The case study in PS11 includes the existential-humanistic perspective (Gestalt principles), the abuse-focused approach (Briere, 1992) and the client-focused approach (Rogers, 1967).
As for the contexts in which the interventions were conducted, care centers and community programmes prevail (PS2, PS4, PS7, PS8, PS10, PS11; 35.29%), along with schools (PS1, PS3, PS6, PS12, PS13; 29.41%), hospitals (PS5; 5.88%), orphanages (PS16; 5.88%) and even the art therapist’s own home (PS9; 5.88%), with three studies not providing such information (PS14,PS15,PS17; 17.65%).
The research results (supplemental material 5) show that the experimental groups improved on all measures compared to the control or comparison groups, and that no experimental group worsened, indicating that the interventions did not have a detrimental effect on children. The benefits of art therapy in children with trauma are mainly evidenced in coping strategies and post-traumatic stress reduction, with special emphasis on emotional (inner world, emotions and feelings, confidence, security, self-esteem, anxiety), social (social skills, communication, collaboration, bonding) and cultural adaptation. These benefits have also been perceived by art therapists, family members, teachers and healthcare providers.
It is interesting to note how research groups contribute to strengthening lines of research and consequently findings. For example, in PS7 and PS8 similar results were obtained on how flow experiences in art therapy support cultural adaptation of South Korean children migrating to the United States. The combination of these therapies offers tools for decision-making, identification of internal resources and task accomplishment. Similarly, sense of time and concentration were modified (flow), while anxiety, fear, boredom, social isolation and frustration decreased. The same happened with PS12 and PS12, also focusing on immigrant and refugee children from diverse cultures attending multi-ethnic schools in Canada, who in the art therapy sessions manifested the home-school and past-present cultural gaps, with representations through mythical references to culture and country of origin.
However, the outcome of some interventions was not as expected. In PS4, resilience did not improve after the art therapy intervention, but self-concept did, albeit partially. Similarly, in PS1, child well-being and resilience improved as perceived by the children themselves and their families, but not as perceived by their teachers. The low self-esteem of sexually abused girls was maintained in both the experimental and the control groups (PS11). Children from families exposed to violence and persecution showed more prosocial behaviors compared to their classmates, and did not show more symptoms than their peers (PS13). On the other hand, although functional problems decreased in both boys and girls, boys scored higher compared to girls (PS3).
Also, few primary studies conducted a follow-up period after the intervention. PS1 found that the Psychosocial Structured Activities programme had a positive impact on the emotional well-being of children in territorial conflict contexts up to one year after the end of the art therapy sessions. Two to three months after the intervention, PS3 confirmed that hope improved and that there was a reduction in all measures (post-traumatic stress, somatic complaints, functional impairment and anxiety). In the same vein, PS13 conducted a follow-up 4 months after the end of the programme for immigrant families with previous experiences of violence with positive findings.
Special attention should be given to the findings of PS5, which highlights the role of the art therapist in the well-being of children hospitalized for leukemia and their families. Parents also requested the support of the art therapist because they felt accompanied and better able to manage the medical procedure. In PS16, the bond with the art therapist also provided a restorative relational space for orphaned children.
Practical Implications for Art Therapy
Likewise, this overview provides several implications for art therapy practice, particularly for clinicians working with children exposed to trauma. First, the synthesis of evidence from systematic reviews and primary studies confirms that art therapy interventions can significantly enhance children’s emotional regulation, self-esteem, and coping mechanisms while reducing post-traumatic stress symptoms. Clinicians can rely on these findings to support the use of art therapy as a complementary or stand-alone intervention in trauma-informed care.
Second, the results highlight the importance of tailoring interventions to the child’s developmental stage and sociocultural background. Art therapists are encouraged to use flexible, creative, and culturally sensitive approaches, incorporating materials, symbols, and narratives that reflect children’s lived experiences and cultural identity. This promotes a sense of safety and agency within the therapeutic process.
Third, the overview emphasizes the central role of the therapeutic relationship. Across the analyzed studies, the art therapist’s presence was described as a restorative and containing figure, particularly valuable for children who have experienced disrupted attachment or loss. Clinicians should therefore prioritize building trust, continuity, and emotional attunement through the creative process.
Fourth, the findings underscore the value of interdisciplinary collaboration. Effective art therapy interventions often occurred in community programs, hospitals, and schools, suggesting that clinicians should work in coordination with educators, psychologists, and social workers to ensure comprehensive trauma-informed support.
Finally, the methodological reflections offered by this overview can guide practitioners who wish to evaluate and document their interventions systematically. By adopting transparent, replicable, and evidence-based procedures, clinicians can contribute to strengthening the empirical foundation of art therapy and advancing professional standards in the field.
Discussion and Conclusion
The production of overviews has increased considerably in recent years, largely in response to the growing proliferation of SRs. Overviews allow evidence from multiple SRs to be combined in an efficient way, which helps to transform and summarize substantial amounts of information. As with primary studies and SRs, overviews must be conducted and presented in compliance with minimum quality standards.
The purpose of this study has been to provide methodological guidelines for the design, development and reporting of qualitative overviews using art therapy interventions with children with trauma as an example. Our aim is to offer guidance to students, academics, and practitioners in this field.
Planning and conducting an overview involve methodological challenges. The strategies implemented in this guide have served as an example to minimize the effects that this method entails, such as the reliability of the findings, the coverage of the evidence, as well as the usability and usefulness as a synthesis of the evidence. Thus, according to the recommendations found in the scientific literature, the research question has been broad enough to include SRs similar or close to the topic, which has favored a more complete view of the existing evidence (Hunt et al., 2018). Another decision was to include SRs regardless of their methodological quality (Gates et al., 2020) and also those that contained information related to the overview question, even if not directly addressing it. In addition, a new risk of bias assessment of the primary studies in the SRs was conducted, since many of them had not been assessed or had been assessed using different tools (Polanin et al., 2017). The effort of searching for primary studies had a significant impact on the quality of the analysis. It enabled cross-checking and supplementing information which, in some cases, was incomplete or contradictory in the SRs involved. In this regard, Pollock et al. (2022) point out that overviews can present the outcome data exactly as they appear in the included SRs, or they can be re-analyzed in a way that differs from the analyses conducted in the SRs.
Working with a specific topic also provides methodological guidance for the discipline. In our case, the small number of SRs on art therapy with children with trauma is evident because it is a relatively recent professional field, as indicated in previous pages and by different authors (Reynolds et al., 2000; Slayton et al., 2010). This scarcity of reviews was, on the one hand, an advantage, as it accelerated the literature review, but at the same time made it difficult to formulate a much more specific question for the overview. Other problems that led us to discard several primary studies were that the effects of art therapy on children and adolescents were analyzed as a whole, rather than in sub-samples or that the ages of infancy or early childhood were not defined, or that the term children was used for samples up to 18 years old.
Likewise, the development of this guideline has yielded valuable lessons for researchers and practitioners, including the following: (1) Establishing a clear objective and scope is essential when planning an overview. Researchers should first determine whether the overview will synthesize evidence from quantitative meta-analyses, qualitative syntheses, or mixed-method reviews. It is also important to explicitly define which types of reviews-such as Cochrane, scoping, narrative, or realist reviews-will be considered eligible for inclusion. (2) Prioritize methodological consistency and justification. Each methodological decision-such as the search strategy, inclusion and exclusion criteria, data extraction process, and quality assessment-should be explicitly justified with reference to established frameworks (e.g., JBI, PRISMA, ENTREQ). Providing a clear rationale for these decisions enhances the transparency, reproducibility, and credibility of the overview. (3) Ensure adherence to quality criteria in qualitative research. It is essential to uphold core qualitative standards such as reflexivity, auditability, triangulation, thick description, credibility, and dependability throughout the overview process. Transparency should be achieved through explicit documentation, reflective practice, and methodological standardization. Potential bias may emerge from methodological shortcuts or interpretive dominance; therefore, the guidelines should incorporate checkpoints at critical stages of the process. Additionally, fostering iterative feedback loops-such as stakeholder consultation and peer review of methods-can further enhance the rigor and trustworthiness of the overview.
On the other hand, in relation to recent evidence, the findings of this overview confirm, qualify, and in some cases problematize current understandings of art therapy in the treatment of childhood trauma reported in the scientific literature. First, the results confirm conclusions from recent systematic reviews and meta-analyses showing that art therapy and other creative arts-based interventions are associated with improvements in emotional regulation, trauma expression, and reductions in post-traumatic stress symptoms in children exposed to adversity, violence, or forced migration (Annous et al., 2022; Morison et al., 2022). These studies also highlight that art therapy is well suited to children’s developmental needs and is especially useful as a non-verbal intervention when verbal expression is limited.
However, the findings of the overview also qualify the predominantly positive view reported in recent literature. Although consistent benefits are observed at emotional and relational levels, effects on variables such as resilience, self-esteem, and overall well-being are not uniform and are not always statistically significant. This lack of consistency is more evident when different informants are considered or when medium- and long-term follow-ups are included. This variability reflects current concerns about methodological heterogeneity, small sample sizes, and the wide range of outcome measures used in recent studies on art therapy and childhood trauma, which limit comparability and the strength of conclusions (Bosgraaf et al., 2020; Cohen-Yatziv & Regev, 2019; Morison et al., 2022).
At the same time, some findings of the overview implicitly challenge common assumptions in the contemporary literature by showing that the absence of negative effects does not mean that art therapy is equally effective for all children or across all outcome domains. Recent studies indicate that outcomes in traumatized children depend on contextual and relational factors, including the type and chronicity of trauma, the therapeutic setting, the duration of the intervention, and the degree of integration with other psychotherapeutic approaches (Bosgraaf et al., 2020; Haeyen & Noorthoorn, 2021; Schouten et al., 2015). In addition, recent qualitative and mixed-methods studies show that meaningful change is not always captured by standardized measures but is often evident in relational, narrative, and symbolic processes relevant to trauma recovery. This supports the need for more longitudinal, context-sensitive, and methodologically integrated research designs (Haeyen & Noorthoorn, 2021; Feniger-Schaal & Orkibi, 2020).
Finally, this overview is not without limitations. One of them is the lack of accessibility to all primary studies. Another difficulty is not having included a step-by-step presentation of how the coding manual was modified, or greater specificity about how the data analysis of the SRs included was conducted, or a presentation on how the discussion section would be, which was not possible due to an issue of extension.
For future research interested in the elaboration of guides or manuals, further exploration is recommended of resources or technological tools for the design and implementation of evidence-based overviews that facilitate the retrieval, comparison and analysis of documents, as well as their quality. Also, following the same quality practices as those applied to other forms of systematic evidence synthesis, it is recommended that an overview protocol be developed to ensure transparency and reduce the possibility of introducing bias into the overviews.
Supplemental Material
Supplemental Material - Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma
Supplemental Material for Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma by Celia Camilli, Mónica Fontana in International Journal of Qualitative Methods.
Supplemental Material
Supplemental Material - Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma
Supplemental Material for Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma by Celia Camilli, Mónica Fontana in International Journal of Qualitative Methods.
Supplemental Material
Supplemental Material - Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma
Supplemental Material for Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma by Celia Camilli, Mónica Fontana in International Journal of Qualitative Methods.
Supplemental Material
Supplemental Material - Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma
Supplemental Material for Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma by Celia Camilli, Mónica Fontana in International Journal of Qualitative Methods.
Supplemental Material
Supplemental Material - Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma
Supplemental Material for Methodological Guide for Conducting an Overview: An Example of Art Therapy for Children with Trauma by Celia Camilli, Mónica Fontana in International Journal of Qualitative Methods.
Footnotes
Ethical Considerations
The Brundibár project was approved by the Research Ethics Committee of the Complutense University of Madrid.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Spanish Ministry of Science and Innovation through the Research and Development (R&D) project “El proceso creador y la arteterapia como vía de bienestar ante el trauma infantil (Brundibár)” (reference: PID2020-114238RB-I00), led by Principal Investigator Dr. Marián López Fernández-Cao.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The authors of the study are committed to open science. The data (supplementary material) necessary for the study have been shared so that they can be further used by other researchers.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
