Sage Journals: Discover world-class research

Abstract

Objectives

Since psychological trauma and the use of technology are common in society, designers and researchers have been applying a trauma-informed (TI) approach to the digital design of technology. A TI approach was developed by the social work field to better serve those who have been through trauma. The purpose of this scoping review was to determine the current state of scholarly research on TI digital design and technology.

Methods

Searches were conducted in five databases for peer-reviewed articles about the design of digital interventions using TI approaches, published from 2000 to 2023. One hundred and eighteen studies were included and examined for their use of TI approaches, settings, types of digital design, and types of organizations and practitioners designing.

Results

Studies and interventions are not explicitly or methodically applying a TI approach to digital design, despite hopes to serve populations with significant trauma (e.g., life-threatening cancer survivors) in various settings. Efforts are sometimes consciously made that align with TI principles (e.g., safety), but other times there's no accounting for trauma and its impacts in the design of interventions.

Conclusion

The review indicates that there is a lack of using the evidence-based framework of a TI approach in digital design.

Keywords

Digital health digital health interventions trauma-informed design trauma-informed care psychological trauma

Introduction

Concept of trauma and trauma-informed approaches

Trauma is a response to “violence, abuse, neglect, loss, disaster, war and other emotionally harmful experiences”¹^(p2) and is a common public health problem that should not be ignored.^1–3 According to survey research from the World Health Organization (WHO) in 24 countries, more than 70% of respondents reported a traumatic event, and about 30% of individuals were exposed to four or more.^4,5 Examples of traumatic events in the WHO survey are the unexpected death of a loved one, a life-threatening illness or injury, a life-threatening event, an automobile accident, and rape.^4,5 The impacts of trauma can include depression, anxiety, substance use, post-traumatic stress disorder (PTSD), cognitive impairment, inability to cope with daily stressors, and problems fostering trusting relationships, among other negative health consequences.^1,6–9 Nearly all people will go through what may be called traumatic experiences; many people's reactions and symptoms resolve in the short term, while other people may experience delayed onset and/or long-lasting adverse effects.^1,9

To better serve people who have experienced trauma, trauma-informed (TI) approaches have become an important concept used in behavioral and mental health services, healthcare, schools, and other types of organizations.^1,10 TI approaches do not aim to treat trauma but seek to create a welcoming, culturally aware environment of safety, choice, empowerment, and collaboration, and avoid recreating harmful dynamics.^1,11,12 Without a TI approach, instead of being helpful, help-seeking experiences can be re-traumatizing and/or fear-inducing for individuals.¹¹ Two well-known TI approaches with principles developed by researchers and clinicians in the United States are those from The Institute on Trauma and Trauma-Informed Care (ITTIC) at the University of Buffalo School of Social Work¹³ and the Substance Abuse and Mental Health Administration (SAMHSA).¹ See the two sets of principles from SAMHSA and ITTIC in Table 1.

Table 1.

SAMHSA and IITIC principles.

SAMHSA principles	ITTIC principles
Safety	Safety
Trustworthiness and transparency	Choice
Peer support	Collaboration
Collaboration and mutuality	Trustworthiness
Empowerment, voice, and choice	Empowerment (ITTIC, 2015).
Cultural, historical, and gender issues (SAMHSA, 2014).

ITTIC: Institute on Trauma and Trauma-Informed Care; SAMHSA: Substance Abuse and Mental Health Administration.

It is important to note that while IITIC does not specifically include principles related to cultural, historical, and gender issues, it may be considered an underlying assumption according to a conversation with social work researcher C. Scott, PhD (April 2024) since the field of social work has a code of ethics with the value of social justice that encourages care related to oppression, cultural, and ethnic diversity.¹⁴ The explicit inclusion of “Cultural, Historical and Genders Issues” as a principle by SAMHSA reflects SAMHSA's goal to be used in a wide variety of settings beyond the social work field, including education, primary healthcare, and the military, among others.¹ The IITIC and SAMHSA TI principles have overlap, and either one offers a useful approach for designers of digital technology.¹⁵ Scott et al.¹⁶ provide a detailed explanation of trauma and TI approaches specifically for those in the technology field to learn more about these concepts.

Social determinants of health include digital technology

Since the late 1970s researchers have asserted that the non-medical factors that influence health are critical components of health outcomes for populations.^17–19 Known as the social determinants of health (SDOH), these factors include, but are not limited to, income, education, housing, unemployment, food insecurity, and social inclusion.¹⁹ The more time spent in stressful social and/or economic circumstances, the less likely people are to be healthy in the long term.²⁰

In the last two decades, a more recent factor affecting health is digital technology. According to the United Nations, in 2023, 5·4 billion people are using the Internet, or about 67% of the world's population.²¹ Researchers and organizations have recognized that digital technology is an increasingly important part of health and is directly related to health equity.^22–24

For example, access to the Internet, telemedicine services, digital literacy, and digital behavior change interventions are all factors, among others, that may affect health outcomes.²³ Similar to how racism has been conceived by Dahlgren and Whitehead²⁵ as a driving force that impacts nearly all SDOHs, digital technology could be viewed as an omnipresent factor in SDOHs as well.²³

For the purpose of this study, we narrowed our focus within digital technology to the design of websites, applications, Internet of Things, and other products used by individuals on computers, smartphones, wearables, and augmented and/or virtual reality devices instead of a broader definition of digital technology that might include the design of robotics, data analysis, cloud computing, e-books, and hardware, among other things. What is yet to be addressed within SDOH literature is how digital technology should be designed for those who have experienced trauma—for example, a website, app, or other digital intervention cannot be considered very effective if it is seen as unusable, untrustworthy, or unsafe by the population it aims to serve. It is common for survivors of traumatic experiences to have impaired cognition.^1,6,9 Unfortunately, problematic website or app design is often cognitively demanding of users.^26,27 Plus it is easy for those in stressful situations to be overwhelmed or confused by online information.^28–30 Emotional stress can also cause other physical symptoms of trauma such as shaky hands which also makes using technology harder to use.³¹ Socially, the design of technology, which includes content and its design, can foster the feeling of being excluded^28,32,33 or even blamed.³⁰ A lack of trust in others, a common impact of interpersonal violence, can add to the challenge of help-seeking.^1,34,35 Trust in digital technology must be earned.

To address safety, trust, access, and other issues that impact those who have experienced trauma, digital technology can be designed in alignment with the TI principles such as safety, trustworthiness, empowerment, etc. Systems and services that are not TI may unintentionally trigger a trauma response in individuals with a trauma history.¹¹ These services and systems would include digital technology as well. In less severe cases that do not inadvertently cause a trauma response, the lack of being TI may cause digital technology to hinder receiving health services or doing necessary daily tasks.^15,28

Guidelines for digital technology have been developed to ensure the needs of more people are met. For example, Web Content Accessibility Guidelines (WCAG) were created in 1991 to make web content more accessible to people with disabilities. These guidelines are continually updated by the World Wide Web Consortium, an international public-interest non-profit.³⁶ While WCAG offers standards, principles, guidelines, success criteria and examples to support the creation of accessible content for everyone, it does not address trauma directly.³⁶

Since digital technology now has an important role in health outcomes, it must be created not only to be accessible to every person but also keep the context of trauma in mind. In the last five years, there has been increased interest in TI digital design in the field of technology.^15,37–40 Design practitioners are attempting to apply TI concepts to both the design research process and products such as websites and mobile applications.^41–44 Social work and trauma researchers are calling for more interdisciplinary partnerships with digital technologists.⁴⁵ However, there is no shared understanding of the methods and progress made, with no standard methodology or known best practices in applying TI principles to the design of technology.

Objective and research questions

To facilitate practice, research, and education related to TI design and technology, this scoping review offers an overview of the body of research at the intersection of TI approaches, technology, and design that is available in peer-reviewed journal articles. Sharing resources and verified research is important to expand knowledge and understanding of TI design and technology in order to improve practice and avoid the aforementioned harms. The objective of this investigation was to assess and characterize the current scholarly literature on TI care as it relates to digital design and technology. Results indicated areas where future research is needed and key themes and guiding questions to consider for a design process or technological product to be TI were identified for practitioners.

To reach these stated goals, we posed the following research question: What is the current state of research on TI digital design and technology? Some of the sub-questions we hoped to answer included:

Which types of digital design and technology (e.g., mobile apps, virtual reality, etc.) are applying the concepts of TI care?

What organizations and types of practitioners are exploring the concept of TI design and technology in their work?

In what settings are the concepts of TI design and technology being applied (e.g., schools, hospitals, online therapy, emergency medical services, etc.)?

Methods

This review was guided by Arksey and O’Malley's⁴⁶ methodological framework and Preferred Reporting Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) was followed.⁴⁷ A protocol was developed a priori and is accessible via the authors’ institutional repository at https://doi.org/10·17615/65cb-pm36.

Eligibility criteria, sources, and search strategy

We included articles that focused on TI design of digital interventions, inclusive of products, services, or technology for all populations. Additionally, we included articles that described the design, development, or adaptation of a digital intervention if the topic area related to, or population of interest was prone to, trauma or adverse childhood experiences. Articles were excluded that focused on trauma-specific care, such as psychotherapy, traumatic brain injury, recovery from bodily trauma, TI architectural design, or those that described TI care without relation to design or technical context.

A health sciences librarian at The University of North Carolina at Chapel Hill developed systematic literature search strategies to identify relevant citations in PubMed (National Institutes of Health, National Library of Medicine), Scopus (Elsevier), PsycInfo (EBSCOhost), and Sociological Abstracts (ProQuest). All databases were searched from their date of inception through the date the searches were executed on February 3, 2023. The search strategy was developed iteratively in PubMed using a combination of text words and controlled vocabulary, when applicable. The search included terms for (1) TI approach (inclusive of violence, adverse childhood experiences, distress, etc.), (2) user-centered design, and (3) digital interventions. The search strategy was validated against a pool of previously identified relevant citations (n = 5) and translated to the remaining databases. Search results were limited to English-language and by publication year, excluding articles published prior to 2000. Search strategies for all databases can be found in the supplemental appendix.

Database searches were supplemented by hand-searches in the Digital Library of the Association for Computing Machinery Digital Library, the Journal of Medical Internet Research, the Journal of Usability Studies, and Google Scholar. Near the end of the study, on February 4, 2024, a hand-search was conducted by one team member, which yielded four additional articles based on title and abstract. Two team members independently reviewed these articles and, upon full-text assessment, determined that they did not meet the inclusion criteria for the study.

Selection process, data charting, and extraction

Citations were exported from each database and imported into Endnote where they were deduplicated. All unique citations were imported into Covidence⁴⁸ and underwent a two-stage screening process. First, title and abstracts of citations were screened by two reviewers based on predefined eligibility criteria. All citations potentially meeting inclusion were screened in a second stage by two independent reviewers using the full-text article. For both stages, each record was independently screened by two blinded reviewers from the team (ME, NK, WAR, KM, JO, EC). Conflicts were resolved by a third reviewer in both screening stages (ME or EC).

Data charting was conducted in Covidence and the form was piloted by all reviewers. Following the two-stage screening process, data items from all included studies were extracted by two, blinded independent reviewers and conflicts were resolved by consensus. After extraction, data were downloaded from Covidence by one reviewer and sent to two other reviewers to review and summarize the data in Excel format. The two reviewers summarized the data by including simple counts (e.g., number of studies from the United Kingdom), grouping data (e.g., age ranges of intended audience), and determining frequencies of occurrences (e.g., how many studies mentioned making a prototype). The two reviewers noted any missing data and asked the first reviewer to fill in missing values and address any errors, returning to the original studies as needed. Counts were updated once all missing data had been identified.

Data items extracted from each included article were year of the publication, types of organizations associated with the co-authors, health topics, country in which the research was conducted, aim of the study, age range intended to be served with the solution, and gender population intended be served with the solution, types of individuals intended be served with the solution, if the study used the term “trauma-informed,” which (if any) sets of TI principles were reference, which TI care principles were referenced in designing of the solution, processes used in designing the solutions, which (if any) theory or framework was used to design the solution, which individuals were involved in the design of the solution, format of the digital solution, if solution utilized artificial intelligence (AI) or machine learning, how many solutions were developed, where the solution is intended to be implemented, if the article explicitly mentions making a “prototype,” if the solution was evaluated in this study, what evaluation designs were used, and if the solution is accessible to the public today.

Organization of results

Following you will find the main research questions addressed and summaries of study characteristics in narrative format such as the year of publication, health topics, organizations and types of practitioners, settings, populations intended to serve, TI care principles referenced in designing solutions, the processes used to design the solution, types of individuals involved in the design of the solution, the format of the solution, the number of solutions developed, type of evaluation (if any), accessibility to the public, number of studies that mentioned making a prototype. For qualitative data, such as the aims of the study and/or which theories or frameworks were used, a researcher conducted an initial thematic analysis to identify overarching findings which were subsequently reviewed and discussed by the entire research team.

Results

A total of 5716 citations were identified from database searches. After removing duplicates (n=1862), a total of 3854 unique citations underwent title and abstract screening and 3591 were determined to not meet inclusion criteria. After the first round of screening, 263 records underwent full-text screening and 145 were excluded for various reasons including not focusing on design or adaptation strategies, being about traumatic brain injury, stroke, or bodily harm, and more. No new items were identified via hand-searching that met the eligibility criteria. A total of 118 reports are included in this review. The study selection process is detailed in Figure 1.

Figure 1.

Preferred Reporting Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 flow diagram.

The list of included studies and all data can be accessed at the public research repository at https://cdr.lib.unc.edu/concern/data_sets/ng451v95m?locale=en.

Study characteristics

The publication years of the 118 articles found in our search spanned from 2007 to 2023, with 75% (n=87) of the articles published in 2018 or later. Our review was limited to articles in the English language, and the majority of the research papers found were from the United States (45%, n=53), with Australia (10%, n=12), the United Kingdom (8%, n=9), and Canada (8%, n=9) also represented. “Other” countries (24%, n=29) contributed articles from around the globe, including Brazil, Sweden, Ghana, South Korea, and Taiwan, among others. Of the health topics addressed, “Cancer” emerged as a significant focus (34%, n=40), while none of the articles explicitly centered on “Covid-19” or “Eating disorders,” despite both areas being highly related to psychological trauma. The “Other” health topics included depression, anxiety, and mental health.

Of the articles reviewed, 70% (82 articles) did not specify gender, indicating a notable gap in this aspect. Of those specifying the gender breakdown revealed 28·8% (34 articles) female, 15% (18 articles) male, and 0·8% (1 article) each for non-binary and unclear gender identification. There was a significant emphasis on individuals with a lived experience of trauma (86%, 101 articles) for the intended populations to be served. Other targeted groups included caregivers for those with trauma experience (20%, 24 articles), medical providers (16%, 19 articles), others (13%, 15 articles), mental health providers (8%, 9 articles), and other community members (2%, 3 articles). The majority of studies did not specify a particular age they aimed to serve with the digital intervention (80%, 94 articles). When specified, the most common age range was 14–18 years (10%, 12 articles). Other age ranges included 19–33 years (6%, 7 articles), 49–64 years (7%, 8 articles), 6–13 years (7%, 8 articles), 65–78 years (5%, 6 articles), 34–48 years (5%, 6 articles), 79–98 years (4%, 5 articles), 3–5 years (2%, 2 articles), and 0–2 years (1%, 1 article). Additionally, some articles fell into the “Other” category (5%, 6 articles).

The design and development of websites (53 articles, 45%) and mobile applications (52 articles, 44%) were predominant in the studies with some digital interventions including both a web platform and mobile application. In another 19 articles, other types of digital interventions were created such as video stories⁴⁹ and a chatbot.⁵⁰ Virtual reality platforms were developed in 2 studies (2%). None of the studies reviewed used AI and machine learning in the design and development of digital solutions.

Among co-authors of the studies reviewed, the creators of these types of digital design interventions were predominantly educational institutions (98%, 116 articles). Corporations (23%, 26 articles), government organizations (16%, 18 articles), and charitable organizations/not-for-profits (14%, 17 articles) also created digital technologies aiming to serve trauma-inclined populations. Those having a lived experience of trauma were the most common (86%, 101 articles) group involved in the design process, which is in line with human-centered design approach. Researchers/scientists (76%, 90 articles), medical providers (48%, 56 articles), and technology professionals (30%, 35 articles) were also contributing parties to the reviewed studies. Implementation settings for proposed solutions varied, with a focus on the general community (43%, 51 articles), followed by hospitals/clinics (24%, 28 articles) and schools (3%, 3 articles). Approximately 44% (52 articles) did not specify the intended setting for the use of the digital intervention related to trauma.

A small amount of studies evaluated the efficacy or the effectiveness of the solution in the study. In total, 87.3% (103 articles) reported no evaluation,9·3% (11 articles) employed a pre-experimental design, and 3·4% (4 articles) used a true experimental design, while no articles mentioned a quasi-experimental design. Studies sometimes referenced that evaluation would be in a future stage of the digital project. As for availability to the public, 25% (29 articles) of the digital interventions developed were accessible, determined by searching the Internet and/or the iOS and Android mobile app stores by researchers. In total, 41% (48 articles) of digital interventions did not appear available to the public as they were not findable. In some cases (35%, 41 articles) it was unclear whether the digital intervention was available or not, for example, the digital intervention may be in use by a hospital but only available to patients of that hospital.

Use of TI approaches and principles

Explicit referencing of TI approaches with associated principles in the design process was nearly non-existent in the studies reviewed. Almost all (99%, n=117) did not reference using a TI approach and associated principles from any organization or group such as the Substance Abuse and Mental Health Services Administration. Only one of the 117 studies, Forehand et al.⁵¹ explicitly used the word “trauma-informed” and shared the process of developing a smartphone app to complement “an in-person trauma-focused knowledge and skills training class for child welfare resource parents (i.e., foster, kinship, and adoptive parents).”⁵¹^(p362) This training class included TI parenting skills, is based on research and information from the National Child Traumatic Stress Network, and is for those caring for children who have experienced trauma.⁵¹ While a participatory design approach was employed, a specific TI approach was not referenced in the development process. However, the reviewers identified TI principles such as safety, trustworthiness and transparency, and peer support addressed in this study.

Though the language “trauma-informed” was not used in any study except for Forehand et al.,⁵¹ TI principles were frequently considered in the design or development process—even though this was not explicitly recognized. Many studies reflected TI principles that are part of the TI approach as they are developing studies and digital solutions. See how frequently TI principles were mentioned and/or alluded to in Table 2.

Table 2.

Trauma-informed care principles referenced in studies.

Principle	N (%)
Safety	74 (63)
Peer support	55 (47)
Empowerment, voice and choice	46 (38)
Trustworthiness & transparency	34 (29)
Collaboration & mutuality	35 (30)
Cultural, historical & gender issues	25 (21)
None of the above	8 (7)

To illustrate, researchers making a smartphone app for the sexual and gender minority community in Brazil deliberately developed app features to increase self-agency and control⁵² which reflected the TI principle of empowerment, voice, and choice. The co-design process used by researchers in Texas to design a digital tool to support harm reduction and overdose response efforts⁵³ embodied the TI principle of collaboration and mutuality. Next, the creators of a digital avatar to support suicidal patients in emergency departments considered the TI principles of safety and peer support.⁵⁴ Moreover, researchers in Wisconsin collaborated with a community organization for veterans to develop a mobile app that focused on peer support but also recognized the critical importance of trust during the design process.⁵⁵ These are examples of TI principles in action.

Themes in study aims, design process, and frameworks used

Since TI approaches were not specifically used in these studies aiming to create digital interventions for populations that experience trauma, it is worthwhile to report on themes in study aims and the approaches used during the creation process instead. Two main themes surfaced as data on the purpose of individual studies were analyzed: helping people help themselves and offering support. About half of the technologies developed for people who experienced trauma fell into one of these two categories.

Almost 25% of studies (n=28) focused on helping people to help themselves. Some studies focused on assistance with decisionmaking, such as a decision aid for caregivers of suicidal youth related to reducing access to lethal methods⁵⁶ and a decision aid for female cancer survivors related to presevring fertility.⁵⁷ Other studies concentrated on educational digital interventions, such as a mobile app that offered evidence-based information about crystal methamphetamine for the general community,⁵⁸ a Web-based psychoeducation program for young people with or at high risk for depression,⁵⁹ and a suite of complementary mobile apps for early identification of unsafe relationship behaviors and safety planning to reduce harm from interpersonal violence.⁶⁰ Self-management digital tools were also created such as an arm and shoulder exercise tool for breast-cancer survivors,⁶¹ a mobile-optimized intervention web program for pain self-management among veterans,⁶² and a self-administered virtual reality experience for teenagers to reduce stress.⁶³

The second theme among reported study aims was offering various kinds of support to specific populations (22%, n=26). Six different articles focused on post-cancer treatment support, for example, a project to create a mobile health interventions for adolescent and young adult survivors of childhood cancer,⁶⁴ and for caregivers of cancer survivors.⁶⁵ Some digital interventions addressed other challenging situations, promoting safety for those experiencing domestic violence,⁶⁶ or fostering peer support for young people experiencing family discord.⁶⁷ Stress-related interventions, such as smartphone apps for people living with cancer⁶⁸ and smartphone apps for junior physicians,⁶⁹ were a sub-theme of the offering support category. Study aims of the remaining articles were wide-ranging and included research that examined chatbot support in South Korea,⁵⁰ that helped perpetrators in the United Kingdom to see other perspectives,⁷⁰ and that combated online child abuse in Namibia.⁷¹

In terms of the design processes used, “User-centered design” was predominant (48%, 55 articles), followed by “Co-design” (23%, 27 articles) and “Participatory design” (20%, 23 articles). Approximately 14% (16 articles) did not specify a particular design process. Other approaches included “Human-centered design” (9·8%, 11 articles), “Community-based participatory research” (5%, 6 articles), “Design thinking” (5%, 6 articles), “Co-creation” (4·5%, 5 articles), and “Participatory action research” (2%, 2 articles). Examples of processes included researchers in Australia who shared how they co-designed a culturally informed early-intervention wellbeing app with Aboriginal and Torres Strait Islander youth.⁷² Other studies contributed information on a community-engaged approach to develop a deliberation tool for pediatric patients and parents,⁷³ the formative development of bidirectional text messaging tool for those with substance use issues and HIV,⁷⁴ an Agile process to develop an app for smoking cessation for cancer patients,⁷⁵ and a multidisciplinary, user-centered approach for creating computer based assistive tools for Alzheimer patients.⁷⁶

Nearly 70% (n=81) of the studies explicitly mentioned making a “prototype” in the study as part of the design process. While reviewers did not extract data on the types of prototypes (digital or not), researchers reported the value of using paper-prototypes to co-design mobile-applications.⁷⁷ The number of digital solutions developed was typically one, with 84% (99 articles) reporting the development of a single solution, 10% (12 articles) developing two solutions, and 4% (5 articles) developing three or more solutions. We considered a digital solution to be a digital intervention developed for a particular platform, e.g., Android, iOS app, or web-based application. Note that solutions for different platforms, e.g., both iOS app and Android app were counted as two solutions. Only 2% (2 articles) did not specify the number of solutions developed.

More than half of the studies we reviewed (57%, n=67) designed the solution using a specific theory or framework. The remaining 51 studies did not reference using a specific theory or framework. A wide range of theories and frameworks were used in the 67 studies with Social Cognitive theory referenced the most (6 articles). Table 3 contains a list of the frameworks that were mentioned more than once. Note that inconsistent use of language, e.g., what researchers consider a framework, makes these results less precise. For example, many studies used user-centered design but only two called it a “framework” so it was not counted as a framework in the table below. Agile methodology was counted in the table but likely other studies may have used an Agile methodology in their iterative process but didn’t report it formally. Other theories and frameworks mentioned were wide-ranging from the fields of psychology, health, and technology—from the transtheoretical pain model⁶² to the Obesity-Related Behavioral Intervention Trials model⁷⁸ to the pedagogical usability framework.⁷⁹

Table 3.

Theory or framework and frequency of use.

Theory or framework	Frequency of use (%)
SCT	6 (5%)
Ottawa decision support framework	5 (4%)
Technology acceptance model	4 (3%)
Grounded theory	4 (3%)
Agile methodology	3 (3%)
C-SHIP model	2 (2%)
UTAUT	2 (2%)
Cognitive behavioral theory	2 (2%)

SCT: social cognitive theory; C-SHIP: cognitive-social health information processing; UTAUT: unified theory of acceptance and use of technology.

Discussion

Our results demonstrate that while many digital technology solutions are designed to serve populations that have typically experienced trauma, e.g., people with life-threatening cancer and those with substance use disorder, a TI framework with associated principles is not being used during research, design, and/or development. The one study that did reference TI principles⁷⁰ did so incidentally because the content focus of the smartphone app was trauma-related, not due to the conscious application of TI principles to the design of technology.

Most studies (85%, 102 articles) detailed specific design processes used during the development of the digital interventions. However, none mentioned a TI approach to design and only one study mentioned TI principles.⁵¹ Nor did any studies report using a TI approach in their research methods, though some studies used related principles and did not report in detail on them. For example, researchers stated they considered safety of participants without explaining how exactly safety was considered. However, many studies had theoretical frameworks or mentioned other approaches to design, for example user-centered design, participatory design, or co-design. We set out to report on the types of technology, organization types, and settings that are using TI principles in the design of technology development, but weren't able to find examples from the peer-reviewed literature. As a subfield, we have a long way to go in providing verified research to guide the practitioners creating technology for those who have experienced trauma.

The TI approach was designed for consumers of behavioral health services originally.^1,6,11 For this reason, it may be more directly translatable to the design of patient-facing technologies than the design of clinical-facing technology. Most of the studies reviewed (86%, 101 articles) were aiming to support individuals with lived experience of trauma. It's advantageous that the field of social work has already developed the TI framework that may be an efficient and appropriate approach to explore and use when designing for individual consumers.

Eight studies appeared to neglect the context of trauma and/or not report on it. For example, no mention of consideration of any TI principles was reported, in the development of a web-based education platform aiming to serve survivors of breast, colorectal, and prostate cancer⁸⁰ or the adaption of an informational and supportive care resource website for Hodgkin Lymphoma survivors.⁸¹ Overlooking the context of trauma may undermine the goals of these digital interventions. With more awareness, researchers may be able to be more explicit and intentional about methodically applying all applicable TI principles to better serve those populations they intend to help.

Following the user-centered design process, many research studies involved people with lived experience of trauma in the design and development of solutions which is likely to result in better digital interventions. Yet it is imperative to note that if researchers are conducting research, especially with people with lived experience of trauma, research practices must also be TI to avoid inadvertent harm. Researchers from the social work,⁸² health,⁸³ and criminal justice fields⁸⁴ offer important considerations and practical suggestions related to conducting research with a TI approach.

These findings suggest that TI design is not yet familiar or fully embraced by researchers, even public health researchers working in these areas with populations who experience significant trauma. This is an area ripe for opportunity and interdisciplinary collaboration. Most researchers are deliberately considering principles that are part of the TI approach during development. Instead of sporadic application of principles or creating a new framework, researchers can learn the TI approach and associated TI principles from SAMHSA and/or IITIC that have been developed by trauma experts, social work practitioners, and those with lived experience of trauma. These approaches can be applied at various stages of the design process as a harm reduction strategy. For example, in Kelly et al.⁸⁵ (excluded from this study since the university-wide intervention didn’t focus on a health topic), researchers and designers adapted TI principles to conduct a heuristic evaluation of a website under development. This methodical application of TI principles and subsequent discussions resulted both in changes to the website design and also helped the team of researchers and designers become more TI.⁸⁵ Following their example is a feasible way for even those unfamiliar with trauma or web design to begin thinking about applying TI approaches.

Conclusion

Once TI principles are applied, researchers can then conduct evaluations on whether the digital interventions being TI affects digital tool utilization and acceptance. This is a promising area to explore for digital interventions as it may increase the benefit to those receiving services. The thoughtful design of digital technology using TI principles can support many fields, including health care, mental health, and education. For example, the intended settings for the digital interventions reviewed varied between the general community and medical facilities, but only a handful of interventions were planned for use in schools. As an institution that serves the public, and with trauma pervasive in the general population, schools would benefit from TI design of technology.

While the individuals reporting on developing these digital interventions were predominately from higher education institutions, they often collaborated with government organizations, corporations (often technology companies) and nonprofits, and sometimes involved people with lived experience of trauma and mental health experts. Further collaboration among these diverse groups is encouraged, especially with those working most closely with the public such as teachers and nurses, whose pragmatic perspectives are essential.

The digital interventions examined in this review were mostly websites and mobile applications for smartphones with only a few articles focused on virtual reality. TI approaches can be applied to various types of technology, but research is needed on how TI features and characteristics might apply to the design of various types of technology. For example, virtual reality likely has additional considerations of safety as it simulates a three-dimensional space and may involve movement. AI was not seen in any of the studies reviewed, but AI is rapidly impacting various fields related to health–how might TI principles apply to digital AI tools used by consumers especially?

With this review, we hope to contribute to a better understanding of how the subfield of TI design is emerging. We encourage researchers to investigate the use and application of a TI approach and related principles when digital interventions are being created, especially when the populations may have experienced traumatic experiences. Based on our findings, trauma is not being directly and consistently addressed despite studies and solutions aiming to help those who have likely experienced trauma. Ignoring the context of trauma while creating digital interventions may have serious consequences, including traumatization, re-traumatization, or merely a bad experience that undermines the goal of the intervention. Fortunately, looking at the TI frameworks and associated principles already developed can provide us with an evidence-based approach to designing digital technology interventions in a more careful, ethical, and potentially effective way.

Limitations and strengths

Scoping reviews offer a broad overview of the existing literature on TI design, an emerging area of design. This study clearly highlights gaps that need attention. Our research identifies practical implications of research findings, which is crucial for fields like design where applying research to real-world situations is necessary. However, database searches were limited to those in English only and by publication date. Our review did not formally assess the quality of the included studies. After data extraction, we realized that our data extraction template erroneously listed the TI principle of “Empowerment & Choice” instead of “Empowerment, Voice, and Choice.” While data extractors were referencing the definitions of the principles, it's important for transparency to note this oversight. We also decided to avoid gray literature, including conference proceedings, due to the volume of peer-reviewed literature we encountered.

In a spirit of self-reflexivity, it is also important to note that our research team is all highly educated, all based in the United States, and hold Western perspectives on trauma that influence our work. Researchers had to use their best judgment during data extraction, identifying what TI principles were reflected in the studies, which could be challenging and required discussion and consensus. Strengths of the research team include that some of us are practicing designers and researchers with experience in the public health, healthcare, and technology fields. We are a multi-racial, multi-generational, multi-gendered research team who brought various perspectives to the study.

This scoping review reflects that there is little evidence in the literature of the formal application of TI principles to studies related to digital technology. This lack of accounting for trauma seems problematic, especially knowing how pervasive trauma is globally⁴ and particularly in sub-populations such as individuals with life-threatening cancer or substance use issues.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076251360925 - Supplemental material for A scoping review of trauma-informed care principles applied in design and technology

Supplemental material, sj-docx-1-dhj-10.1177_20552076251360925 for A scoping review of trauma-informed care principles applied in design and technology by Melissa Eggleston, Emily P Jones, Nashmia Khan, William A Romani, Kyle McQuillan, Jessica Otero and Elizabeth Chen in DIGITAL HEALTH

Footnotes

ORCID iDs

Melissa Eggleston

Emily P Jones

Nashmia Khan

Kyle McQuillan

Jessica Otero

Elizabeth Chen

Ethical considerations

Because this is a scoping review, there are no human participants in this article and informed consent is not required.

Author contributorship

Three authors were involved in conceptualizing the study—Melissa Eggleston, Emily Jones, and Elizabeth Chen

Four authors were involved in data curation and data analysis—Melissa Eggleston, Elizabeth Chen, Emily Jones, and Nashmia Khan

All seven authors were involved in data collection.

Three authors were responsible for methodology—Melissa Eggleston, Emily Jones, and Elizabeth Chen

One author was responsible for project planning and administration—Melissa Eggleston

Two authors were responsible for supervision and mentorship—Elizabeth Chen and Emily Jones

Two authors were responsible for validation work—Melissa Eggleston and Elizabeth Chen

Three authors were responsible for visualization work—Melissa Eggleston, Emily Jones, and Nashmia Khan

Three authors drafted the original article—Melissa Eggleston, Emily Jones, and Nashmia Khan

All seven authors were involved in the review and editing process

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Declaration of patient or public involvement

No patients or members of the public were involved in the design, conduct, or reporting of this scoping review.

Data availability

The data used for this review is available at

Supplemental material

Supplemental material for this article is available online.

References

Substance Abuse and Mental Health Services Administration. SAMHSA’s Concept of Trauma and Guidance for a Trauma-Informed Approach. HHS Publication No. (SMA) 14-4884. Rockville, MD: SAMHSA, 2014.

Bloom

. Advancing a national cradle-to-grave-to-cradle public health agenda. J Trauma Dissociation 2016; 17: 383–96.

Herman

. Truth and repair: How trauma survivors envision justice. UK: Hachette, 2023.

Benjet

Bromet

Karam

, et al. The epidemiology of traumatic event exposure worldwide: Results from the world mental health survey consortium. Psychol Med 2016; 46: 327–343.

Kessler

Aguilar-Gaxiola

Alonso

, et al. Trauma and PTSD in the WHO world mental health surveys. Eur J Psychotraumatol 2017; 8: 1353383.

Bloom

. Creating Sanctuary: Toward the Evolution of Sane Societies. Revised Edition. New York and London: Routledge, 2013.

Felitti

Anda

Nordenberg

, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The adverse childhood experiences (ACE) study. Am J Prev Med 1998; 14: 245–258.

Ford

Courtois

. Defining and understanding complex trauma and complex traumatic stress disorders. In: Ford

Courtois

(eds) Treating complex traumatic stress disorders in adults: Scientific foundations and therapeutic models. 2nd ed. New York: The Guilford Press, 2020, pp.3–34.

Van der Kolk

. The body keeps the score: Brain, mind, and body in the healing of trauma. New York: Viking, 2014.

10.

Substance Abuse and Mental Health Services Administration. Practical guide for implementing a trauma-informed approach. SAMHSA Publication No. PEP23-06-05-005. Rockville, MD: National Mental Health and Substance Use Policy Laboratory, Substance Abuse and Mental Health Services Administration; 2023.

11.

Harris

Fallot

. Envisioning a trauma-informed service system: a vital paradigm shift. New Dir Ment Health Serv 2001; 2001: 3–22.

12.

Levenson

. Trauma-informed social work practice. Soc Work 2017; 62: 105–113.

13.

Koury

Green

. Trauma-informed organizational change manual. Institute on Trauma and Trauma-Informed Care 2021: 26–29.

14.

National Association of Social Workers, Washington, Dc. Code of ethics of the National Association of Social Workers. NASW Distribution Center; 2021.

15.

Eggleston

Noel

. Repairing the harm of digital design using a trauma-informed approach. Diseña 2024;24: 7.

16.

Scott

Marcu

Anderson

, et al. Trauma-informed social media: towards solutions for reducing and healing online harm. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 2023 Apr 19, pp.1–20. New York: Association For Computing Machinery.

17.

Feinstein

. The relationship between socioeconomic status and health: a review of the literature. Milbank Q 1993; 71: 279–322.

18.

Link

Phelan

. Social conditions as fundamental causes of disease. J Health Soc Behav 1995: 80–94.

19.

World Health Organization Commission on Social Determinants of Health. Closing the gap in a generation: health equity through action on the social determinants of health. Final report of the Commission on Social Determinants of Health. Geneva: World Health Organization, 2008.

20.

World Health Organization Regional Office for Europe. Social determinants of health: the solid facts. 2nd ed. Wilkinson

Marmot

(eds) Copenhagen, Denmark: World Health Organization, 2003.

21.

International Telecommunication Union (ITU). Measuring Digital Development: Facts and Figures 2023. United Nations; 2023. Available from: https://www.itu.int/en/ITU-D/Statistics/Pages/facts/default.aspx. Accessed 2024 Jul 15.

22.

Chidambaram

Jain

, et al. An introduction to digital determinants of health. PLOS Digital Health 2024; 3: e0000346.

23.

Jahnel

Dassow

H-H

Gerhardus

, et al. The digital rainbow: digital determinants of health inequities. Digit Health 2022; 8: 20552076221129093.

24.

World Health Organization. Global strategy on digital health 2020–2025. Geneva: World Health Organization; 2021. Licence: CC BY-NC-SA 3·0 IGO

25.

Dahlgren

Whitehead

. The dahlgren-whitehead model of health determinants: 30 years on and still chasing rainbows. Public Health 2021; 199: 20–24.

26.

Krug

. Don't make me think, revisited: A common sense approach to web usability. 3. Indianapolis, IN, USA: New Riders, 2013.

27.

Nielsen

. Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, Boston, USA, 24 April 1994, pp.152–158. New York: Association for Computing Machinery.

28.

Meyer

Wachter-Boettcher

. Design for real life. New York, NY: A Book Apart, 2016.

29.

Rabaan

Dombrowski

. Survivor-Centered transformative justice: an approach to designing alongside domestic violence stakeholders in US muslim communities. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 23–26 April 2023, pp.1–19. New York: Association for Computing Machinery.

30.

Eno

Armstrong

Levitsky

, et al. How is a student to know who they can talk to?”: university website communication about sexual assault in the context of compelled disclosure. Rev High Educ 2023; 46: 373.

31.

Tseng

Freed

Engel

, et al. A digital safety dilemma: analysis of computer-mediated computer security interventions for intimate partner violence during COVID-19. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 8–13 May 2021, pp.1–17. New York: Association for Computing Machinery.

32.

Costanza-Chock

. Design justice: Community-led practices to build the worlds we need. Cambridge, MA, USA: The MIT Press, 2020.

33.

Holmes

. Mismatch: How inclusion shapes design. Cambridge, MA, USA: The MIT Press, 2018.

34.

Herman

. Trauma and Recovery. 2022 ed. New York, NY: Basic Books, 2022.

35.

Truss

Liao Siling

Phillips

, et al. Barriers to young people seeking help for trauma: a qualitative analysis of internet forums. Psychol Trauma: Theory, Res, Pract Policy 2023; 15: S163.

36.

World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 3·0. https://www.w3.org/TR/wcag-3·0/ . (2024, accessed 24 September 2024).

37.

Dietkus

. The call for trauma-informed design research and practice. Des Manag Rev 2022; 33: 26–31.

38.

Fathallah

. Trauma Responsiveness in Participatory Research. https://www.thinkof-us.org/blog/trauma-responsiveness-in-participatory-research . (2022, accessed 24 September 2024).

39.

Wechsler

. Trauma-informed design research. UX Collective on Medium. 2021. Available from: https://uxdesign.cc/trauma-informed-design-research-69b9ba5f8b08

40.

Winfield

. Keeping Survivors Safe During UX Research. Chayn. https://blog.chayn.co/keeping-survivors-safe-during-ux-research-eb20c1ee1f04 . (2023, accessed 24 September 2024).

41.

Bernius

Diektus

. Exploring Trauma-informed Research Practices. Code For America. https://codeforamerica.org/news/exploring-trauma-informed-research-practices (2023, accessed 24 September 2024).

42.

Eggleston

. Is Your Website Trauma-Informed? Melissa Eggleston. https://www.melissaegg.com/blog/is-your-website-trauma-informed (2017, accessed 24 September 2024).

43.

Kelly

Lauren

Nguyen

. Trauma-informed web heuristics for communication designers. In: Proceedings of the 39th ACM International Conference on Design of Communication, Virtual, 12 October–14 October 2021, pp. 172–176. New York, USA: Association for Computing Machinery.

44.

Slate

. Trauma Informed Design Course. User Experience Professionals Association. https://uxpa.org/product/trauma-informed-design/ . (2021, accessed 13 February 2024).

45.

Storer

Scott

Eggleston

, et al. Reimagining social work’s digital future: the critical role of interdisciplinary tech partnerships. J Soc Work Educ 2023; 59: S105–S120.

46.

Arksey

O'Malley

. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8: 19–32.

47.

Tricco

Lillie

Zarin

, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169: 467–473..

48.

Veritas Health Innovation. Covidence. Melbourne, Australia: Veritas Health Innovation; 2024. Available from: www.covidence.org.

49.

Honary

McNaney

Lobban

. Designing video stories around the lived experience of severe mental illness. In: Proceedings of the 10th Nordic Conference on Human-Computer Interaction, Oslo Norway, 29 September–3 October 2018, pp. 25–38. New York, USA: Association for Computing Machinery.

50.

Maeng

Lee

. Designing and evaluating a chatbot for survivors of image-based sexual abuse. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, Virtual, Online, 30 April–5 May 2022, p. 344, pp.1-21. New York, USA: Association for Computing Machinery.

51.

Forehand

Schisler-Blizzard

Deaver

, et al. Triangulating perspectives to inform the development of a smartphone application for foster, kinship, and adoptive parents. J Technol Hum Serv 2019; 37: 362–394.

52.

Silva

Malta

da Silva

, et al. The Dandarah App: an mHealth platform to tackle violence and discrimination of sexual and gender minority persons living in Brazil. Int J Environ Res Public Health 2022; 20: 280.

53.

Claborn

Creech

Whittfield

, et al. Ethical by design: engaging the community to co-design a digital health ecosystem to improve overdose prevention efforts among highly vulnerable people who use drugs. Front Digit Health 2022; 4: 880849.

54.

Dimeff

Jobes

Chalker

, , et al. A novel engagement of suicidality in the emergency department: virtual collaborative assessment and management of suicidality. Gen Hosp Psychiatry 2020; 63: 119–126.

55.

Rizia

Johnson

Patwary

ABMK

, et al. Mobile peer-mentoring: an approach to make veterans seek mental health-care support a normality. In: Proceedings of the 10th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, Miami, USA, 22 October–25 October 2015, pp. 437–441. IEE.

56.

Asarnow

Zullo

Ernestus

, et al. “Lock and Protect”: Development of a Digital Decision Aid to Support Lethal Means Counseling in Parents of Suicidal Youth. Front Psychiatry 2021; 12.

57.

Woodard

Hoffman

Covarrubias

, , et al. The pathways fertility preservation decision aid website for women with cancer: development and field testing. J Cancer Surviv 2018; 12(1): 101–114.

58.

Birrell

Deen

Champion

, et al. A mobile app to provide evidence-based information about crystal methamphetamine (Ice) to the community (cracks in the ice): co-design and beta testing. JMIR Mhealth Uhealth 2018; 6: e11107.

59.

Bevan Jones

Thapar

Rice

, , et al. A web-based psychoeducational intervention for adolescent depression: design and development of MoodHwb. JMIR Ment Health 2018; 5: e13.

60.

O’Campo

Velonis

Buhariwala

, et al. Design and development of a suite of intimate partner violence screening and safety planning web apps: user-centered approach. J Med Internet Res 2021; 23: e24114.

61.

Harder

Holroyd

Burkinshaw

, et al. A user-centred approach to developing bWell, a mobile app for arm and shoulder exercises after breast cancer treatment. J Cancer Surviv 2017; 11: 732–742.

62.

Johnson

Levesque

Broderick

, et al. Pain self-management for veterans: development and pilot test of a stage-based Mobile-optimized intervention. JMIR Med Inform 2017; 5: e40.

63.

Björling

Cicero

Sankar

, et al. Thought disposal: co-designing a virtual interaction to reduce stress in teens. Proceedings of the 18th ACM International Conference on Interaction Design and Children, IDC 2019. 2019; pp. 562–567. New York, USA: Association for Computing Machinery.

64.

Schwartz

Psihogios

Henry-Moss

, et al. Iterative development of a tailored mHealth intervention for adolescent and young adult survivors of childhood cancer. Clin Pract Pediatr Psychol 2019; 7: 31–43.

65.

Oakley-Girvan

Davis

Kurian

, et al. Development of a Mobile health app (TOGETHERCare) to reduce cancer care partner burden: product design study. JMIR Form Res 2021; 5: e22608.

66.

Orji

Hernandez

Selema

, et al. Tidyhome: a persuasive app for supporting victims and survivors of domestic violence. In: International Conference on Human-Computer Interaction 2022, Virtual, 26 June–1 July 2022, pp. 400–415. Cham: Springer International Publishing.

67.

Campbell

Ridout

Amon

, et al. A customized social network platform (kids helpline circles) for delivering group counseling to young people experiencing family discord that impacts their well-being: exploratory study. J Med Internet Res 2019; 21: e16176.

68.

Børøsund

Mirkovic

Clark

, et al. A stress management app intervention for cancer survivors: design, development, and usability testing. JMIR Form Res. 2018; 2: e19.

69.

Counson

Bartholomew

Crawford

, et al. Development of the shift smartphone app to support the emotional well-being of junior physicians: design of a prototype and results of usability and acceptability testing. JMIR Form Res 2021; 5: e26370.

70.

Bellini

Forrest

Westmarland

, et al. Choice-Point: fostering awareness and choice with perpetrators in domestic violence interventions. In: Proceedings of the 2020 ACM CHI Conference on Human Factors in Computing Systems, Honolulu, USA, 25–30 April 2020, Paper no. 3376386, pp. 1–14. New York, USA: Association for Computing Machinery.

71.

Mikka-Muntuumo

Peters

. Designing an interactive game for preventing online abuse in Namibia. In: 2021 3rd International Multidisciplinary Information Technology and Engineering Conference, IMITEC 2021, Windhoek, Namibia, 23-25 November 2021, pp. 1–6. IEEE.

72.

Povey

Sweet

Nagel

, et al. Drafting the aboriginal and islander mental health initiative for youth (AIMhi-Y) app: results of a formative mixed methods study. Internet Interv 2020; 21: 100318.

73.

Moultrie

Lewis

Paquin

, et al. An evidence-based, community-engaged approach to develop an interactive deliberation tool for pediatric neuromuscular trials. J Genet Couns 2018; 27: 416–425.

74.

Ingersoll

Dillingham

Reynolds

, et al. Development of a personalized bidirectional text messaging tool for HIV adherence assessment and intervention among substance abusers. J Subst Abuse Treat 2014; 46: 66–73.

75.

Bricker

Watson

Heffner

, et al. A smartphone app designed to help cancer patients stop smoking: results from a pilot randomized trial on feasibility, acceptability, and effectiveness. JMIR Form Res 2020; 4: e16652.

76.

Gowans

Dye

Alm

, et al. Designing the interface between dementia patients, caregivers and computer-based intervention. Design Journal 2007; 10: 12–23.

77.

Cardoso

Hawk

Cross

. What young people need to make better-informed decisions when communicating with digital images: implications for mental health and well-being. Health Educ Behav 2020; 47: 29–36.

78.

Dorfman

Shelby

Stalls

, et al. Improving symptom management for survivors of young adult cancer: development of a novel intervention. J Adolesc Young Adult Oncol 2023; 12: 472–487.

79.

Hughes

Musselman

Walsh

, et al. The mPOWERED electronic learning system for intimate partner violence education: mixed methods usability study. JMIR Nurs 2020; 3: e15828.

80.

O’Malley

Davis

Devine

, et al. Development and usability testing of the e-EXCELS tool to guide cancer survivorship follow-up care. Psychooncology 2020; 29: 123–131.

81.

Amweg

McReynolds

Lansang

, et al. Hodgkin lymphoma survivor wellness: development of a web-based intervention. Clin J Oncol Nurs 2020; 24: 284–289.

82.

Voith

Hamler

Francis

, et al. Using a trauma-informed, socially just research framework with marginalized populations: practices and barriers to implementation. Soc Work Res 2020; 44: 169–181.

83.

Edelman

. Trauma and resilience informed research principles and practice: a framework to improve the inclusion and experience of disadvantaged populations in health and social care research. J Health Serv Res Policy 2023; 28: 66–75.

84.

Houston-Kolnik

Vasquez

. Cognitive interviewing: lessons learned and recommendations for structured interviews with survivors of crime. J Fam Violence 2022; 37: 325–335.

85.

Kelly

Lauren

Nguyen

. Trauma-informed web heuristics for communication designers. In: Proceedings of the 39th ACM International Conference on Design of Communication, Virtual, Online, 12–14 October, 2021. pp. 172–176. New York, USA: Association for Computing Machinery.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB