Sage Journals: Discover world-class research

Abstract

The purpose of this study is to describe how a mixed methods approach was used to gain a better understanding of functioning in schizophrenia. A two-phase design was employed. In the first phase, four independent qualitative and quantitative preparatory studies were concurrently carried out to identify areas of convergence. In the second phase, we held a consensus conference with an international panel of experts to explore how these preparatory studies contributed to the final list of areas of functioning in schizophrenia. The data of the preparatory studies were complementary, and the qualitative methodology (i.e., focus groups with patients and families) was the main contributor to the final list. The experience of the conference of experts highlights the importance of the consensus process for capturing a range of cultural differences.

Keywords

community based research focus groups mixed methods qualitative meta-analysis/synthesis secondary data analysis

Introduction

In mixed methods research, elements of qualitative and quantitative research approaches are combined to obtain breadth and depth of understanding and corroboration (Granikov et al., 2020; Johnson & Onwuegbuzie, 2007). One of the main benefits of mixed methods studies is that they can offer a richer description of a phenomenon by providing a holistic view point (Poth and Onwuegbuzie, 2015), so that the weaknesses of one method are compensated by the strengths of the other methods (Johnson & Onwuegbuzie, 2007; Muñoz Poblete, 2013). The correct integration of the qualitative and quantitative components is therefore crucial (Granikov et al., 2020), and there are different integration designs depending on the study objectives (i.e., models to develop instruments, explanatory models, triangulation models, and longitudinal transformation models) (Hamui-Sutton, 2013). Mixed methods approaches have been used to create high quality measurement instruments that can evaluate a construct of interest in its entirety (David et al., 2016). To give some examples, Keetharuth et al. (2018) developed an approach based on a traffic light pictorial format to present quantitative and qualitative data to determine the content validity of a scale; Benítez et al. (2016) obtained a comprehensive perspective of functioning by combining information from psychological scales and focus groups with patients and relatives; and other authors used the Delphi methodology to develop scales based on mixed methods (Taghipoorreyneh and de Run, 2020). The present study incorporates a mixed methods approach to acquire a better understanding of functioning in schizophrenia as an initial step for developing an assessment tool.

Schizophrenia is a severe psychiatric disorder characterized by positive symptoms (e.g., hallucinations and delusions), negative symptoms (e.g., blunted affect) and cognitive impairments (e.g., social cognition deficits) (American Psychiatric Association, 2013; Birnbaum et al., 2018; Singh et al., 2016). People who live with schizophrenia commonly experience significant impairments in multiple areas of daily functioning, including personal care, personal relationships, education, and employment (Bozzatello et al., 2018; Cook et al., 2009; Foruzandeh and Parvin, 2013); this is why schizophrenia is considered a highly disabling mental illness (Ferrari et al., 2016). In this sense, schizophrenia has a high social and economic burden, not only for people with a diagnosis of schizophrenia, but also for their families, caregivers, and the wider society (Charlson et al., 2018; Chong et al., 2016).

A multidisciplinary treatment approach (e.g., pharmacological, psychoeducation, physical activity, psychosocial functioning training) can significantly improve clinical outcomes for patients with schizophrenia (Deenik et al., 2018). In any of the treatment programs, daily-life functioning plays an essential role, and consequently, areas of functioning need to be comprehensively assessed and later treated (Si et al., 2019). Although there are instruments for evaluating cognitive, behavioral and social functioning, a multidisciplinary assessment tool for evaluating functioning in patients with schizophrenia that efficiently captures all the relevant areas of functioning is still necessary (Barrios et al., 2019).

The International Classification of Functioning, Disability, and Health (ICF), developed by the WHO (2001), offers a framework for developing such an assessment tool. The ICF provides a common and universal language for describing functioning, disability, and health in people with any health condition. It is an internationally accepted classification standard based on a biopsychosocial model, in which the areas of functioning related to a health condition are understood according to the following components: (1) Body functions and structures, (2) Activities and participation, (3) Environmental factors, and (4) Personal factors (WHO, 2001). These areas of functioning are represented by + 1400 categories, hierarchically arranged according to levels of detail, which are identified using an alphanumeric code (i.e., the letter refers to each component and the number refers to the specific category).

However, since not all ICF categories are relevant for a specific health condition, ICF Core Sets (ICF-CSs) have been developed for specific health conditions, including schizophrenia (Gómez-Benito et al., 2017). The World Health Organization’s guidelines propose combining four different perspectives (i.e., researchers, experts, clinicians, and people diagnosed with schizophrenia and their relatives and caregivers) in the selection of core elements of functioning (Selb et al., 2015). In the context of schizophrenia, it has been widely recognized that the inclusion of diverse perspectives from an interdisciplinary team of experts is essential. This team usually comprises professionals including psychiatrists, psychologists, nurses, occupational therapists, social workers, and physiotherapists) (Nuño et al., 2021). Additionally, studies have underscored the significance of considering the viewpoints of patients, caregivers, and experts (Jacob et al., 2017). This is crucial because the needs of individuals affected by schizophrenia and their families tend to be systematically overlooked (Alyahya et al., 2022; Davidson et al., 2006).

Briefly, the process of developing the ICF-CSs consisted in two phases. In a first phase, four independent preparatory studies were carried out (i.e., three studies applying a qualitative methodology and one study applying a quantitative methodology), each representing a different perspective (i.e., researchers, experts, clinicians, and people diagnosed with schizophrenia and their relatives and caregivers). In a second phase, a consensus conference was held in which an international panel of experts, based on the results of the previous four preparatory studies, agreed on which categories should be represented in the ICF-CSs.

The Comprehensive ICF-CS for schizophrenia includes 97 categories that cover the spectrum of typical problems in functioning that people living with schizophrenia experience. The Brief ICF–CS is a subset of 25 of these 97 categories considered essential for evaluating and treating this population. This design, characterized by combining qualitative and quantitative methodologies, provided an ideal scenario for applying a mixed methods approach to assess whether the ICF-CSs for schizophrenia capture all the highly relevant areas of functioning in schizophrenia. More specifically, this two-phase study allowed us to integrate the data collected in the four preparatory studies (first phase) and compare it to the ICF-CSs for schizophrenia that resulted from the consensus conference of experts (second phase).

The research questions of the present study are:

1. How do the results of the four preparatory studies complement each other?

2. How did each preparatory study influence the development of the ICF-CSs for schizophrenia during the second phase?

Method

This study applies a mixed methods approach defined as a type of research that combines elements from qualitative and quantitative studies in order to obtain a profound understanding and to corroborate data (Schoonenboom and Johnson, 2017). Specifically, we adopted a multiphase mixed methods approach because our study assesses qualitative and quantitative data in different phases over a period of time (Creswell and Plano, 2011).

First phase: Preparatory Studies

Between 2013 and 2015, four independent preparatory studies were conducted to collect data on relevant areas of functioning in schizophrenia, reflecting different and complementary perspectives (i.e., researchers, experts, clinicians, and people with schizophrenia and their relatives and caregivers). For the researchers’ perspective, a systematic review was conducted to analyze areas of functioning identified in 206 papers published in peer reviewed journals (Gorostiaga et al., 2017). The experts’ perspective was reflected with an internet-based survey with participants representing different disciplines (i.e., psychologists, psychiatrists, nurses, social workers, and occupational therapists) from the six WHO regions (i.e., Africa, Eastern Mediterranean, Europe, South-East Asia, The Americas, and Western Pacific) (Barrios et al., 2017). The clinical perspective was captured by conducting a multi-center empirical study at five different health centers, in which clinicians filled out a standardized record form on the patients' functioning (Barrios et al., 2018). Lastly, the perspective of people with schizophrenia and their relatives and caregivers was captured by conducting focus groups with individuals with schizophrenia, relatives, and caregivers in hospitals in Spain and the USA (Benítez et al., 2016). A total of 11 focus groups have been included (seven with patients, three with relatives and one with institutional caregivers) using a topic guide of six pre-defined open-ended questions covering all of the ICF components (Gómez-Benito et al., 2017). These four preparatory studies were performed to pre-select a set of so-called “candidate” ICF categories that were considered the most relevant for describing the functioning of people with schizophrenia.

Second phase: Consensus Conference to Develop the ICF-CSs for Schizophrenia

In 2015, an international consensus conference was held in Barcelona (Spain) with a panel of 20 experts on schizophrenia representing five of the six WHO regions (i.e., all but South-East Asia) and diverse professional backgrounds (i.e., psychiatrists, psychologists and neuropsychologists, nurses, social workers, and occupational therapists). All of the experts had a minimum of five years of experience in research and/or treatment or care of people with schizophrenia. In this meeting, the set of candidate ICF categories identified in the preparatory studies was presented to the participants, who decided on the final list of categories to be included in the ICF-CSs for schizophrenia in an iterative decision-making and consensus process (Gómez-Benito et al., 2017).

Contribution to Mixed Methods Research

The type of data collected in each preparatory study represents the complementary perspectives of the researchers, experts, clinicians, and individuals with schizophrenia, relatives and caregivers (Gómez-Benito et al., 2017). The three professionals’ perspectives are reflected in the quantitative data, while the perspective of the individuals with schizophrenia, relatives and caregivers is captured with qualitative data. Therefore, the main purpose for implementing a mixed methods approach is for verifying the complementation and development of the preparatory studies (first phase) and the creation of the ICF-CSs for schizophrenia (second phase). For this study, complementation refers to clarifying and illustrating the results from one method (quantitative) with the results from another method (qualitative), while development refers to using the results to develop or clarify a construct of interest (Greene et al., 1989). Therefore, by using a mixed methods approach, the present study could provide an answer to the two research questions (i.e., how do the results of the four preparatory studies complement each other? How did each preparatory study influence the development of the ICF-CSs for schizophrenia during the second phase?).

Design

A multiphase mixed methods design was implemented. Multiphase designs combine concurrent and sequential approaches and are typically implemented when an individual or group of researchers aim to examine a topic or problem by integrating quantitative and qualitative data in which each stage is built on what has been previously learned (Granikov et al., 2020). Multiphase designs are useful for evaluating programs, developing measurement instruments, and carrying out large-scale studies (Creswell and Plano, 2011). In the present study, the preparatory studies (first phase) were implemented to obtain information for developing the ICF-CSs (second phase). Figure 1 shows the multiphase mixed methods design implemented.

Figure 1.

The multiphase mixed methods approach.

Timing

Preparatory studies of the first phase were implemented concurrently and independently. Concurrence refers to the fact that both the quantitative and qualitative components were performed simultaneously, while independence implies that the implementation of the studies did not depend on the results of the other components (Schoonenboom and Johnson, 2017). Therefore, the preparatory studies were conducted at the same time, without interfering with each other.

On the contrary, the development of the ICF-CSs for schizophrenia in the second phase is considered to be a sequential and dependent approach (Schoonenboom and Johnson, 2017). It is a sequential approach since it was preceded by another study or set of studies, and it is dependent because the development of the ICF-CSs depends on the results of the preparatory studies.

Point of Interference

The point of interference is the step within the research process where the quantitative and qualitative data are brought together (Creswell and Plano, 2011), typically selected at the time of data analysis. This data analysis point of interference is applied when the qualitative and quantitative data are analyzed independently and are then brought together (Creswell and Plano, 2011). In this study, it was constituted by the number of categories that emerged from each preparatory study, as discussed below.

Data Analysis

The meaningful concepts collected in the systematic review, expert survey, and focus groups were linked to the corresponding ICF categories according to established linking rules (Cieza et al., 2005). In the case of the empirical study, ICF categories were extracted directly. In all these preparatory studies, we obtained the frequency and percentage of each ICF category. The categories that showed a frequency of at least 5% in the systematic literature review, the expert survey, and the empirical study were selected to be included in a set of candidate categories, which were presented in the consensus conference. In the qualitative study, however, those categories that appeared in at least one focus group were included in the set of candidate categories.

To explore how the results of the four preparatory studies complement each other (Research question 1), we obtained the number of categories that emerged from each preparatory study as a whole and by component. In addition, the percentages of uniqueness of each preparatory study and commonalities and discordances were calculated between the four studies, two by two.

To study how each preparatory study influenced the development of the ICF-CSs for schizophrenia (Research question 2), the contribution of each study was explored in terms of frequency and percentage of candidate categories that were finally included in the ICF-CSs. Furthermore, for each ICF component, the percentage of occurrence of each category was averaged between the different preparatory studies, allowing us to assess the contribution of each category to the development of the ICF-CSs. The data were processed and analyzed with the R software.

Results

Research question 1: How do the results of the four preparatory studies complement each other?

One-hundred and eighty-four unique ICF categories were identified in the four preparatory studies. Table 1 shows that Body functions were represented by 52 ICF categories (28.3%), Body structures by 13 (7.1%), Activities and participation by 70 (38.0%) and Environmental factors by 49 ICF categories (26.6%). As shown in Table 1, the number of categories that emerged from each preparatory study and their distribution across the ICF components differed considerably between them. Specifically, the highest number of categories were derived from the focus groups (n = 149), followed by the expert survey and the empirical study (n = 95), and finally the systematic review (n = 70).

Table 1.

Number of Categories That Were Obtained in Each Preparatory Study and Their Distribution Across ICF Components.

Perspective/Source	ICF categoriesN	Body functions n (%a, %^b)	Body structures	Activities and participation	Environmental factors
Researchers: Systematic review	70	30 (42.9%, 57.7%)	2 (2.9%, 15.4%)	34 (48.6%, 48.6%)	4 (5.7%, 8.2%)
Experts: Expert survey	95	19 (20.0%, 36.5%)	7 (7.4%, 53.8%)	33 (34.7%, 47.1%)	36 (37.9%, 73.5%)
Clinicians: Empirical study	95	32 (33.7%, 61.5%)	0 (.0%, .0%)	45 (47.4%, 64.3%)	18 (18.9%, 36.7%)
People with schizophrenia, relatives, and caregivers: Focus groups	149	45 (30.2%, 86.5%)	6 (4.0%, 46.2%)	60 (40.3%, 85.7%)	38 (25.5%, 77.6%)
Total	184	52 (28.3%^c)	13 (7.1%)	70 (38.0%)	49 (26.6%)

^aPercentage based on the total number of ICF categories identified in each preparatory study.

^bPercentage based on the total number of ICF categories identified in each component.

^cPercentage based on the total number of ICF categories in the four preparatory studies (n = 184).

The relevance of each component varied substantially across the preparatory studies. While the expert perspective (expert survey study) mainly focused on Activities and participation categories and Environmental factors, the scientific literature perspective (systematic review study) and clinicians (empirical study) highlighted Body functions and Activities and participation categories. Finally, people with schizophrenia, relatives, and caregivers (focus groups) described functioning and disability mainly distributed in three components: Body functions, Activities and participation, and Environmental factors.

Table 2 shows the uniqueness of each preparatory study and commonalities and discordances between the preparatory studies. The focus groups study with patients, relatives, and caregivers (n = 43; 23.4%) was the preparatory study that contributed the most to the list of ICF categories. A total of 42 ICF categories (22.8%) were common across all preparatory studies (i.e., 17 Body functions, 21 Activities and participation, and 4 Environmental factors). Two-by-two percentages of common categories identified by the different preparatory studies ranged between .5% (systematic review vs. expert survey) and 9.8% (empirical study vs. focus group).

Table 2.

Number and Percentage of Categories Uniquely Identified by Each Preparatory Study (Diagonal) and of Common (Green) and Discordant (red) Categories Between Them.

Research question 2: How did each preparatory study influence the development of the ICF-CSs for schizophrenia during the second phase?

We examined the contribution of the preparatory studies to the consensus process for developing the ICF-CSs for schizophrenia. As shown in Table 3, from the 184 categories that were identified in the preparatory studies (first phase), 97 categories (52.7%) were selected for the ICF-CSs for schizophrenia (second phase), and the contribution ranged from 51.1% of the categories identified in the systematic review to 93.8% of those identified in the focus groups. If we focus on the different total number of ICF categories identified in the preparatory studies, each study made a similar contribution to the development of the ICF-CSs for schizophrenia, with percentages of included categories ranging from 61.0%, in the study with focus groups (i.e., 91 categories out of the 149 identified), to 71.5%, in the expert survey (i.e., 68 out of the 95 identified).

Table 3.

Contribution of the Preparatory Studies (First phase) to the ICF-CSs (Second phase).

Perspective/Source	Categories from the preparatory studies (1^st phase)	Contribution to the ICF-CSs (2^nd phase)
Perspective/Source	n categories	n categories (%^a, %^b)
Researchers: Systematic review	70	50 (71.4%, 51.5%)
Experts: Expert survey	95	68 (71.5%, 70.1%)
Clinicians: Empirical study	95	65 (68.4%, 67.0%)
Persons with schizophrenia, relatives, and caregivers: Focus groups	149	91 (61%, 93.8%)
Total	184	97 (52.7%)

^aPercentage based on the total number of ICF categories identified in each preparatory study.

^bPercentage based on the total number of ICF categories included in the second phase (n = 97).

However, the contribution of the preparatory studies to the selection of categories of the ICF-Cs varied by component (see Figure 2).

Figure 2.

Contribution of the preliminary studies (first phase) to the ICF-CSs for schizophrenia (second phase). Categories from the Body functions component (Panel A), Activities and participation (Panel B) and Environmental factors (Panel C). Note. o = Categories included the ICF-CSs that did not emerge from all four preparatory studies; x = Categories that emerged from all four preparatory studies, but were not included in the ICF-CSs; Categories not marked represent those categories that emerged from all four preparatory studies and were included in the ICF-CSs. All = Preparatory studies; ES = Empirical study; Esurvey = Expert survey; FG = Focus groups; SR = Systematic review.

Of all the categories from the Body functions component that were selected for the ICF-CSs (n = 17), 14 of them (82%) emerged in all preparatory studies, with the contribution percentages of each study ranging from 18% to 85% (Figure 2(A)). This result indicates that most of the Body functions categories selected for the ICF-CS were reported by all preparatory studies, suggesting a high congruence between studies for this component. Three categories (18%) that were included in the ICF-CSs for schizophrenia were only provided by the focus groups, the empirical study, and the systematic review, with contribution percentages of each study ranging from 25% to 39%. Moreover, three categories that emerged from all four preparatory studies were not included in the ICF-CSs for schizophrenia, despite some of their contribution percentages being relatively high (e.g., the category b126 Temperament and personality functions, which obtained a contribution percentage of 70% but was not finally selected for the ICF-CSs).

From all the Activities and participation categories that were selected for the ICF-CSs (n = 48), 20 (42%) were provided by all preparatory studies, with contribution percentages ranging from 26% to 69% (Figure 2(B)). Thus, the remaining 28 categories (58%) that were selected for the ICF-CSs did not appear in all studies, with study contribution percentages ranging from 2% to 31%. In addition, one category that was provided by all preparatory studies was not included in the ICF-CSs (i.e., d550 Eating).

Of all the categories from the Environmental factors component that were selected for the ICF-CSs (n = 32), three of them (9%) were provided by all preparatory studies, with contribution percentages ranging from 68% to 84% (Figure 2(C)). This result indicates low congruence between preparatory studies in categories from this component. Furthermore, one category was provided by all preparatory studies (i.e., e115 Products and technology for personal use in daily living) but was not included in the ICF-CSs for schizophrenia.

Discussion

The current study aimed to identify relevant areas of functioning in schizophrenia based on different perspectives (i.e., systematic review to obtain the researchers’ perspective, expert survey study with specialists in schizophrenia, empirical study with clinicians, and focus groups with people diagnosed with schizophrenia, relatives, and caregivers), and analyze the contribution of each perspective to the development of the ICF-CSs for schizophrenia. To achieve this objective, a mixed methods approach was employed using a two-phase design. In the first phase, four independent qualitative and quantitative preparatory studies were carried out concurrently. In the second phase, a consensus conference with an international panel of experts was held, from which a consensual list of relevant areas of functioning in schizophrenia (i.e., ICF-CSs for schizophrenia) was obtained. In this second phase, we explored how these preparatory studies contributed to the decision about which categories should be included in the ICF-CSs for schizophrenia.

A noteworthy contribution of this study is that every area of functioning identified in each preparatory study was translated into the ICF language, which facilitates comparison across studies. ICF is gaining increasing worldwide acceptance and provides a universal language for professionals who deal with functioning and disability in different health conditions, including schizophrenia (Gómez-Benito et al., 2017). In addition, we followed a rigorous data collection methodology (Selb et al., 2015) to gather information for developing the ICF-CSs for schizophrenia. We also carried out rigorous analyses based on mixed methods to construct scales (Taghipoorreyneh and de Run, 2020), which have made it possible to correctly integrate and complement the available quantitative and qualitative data (Granikov et al., 2020).

In the first phase of the study, several ICF categories were identified in the preparatory studies to describe functioning in schizophrenia. The data showed that all preparatory studies complemented each other, since although a considerable number of ICF categories were agreed on by all preparatory studies, each perspective provided a unique set of ICF categories. Thus, each preparatory study highlighted different areas of functioning, showing the importance of including different perspectives to obtain a complete view of functioning and disability experienced by individuals with schizophrenia (Matvienko-Sikar et al., 2020; Selb et al., 2015).

Our results also show that, although several categories were reported in all four preparatory studies, the frequency with which an ICF category was identified in each preparatory study varied substantially across studies. This indicates that, despite some functioning areas being considerably relevant in all four preparatory studies, their importance varied depending on whose perspective was being considered (i.e., categories that appeared in at least one focus group were included, while in the rest of the quantitative studies they had to appear in at least 5%). In this sense, the focus groups of patients, relatives and caregivers provided a substantially higher number of categories than the remaining preparatory studies, placing particular emphasis on Activities and participation (Schneider, 2012). In line with this result, Benítez et al. (2016) found evidence that there are variables that can only be gathered after the deep description provided by participants or relatives in focus groups, which facilitate going directly to the root of the problem. On the other hand, researchers, and clinicians, in addition to Activities and participation, also attached importance to the Body functions component, just as experts did with Environmental factors. The fact that none of the four preparatory studies was able to capture all the final categories included, firstly, highlights the need to consider all relevant perspectives when the functional profile of individuals with schizophrenia is described; and, secondly, evidences that the different perspectives emphasize and prioritize certain aspects of functioning but not others. Thus, experts, clinicians, persons with schizophrenia, and relatives and caregivers can provide complementary views of the disease that could be essential when goal-oriented interventions are planned.

In the second phase of the study, the results showed that, despite the overall similarity of the contribution with respect to the number of categories initially identified in each preparatory study to the development of the ICF-CSs for schizophrenia, this varied depending on which ICF component was considered.

In the case of the Body functions component, preparatory studies seemed consistent in their contribution to the ICF-CSs, since almost all the categories selected for the ICF-CSs were provided by all of the studies, with high agreement in the categories related to b1 Mental Functions. This may be a consequence of evidence consistently found in scientific research that disruption in a cognitive process can disturb the functioning of a person diagnosed with schizophrenia (Conn et al., 2020; Keefe et al., 2016).

A considerable proportion of Activities and participation categories from the preparatory studies was finally included in the ICF-CSs for schizophrenia. However, there were differences between preparatory studies. Many categories emerged in some but not all preparatory studies. In the categories that coincided in the four preparatory studies (mostly from d7 Interpersonal interactions and relationships) the percentage of agreement was relatively high compared to the Body functions component. The variation between preparatory studies can be explained by the multidisciplinary approach (e.g., researcher, people with schizophrenia, etc.) that has been used to create the ICF-CSs. On a positive note, this mixed methods approach was beneficial because it managed to identify categories that in another study would not have been specified. In addition, this reflects that schizophrenia has a huge implication in everyday functioning, in what a person can do and what they cannot do (Slade and Longden, 2015). This also suggests that other areas beyond clinical symptoms (e.g., positive and negative symptomatology), should be assessed and treated, such as the promotion of healthier lifestyles (e.g., decreasing sedentary behavior, increasing physical activity and improving dietary habits) (Deenik et al., 2018).

Likewise, regardless of the perspective considered, it has been reported that there are certain sections of the ICF in which hardly any categories have been included, as is the case of body structures (Nuño et al., 2019). This can be attributed to the fact that we are facing a mental illness, with a predominance of behavioral disorders. Further, the observed structural alterations in schizophrenia are predominantly limited to the brain (Naim-Feil et al., 2018), despite being substantial differences among individuals in terms of the specific brain regions affected (Brugger and Howes, 2017).

For the Environmental factors component, only a small portion of the categories that were provided by all of the preparatory studies (9%) were finally included in the ICF-CSs for schizophrenia. Thus, this emphasizes that health professionals need to recognize environmental factors as an important aspect in relevant areas of functioning (Galderisi et al., 2018), both in the evaluation process and in treatment design. In schizophrenia, environmental factors, including sociocultural factors, are of particular importance in prognostic features (Gurak and Weisman de Mamani, 2016; Niendam et al., 2018). Therefore, our study has tried to take into account different sociocultural settings, integrating an international perspective of the disease from the six WHO regions. There were experts from the six WHO regions in the consensus conference with the intention of capturing all these cultural differences. This highlights the importance of the consensus process among the experts during the consensus conference (Peuskens and Gorwood, 2012).

In conclusion, the ICF-CSs for schizophrenia were created by combining mixed methods and a multidisciplinary approach. Moreover, involving researchers, experts, and clinicians as well as patients, family members and caregivers has made it possible to obtain complementary visions of functioning, avoiding the omission of certain relevant categories. In addition, mixed methods allowed us to obtain a broad view of relevant areas of functioning by combining quantitative and qualitative data from the four preparatory studies. In this regard, we found that the focus group study (patients’ and caregivers’ perspective) was the study that presented the largest number of categories. Concurring with the conclusions of Benítez et al. (2016), qualitative data can play an important role in understanding the areas of functioning in patients with schizophrenia. Moreover, it is confirmed that this mixed methods approach helped in the process of developing the ICF-CSs for schizophrenia, implying that mixed methods studies are useful for developing an assessment instrument (David et al., 2016; Taghipoorreyneh and de Run, 2020). Therefore, this research is proof that mixed methods studies can be used to obtain a broad view of a given phenomenon (Granikov et al., 2020).

Limitations

There could be some issues about the applicability of the ICF-CSs for schizophrenia in other cultures, since they represent a mainly European and American perspective of functioning, due to the origins of the study participants in the first phase. This issue was mitigated at the consensus conference (second phase), as the expert panel was composed of members from all six WHO regions. Furthermore, it is worth noting that recent research is centered on cross-culturally validating the ICF-CSs in large samples with diverse sociocultural backgrounds (Nuño et al., 2021).

Implications

Numerous studies have found that the relationship between patient and clinician ratings of the same outcome measure has frequently been reported to be low or moderate (Lahera et al., 2018). Crespo-Facorro et al. (2017) pointed out that outcome measures should be multidimensional taking into account both clinical symptoms and social outcomes. In our study, a substantial number of categories have been identified to describe the functioning of individuals with schizophrenia, covering the different ICF components. This set of categories provides a list of issues to consider when an outcome measure is selected or when intervention programs are designed and applied. For instance, for research purposes, the results of our qualitative study showed that the contribution from people with schizophrenia, families, and caregivers provides a more subjective view of the experience of illness, either as a person diagnosed or as a family member or caregiver. This information can be important when treatment outcomes are selected. Moreover, the selection of outcomes that are meaningful and relevant for people with schizophrenia and their families could improve adherence to treatments and satisfaction. Similarly, in an intervention program context, working towards goals that are relevant for families and caregivers could improve the involvement of influential family members or caregivers in the intervention programs, which could lead to people with schizophrenia adhering more to the program.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Spain’s Ministry of Science, grant number PID2019-109887GB-100, and the Agency for the Management of University and Research Grants of the Government of Catalonia, grant number 2021SGR01071.

ORCID iDs

Maite Barrios

Alba Aza

José R. Chimelis-Santiago

Juana Gomez-Benito

Georgina Guilera

References

Alyahya

N. M.

Munro

Moss

(2022). The experience of psychosis and recovery from consumers’ perspectives: An integrative literature review. Journal of Psychiatric and Mental Health Nursing, 29(1), 99–115. https://doi.org/10.1111/JPM.12696

American Psychiatric Association . (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Washington, DC: American Psychiatric Association.

Barrios

Gómez-Benito

Pino

Rojo

Guilera

(2018). Functioning in patients with schizophrenia: A multicentre study evaluating the clinical perspective. Psychiatry Research, 270, 1092–1098. https://doi.org/10.1016/j.psychres.2018.05.079

Barrios

Guilera

Hidalgo

M. D.

Cheung

E. C. F.

Chan

R. C. K.

Gómez-Benito

(2019). The most commonly used instruments in research on functioning in schizophrenia. European Psychologist, 25(4), 283–292. https://doi.org/10.1027/1016-9040/a000386

Barrios

Guilera

Selb

Gómez-Benito

(2017). Identification of problems in the functioning of individuals with schizophrenia from the expert perspective: An internet-based survey. Disability and Rehabilitation, 39(20), 2055–2062. https://doi.org/10.1080/09638288.2016.1217073

Benítez

Pino

Padilla

J. L.

Cuevas-Parra

(2016). Integrating scale data and patient perspectives for assessing functionality in schizophrenia. Community Mental Health Journal, 52(8), 914–920. https://doi.org/10.1007/s10597-015-9959-0

Birnbaum

Jaffe

A. E.

Chen

Shin

J. H.

Kleinman

J. E.

Hyde

T. M.

Weinberger

D. R.

(2018). Investigating the neuroimmunogenic architecture of schizophrenia. Molecular Psychiatry, 23(5), 1251–1260. https://doi.org/10.1038/mp.2017.89

Bozzatello

Bellino

De Marzi

Macrì

Piterà

Montemagni

Rocca

(2018). Effectiveness of psychosocial treatments on symptoms and functional domains in schizophrenia spectrum disorders: A prospective study in a real-world setting. Disability and Rehabilitation, 41(23), 2799–2806. https://doi.org/10.1080/09638288.2018.1480666

Brugger

S. P.

Howes

O. D.

(2017). Heterogeneity and homogeneity of regional brain structure in schizophrenia: A meta-analysis. JAMA Psychiatry, 74(11), 1104–1111. https://doi.org/10.1001/jamapsychiatry.2017.2663

10.

Charlson

F. J.

Ferrari

A. J.

Santomauro

D. F.

Diminic

Stockings

Scott

J. G.

McGrath

J. J.

Whiteford

H. A.

(2018). Global epidemiology and burden of schizophrenia: Findings from the global burden of disease study 2016. Schizophrenia Bulletin, 44(6), 1195–1203. https://doi.org/10.1093/schbul/sby058

11.

Chong

H. Y.

Teoh

S. L.

D. B.-C.

Kotirum

Chiou

C.-F.

Chaiyakunapruk

(2016). Global economic burden of schizophrenia: A systematic review. Neuropsychiatric Disease and Treatment, 12, 357–373. https://doi.org/10.2147/NDT.S96649

12.

Cieza

Geyh

Chatterji

Kostanjsek

Üstün

Stucki

(2005). ICF linking rules: An update based on lessons learned. Journal of Rehabilitation Medicine, 37(4), 212–218. https://doi.org/10.1080/16501970510040263

13.

Conn

K. A.

Burne

T. H.

Kesby

J. P.

(2020). Subcortical dopamine and cognition in schizophrenia: Looking beyond psychosis in preclinical models. Frontiers in Neuroscience, 14, 542. https://doi.org/10.3389/fnins.2020.00542

14.

Cook

J. A.

Copeland

M. E.

Hamilton

M. M.

Jonikas

J. A.

Razzano

L. A.

Floyd

C. B.

Hudson

W. B.

Macfarlane

R. T.

Grey

D. D.

(2009). Initial outcomes of a mental illness self-management program based on wellness recovery action planning. Psychiatric Services, 60(2), 246–249. https://doi.org/10.1176/ps.2009.60.2.246

15.

Crespo-Facorro

Bernardo

Argimon

J. M.

Arrojo

Bravo-Ortiz

M. F.

Cabrera-Cifuentes

Carretero-Román

Franco-Martín

M. A.

García-Portilla

Haro

J. M.

Olivares

J. M.

Penadés

Del Pino-Montes

Sanjuán

Arango

(2017). Effectiveness, efficiency and efficacy in the multidimensional treatment of schizophrenia: Rethinking project. Revista de Psiquiatía y Salud Mental, 10(1), 4–20. https://doi.org/10.1016/j.rpsm.2016.09.001

16.

Creswell

Plano

C. V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.

17.

David

S. L.

Hitchcock

J. H.

Ragan

Brooks

Starkey

(2016). Mixing interviews and rasch modeling: Demonstrating a procedure used to develop an instrument that measures trust. Journal of Mixed Methods Research, 12(1), 75–94. https://doi.org/10.1177/1558689815624586

18.

Davidson

O’Connel

Tondora

Styron

Kangas

(2006). The top ten concerns about recovery encountered in mental health system transformation. Psychiatric Services, 57(5), 640–645. https://doi.org/10.1176/PS.2006.57.5.640

19.

Deenik

Tenback

D. E.

Tak

E. C. P. M.

Hendriksen

I. J. M.

van Harten

P. N.

(2018). Improved psychosocial functioning and quality of life in inpatients with severe mental illness receiving a multidisciplinary lifestyle enhancing treatment. The MULTI study II. Mental Health and Physical Activity, 15, 145–152. https://doi.org/10.1016/j.mhpa.2018.10.004

20.

Ferrari

A. J.

Stockings

Khoo

J. P.

Erskine

Degenhardt

Vos

Whiteford

A. H.

(2016). The prevalence and burden of bipolar disorder: findings from the Global Burden of Disease Study 2013. Bipolar disorders, 18(5), 440–450. https://doi.org/10.1111/bdi.12423

21.

Foruzandeh

Parvin

(2013). Occupational therapy for inpatients with chronic schizophrenia: A pilot randomized controlled trial. Japan Journal of Nursing Science, 10(1), 136–141. https://doi.org/10.1111/j.1742-7924.2012.00211.x

22.

Galderisi

Rucci

Kirkpatrick

Mucci

Gibertoni

Rocca

Rossi

Bertolino

Strauss

G. P.

Aguglia

Bellomo

Murri

M. B.

Bucci

Carpiniello

Comparelli

Cuomo

De Berardis

Dell’Osso

Di Fabio

Bozzatello

(2018). Interplay among psychopathologic variables, personal resources, context-related factors, and real-life functioning in individuals with schizophrenia a network analysis. JAMA Psychiatry, 75(4), 396–404. https://doi.org/10.1001/jamapsychiatry.2017.4607

23.

Gómez-Benito

Guilera

Barrios

Rojo

Pino

Gorostiaga

Balluerka

Hidalgo

M. D.

Padilla

J. L.

Benítez

Selb

(2017). Beyond diagnosis: The core sets for persons with schizophrenia based on the world health Organization’s international classification of functioning, disability, and health. Disability and Rehabilitation, 40(23), 2756–2766. https://doi.org/10.1080/09638288.2017.1356384

24.

Gorostiaga

Balluerka

Guilera

Aliri

Barrios

(2017). Functioning in patients with schizophrenia: A systematic review of the literature using the international classification of functioning, disability and health (ICF) as a reference. Quality of Life Research, 26(3), 531–543. https://doi.org/10.1007/s11136-016-1488-y

25.

Granikov

Hong

Q. N.

Crist

Pluye

(2020). Mixed methods research in library and information science: A methodological review. Library and Information Science Research, 42(1), 101003. https://doi.org/10.1016/j.lisr.2020.101003

26.

Greene

J. C.

Caracelli

V. J.

Graham

W. F.

(1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. https://doi.org/10.3102/01623737011003255

27.

Gurak

Weisman de Mamani

(2016). Risk and protective factors, perceptions of family environment, ethnicity, and schizophrenia symptoms. The Journal of Nervous and Mental Disease, 204(8), 570–577. https://doi.org/10.1097/NMD.0000000000000558

28.

Hamui-Sutton

(2013). Un acercamiento a los métodos mixtos de investigación en educación médica. Investigación En Educación Médica, 2(8), 211–216. https://doi.org/10.1016/s2007-5057(13)72714-5

29.

Jacob

Munro

Taylor

B. J.

Griffiths

(2017). Mental health recovery: A review of the peer-reviewed published literature. Collegian, 24(1), 53–61. https://doi.org/10.1016/J.COLEGN.2015.08.001

30.

Johnson

R. B.

Onwuegbuzie

A. J.

Turner

L. A.

(2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. https://doi.org/10.1177/1558689806298224

31.

Keefe

R. S. E.

Davis

V. G.

Atkins

A. S.

Vaughan

Patterson

Narasimhan

Harvey

P. D.

(2016). Validation of a computerized test of functional capacity. Schizophrenia Research, 175(1–3), 90–96. https://doi.org/10.1016/j.schres.2016.03.038

32.

Keetharuth

A. D.

Buck

E. T.

Acquadro

Conway

Connell

Barkham

Carlton

Ricketts

Barber

Brazier

(2018). Integrating qualitative and quantitative data in the development of outcome measures: The case of the recovering quality of life (ReQol) measures in mental health populations. International Journal of Environmental Research and Public Health, 15(7), 1342. https://doi.org/10.3390/ijerph15071342

33.

Lahera

Gálvez

J. L.

Sánchez

Martínez-Roig

Pérez-Fuster

J. V.

García-Portilla

Herrera

Roca

(2018). Functional recovery in patients with schizophrenia: Recommendations from a panel of experts. BMC Psychiatry, 18(1), 1–10. https://doi.org/10.1186/s12888-018-1755-2

34.

Matvienko-Sikar

Terwee

C. B.

Gargon

Devane

Kearney

P. M.

Byrne

(2020). The value of core outcome sets in health psychology. British Journal of Health Psychology, 25(3), 377–389. https://doi.org/10.1111/bjhp.12447

35.

Muñoz Poblete

(2013). Métodos mixtos: Una aproximación a sus ventajas y limitaciones en la investigación de sistemas y servicios de salud. Revista Chilena de Salud Pública, 17(3), 218. https://doi.org/10.5354/0719-5281.2013.28632

36.

Naim-Feil

Rubinson

Freche

Grinshpoon

Peled

Moses

Levit-Binnun

(2018). Altered brain network dynamics in schizophrenia: A cognitive electroencephalography study. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 3(1), 88–98. https://doi.org/10.1016/j.bpsc.2017.03.017

37.

Niendam

T. A.

Tully

L. M.

Iosif

A.-M.

Kumar

Nye

K. E.

Denton

J. C.

Zakskorn

L. N.

Fedechko

T. L.

Pierce

K. M.

(2018). Enhancing early psychosis treatment using smartphone technology: A longitudinal feasibility and validity study. Journal of Psychiatric Research, 96, 239–246. https://doi.org/10.1016/j.jpsychires.2017.10.017

38.

Nuño

Barrios

Moller

M. D.

Calderón

Rojo

Gómez-Benito

Guilera

(2019). An international survey of psychiatric-mental-health nurses on the content validity of the international classification of functioning, disability and health core sets for schizophrenia. International Journal of Mental Health Nursing, 28(4), 867–878. https://doi.org/10.1111/inm.12586

39.

Nuño

Guilera

Rojo

Gómez-Benito

Barrios

(2021). An integrated account of expert perspectives on functioning in schizophrenia. Journal of Clinical Medicine, 10(18), 4223. https://doi.org/10.3390/jcm10184223

40.

Peuskens

Gorwood

, & (2012). How are we assessing functioning in schizophrenia? A need for a consensus approach. European Psychiatry, 27(6), 391–395. https://doi.org/10.1016/j.eurpsy.2011.02.013

41.

Poth

Onwuegbuzie

A. J.

(2015). Special issue: Mixed methods. International Journal of Qualitative Methods, 14(2), 1–4. https://doi.org/10.1177/160940691501400201

42.

Schneider

(2012). Participatory action research, mental health service user research, and the Hearing (our) Voices projects. International Journal of Qualitative Methods, 11(2), 152–165. https://doi.org/10.1177/160940691201100203

43.

Schoonenboom

Johnson

R. B.

(2017). How to construct a mixed methods research design. KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 69(Suppl 2), 107–131. https://doi.org/10.1007/s11577-017-0454-1

44.

Selb

Escorpizo

Kostanjsek

Stucki

Üstün

Cieza

(2015). A guide on how to develop an international classification of functioning, disability and health core set. European Journal of Physical and Rehabilitation Medicine, 51(1), 105–117.

45.

T. M.

ZhanglongFeng

Y. Y.

Zhuo

J. M.

Cai

Zhang

(2019). Association between symptom control and functional improvement in patients with acute schizophrenia: A post hoc analysis of an open-label, single-arm, multi-center study of paliperidone-extended release formulation. Psychiatry Research, 274, 301–305. https://doi.org/10.1016/j.psychres.2019.02.036

46.

Singh

Kurki

M. I.

Curtis

Purcell

S. M.

Crooks

McRae

Suvisaari

Chheda

Blackwood

Breen

Pietiläinen

Gerety

S. S.

Ayub

Blyth

Cole

Collier

Coomber

E. L.

Craddock

Daly

M. J.

Barrett

J. C.

(2016). Rare loss-of-function variants in SETD1A are associated with schizophrenia and developmental disorders. Nature Neuroscience, 19(4), 571–577. https://doi.org/10.1038/nn.4267

47.

Slade

Longden

(2015). Empirical evidence about recovery and mental health. BMC Psychiatry, 15(1), 285. https://doi.org/10.1186/s12888-015-0678-4

48.

Taghipoorreyneh

de Run

E. C.

(2020). Using mixed methods research as a tool for developing an indigenous cultural values instrument in Malaysia. Journal of Mixed Methods Research, 14(3), 403–424. https://doi.org/10.1177/1558689819857530

49.

World Health Organization (2001). International classification of functioning, disability and health. World Health Organization.

A Mixed Methods Approach for Studying Relevant Areas of Functioning in Schizophrenia

Abstract

Keywords

Introduction

Method

First phase: Preparatory Studies

Second phase: Consensus Conference to Develop the ICF-CSs for Schizophrenia

Contribution to Mixed Methods Research

Design

Timing

Point of Interference

Data Analysis

Results

Research question 1: How do the results of the four preparatory studies complement each other?

Research question 2: How did each preparatory study influence the development of the ICF-CSs for schizophrenia during the second phase?

Discussion

Limitations

Implications

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

References