Abstract
The use of the Cultural Formulation Interview (CFI), a semi-structured 16-item cultural assessment developed for the DSM-5, has not yet been evaluated in the context of post-secondary education campuses (Cégeps and universities). The aim of this study was to assess whether a 2-h training session on the CFI improved cultural competence in clinicians working in student counseling centers throughout the province of Québec as well as to evaluate the clinicians’ attitudes toward the adoption of the CFI and their perceptions of the tool's feasibility, acceptability, and clinical utility. Thirty-nine clinicians from seven Cégeps and universities completed the pre-training questionnaires, participated in the 2-h training session, and then completed the post-training questionnaires. Paired-samples t tests were used to compare change in the scores of self-reported cultural competence pre and post training. Clinicians favorably rated the assessment tool, and most clinicians (89%) reported that they planned to incorporate the CFI into their routine clinical practice. There was an increase in multicultural counseling knowledge post training session, as well as a decrease in the multicultural counseling relationship subscale. Potential barriers to the use of CFI were also identified. More research is needed to explore the use of the CFI in different clinical contexts, as well as to evaluate the impact of cultural competence training on clinical practice.
Introduction
Research in the field of culture and mental health continues to underline the importance of adopting culturally competent care in mental health evaluation and treatment. Indeed, the undeniable influence that cultural factors have in the therapeutic process has been thoroughly described in the literature. Differences in age, sex, socioeconomic status, religion, language, or ethnicity of both the clinicians involved and the client can affect when, where, how, and to whom the patient chooses to disclose their suffering (Ayonrinde, 2003; Kirmayer, 2006), the pattern of symptoms experienced by the client (Kleinman, 1977; Leanza, 2011), and the client's perceptions of care, including the type(s) and duration of treatment that are acceptable (Lewis-Fernández et al., 2013). Clinicians who fail to adequately consider cultural factors during intake interviews and treatment planning may have several negative impacts on the therapeutic outcome such as diagnostic errors or inaccuracy in the assessment of illness severity (Adeponle et al., 2012), may create barriers in communication in the clinician–client dyad, and may negatively impact the client's engagement in the therapeutic process (Bhui & Bhugra, 2002), potentially leading to premature termination of mental health services (Alegria et al., 2012).
Cultural competence, defined as a set of abilities, capacities, and skills that aid in providing culturally appropriate services for a culturally diverse clientele (Bhui et al., 2007), is now considered to be an essential competency to be developed during training in many mental health fields. For example, among psychologists, the American Psychological Association (APA) has listed being competent in “individual and cultural diversity” as one of the foundational competencies for practitioners to develop (Clay, 2010). However, the academic curriculum currently offered to students in helping professions is rarely offered to an appropriate degree to ensure that this field of competence has been adequately acquired at the end of future clinicians’ training (Beaudoin-Julien et al., 2022; Newell et al., 2010). Furthermore, research evaluating the impact of training on cultural factors on clinicians’ level of cultural competence remains scarce, making it difficult to determine the actual effectiveness of these programs (Beach et al., 2005; Crosson et al., 2004; Lim et al., 2008; Price et al., 2005). Indeed, a review of the literature by Bhui and colleagues (2007) demonstrates that only nine articles out of a potential 109 provided information on the effectiveness of their training on professionals’ levels of cultural competence, indicating that we do not know whether training programs are accomplishing their goal of increasing the levels of cultural competence of clinicians working in cross-cultural settings. Results such as those yielded by Bhui and colleauges (2007) have been found countless times in reviews of the current literature about evaluating the effectiveness and impacts of cultural competence training (e.g., Gallagher & Polanin, 2015; Jongen et al., 2018; Renzaho et al., 2013; Truong et al., 2014). Amongst the small body of research examining the effects of these trainings, most of the studies have been focused on addressing the impact on the development of the provider's competencies, but very few studies have been able to show how cultural competency training may affect patient outcomes, such as patient engagement, satisfaction, or engagement in health-promoting behaviors (Lie et al., 2011).
A growing body of evidence has surfaced in previous years supporting the use of the Cultural Formulation Interview (CFI) (American Psychiatric Association, 2013) as an adequate tool to better assess cultural factors in mental health evaluations (e.g., Jarvis et al., 2020; Lewis-Fernández et al., 2017, 2020). The CFI, created by the Cross-Cultural Issues Subgroup Task Force of the American Psychiatric Association's Diagnostic and Statistical Manual, fifth edition, and published in 2013, is a semi-structured interview of 16 questions destined to be used during the intake of new patients in a mental health setting (American Psychiatric Association, 2013). In some respects, one of the goals of the CFI is to increase the cultural competence and sensitivity of practitioners during the initial assessment of clients. However, there is a limited body of knowledge currently available to assess whether CFI training leads to an increase in perceived cultural competence among clinicians. To our knowledge, there exist only two studies examining the effects of a 1-h training session on the CFI on psychiatry residents’ perceived levels of cultural competence, with results yielding significant increases in cultural competence after the training session (Mills et al., 2016, 2017).
Post-secondary education campuses are usually the most culturally diverse locations in the majority of Québec's cities. In fact, as an example, 20% of students from the Université du Québec à Chicoutimi (UQAC) are international students, whereas the region of Saguenay Lac St-Jean, the administrative region where the UQAC is located, is one of the regions welcoming the least number of migrants, with 1.1% of the total population of this region being immigrants (Statistics Canada, 2017). Students attending universities and Cégeps (Collège d’enseignement géneral et professionnel, translated in English in Québec to general and vocational colleges), known as publicly funded colleges in the province, are also among the population reporting the highest rates of psychological distress (Zochil & Thorsteinsson, 2018), with 81% of students showing signs of psychological distress in the fall 2020 semester (Québec Student Union, 2020). Taken together, student counseling centers present an important need for cultural competency training and tools to adapt their practices.
This study presented two main objectives: (a) to assess whether a 2-h training session on the DSM-5 CFI affects levels of cultural competence in clinicians working in student counseling centers in universities and Cégeps in the province of Québec; and (b) to evaluate whether clinicians perceived the CFI to be a useful, acceptable, and feasible tool to be incorporated in their practice, and whether this perception was affected by their attitudes toward adopting new treatments, interventions, and practices.
The three hypotheses of this study were as follows: (a) the cultural competence scores of clinicians working in student counseling centers will increase after having taken part in the 2-h training session on the CFI as measured by the Multicultural Counseling Inventory (MCI); (b) clinicians will perceive the CFI to be a useful, feasible, and acceptable assessment tool; and (c) clinicians who are more open to adopting new treatments, interventions, and practices will perceive the CFI more favorably.
Methodology
Setting and study design
The primary author contacted coordinators of student counseling centers in both English- and French-speaking universities and Cégeps throughout the province of Québec to assess the potential interest in participating in the study via email. A brief 30-min meeting was organized with interested centers to further explain the purpose and implication of the study. Voluntary clinicians of each center were then invited to participate in the 2-h training session, which was held via the videoconference app Zoom due to the health and safety measures enforced by the government amidst the COVID-19 pandemic. Recruitment began in February 2021 and ended in February 2022. The training session was offered in five universities (Bishop's University, Laval University, Sherbrooke University, Université du Québec à Chicoutimi, and HEC Montréal) and two Cégeps (Cégep Ste-Foy and Cégep de Lévis). According to enrollment statistics, the offered training sessions would have reached approximately 15% of students enrolled in every higher education institution in the province (Federation of Cégeps, 2020; Statista, 2022). This study collected data from two time points (pre-post study design) (Thiese, 2014). Clinicians were first asked to provide informed consent and to complete pre-training questionnaires before the beginning of the training session (T1). These questionnaires required approximately 20 min to complete. Participants were also asked to complete post-training questionnaires (T2), requiring approximately 15 min, immediately following the training session. The 2-h training session was developed and offered by the primary author, who was a fourth-year PhD student in clinical psychology. The training was interactive, was offered with the visual aid of a PowerPoint presentation, and consisted of (a) exploring the importance of considering cultural factors during psychological evaluation; (b) presenting the written guidelines of the CFI and the 16 questions; (c) clinical vignettes; and (d) a question-and-answer period. This study received ethical approval from necessary ethics boards.
Participants
Eligible participants were mental health practitioners working in student counseling centers in Cégeps and universities throughout the province of Québec. Clinicians were required to have a terminal degree allowing them to practice independently; this group consisted of registered clinical psychologists and social workers. Clinicians who could not participate in the entirety of this training session were excluded from the study. Participants who did not complete both the pre- and post-training questionnaires were also removed from the analysis.
Measures
Pre-training questionnaires (T1)
Before partaking in the training session, clinicians were asked to complete a set of questionnaires, including: 1) a sociodemographic questionnaire including questions on past cultural training and exposure to different significant cultural experiences (i.e., significant travel experience, being in an interethnic relationship, migration, training abroad). These items were derived from earlier work from the same team working on the development of cultural competence in professionals (Demers et al., 2022); 2) the MCI, a 40-item scale assessing behaviors and attitudes in relation to multicultural competencies through a 4-point Likert scale (very inaccurate to very accurate) on four subscales (Multicultural Awareness, Multicultural Counseling Knowledge, Multicultural Counseling Skills, and Multicultural Counseling Relationship) (Sodowsky et al., 1994); and 3) the Evidence-Based Practice Attitude Scale (EBPAS), a 15-item scale rating clinicians’ attitudes toward adopting new treatments, interventions, and practices through a 5-point Likert scale, with options ranging from 0 (not at all) to 4 (to a very great extent) on four subscales: Requirements to use evidence-based practices, Appeal of evidence-based practices, Openness to innovation, and Perceived divergence of evidence-based practices with usual practice (Aarons, 2004). As these questionnaires were only available in English and most participants recruited in this study completed the questionnaires in French, the scientific adaption process was used to ensure an adequate linguistic adaptation of the measures. This process was composed of four stages: 1) initial translation, where each item is translated in its literal sense; 2) internal revision, which consists of completing a back-translation with the aid of people who are fluent in both languages; 3) external revision, a step consisting of soliciting feedback from representatives of the group of participants with whom the measures will be used (i.e., social workers and psychologists); and 4) testing (René de Cotret, 2019).
Post-training questionnaires (T2)
Following the 2-h training session, participants were asked to complete the MCI for a second time. Clinicians were also asked to identify potential perceived barriers in implementing the CFI in their routine clinical practice (i.e., I do not believe the CFI to be useful, I don’t understand the goal of the CFI, I do not want to change my practice). At this stage, participants were also asked to complete the Debriefing Instrument for Clinicians (DIC; Lewis-Fernández et al., 2016), an 18-item self-report 4-point Likert-type scale (strongly disagree to strongly agree) on three subscales (feasibility, acceptability, and clinical utility), which was created and used by the research team in the international field trials of the CFI.
Statistical analyses
Only completed questionnaires were analyzed. Paired-samples t tests were used to examine within-participant pre- to post-training change on the cultural competence subscales. Means on the feasibility, acceptability, and clinical utility subscales of the DIC were calculated. A linear regression was performed to assess whether clinicians’ attitudes towards adopting new treatments, interventions, and practices (EBPAS scores) predicted the participants’ perceptions (of the feasibility, acceptability, and clinical utility) of the CFI.
Results
In total, 45 participants completed the pre-training questionnaires. However, six participants did not complete the post-training questionnaires and were therefore not included in the analysis, leaving 39 participants who completed all the required questionnaires. Specifically, 13 clinicians were working at Laval University, seven at Bishop's University, five at HEC Montréal, five at Université du Québec à Chicoutimi, four at Cégep de Lévis, three at Cégep Ste-Foy, and two at Sherbrooke University (see Table 1 for the clinicians’ sociodemographic characteristics). Of the 39 clinicians included in this study, 30.8% reported that their academic education had adequately prepared them to work in intercultural settings, with younger clinicians reporting being significantly more prepared than older clinicians, r(36) = −.46, p = .003. Clinicians with a higher number of significant cultural experiences also reported being better prepared for intercultural intervention, r(37) = .37, p = .02. Regarding the clinicians’ comfort level in treating clients from different cultural backgrounds, 28.2% of participants reported being moderately comfortable, 51.3% reported being comfortable, and 20.5% said they were very comfortable.
Participant sociodemographic information.
Note. a Other clinicians: Unspecified clinician (n = 1). b Examples of significant cultural experiences include significant travel experience, migration, and adoption. c Examples of cultural training sessions range from a one-hour lecture in the classroom to multi-day training.
Multicultural Counseling Inventory and pre to post changes in cultural competency
Cronbach's alpha of the measure in this sample was high (Cronbach's α = .89 for the pre-test measure and α = .86 for the post-test measure). One-tailed paired-samples t tests were conducted to explore differences in pre- and post-training scores within individuals for the overall questionnaire score and each subscale of the measure. No significant difference was found in the total MCI score pre (M = 3.14, SD = .33) and post (M = 3.18, SD = .34) training on the CFI, t(38) = −1.07, p = .15. However, significant improvement in multicultural counseling knowledge (subscale 4) was observed between pre (M = 3.05, SD = .51) and post (M = 3.25, SD = .42) training, t(38) = −2.39, p = .01. Contrarily, results demonstrated a significant decrease in the multicultural counseling relationship subscale between the pre-training measure (M = 3.05, SD = .47) and post-training measure (M = 2.93, SD = .55), t(38) = 2.02, p = .025. This subscale measures the clinician's interactional process with minority clients including trustworthiness, comfort level, stereotypes, and the clinician's worldview. No significant differences were found between pre to post measures on the multicultural awareness or multicultural counseling skills subscales (Figure 1).
Mean MCI subscale scores pre and post CFI training.
Clinicians’ perceptions of the CFI's feasibility, acceptability, and clinical utility
Internal consistency for the DIC subscales was acceptable, with Cronbach's alpha values of .78 (acceptability), .91 (clinical utility), and .68 (feasibility). Clinicians’ ratings of feasibility (M = 3.09, SD = .52), acceptability (M = 3.25, SD = .60), and clinical utility (M = 3.26, SD = .58) were positive. Overall, the feasibility scale was significantly lower than both acceptability and clinical utility (see Figure 2). In total, 89.3% (25/28) of clinicians reported that they plan to use the CFI in their clinical practice. Regarding potential barriers to the use of the CFI in clinical practice, 50% of clinicians reported not noting any barriers at all to using the CFI in their clinical practice. The most commonly reported barrier to implementing the CFI (9/32) was difficulty differentiating the CFI from other assessment tools (i.e., “I already ask similar questions” and “the CFI is not different than other assessment tools”), followed by administrative barriers (5/32) (i.e., “I will not have any new clients”) and the CFI not being realistic (3/32). The linear regression conducted to assess whether clinicians’ perceptions of the feasibility, acceptability, and clinical utility were predicted by their openness toward adopting new treatments, interventions, and practices did not yield any significant results.
Mean DIC scores reported by clinicians.
Discussion
Main findings
This study sought to explore whether a training session on the CFI affected levels of cultural competence in 39 clinicians working in student counseling centers throughout the province of Québec. This study also sought to evaluate clinicians’ perceptions of the CFI's feasibility, acceptability, and clinical utility and whether these perceptions were predicted by the practitioner's attitudes toward adopting new treatments, interventions, and practices.
Overall, a training session on the DSM-5's CFI was shown to have a significant impact on levels of perceived cultural competence pre and post training session in clinicians working in student counseling centers on university and Cégep campuses throughout the province of Québec. Specifically, participants reported higher levels of multicultural counseling knowledge post intervention. This result indicates that the 2-h training session provided clinicians with information regarding relevant case conceptualization and treatment strategies, as well as cultural information (Sodowsky et al., 1994). However, levels of the multicultural counseling relationship diminished after intervention. This subscale is defined as the clinician's interactions with minority clients, including the professional's trustworthiness, levels of comfort in working with minority clients, stereotypes, and worldview (Sodowsky et al., 1994). A decrease in levels of this subscale taken in conjecture with 72% of clinicians initially reporting being either comfortable or very comfortable with multicultural intervention despite most admitting to not having had the proper training to do so may hint towards clinicians’ realizing they may have overestimated their initial competency in interacting with clients from diverse cultural backgrounds and that the training sessions provided them with awareness of this deficiency. This tendency to overestimate one's level of competence, especially in areas where they have little or no experience, is well documented in the literature, notably by the Dunning-Kruger effect (Kruger & Dunning, 1999). The Dunning-Kruger effect seems to be particularly present in Western populations (Heine & Hamamura, 2007), making our sample of white western practitioners especially vulnerable to overestimating their competencies.
Regarding the Skills subscale, the lack of significant results is logical as the post-test was conducted directly after the training session; practitioners did not have time to practice using the tool in their clinical practice. This potential explanation of the null finding can also be generalized to the multicultural awareness subscale, which is defined as “suggesting proactive multicultural sensitivity and responsiveness, extensive multicultural interactions and life experiences, broad-based cultural understanding, advocacy within institutions, enjoyment of multiculturalism, and an increase in minority caseload” (Sodowsky et al., 1994, p. 142). Re-testing the clinicians another time (e.g., a few months later) would have been interesting to observe the self-reported changes in this subscale after use of the CFI with new clients. The lack of significant effect on the multicultural awareness and multicultural counseling skills subscales as well as on the total inventory score may also be due to the small sample size and thus a lack of statistical power, making detecting small differences difficult.
Clinicians rated the CFI favorably regarding feasibility, acceptability, and clinical utility. Overall, practitioners rated the CFI's feasibility significantly lower than its acceptability and clinical utility. This result is curious considering that half of clinicians reported not perceiving any potential barriers to implementing the CFI in routine clinical practice. However, the most frequent barrier reported by more than 25% of practitioners, having difficulty differentiating the CFI from other assessment tools, has also been found in other studies exploring potential barriers to implementing the CFI (e.g., Aggarwal et al., 2013). Clinicians’ attitudes toward adopting new treatments, interventions, and practices did not predict their perceptions of the CFI's utility, acceptability, or feasibility. Our small sample size and ultimately a lack of statistical power to carry out certain analyses may play a part in explaining a lack of significant results. At the individual worker level, factors such as a practitioner's years of experience, their educational attainment, and their attitudes, opinions, and beliefs all contribute to the successful implementation of evidence-based practices (Aarons, 2005; Aarons et al., 2010; Stahmer & Aarons, 2009; Wolf et al., 2014) and thus may have influenced clinicians’ resistance to change their practices, potentially explaining the lower feasibility rating. It is also important to note that cultural competence is not only something that clinicians should strive for on an individual level, but this quest must also be supported by organizational- and institutional-level practices (Kirmayer, 2012; Wells, 2000). It is possible that practitioners working in the student counseling centers did not feel supported by the administration, making it hard for them to envision implementing new tools. Further research is needed to explore optimal ways to ensure clinicians can implement real changes in practice. Qualitative studies should also be conducted exploring clinicians’ perspectives on administrative support in the promotion of organizational practices that help meet the needs of diverse populations.
Limitations
This study has several limitations. First, though seven sites were included in the present study throughout the province and both social workers and psychologists participated, the generalizability of these findings to other settings and provinces remains low. However, as no other studies exist examining the use of the CFI in university and Cégep campuses, we believe this study is an important first step in demonstrating the pertinence of CFI training in this population. Second, as the training sessions were offered at different moments with different groups of participants, it is possible that, depending on the topics of discussions or the questions asked by practitioners during each group, the training sessions were not exactly similar from one campus to another, ultimately potentially impacting the clinicians’ perceptions of the tool. Third, the lack of a control group in the study design is also a limitation. Due to this, differences in self-assessed levels of cultural competence pre and post training session could have been due to other factors not pertaining to specific content in the CFI, such as simply discussing the topic of culture and mental health assessments. Fourth, none of the clinicians in this study identified as being part of a visible ethnic minority group, limiting the ethnic diversity background of participants. Fifth, the method of recruitment of participants carried a risk of self-selection bias (Smart, 1966). Indeed, since recruitment was voluntary, clinicians with an already existing interest in cultural issues may have been more likely to participate in the 2-h training session on the subject. This may have had an impact on scores for perceived usefulness, feasibility, and acceptability of the CFI as well as scores related to attitudes toward the adoption of new treatments, interventions, and practices. Randomized controlled studies would be relevant to minimize this bias in future research (Larzelere et al., 2004). Finally, cultural competence in this study was measured using a self-report scale. As no objective and observable data on the gain of cultural competence was obtained, we do not know if actual behavioral changes were made in the clinicians’ routine clinical practice. It is also unknown if changes in cultural competence were maintained long term, as no longitudinal measures were taken.
Future directions
Despite having certain limitations, this is, to our knowledge, the first study collecting data on the effects of a training session on the DSM-5's CFI on self-assessed levels of cultural competence in mental health practitioners working in student counseling centers. Qualitative research should be carried out to gain better insight into clinicians’ perspectives on the CFI. Results from this study should also be used to further examine optimal ways to implement cultural competency training as well as increasing the probability of using the CFI in routine clinical practice for mental health workers. Further research is also needed to continue exploring the effects of cultural competence training, specifically training on the CFI, on actual routine clinical practice. Exploring how cultural competence training may impact patient outcomes is also an important next step in this field of research. This study is also the first to explore the use of the CFI in a primarily French-speaking sample; further research on clinicians employing the CFI in a wide array of languages, including French, is necessary to determine whether the tool is perceived positively by non-English-speaking clinicians.
Conclusion
In summary, a 2-h training session on the CFI improved clinicians’ subjective levels of cultural competence, more specifically multicultural counseling knowledge. It was also shown that clinicians perceive the CFI favorably and perceive few barriers to implementing the tool in their routine clinical practice. Results from this study validate the relevance and importance of offering training on cultural issues to mental health workers.
Implications
There are currently over 175,000 students currently enrolled in one of the 48 Cégeps throughout the province (Federation of Cégeps, 2020) and over 500,000 students enrolled in universities (Statista, 2022). Training sessions offered to clinicians in the seven institutions enrolled in this study have the potential to impact over 100,000 students throughout the province. Indeed, systematic CFI training among professionals working in student counseling centers could have a beneficial impact on the totality of the student population, which is, as previously mentioned, known for being diverse and prone to mental health issues. Results from this study are an important first step in starting discussions and reflections on the importance of offering adequate training to practitioners on culture-related issues in mental health settings on both individual and institutional levels.
Footnotes
Acknowledgements
The authors warmly thank the professionals who agreed to take part in the research. This first author was supported by a scholarship from the Canada Social Sciences and Humanities Research Council.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Social Sciences and Humanities Research Council of Canada.
Ethical approval
The project presented in this paper has obtained an ethics certificate from the Université Laval Ethic Review Board (Psychology and Education committee). The certificate number is 2020-142 A-2 R-1.
Author biographies
). His research interests are focused on the activity of health professionals in a context of diversity and interpreted interactions.