Sage Journals: Discover world-class research

Abstract

As trainees’ development of multicultural counseling skills has become integral to graduate training in the mental health professions, so has the necessity of competent supervision for developing these skills. A critical component of supervision is evaluation, the process of observing, judging, and providing feedback to supervisees about their development. However, there is a paucity of guidance and resources available to assist supervisors in evaluating the multicultural skill development of supervisees. Such a vacuum opens the door for considerable bias as well as judgmental and inferential errors, resulting in the real possibility of unsound, unreliable, and invalid evaluations of supervisee development. This article presents the Multicultural Counseling Skills Evaluation Form (MCSEF), a 23-item instrument designed to fill the vacuum. The conceptual foundation of the form, development of items and revisions through expert review, potential utility in training and evaluation, and limitations are discussed.

Keywords

multicultural skills supervision clinical training multicultural competencies professional psychology

Significance of the Scholarship to the Public

Trainee development of multicultural skills has become integral to the training of mental health professionals. Yet, there is limited guidance and few resources for supervisors to evaluate trainees’ development of these skills, even though supervisory evaluation is critical to this process. We present The Multicultural Skills Evaluation Form (MCSEF) as a tool for supervisors to fill this gap.

Clinical supervision plays a critical role in the development of graduate trainees in health service psychology and related mental health care professions (Falender, 2018; Gonsalvez et al., 2017). Supervision has been described variously as the cornerstone, core value, and sine qua non of graduate training (Falender et al., 2013; Tugendrajch et al., 2021). With an increased emphasis on competency-based training models for supervisees, the need for bolstering supervisors’ competence in supervision is greater than ever (American Psychological Association [APA], 2015; Falender & Shafranske, 2017; Gonsalvez & Calvert, 2014). Competency-based training requires supervisors to identify the knowledge, awareness, and skills related to clinical competence, in addition to their use of instructional strategies and evaluations that contribute to their supervisees’ development.

A key component of clinical supervision is evaluation, the process by which supervisors observe, judge, and give feedback to supervisees. This process is indispensable in helping supervisees develop their competence as well as cultivate professional attitudes and values (Borders et al., 2014; Gonsalvez & Crowe, 2014). Typically, evaluation occurs throughout all of the stages of training, as it can be formative or summative (Bernard & Goodyear, 2019). This understanding of evaluation, however, assumes supervisors are adept in utilizing their observations to inform their ratings of supervisees (Kemer et al., 2022; Tugendrajch et al., 2021). We maintain that the assumption is questionable due to the missing evidence to support the supervisors’ adeptness in evaluation. This should be of concern to a profession that touts evidence-based practice.

Given the importance of sound evaluation, supervisors should anchor their judgments in identifiable criteria of counseling competence and professionalism as opposed to idiosyncratic perceptions and opinions. Ironically, like clinicians who are subject to personal biases and judgmental and inferential errors (Lilienfeld & Basterfield, 2020; Ridley et al., 2011), clinical supervisors are similarly susceptible to rater bias, such as leniency error and the halo effect (Gonsalvez & Crowe, 2014). Rater bias, in turn, translates into unsound evaluations, the antithesis of markers of competent supervision. Alternatively, to reflect soundness, evaluations should be grounded in competency benchmarks and behavioral anchors (Falender & Shafranske, 2017). This benchmarking and anchoring enable an evidentiary basis and debiasing of evaluation.

Our interest lies in the evaluation of supervisees’ multicultural counseling competence (MCC) and stems from its advancement to the forefront of supervision discourse (Benuto et al., 2019; Falender et al., 2013; Watkins et al., 2019; Wiley et al., 2021). We acknowledge that this competence is mandated for psychologists as health service providers (see APA, 2017). We also recognize the necessity of intentional integration of multiculturalism into the supervisory process (see Borders et al, 2014). What provokes our interest even more is the paucity of guidance and resources for supervisors to evaluate their supervisees’ MCC. This condition raises several important questions. How do supervisors demonstrate their competence to evaluate MCC? What knowledge and skills are necessary for supervisors to impartially evaluate supervisees? Is the general state of multicultural supervision adequate to meet the developmental needs of supervisees?

The purpose of this article is to introduce the Multicultural Counseling Skills Evaluation Form (MCSEF), a tool designed to improve the evaluative competence of clinical supervisors. Our collective experiences as professors, supervisors, and supervisees across multiple settings have found the evaluation of MCC to be woefully inadequate. We consider this situation to be untenable, for it serves as an impediment to the profession’s multicultural agenda. This prompts us to argue that a correction, and perhaps even an attempt at standardizing the process, is necessary to advance the agenda. To achieve our objective, we organized the article into five major sections. First, we offer a rationale for such an evaluation tool. Second, we describe the development process of the MCSEF. Third, we give a description of the MCSEF. Fourth, we offer guidance on the use of the tool. Fifth, we discuss the strengths and limitations of the tool. We conclude with a discussion on the implications for training, practice, advocacy, and research, and then present MCSEF in the online supplements.

Rationale for an Evaluation Tool

A confluence of interrelated issues underscore the rationale for a tool such as the MCSEF. Five are especially compelling: (a) the integral role of clinical supervision in multicultural training, (b) the lack of a consensus definition of MCC, (c) the nonprescriptive best practices of MCC supervision, (d) the inexplicit evaluation criteria of MCC, and (e) the limitations of existing instruments. These issues highlight the need for specificity, clarity, and purposiveness in the evaluation of multicultural counseling skills.

Clinical Supervision as Integral to Multicultural Training

Clinical supervision plays an integral role in the training of multicultural and social justice-oriented health service psychologists (Benuto et al., 2019; Rocha & Kemer, 2022). A multicultural emphasis in supervision entails the intentional integration of sociocultural influences and contexts into the triadic relationship of client, clinician, and supervisor, the goal of which is to maximize the process and outcome of supervision (Bernard & Goodyear, 2019; King et al., 2020). This integration involves identifying salient cultural identities and related influences of power, privilege, and oppression within the triadic relationship. For example, social justice supervision models encourage supervisors to recognize the inherent power differentials in the supervisory relationship and acknowledge this dynamic with supervisees by clarifying supervisees’ rights and responsibilities, collaborating on goal setting, and modeling equitable service delivery (Glosoff & Durham, 2010; Peters, 2017).

Trainees in graduate programs develop their MCC through coursework, supervised clinical experiences, and to a lesser extent, in-class exercises such as case conceptualizations, role plays, and modeling (Benuto et al., 2019). Although there is considerable variation in the integration of multicultural training within and across graduate programs (Rogers & O’Bryon, 2014), trainees have identified faculty modeling behaviors consistent with cultural responsiveness and encouraging trainees’ incorporation of culture in therapy as important components in developing their MCC (Wilcox et al., 2021). The benefits of the explicit integration of multicultural factors in training cannot be overstated since a mounting body of research finds that this type of clinical supervision can help facilitate supervisees’ openness, understanding, and application of culturally responsive care in clinical practice (Jones & Branco, 2020; Soheilian et al., 2014; Wilcox et al., 2023). Depending on the supervisory competence, there is considerable variation in the instruction and evaluation of MCC. Tran (2021) found that supervisors reflected on their own experiences in supervision as: (a) rife with inadequate guidance on multicultural counseling knowledge and skills; (b) consisting of wide variation in supervisors’ incorporation of culture in the supervision process; and (c) resulting in their underdevelopment as supervisors in multicultural knowledge, awareness, and skills.

Competency-based models of supervision and guidelines for best practices in supervision emphasize the clinical supervisor’s role in developing and evaluating supervisees’ clinical competencies (Falender & Shafranske, 2017). Inclusive of this role is the development and evaluation of MCC. Accordingly, culturally responsive and socially just supervision reflects on supervisor competence in MCC (DeAngelis, 2023). Given the variability in multicultural training and competence of supervisors to evaluate MCC, any viable tool that assists supervisors in fulfilling their evaluative role is a clear benefit to the profession.

Lack of Consensus Definition of Multicultural Counseling Competence

Assuming supervisors operate from consensus definitions of multicultural constructs is erroneous and detrimental to supervisory evaluations. Ridley et al. (2021a) enumerated 10 problems associated with the definition of MCC, revealing irrefutable inconsistencies and therefore, the failure of the profession to provide a conceptually coherent definition of the construct. Without a uniform, profession-wide definition, it is virtually impossible to know which iteration of the construct supervisors and supervisees employ and whether both employ the same definition. Unless we address the problem of definitional inconsistencies, we cannot legitimately hold either accountable to a common standard of MCC. Supervisors then must rely on their ideological preferences, personal and professional experiences, unique interactions with culturally diverse groups, or inklings drawn from their previous clinical supervisors. The unintended consequence is supervisors’ vulnerability to their personal biases and misjudgments about supervisees’ development of MCC.

Consider a supervisory relationship in which the supervisor and supervisee operate from different perspectives on the role of culture in MCC (Ridley et al., 2021b). The supervisor endorses a culturally general definition, a view of the competence as applicable to all clients, regardless of their backgrounds, whereas the supervisee has a culturally specific definition, a view of the competence as applicable to a particular racial, ethnic, or cultural group. Unless supervisors are astute enough to recognize the definitional differences and align both their perspectives, the maximal benefits of supervision are unattainable. This lack of a consensus definition serves as an overarching impediment to the trainee’s development of MCC and soundness of the supervisor’s evaluation.

Nonprescriptive Guidance for Multicultural Counseling Competence Supervision

Considerable supervisory models employing a variety of strategies, guidelines, and recommendations for multiculturally responsive and social justice-oriented supervision have been put forth (e.g., Dollarhide et al., 2021; Mitchell & Butler, 2021; Singh & Chun, 2010; Upshaw et al., 2020; Watkins et al., 2019). For supervisors to provide training in and evaluation of MCC, they must demonstrate their own multicultural counseling knowledge, awareness, skills, and humility (DeAngelis, 2023). However, many of the major documents hailing the importance of supervisory MCC (e.g., APA, 2017; Borders et al, 2014) only offer descriptive guidance.

The Supervision Best Practice Guidelines (Borders et al., 2014) exemplify the problem of descriptive guidance. One of the twelve guidelines is Diversity and Advocacy Considerations. This guideline has two major components: (a) “the supervisor recognizes that all supervision is multicultural supervision and infuses multicultural considerations into his/her approach to supervision” (p. 38) and (b) “the supervisor encourages supervises to infuse diversity and advocacy considerations in their work with clients” (p. 38). The operative actions are to Recognize, Infuse, and Encourage, but how do supervisors transpose these directives into concrete supervisory behaviors? Supervisors must surmise for themselves how to implement each of these best practice guidelines. Although such descriptive guidance allows for flexibility due to variations in training settings, supervisors, and supervisees, it leaves supervisors with mandates to demonstrate multicultural supervisory competence without the cumulative instructions, explicit guidance, and behaviors to execute the mandates. These demonstrations of supervisor competence range from surface level recognition and infusion of multicultural considerations to deep-structure incorporation of culture.

The above critique aligns with the assertion that competent multicultural supervision is elusive (Falender et al., 2013). It also corresponds with Tran’s (2021) conclusions of the need for: (a) prescriptive guidance for trainees from supervisors to transform theoretical concepts into MCC and clinical practice, (b) implementation of clinical skills and practices transformed from theory early in the process, and (c) ongoing and reflective evaluation on the part of trainees and supervisors. The nonprescriptive guidance found in important supervision resources can adversely affect supervisors’ ability to evaluate their supervisees’ MCC.

Inexplicit Evaluation Criteria of Multicultural Counseling Competence

Because supervisory evaluation entails observing, judging, and providing constructive feedback, a sound process must connect a coherent definition of MCC to a set of related skills. Translated into overt behaviors, the skills become observable, measurable, and repeatable. Based on their observations, supervisors can determine the frequency of qualified skills and how well they are performed. In this way, their evaluation moves beyond a conceptual abstraction to an operationalization based on behavioral criteria that is communicated to supervisees at the start of supervision (Bernard & Goodyear, 2019; Jacobs et al., 2011). Their evaluation also should include positive and corrective feedback, the purpose of which is to monitor supervisees’ progress and maximize their development (Hattie & Timperley, 2007). The necessity of these best practices is exemplified by the difficulties supervisors experience in providing clear and helpful feedback (Bernard & Goodyear, 2019; Motley et al., 2014), especially corrective feedback in cross-cultural supervision (Burkard et al., 2014, Hall & Spencer, 2017). Given the definitional difficulties of MCC, any existing behavioral anchors or benchmarks are connected to an incoherent definition of the construct and call into question any empirical support based on those anchors. More than likely, this condition results in unreliable inferences, and unreliable inferences result in unsound evaluations. For this reason, supervisors need a set of theory-driven, behaviorally anchored criteria to evaluate MCC, the utility of which enables fairer discrimination among the levels of skill performance.

Limitations of Existing Instruments

Several instruments to evaluate MCC have been developed. However, notable limitations restrict their utility as evaluation tools in supervision. The Multicultural Counseling Inventory (Sodowsky et al., 1994) is a 40-item, self-report measure of four factors of MCC: multicultural counseling skills, awareness, relationship, and knowledge. Although the instrument has moderately high internal consistency and moderate interfactor correlations, it has low accuracy in evaluating cultural competence. The problem is common among self-report scales, such as the Multicultural Awareness-Knowledge-Skills Survey (D’Andrea et al., 1991). This survey consists of 60 multiple-choice, self-report items, divided across three domains—multicultural counseling knowledge, awareness, and skills. Both measures rely on the individual’s perception of their competencies and do not require a demonstration of evidence (e.g., observable skills) to inform ratings. This creates ambiguity and thereby weakens the accuracy of the scales. Both supervisees and supervisors must be able to rely on evaluation tools to cover the complexities of multicultural counseling accurately and comprehensively.

The Multicultural Counseling Knowledge and Awareness Scale (Ponterotto et al., 2002), a 32-item scale, in which knowledge and skills are loaded together under one domain, separate from awareness. Although the measurement of multicultural knowledge and awareness is befitting, the scale does not measure multicultural counseling skills, i.e., transforming knowledge and awareness into practice. This oversimplification and lack of integration (Ridley et al., 2021a) undermines the construct validity of the scale to measure MCC. Attempts to resolve this issue of measurement have been made, but their use may be impractical or inappropriate for supervision. For example, The Cross-Cultural Counseling Inventory–Revised (LaFromboise et al., 1991) is a 20-item scale that uses observer report to evaluate cross-cultural counseling skills, sociopolitical awareness, and cultural sensitivity. The authors cautioned against its use with new trainees due to anxiety and early-stage counseling development. We suggest that seasoned professionals also may lack multicultural counseling skills and display discomfort in their navigation of multicultural issues in counseling, raising questions of its viability as an evaluation tool.

Although these four instruments have been used extensively in research, they have notable limitations in their utilization for supervisory evaluation. To some extent, each instrument is based on the tripartite model of awareness–knowledge–skills (Sue et al., 1982, 1992). This groundbreaking model has been an assiduous catalyst in multicultural research and scholarship, but its translation of multicultural skills into counseling practice is not well-established. Foremost, these instruments do not enable users to directly observe behavior and connect their observations to the ratings of multicultural counseling skills in a clinical context. Furthermore, some only provide an overall profile of multicultural counseling knowledge and awareness. Such a focus assumes that acquisition of knowledge and awareness translates into skills.

In critiquing the significant challenges these instruments pose in the evaluation of MCC, Frisby (2018) notated the “lack of significant conceptual and/or empirical relationships among scales purporting to measure the same construct” and the “lack of correspondence between cultural competency self-reports and observer ratings” (p. 67). A sound evaluation of multicultural counseling skills must be guided by focused observation, and supervision provides this context to observe supervisees’ demonstration of these skills. This rationale was the impetus for the development of an evaluation tool that is anchored in observable indicators rather than exclusive reliance on self-report and presumption of clinical utility.

Development of the Multicultural Counseling Skills Evaluation Form

We present the MCSEF as an evaluation tool to overcome the challenges with multicultural counseling skills evaluation in training and supervision. Following are the key components of the development of the tool.¹

Item Generation

During the composition of the MCSEF, the team created 22 items that reflected the clinical operations of the process model. The team revised and reconceptualized the items to have a preliminary form to submit for expert review. After obtaining Institutional Review Board approval (IRB #2022-0596M), the team recruited 15 participants through purposive and snowball sampling methods from September 2022 to July 2023. We defined expertise in this area as having a doctoral degree in professional psychology and holding a position as either a clinical supervisor or academic faculty with a multicultural, social justice, or liberation psychology theoretical orientation.

Sample

Fifteen experts in multicultural psychology provided feedback on the practicality and utility of the MCSEF through the online Qualtrics survey. Most experts identified as women (n = 11).

Their self-described ethnic identity responses included Hispanic/Latinx/Latine (n = 3), White and Jewish American (n = 2), South Asian (n = 1), American Arab, Middle Eastern, and North African (n = 1), Asian and White (n = 1), Scots and Irish (n = 1). Most reported holding academic positions (e.g., professors, lecturers; n = 11), clinical positions (i.e., psychologists or counselors; n = 3), and post-doctoral fellow (n = 1). Their professional experience ranged from 6 to 40 years, and they reported a wide range of theoretical orientations, including humanistic, feminist, relational-cultural, integrative, liberation-psychodynamic, and cognitive behavioral.

Expert Feedback

Experts were asked to rate each item of the MCSEF and provide qualitative feedback in an open-ended response section. The items were rated on accuracy, goodness of fit, clarity, and the overall coverage of the construct on a 7-point Likert-type scale ranging from 1 (not at all) to 7 (completely). Accuracy measured the degree to which an item distinctly represented domain specific multicultural counseling skills in a mental health setting. Goodness of fit determined if the item contributes to the overall assessment of the specific domain. Items were rated on clarity (i.e., if they were understandable and had clear phrasing). For the last criteria, the overall coverage of the construct asked how well all the items under the specific domain thoroughly assessed that domain.

Revision of the Multicultural Counseling Skills Evaluation Form

Overall, the average ratings of MCSEF items on accuracy, goodness of fit, and clarity ranged between 4.00 and 6.57 points (see Table 1). Given most items received relatively high ratings on the three criteria, we determined that items or domains rated on average below 6 on accuracy, goodness of fit, clarity, and overall coverage; and those with a single item rating of 1 (not at all) or 2 (almost not at all) on the first three criteria were considered items “of concern.” These review criteria were aligned with the experts’ written feedback, which included their specific concerns, questions, and recommendations for revision. We identified 10 items and four domains that needed revisions, for which we independently reviewed the written feedback and then met as a team for discussions. Through considerable deliberation and consensus building, the team revised the wording and order of the items within and across domains to improve the clarity, accuracy, and goodness of fit of the items. The reordering of items was to improve the logical sequence of actions related to the domains (i.e., clinical operations of the process model) and address expert concerns about redundancy and overlap in the item’s coverage of the construct. This revision resulted in the final tool consisting of 23 items and seven domains.

Table 1.

Item Ratings of the Multicultural Counseling Skills Evaluation Form

Item	Mean	SD	Item	Mean	SD	Item	Mean	SD
Item 1*			Item 8			Item 15
Accuracy	6.00	1.36	Accuracy	6.33	0.81	Accuracy	6.43	0.76
Fit	5.80	1.52	Fit	6.40	0.83	Fit	6.50	0.76
Clarity	5.40	1.45	Clarity	6.27	0.96	Clarity	6.57	0.85
Item 2*			Item 9			Item 16*
Accuracy	5.93	1.28	Accuracy	5.80	1.32	Accuracy	5.07	1.85
Fit	5.47	1.59	Fit	5.60	1.50	Fit	5.21	1.92
Clarity	5.67	1.58	Clarity	5.13	1.55	Clarity	4.00	2.03
Item 3*			Item 10*			Item 17
Accuracy	6.27	0.80	Accuracy	5.93	1.28	Accuracy	6.57	0.65
Fit	6.47	0.74	Fit	5.93	1.28	Fit	6.50	0.76
Clarity	5.87	0.83	Clarity	5.87	1.45	Clarity	6.21	0.89
Item 4			Item 11			Item 18
Accuracy	6.27	0.80	Accuracy	6.33	0.98	Accuracy	6.57	0.76
Fit	6.20	0.94	Fit	6.27	1.03	Fit	6.50	0.94
Clarity	6.20	0.94	Clarity	6.47	0.83	Clarity	6.43	1.01
Item 5*			Item 12			Item 19
Accuracy	5.79	1.12	Accuracy	6.40	0.74	Accuracy	6.07	0.99
Fit	5.86	1.29	Fit	6.33	0.72	Fit	6.15	1.14
Clarity	5.79	1.18	Clarity	6.13	1.06	Clarity	6.14	1.02
Item 6*			Item 13			Item 20
Accuracy	5.87	1.12	Accuracy	6.40	0.83	Accuracy	6.07	1.14
Fit	5.80	1.26	Fit	6.40	0.74	Fit	6.14	1.09
Clarity	5.60	1.54	Clarity	6.47	0.92	Clarity	6.29	1.26
Item 7*			Item 14*			Item 21
Accuracy	5.47	1.40	Accuracy	6.21	0.70	Accuracy	6.43	0.85
Fit	5.53	1.64	Fit	6.29	0.73	Fit	6.43	0.85
Clarity	5.20	1.85	Clarity	5.79	1.18	Clarity	6.57	0.76

Note. Ratings are for the original items by experts during the expert review. Items with a “*” denote those revised by the team based on the expert feedback.

Research Team

We recognize our sociocultural identities, social positions, and related experiences are the lens that have shaped this research. The first author identifies as a South Asian, heterosexual, able-bodied, cis woman, and nonimmigrant worker, who understands experiences using a cultural relativism lens. The second author identifies as a White, college-educated, disabled, non-binary person from the South in the United States, critically reflecting on their areas of privilege and the significance of visible and invisible identities in their professional work and relationships. The third author identifies as a Christian, male, and African American who is a senior scholar and researcher in multicultural counseling. The fourth author identifies as an Asian American, middle-class, heterosexual cisgender woman, daughter of immigrants, and early career psychologist who holds a deep commitment to fostering the growth and development of therapists-in-training. Our collective experiences in research, teaching, supervision, and clinical practice center seriously on multiculturalism. A guiding principle of our work is the translation of multicultural research and theory into competent clinical practice.

Description of the Multicultural Counseling Skills Evaluation Form

The MCSEF, a 23-item, behaviorally anchored tool, assists supervisors in the evaluation of supervisees’ multicultural counseling skills (see Online Supplement). It was conceived as an alternative to the limited guidance in multicultural supervision and designed to provide constructive feedback to supervisees. The tool can help supervisors affirm strengths, notate acceptable levels of performance, and improve supervisees’ areas of growth. In essence, the tool has practicality. The aim of the MCSEF is to enhance the competence of supervisees to facilitate therapeutic change. Supervisors achieve this aim by monitoring the progress of their supervisees in supervision. The ensuing flexibility allows correction and adjustment along the trainees’ development trajectory.

Conceptual Foundation

The conceptualization underlying the form is The Process Model of Multicultural Counseling Competence (Ridley et al., 2021b). The model offers a coherent definition of MCC as “the facilitation of therapeutic change through the deep-structure incorporation of culture into counseling and psychotherapy” (p. 536). Based on this definition, the model has four major components: phases of counseling, clinical multilevel dynamic interactions, clinical operations, and therapeutic outcome. The phases of counseling reflect the time dimension of counseling and comprise of preparation, starting at intake, continuing in sessions, and ending with termination. Throughout the process, clinicians engage in various clinical operations that have multilevel dynamic interactions to facilitate therapeutic change. These clinical operations are: (a) Deep-Structure Incorporation of Culture, (b) Developing of a Therapeutic Alliance, (c) Infusing Preparation, (d) Adapting Interventions, and (e) Evaluating Process & Outcome.

The model proposes Deep-Structure Incorporation of Culture as its superordinate clinical operation and central component. Ridley et al. (2021b) assert that “determining the influence of cultural values and beliefs on client behavior, managing their influence on the dynamics of the therapeutic relationship, and leveraging these aspects of culture to facilitate therapeutic change is the purview of multicultural counseling competence” (p. 538). Deep-Structure Incorporation of Culture has three competencies: identifying cultural data, interpreting cultural data, and integrating cultural data. Developing a Therapeutic Alliance, Infusing Preparation, Adapting Interventions, and Evaluating Process and Outcome are the subordinate clinical operations. Developing a Therapeutic Alliance requires collaborating, negotiating, and goal setting with clients. Adapting Interventions entails tailoring counseling to the unique cultural needs of clients. Infusing Preparation involves “clinicians using the knowledge they gained through training and study outside of therapy to inform their clinical activities inside therapy” (Ridley et al., 2021b, p. 556). Evaluating Process and Outcome involves formative and summative evaluations, allowing clinicians to monitor the course of therapy and its alignment with the therapeutic goals. These operations have dynamic multilevel interactions with Deep Structure Incorporation of Culture to facilitate therapeutic outcomes. The authors assert clinicians must “understand that the model does not replace but complements counseling and psychotherapy” (Ridley et al., 2021b, p. 544). The skills to evaluate in the MCSEF are derived from the superordinate and subordinate clinical operations of the model.

Design

The 23 items of the MCSEF are organized into seven domains: (a) Identifying Cultural Data, (b) Interpreting Cultural Data, (c) Integrating Cultural data, (d) Infusing Preparation, (e) Developing a Therapeutic Alliance, (f) Adapting Interventions, and (g) Overall Rating. Each item is rated on a Likert scale ranging from 1 (below average for level) to 5 (above average for level) based on observed clinical performance. The Likert-type scale of each item is followed by a Behavioral Evidence, which allows qualitative feedback to support supervisors’ ratings. By documenting their observations of their supervisees’ skill demonstrations, supervisors can ground their evaluations and adjust their supervision strategies in developing these skills.

Guidance for Using the Multicultural Counseling Skills Evaluation Form

The MCSEF is a tool to evaluate demonstrable skills for MCC in counseling. A brief description of each domain is provided along with examples of qualitative feedback. The tool is useful for observations of supervisees in real-time counseling, recorded session reviews, case presentations, clinical documentations, and discourse in individual and group supervision. In Behavioral Evidence, supervisors give numeric ratings and notate their observations of supervisee skills. The tool is also useful for formative and summative evaluations (Bernard & Goodyear, 2019). We recommend supervisors familiarize themselves with the form and its foundational model before they begin a supervisory relationship. They can use this knowledge and present the MCSEF to orient supervisees to supervision and evaluative process.

The Likert-type scale ratings should be commensurate with supervisees’ developmental level, with a rating of 3 being indicative of supervisees meeting expectations at their level of training. For example, beginning level supervisors are likely to demonstrate skills in identifying the client’s salient identities and related life experiences, but may have a limited understanding of how systems of power and oppression impact individual well-being; hence, they are unable to identify the client’s micro- and macro-systems and their related influences. The numeric ratings allow supervisors to note areas that supervisees demonstrate skills above expectation at their training level and vice versa. This evaluation form can help supervisors identify supervisees’ specific areas of improvement and key strengths and work collaboratively in formulating training goals.

The multicultural counseling skill of identifying cultural data can serve as an illustration of this usage. This skill typically is a prerequisite to interpreting clients’ cultural data, formulating a case conceptualization, and then integrating that formulation into counseling through treatment planning. For example, supervisors can observe this skill of identifying cultural data directly through observation windows in the training clinic or indirectly through recorded counseling sessions. Supervisors can note how supervisees inquire about their clients’ cultural values, beliefs, and worldviews using broaching dimensions and techniques (Day-Vines et al., 2021). Does the supervisee ask additional questions to clarify their understanding of a client’s cultural upbringing? Supervisors can anchor their rating in these observations and use the Behavioral Evidence to document examples of demonstrated (or lack of) skills. Unless the skill of identifying cultural data is rated as an explicit metric of MCC, supervisors cannot fairly discriminate the performance of this multicultural counseling skill.

Thus, MCSEF has the potential to delineate the demonstrable skills of MCC and provide specificity and guidance in its evaluation. A potentially helpful use of the form is to encourage dual evaluation inputs. Supervisors can complete the form and have their supervisees do so independently. This arrangement allows supervisors to compare their respective evaluations, from which invaluable insights may emanate from the discussions of similarities and differences in their ratings of the skills. For optimal soundness of evaluations, supervisors can evaluate supervisees in each of the multicultural skills under the seven domains. We describe the seven domains, list the items under each domain, and provide examples of concrete evidence of demonstrations of skills in the next section.

Identifying Cultural Data

Identifying cultural data allows supervisees to recognize important pieces of multicultural information that impact clients’ psychological presentation. Crucially, interpretation and integration cannot occur without first identifying clients’ cultural data. The supervisee must demonstrate skills in broaching how a client’s unique experiences of culture influence their values and beliefs, including how clients’ cultural experiences relate to their clinical presentation. Failure to identify cultural data appropriately may lead to a narrow and limited conceptualization of the client’s presenting concern, a break in rapport or therapeutic alliance, and/or misdiagnosis and inappropriate treatment approaches.

Item 1

Gathers data about clients’ unique experiences of their culture using a range of sources.

Behavioral Evidence Example: Supervisee gathers all necessary information on clients’ sociocultural identities during the intake and through their course of treatment. They identify cultural data the client provided as well as consider cultural information that the client did not explicitly state, for example, connecting the client’s clinical concern to ongoing sociopolitical conflicts that affect client’s ethnic group. They articulated them well in supervision and recognized areas related to the client’s concerns that require further exploration.

Item 2

Clarifies one's understanding of clients’ unique experiences of their culture using follow-up probes.

Behavioral Evidence Example: Supervisee often asks the client additional questions to clarify anything they may have misunderstood and avoid overgeneralizations and/or stereotyping of the client’s experiences.

Interpreting Cultural Data

After identifying cultural data, the supervisee should explore the meaning of clients’ intersecting identities, cultural values, beliefs, and worldviews related to their clinical presentation. Clients’ cultural context serve as a foundation of their psychological well-being and provide insightful connections to their clinical concern. Accurate interpretation therefore suggests improved conceptualization, communication, and intervention development. This involves contextualizing the client’s clinical concern using cultural explanations. Failure to accurately interpret the cultural data may lead to inadequate interventions.

Item 3

Facilitates clients’ insight into how cultural values, beliefs, and norms influence their thoughts, behaviors, and affective experience.

Behavioral Evidence Example: Supervisee spends time interpreting how their client’s cultural values influence their clinical presentation during treatment planning and session note writing. They engage in an ongoing exploration of their client’s cultural frame of reference using the client’s sexual orientation, religious, socioeconomic, racial, and other salient identities.

Item 4

Explores the meaning of intersections of clients’ identities as it relates to their clinical presentation.
Behavioral Evidence Example: Supervisee often asks questions during individual and group supervision seeking to understand how client’s cultural upbringing influences their thoughts, behaviors, and affective experiences. Supervisee broaches cultural topics in session.

Item 5

Contextualizes clients’ clinical presentation using appropriate/accurate cultural explanations.
Behavioral Evidence Example: Supervisee draws accurate conclusions from generated clinical hypotheses on the etiology of the client’s clinical presentation. They check in with the client if the proposed hypothesis and conclusions accurately reflect the client’s experiences.

Item 6

Identifies stereotypes and biases related to clients’ cultural background that may result in pathologized interpretations of their clinical presentation.
Behavioral Evidence Example: Supervisee is intentional in identifying stereotypes and biases related to the client’s religious background. They discuss how it may result in pathologizing client’s thoughts and behaviors when they fail to utilize the client’s cultural frame of reference.

Integrating Cultural Data

Cultural data must be repeatedly addressed and integrated throughout the course of therapy, not solely in the intake session. Using the identified and interpreted cultural data, supervisees engage in the process of generating and testing their clinical hypotheses related to clients’ clinical presentations. Skilled supervisees utilize the identified cultural data throughout the phases of counseling to revise their conceptualizations of the client and subsequently inform their diagnoses, treatment plan, and interventions. In addition, they should periodically check whether their mode of engagement is consistent with the experiences of their clients.

Item 7

Generates clinical hypotheses that incorporate clients’ culture, values, beliefs, and norms.
Behavioral Evidence Example: Supervisee’s initial conceptualizations demonstrate that they consider their client’s environment and sociopolitical contexts. For example, supervisee discusses how client’s attending a predominantly White institution in a small college town may interfere with or influence their concerns of anxiety and panic when interacting with unfamiliar individuals.

Item 8

Tests clinical hypotheses that incorporate clients’ culture, values, beliefs, and norms.
Behavioral Evidence Example: Supervisee attempts to discuss with the client how the cultural information is important in understanding and addressing the clinical concerns. For example, they discuss with their client how their anxiety and high expectations of themselves stems from their desire to prove to society that Black women are “smart” and “capable” in response to challenging negative stereotypes.

Item 9

Revises case conceptualization based on clinical hypothesis testing.
Behavioral Evidence Example: Supervisee solicits feedback during supervision on how they are integrating the information the client provides into their case conceptualization. For example, the client may be exhibiting severe anxiety because of academic performance, barriers and challenges faced, and additional demands and expectations to succeed as an undocumented immigrant. The supervisee acknowledges both value orientation and systemic barriers for undocumented individuals influencing the client’s anxiety.

Item 10

Evaluates one's patterns of integrating or omitting cultural data using self-reflective practices.
Behavioral Evidence Example: Supervisee routinely uses supervision to explore if they are likely neglecting or overlooking cultural data. They spend time intentionally reexamining their conceptualization and plans for intervention and seek additional guidance during individual and group supervision.

Infusing Preparation

Supervisees must apply their acquired multicultural training into the provision of their clinical services (e.g., therapy, consultation, assessment). It is imperative for supervisees to continually acquire additional training and knowledge throughout their professional careers, as the literature expands its understanding of culture and its intersectionality with other aspects of health and identity. Focusing on improving their multicultural awareness, knowledge, and skills will facilitate a supervisee’s intentional actions toward preparation. Examples of this include attending conferences, training seminars, and workshops; reading literature; and engaging with diverse communities to learn about their sociocultural experiences and activism efforts.

Item 11

Applies one’s acquired multicultural knowledge, awareness, and skills when engaging in clinical services.
Behavioral Evidence Example: Supervisee has done a sufficient level of preparation in their multicultural course(s) and other related training and can implement their skills and knowledge in their clinical work. For example, supervisee applies the ADDRESSING framework during intake session to explore various domain of cultural influences with the client.

Item 12

Seeks additional consultation, educational, or training experiences to help understand cultural group values, dynamics, and historical/social issues.
Behavioral Evidence Example: Supervisee often seeks supervision and/or consultation when they are challenged by differences in values, historical and sociocultural lived realities or other cultural dynamics in session. For example, as a Black American non-binary clinician counseling a Vietnamese American cis man with strong values tied to his religious and ethnic (Buddhist) community, supervisee sought information on relevant social issues, cultural values and beliefs, and local resources for Vietnamese American Buddhist community in that region.

Item 13

Initiates furthering knowledge through literature sources, continuing education, and conversations between peers and/or members from various sociocultural groups.
Behavioral Evidence Example: Supervisee reviews academic literature when they are not familiar with a certain culture without excessive prompting. They actively engage in discussions during seminars on various multicultural groups, issues surrounding power, privilege, and oppression. They have joined cultural organizations and advocacy groups to further their knowledge and awareness.

Item 14

Seeks to understand how power, privilege, and oppression impact various populations.
Behavioral Evidence Example: Supervisee attends to how power and oppression may impact their clients. They engage in privilege examination as it relates to their clinical work through discourse in supervision. They are aware of how external factors may prevent or impede their clients and identify these factors while discussing their clients during supervision.

Developing a Therapeutic Alliance

The therapeutic relationship is essential in the therapy process (Baier et al., 2020), and supervisees must attend to interpersonal dynamics with their clients. It is necessary for supervisees to explore their cultural values, beliefs, and norms, and consider how they may impact their interpersonal communication and approach to their clients. Addressing similarities and differences may enhance alliance building and increase the level of trust and quality of the relationship. Examples of this include broaching culture, addressing misunderstandings or lack of knowledge, using self-disclosure, or validating self-experiences.

Item 15

Thinks critically about one's own cultural values, identities, beliefs, worldview, stereotypes, and biases.
Behavioral Evidence Example: Supervisee is introspective and often wonders how their own cultural values and beliefs may impact their clients. They critically reflect on their sociocultural identities and related lived experiences, demonstrating a growing cultural self-awareness. For example, supervisee noted how their cultural upbringing values independence and autonomy influenced taking a problem-solving approach with their Muslim American cis woman client, encouraging her to move out of state, away from her family, to pursue her career. Supervisee notes how this encouragement disregards the client’s family-oriented values.

Item 16

Examines how one's attitudes towards cultural similarities and differences may influence the therapeutic alliance.
Behavioral Evidence Example: Supervisee discusses their affective and/or behavioral reactions to clients with shared/different identities during supervision. They examine differences in their interpersonal communication and approach using recorded session tapes. For example, during supervision, supervisee explores feelings of warmth and kinship and identifying noticeable change in their body language (leaning forward) with client who shares their ethnic and immigrant identity.

Item 17

Challenges one's attitudes towards cultural similarities and differences that may rupture the therapeutic alliance.
Behavioral Evidence Example: Supervisees recognize and challenge personal values and attitudes that do not support differences between them and their client. They attempt to work through their reactions (e.g., feelings of frustration) to these differences and challenging their attitudes when it may rupture the therapeutic alliance (e.g., pushing their clients in a non-therapeutic way). For example, supervisees express an understanding of how their feminist beliefs may not align well with their client’s ascription to more traditional gender roles. Using supervision, they challenged their negative attitudes towards the value difference.

Item 18

Demonstrates intentional changes in response to the challenges to one's attitudes and beliefs.
Behavioral Evidence Example: Supervisor discusses ways in which they can make changes within therapy that is client-centered and facilitates a trusting, open, and safe therapeutic relationship. After discussing how they may impose their attitudes and beliefs when evaluating the client’s behaviors, the supervisee explores the utility of those behaviors with curiosity and non-judgment in session. They inquisitively approach their client and ask open-ended questions when they do not understand or agree with the client.

Item 19

Demonstrates respect and cultural humility when inquiring about cultural experiences to enhance the quality of the therapeutic alliance.
Behavioral Evidence Example: Supervisee recognizes that they do not know everything about their client’s culture. They maintain an open, empathic, respectful, and humble stance in therapy. For example, they disclose their lack of knowledge of sociopolitical history of the Native American tribes in their region and invite the client to share their knowledge, experiences, and meaning the identities hold for the client.

Adapting Interventions

Adapting interventions to the client’s needs and uniqueness is critical, as it contributes to optimized communication between the clinician and the client, encourages active engagement of the client in therapy, and aligns therapeutic interventions with the client’s unique needs (Ridley et al., 2021b). Adaptation within therapy occurs within a holistic framework, which requires the supervisee to consider multiple aspects of a client’s clinical presentation and background when adapting interventions. These adaptations require modifications to therapeutic interventions and processes to provide culturally responsive care.

Bernal et al.’s (2009) eight dimensions of modifications to language, persons, metaphors, content, concepts, goals, methods, and context can guide cultural adaptations in therapy. These modifications to interventions and processes entail communicating effectively with clients using linguistic and culturally appropriate idioms. Additionally, a collaborative approach to therapy that continually solicits client feedback on the progress and process of therapy is necessitated. The modifications create a more dynamic approach to culturally responsive counseling and may appear as implementing a culture-specific intervention, using cultural idioms to explain concepts, and/or adjusting one’s clinical approach to align with the client’s cultural expressions.

Item 20

Modifies or tailors interventions to align with the client's cultural experiences.
Behavioral Evidence Example: Supervisee makes necessary adjustments when needed by changing their interventions to fit their client’s needs. They discuss modifications to treatment strategies to align with client’s needs and cultural values with their supervisors. For example, they include faith-based activities in addressing symptoms of depression because their client stated how important religion is to her.

Item 21

Collaborates with clients to set therapeutic goals, apply interventions, and terminate therapy.
Behavioral Evidence Example: Supervisee always involves the client in treatment planning and checks in with the client to make sure they are comfortable. They allow the client to bring up concerns and take the time to make their clients feel empowered to make decisions.

Item 22

Solicits client feedback on the progress and process of therapy.
Behavioral Evidence Example: Supervisee engages in ongoing feedback with the client by checking in with them on the process and progress in therapy. This entails revisiting therapeutic goals, progressing towards stated goals, and modifying aspects of therapy to be congruent with the client’s cultural experiences.

Overall Rating

This section evaluates the supervisee’s overall understanding and conceptualization of culture as a dynamic system that is key in shaping human experiences. This demonstration goes beyond surface-level incorporation of culture such as simply asking about demographic information or matching clinicians and clients based on their shared race, ethnicity, or culture. Instead, it seeks to identify, interpret, and integrate cultural data in case conceptualizations and treatment planning. Accordingly, cultural “values, beliefs, assumptions, and norms make up the primary data of interest because they are integral to individuals’ psychological presentations and influence the counseling relationship” (Ridley et al., 2021b, p. 548). Supervisees can ground their understanding of cultural complexities and their role in shaping their thoughts, behaviors, and emotional expressions.

Item 23

Conceives of culture as a complex and dynamic system integral to clients’ psychological presentation.
Behavioral Evidence Example: Supervisees attend to culture in their intake and how it influences their clients by incorporating an intentional exploration of cultural upbringing and how it influences their clients. They approach their clients through a multicultural lens that recognizes how an individual’s lived experiences are shaped through overlapping and intersecting identities.

Incorporating cultural data deeply into the therapeutic process is essential to developing supervisees’ skills, self-awareness, and intentionality as multiculturally responsive clinicians. The domains under the MCSEF attend to the intentional incorporation of culture by using behavioral anchors to provide a foundation on which to evaluate theoretically aligned, demonstrable skills. This evaluation based on observable demonstrations of multicultural skills may help minimize supervisors’ biases in the evaluation of these skills.

Strengths and Limitations of the Multicultural Counseling Skills Evaluation Form

The MCSEF has the potential to improve supervisors’ evaluation of MCC due to its anchoring in the conceptually coherent definition of MCC as defined by the process model (Ridley et al., 2021b). Specific strengths, potential utility, and limitations should be considered prior to its usage. Item development and analysis were intentionally kept at the expert review level; hence, revisions of the MCSEF were derived from a small sample. Although such numbers are typical for a scale expert review (Presser et al., 2004), the average ratings for each item on the scale may have been influenced by significantly low or high ratings from one or two individuals. Acknowledging this potential for skewed values, we also reviewed any items that received even a single rating of 1 or 2 irrespective of the item’s average rating.

We provided experts with definitions for each domain of the MCSEF before presenting the respective items to establish conceptual grounding within the process model. Nonetheless, a lack of familiarity with this conceptual foundation among experts might have led to challenges in clearly defining and operationalizing their ratings. This is reflected in some of the experts’ suggested phrasing, which encouraged supervisors to adopt an intersectional perspective when evaluating to emphasize the importance of considering the combined effects of multiple marginalized identities rather than a singular focus on individual cultural aspects. The items under the MCSEF do not encourage taking a siloed identity approach but instead recommend examining all of the influences of clients’ salient sociocultural identities on their presenting concerns. Using the words “identities” and “culture” are meant to examine multiple and additive influences. The MCSEF does not give specific instructions for the combination of sociocultural influences, recognizing that the breadth and depth of their salience vary across individuals. Additionally, the MCSEF explicitly evaluates the supervisee’s understanding of the impact of systems of power, privilege, and oppression within clinical work—inherent to which are intersectional experiences of marginalization. We entrust that supervisors are aware of the varying degrees of ability in trainees to identify, consider, and incorporate concepts of intersectionality in clinical work.

It is important to note that as the process model is an augment rather than a substitution for counseling, the MCSEF is transtheoretical and applies to any therapeutic orientations endorsed by supervisors and supervisees. We created this form with utility and application within training contexts in mind. The tool is intended to be applicable with supervisees across graduate programs (e.g., masters, doctoral, post-doctoral) and training developmental levels. The tool has yet to be piloted in a clinical training setting, limiting our ability to attest to its practicality in skills evaluation within training contexts. For example, the requirement for anchoring evaluations on each item on observable skills may be difficult for supervisors overseeing the training of multiple trainees in clinically demanding mental health clinics. Field testing across a multitude of supervisory contexts would help establish the reliable, practical, and sound applications of the MCSEF and is discussed further in the implications section.

The MCSEF requires supervisors to evaluate multicultural counseling skills that are anchored in specific observable behavioral competencies. This manuscript provides a definition of the MCSEF domains and items under each domain of the MCSEF, with examples of behavioral evidence as guidance for evaluating multicultural counseling skills. Nevertheless, supervisors may still interpret the items outside of what is intended, subsequently impacting their evaluations. Certainly, there is an element of supervisor judgment in the observation of skill demonstrations, and such subjectivity can introduce variability in evaluation. As expected in supervision, consultation with colleagues and reference to the clinical and programmatic expectations for supervisees at each stage of development is encouraged to reduce the subjective interpretation of noted behavioral anchors. We acknowledge that some subjectivity allows for variations in supervisory expertise, training styles, and expected level of performance at trainees’ developmental levels. In many ways, supervisor subjectivity, which stems from diversity in lived experience, variance in training backgrounds, and differences in theoretical and supervision orientations, can offer trainees opportunities to receive feedback on their multicultural counseling skills from multiple perspectives. Additionally, the feedback from experts demonstrated a degree of understanding of MCSEF items corresponding to their respective domains and illustrated the utility of allowing for some subjectivity in this process—without the compiled feedback from these experts of various theoretical backgrounds, the significant revisions of the scale would not have been possible.

The MCSEF provides opportunities to evaluate and provide feedback on the supervisee’s continuum of skill development and supervisors can use the observed skills demonstrations to discriminate (e.g., below average, average, and above average) performance. Some supervisors may experience ambiguity in discerning what is considered a developmentally appropriate skill demonstration. For example, identifying expected skills demonstration for integrating cultural data or what is deemed developmentally appropriate for a beginning practicum supervisee versus a supervised predoctoral intern? This may be partly ameliorated by reference to the standards set by the supervisor’s institution or accrediting body in determining the expected level of skills at each training level. Although ambiguity can be a pitfall in operationalization and evaluation, some degree of ambiguity can be expected when working in psychological sciences (Gard & Leung, 2020). Rather than attempting to eliminate variability, the MCSEF offers opportunities to engage in productive discussions on the supervisee’ and supervisor’s interpretations of the items, as well as the behavioral evidence(s) noted in observations. The form offers a rating scale and an open-ended comments section for each item comprising the various domains of evaluation. This allows supervisors to indicate examples of skill demonstrations as evidence of support for their quantitative ratings, aiding in formative and summative evaluations. Such discourse and collaboration are integral in establishing the supervisory alliance and delivery of effective supervision (Ladany et al., 2013; Watkins, 2014). Referring to the standards set forth by institutional bodies and discussing conceptualizations of expected skill demonstrations are some ways of addressing the ambiguity present in evaluation.

Additionally, supervisors’ multicultural training and competence in MCC will influence their ability to recognize, provide training in, and evaluate MCC skills (Adams et al., 2023; Tran, 2021). Hence, if supervisors demonstrate a lack of knowledge and skills in this domain, it will subsequently influence the rating of their supervisee’s skills. We strongly encourage supervisors to become familiar with the relevant research literature (e.g., the process model, corrective feedback in supervision) before using the MCSEF. Much like trainees, licensed psychologists in supervisory roles should remain aware of the need to engage in self-reflection, evaluation, and improvement of their knowledge and skills under MCC.

The MCSEF is not intended to be the singular piece of MCC evaluation within a training context. Programs and clinics maintain their own method of documentation and tracking of training outcomes and expected level of performance by trainees, following the guidelines set forth by their professional accrediting organizations. Accreditation bodies, such as the American Psychological Association, require evaluation to include several aspects and diverse sources of demonstrating training outcomes. Although we do not suggest the MCSEF is the ultimate supervisory tool, it takes a comprehensive and nuanced approach to multicultural counseling skill evaluation.

Implications for Training, Practice, Advocacy, and Research

As an evaluation tool for supervisors, the MCSEF has important implications. In addition to its advantages of behavioral anchoring and numeric ratings, the form aligns with the recommendations of transparency under the framework of multiculturally responsive and social justice-oriented supervision (DeAngelis, 2023; Dollarhide et al., 2021). In this section, we highlight seven implications of using the MCSEF.

Flexible Multicultural Application

The MCSEF enables supervisors to collect and collate data across multiple supervisees’ clinical cases. Not only does it support evaluation of supervisees’ multicultural counseling skills in clinical work, but also supervisors can identify supervisee’s patterns of case formulation, use of skills, and interventions. For example, a supervisor observing a supervisee who explores intersecting identities only with clients who have minoritized identities, such as sexual and gender minorities. The supervisor can use this pattern to discuss the importance of exploring and integrating cultural influences for all clients such as those who have majority-group and/or privileged identities or religious affiliations. The supervisor can tailor supervisory goals and strategies, such as broaching (Day-Vines et al., 2021), modeling cultural humility (Jones & Branco, 2020), and interpersonal recall to increase supervisees’ multicultural awareness (Ivers et al., 2017).

Across Stages of Development

The MCSEF has utility regardless of supervisees’ stage of development. Therefore, the tool can be used to monitor, track, and document supervisees’ developmental trajectory throughout their years’ training. For example, programs with in-house training clinics can use the MCSEF as the baseline evaluation of supervisees at the outset of their training. Results can then be extended to external practicum sites for continued evaluation. Supervisees at an advanced level of their training may evaluate themselves, demonstrating self-efficacy in ongoing evaluation of their MCC.

Self-Evaluation

Juxtaposed to the constructive feedback supervisees receive from supervisors, the MCSEF offers a unique opportunity for supervisees to engage in independent self-reflection and self-evaluation. Self-reflection exercises like journaling, writing reaction papers, participating in cultural immersion experiences or difficult dialogues, and reading thought provoking literature are often encouraged or required in graduate programs. Psychologists at all levels of training and experience are expected to “engage in personal reflection on multicultural issues” (APA, 2017, p. 31). The advantage of our tool is the provision of objective behavioral data through observable demonstrations of MCC in clinical work. The MCSEF helps to overcome this disadvantage by exposing individuals to their own objective behavioral data. This exposure can play a crucial role in heightening their self-awareness, objectifying personal skill levels, and providing a reliable foundation on which to evaluate change.

Program Evaluation

The MCSEF can extend beyond individual supervision to evaluate training outcomes in graduate programs. The evaluations can be stored in a database, allowing programs to monitor their trainees’ overall performance of MCC, which facilitates both annual reviews and remediation. These data can help programs track aggregate trends in trainees’ development and inform changes in the training curriculum. Finally, this data potentially can be used to help fulfill accreditation requirements by integrating the findings alongside evidence from various sources into self-study program reports to accrediting organizations.

Advocacy

We acknowledge that the current sociopolitical and legal climate presents new challenges for advocates of diversity and social justice. The MCSEF primarily uses the word “culture” to broadly represent multicultural influences and infrequently uses terminology that some jurisdictions find restrictive. We are mindful of the scrutiny on some institutions even for the use of the words “privilege,” “oppression,” “stereotype and biases based on race and sex.” Nevertheless, the language in our tool is consistent with that of profession-wide competencies of health service psychology (APA, 2017). Therefore, supervisors in restrictive jurisdictions have a justification for evaluating these skills, and we encourage them to familiarize themselves with the laws and policies of their jurisdictions (e.g., institutional, local, state). We also recommend they consult with professional organizations and prevail upon resources for assistance (e.g., Adams et al., 2023).

Future Research

A program of research to investigate the utility of the MCSEF has the principal goal of establishing its evidence basis. To establish the most robust evidence, research can be conducted under a host of conditions: diverse clinical training contexts (e.g., university counseling centers, mental health consortiums), supervisees at varying levels of training, supervisors of varying therapeutic orientations, and supervision dyads of varying racial and cultural compositions. Recent studies examining the harmful supervisee experiences and inadequate supervisory feedback (Hutman et al., 2023; Ladany et al., 2013) and promoting supervisor cultural humility, supervisory working alliance, and collaboration (Ertl et al., 2023; Upshaw et al, 2020) suggest new avenues to use the tool in research. A far-reaching research agenda would attempt to establish the linkage of supervisees’ development of MCC to their facilitation as therapists of therapeutic outcomes. This demonstrated linkage would be a monumental advancement since we intuitively make the double inference (but without supporting evidence) that multicultural training leads to MCC, which, in turn, leads to positive treatment outcomes (Ridley et al., 2021b). Using the MCSEF to promote self-reflective practices is another potential line of study in examining its application outside of training contexts. Finally, we envision research networks that use the MCSEF to advance the body of knowledge regarding multicultural supervision.

Supplemental Material

Supplemental Material - Multicultural Counseling Skills Evaluation Form: A Tool for Clinical Supervision

Supplemental Material for Multicultural Counseling Skills Evaluation Form: A Tool for Clinical Supervision by Ankita Sahu, Anne-Stuart Bell, Charles R. Ridley, and Vy Tran in The Counseling Psychologist

ORCID iDs

Ankita Sahu https://orcid.org/0009-0009-1799-974X

Anne-Stuart Bell https://orcid.org/0009-0005-9813-5214

Charles R. Ridley https://orcid.org/0000-0003-2259-376X

Footnotes

Supplemental Material

Supplemental material for this article is available online.

Note

Author Biographies

Ankita Sahu, PhD, is an assistant professor in the School of Psychological and Behavioral Sciences at Southern Illinois University Carbondale. She received her doctorate in counseling psychology from Texas A&M University. Her research interests examine the application of multicultural theories and principles in graduate training and supervision to improve skills-based outcomes of multicultural case conceptualization and multicultural counseling skills.

Anne-Stuart Bell, MEd, is a doctoral student in the counseling psychology program at Texas A&M University. Their current research interests are in multicultural issues in counseling, rural mental health, and the use of narratives in psychological research. They graduated from Sewanee: The University of the South with a BA in psychology and women’s & gender studies.

Charles R. Ridley, PhD, is a professor in the counseling psychology program at Texas A&M University. He received his doctorate from the University of Minnesota. His research interests include multicultural counseling competence, thematic mapping in case conceptualization, and the scientist–practitioner model.

Vy Tran, PhD, is a recent graduate of Texas A&M University’s counseling psychology doctoral program. She is currently a full-time clinician in private practice. Her areas of clinical interest include complex trauma recovery, identity development, multiculturalism, and self-actualization.

References

1.
Adams L. J. MacLean R. R. Portnoy G. A. Beauvais J. Stacy M. A. (2023). Psychology trainee and supervisor perspectives of multicultural supervision. Psychological Services, 20(S1), 78–85. https://doi.org/10.1037/ser0000643

2.
American Psychological Association . (2015). Guidelines for clinical supervision in health service psychology. American Psychologist, 70(1), 33–46. https://doi.org/10.1037/a0038112

3.
American Psychological Association . (2017). Multicultural guidelines: An ecological approach to context, identity, and intersectionality. https://www.apa.org/about/policy/multicultural-guidelines.pdf

4.
Baier A. L. Kline A. C. Feeny N. C. (2020). Therapeutic alliance as a mediator of change: A systematic review and evaluation of research. Clinical Psychology Review, 82, 101921. https://doi.org/10.1016/j.cpr.2020.101921

5.
Benuto L. T. Singer J. Newlands R. T. Casas J. B. (2019). Training culturally competent psychologists: Where are we and where do we need to go? Training and Education in Professional Psychology, 13(1), 56–63. https://doi.org/10.1037/tep0000214

6.
Bernal G. Jiménez-Chafey M. I. Domenech Rodríguez M. M. (2009). Cultural adaptation of treatments: A resource for considering culture in evidence-based practice. Professional Psychology: Research and Practice, 40(4), 361–368. https://doi.org/10.1037/a0016401

7.
Bernard J. M. Goodyear R. K. (2019). Fundamentals of clinical supervision (6th ed.). Pearson.

8.
Borders L. D. Glosoff H. L. Welfare L. E. Hays D. G. DeKruyf L. Fernando D. M. Page B. (2014). Best practices in clinical supervision: Evolution of a counseling specialty. The Clinical Supervisor, 33(1), 26–44. https://doi.org/10.1080/07325223.2014.905225

9.
Burkard A. W. Knox S. Clarke R. D. Phelps D. L. Inman A. G. (2014). Supervisors’ experiences of providing difficult feedback in cross-ethnic/racial supervision. The Counseling Psychologist, 42(3), 314–344. https://doi.org/10.1177/0011000012461157

10.
D’Andrea M. Daniels J. Heck R. (1991). Evaluating the impact of multicultural counseling training. Journal of Counseling & Development, 70(1), 143–150. https://doi.org/10.1002/j.1556-6676.1991.tb01576.x

11.
Day-Vines N. L. Cluxton-Keller F. Agorsor C. Gubara S. (2021). Strategies for broaching the subjects of race, ethnicity, and culture. Journal of Counseling & Development, 99(3), 348–357. https://doi.org/10.1002/jcad.12380

12.
DeAngelis T. (2023). Increasing supervisor savvy around culture, race, and identity. Monitor on Psychology, 54(1), 34. https://www.apa.org/monitor/2023/01/culturally-responsive-supervision.

13.
Dollarhide C. T. Hale S. C. Stone-Sabali S. (2021). A new model for social justice supervision. Journal of Counseling & Development, 99(1), 104–113. https://doi.org/10.1002/jcad.12358

14.
Ertl M. M. Ellis M. V. Peterson L. P. (2023). Supervisor cultural humility and supervisee nondisclosure: The supervisory working alliance matters. The Counseling Psychologist, 51(4), 590–620. https://doi.org/10.1177/00110000231159316

15.
Falender C. A. (2018). Clinical supervision—The missing ingredient. American Psychologist, 73(9), 1240–1250. https://doi.org/10.1037/amp0000385

16.
Falender C. A. Burnes T. R. Ellis M. V. (2013). Multicultural clinical supervision and benchmarks: Empirical support informing practice and supervisor training. The Counseling Psychologist, 41(1), 8–27. https://doi.org/10.1177/0011000012438417

17.
Falender C. A. Shafranske E. P. (2017). Competency-based clinical supervision: Status, opportunities, tensions, and the future. Australian Psychologist, 52(2), 86–93. https://doi.org/10.1111/ap.12265

18.
Frisby C. L. (2018). History and development of cultural competence evaluation in applied psychology. In Frisby C. L. O’Donohue W. T. (Eds.), Cultural competence in applied psychology: An evaluation of current status and future directions (pp. 123–138). Springer.

19.
Gard D. E. Leung L. N. (2020). How meaning is made: Ambiguity tolerance as a central, operationalizable concept for psychotherapy integration. International Journal of Integrative Psychotherapy, 11(1), 97–123. https://www.integrative-journal.com/index.php/ijip/article/view/155/111

20.
Glosoff H. L. Durham J. C. (2010). Using supervision to prepare social justice counseling advocates. Counselor Education and Supervision, 50(2), 116–129. https://doi.org/10.1002/j.1556-6978.2010.tb00113.x

21.
Gonsalvez C. J. Calvert F. L. (2014). Competency-based models of supervision: Principles and applications, promises and challenges. Australian Psychologist, 49(4), 200–208. https://doi.org/10.1111/ap.12055

22.
Gonsalvez C. J. Crowe T. P. (2014). Evaluation of psychology practitioner competence in clinical supervision. American Journal of Psychotherapy, 68(2), 177–193. https://doi.org/10.1176/appi.psychotherapy.2014.68.2

23.
Gonsalvez C. J. Wahnon T. Deane F. P. (2017). Goal-setting, feedback, and assessment practices reported by Australian clinical supervisors. Australian Psychologist, 52(1), 21–30. https://doi.org/10.1111/ap.12175

24.
Hall J. C. Spencer R. E. (2017). Illuminating the phenomenological challenges of cross-cultural supervision. Smith College Studies in Social Work, 87(2–3), 238–253. https://doi.org/10.1080/00377317.2017.1324108

25.
Hattie J. Timperley H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

26.
Hutman H. Ellis M. V. Moore J. A. Roberson K. L. McNamara M. L. Peterson L. P. Taylor E. J. Zhou S. (2023). Supervisees’ perspectives of inadequate, harmful, and exceptional clinical supervision: Are we listening? The Counseling Psychologist, 51(5), 719–755. https://doi.org/10.1177/0011000023117254

27.
Ivers N. N. Rogers J. L. Borders L. D. Turner A. (2017). Using interpersonal process recall in clinical supervision to enhance supervisees’ multicultural awareness. The Clinical Supervisor, 36(2), 282–303. https://doi.org/10.1080/07325223.2017.132053

28.
Jacobs S. C. Huprich S. K. Grus C. L. Cage E. A. Elman N. S. Forrest L. Schwartz-Mette R. Shen-Miller D. S. Van Sickle K. S. Kaslow N. J. (2011). Trainees with professional competency problems: Preparing trainers for difficult but necessary conversations. Training and Education in Professional Psychology, 5(3), 175–184. https://doi.org/10.1037/a0024656 (Retraction published 2012, Training and Education in Professional Psychology, 6[4], 219)

29.
Jones C. T. Branco S. F. (2020). The interconnectedness between cultural humility and broaching in clinical supervision: Working from the multicultural orientation framework. The Clinical Supervisor, 39(2), 198–209. https://doi.org/10.1080/07325223.2020.1830327

30.
Kemer G. Li C. Attia M. Chan C. D. Chung M. Li D. Colburn A. N. Peters H. C. Ramaswamy A. Sunal Z. (2022). Multicultural supervision in counseling: A content analysis of peer-reviewed literature. Counselor Education and Supervision, 61(1), 2–14. https://doi.org/10.1002/ceas.12220

31.
King K. M. Borders L. D. Jones C. T. (2020). Multicultural orientation in clinical supervision: Examining impact through dyadic data. The Clinical Supervisor, 39(2), 248–271. https://doi.org/10.1080/07325223.2020.1763223

32.
Ladany N. Mori Y. Mehr K. E. (2013). Effective and ineffective supervision. The Counseling Psychologist, 41(1), 28–47. https://doi.org/10.1177/0011000012442648

33.
LaFromboise T. D. Coleman H. L. K. Hernandez A. (1991). Development and factor structure of the Cross-Cultural Counseling Inventory—Revised. Professional Psychology: Research and Practice, 22(5), 380–388. https://doi.org/10.1037/0735-7028.22.5.380

34.
Lilienfeld S. O. Basterfield C. (2020). Reflective practice in clinical psychology: Reflections from basic psychological science. Clinical Psychology: Science and Practice, 27(4), Article e12352. https://doi.org/10.1111/cpsp.12352

35.
Mitchell M. D. Butler S. K. (2021). Acknowledging intersectional identity in supervision: The multicultural integrated supervision model. Journal of Multicultural Counseling and Development, 49(2), 101–115. https://doi.org/10.1002/jmcd.12209

36.
Motley V. Reese M. K. Campos P. (2014). Evaluating corrective feedback self-efficacy changes among counselor educators and site supervisors. Counselor Education and Supervision, 53(1), 34–46. https://doi.org/10.1002/j.1556-6978.2014.00047.x

37.
Peters H. C. (2017). Multicultural complexity: An intersectional lens for clinical supervision. International Journal for the Advancement of Counselling, 39(2), 176–187. https://doi.org/10.1007/s10447-017-9290-2.

38.
Ponterotto J. G. Gretchen D. Utsey S. O. Rieger B. P. Austin R. (2002). A revision of the multicultural counseling awareness scale. Journal of Multicultural Counseling and Development, 30(3), 153–180. https://doi.org/10.1002/j.2161-1912.2002.tb00489.x

39.
Presser S. Couper M. P. Lessler J. T. Martin E. Martin J. Rothgeb J. M. Singer E. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130. https://doi.org/10.1093/poq/nfh008

40.
Ridley C. R. Kelly S. M. Mollen D. (2011). Microskills training: Evolution, reexamination, and call for reform. The Counseling Psychologist, 39(6), 800–824. https://doi.org/10.117/0011000010378438

41.
Ridley C. R. Mollen D. Console K. Yin C. (2021a). Multicultural counseling competence: A construct in search of operationalization. The Counseling Psychologist, 49(4), 504–533. https://doi.org/10.1177/0011000020988110

42.
Ridley C. R. Sahu A. Console K. Surya S. Tran V. Xie S. Yin C. (2021b). The process model of multicultural counseling competence. The Counseling Psychologist, 49(4), 534–567. https://doi.org/10.1177/0011000021992339

43.
Rocha J. Kemer G. (2022). What must supervisees do to promote supervision? An exploratory sequential mixed methods approach. The Clinical Supervisor, 41(1), 70–87. https://doi.org/10.1080/07325223.2022.2045526

44.
Rogers M. R. O’Bryon E. C. (2014). Multicultural training models and curriculum. In Leong F. T. L. Comas-Díaz L. Nagayama Hall G. C. McLoyd V. C. Trimble J. E. (Eds.). APA handbook of multicultural psychology, Vol. 2: Applications and training. (pp. 659–679). American Psychological Association.

45.
Singh A. Chun K. Y. S. (2010). “From the margins to the center”: Moving towards a resilience-based model of supervision for queer people of color supervisors. Training and Education in Professional Psychology, 4(1), 36–46. https://doi.org/10.1037/a0017373

46.
Sodowsky G. R. Taffe R. C. Gutkin T. B. Wise S. L. (1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41(2), 137–148. https://doi.org/10.1037/0022-0167.41.2.137

47.
Soheilian S. S. Inman A. G. Klinger R. S. Isenberg D. S. Kulp L. E. (2014). Multicultural supervision: Supervisees’ reflections on culturally competent supervision. Counselling Psychology Quarterly, 27(4), 379–392. https://doi.org/10.1080/09515070.2014.961408

48.
Sue D. W. Arredondo P. McDavis R. J. (1992). Multicultural competencies and standards: A call to the profession. Journal of Counseling & Development, 70(4), 477–486. https://doi.org/10.1002/j.1556-6676.1992.tb01642.x

49.
Sue D. W. Bernier J. B. Durran A. Feinberg L. Pedersen P. Smith E. J. Vasquez-Nuttall E. (1982). Position paper: Cross-cultural counseling competencies. The Counseling Psychologist, 10(2), 45–52. https://doi.org/10.1177/0011000082102008

50.
Tran V. T. (2021). Clinical supervisors’ conceptualization and evaluation of supervisee’s multicultural counseling competence: An exploratory study. (Publication No. 29241849). [Doctoral dissertation, Texas A&M University]. ProQuest Dissertations & Theses Global. https://www.proquest.com/dissertations-theses/clinical-supervisors-conceptualization-evaluation/docview/2669542663/se-2

51.
Tugendrajch S. K. Sheerin K. M. Andrews J. H. Reimers R. Marriott B. R. Cho E. Hawley K. M. (2021). What is the evidence for supervision best practice? The Clinical Supervisor, 40(1), 68–87. https://doi.org/10.1080/07325223.2021.1887785

52.
Upshaw N. C. Lewis D. E. Jr. Nelson A. L. (2020). Cultural humility in action: Reflective and process-oriented supervision with Black trainees. Training and Education in Professional Psychology, 14(4), 277–284. https://doi.org/10.1037/tep0000284

53.
Watkins C. E. Jr. (2014). The supervisory alliance as quintessential integrative variable. Journal of Contemporary Psychotherapy, 44(3), 151–161. https://doi.org/10.1007/s10879-013-9252-x

54.
Watkins C. E. Jr. Hook J. N. Owen J. DeBlaere C. Davis D. E. Van Tongeren D. R. (2019). Multicultural orientation in psychotherapy supervision: Cultural humility, cultural comfort, and cultural opportunities. American Journal of Psychotherapy, 72(2), 38–46. https://doi.org/10.1176/appi.psychotherapy.20180040

55.
Wilcox M. M. Gale M. M. McLaughlin K. L. Squyres E. Burish E. C. Khojasteh J. (2021). Development and initial validation of the Multicultural Training and Education Questionnaire. Training and Education in Professional Psychology, 16(1), 55–66. https://doi.org/10.1037/tep0000344

56.
Wilcox M. M. Winkeljohn Black S. Farra A. Zimmerman D. Drinane J. M. Tao K. W. DeBlaere C. Hook J. H. Davis D. E. Watkins C.E. Jr. Owen J. (2023). Cultural humility, cultural comfort, and supervision processes and outcomes for BIPOC supervisees. The Counseling Psychologist, 51(7), 1037–1058. https://doi.org/10.1177/00110000231188337

57.
Wiley E. D. Phillips J. C. Palladino Schulthesis D. E. (2021). Supervisors’ perceptions of their integration of strength-based and multicultural approaches to supervision. The Counseling Psychologist, 49(7), 1038–1069. https://doi.org/10.1177/00110000211024595

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.52 MB