Sage Journals: Discover world-class research

Abstract

OBJECTIVES

In recent years, Indigenous health curricula have been integrated into medical education in response to international calls to improve Indigenous health care. Instruments to evaluate Indigenous health education are urgently needed. We set out to validate a tool to measure self-reported medical student preparedness to provide culturally safe care to Indigenous Peoples. We then applied the tool to evaluate the effectiveness of the Northern Ontario School of Medicine University's (NOSM U) Indigenous health curriculum.

METHODS

We conducted psychometric testing of a 46-item draft NOSM Cultural Competency and Safety Tool (CAST). Testing included principal components analysis, subscale and item analysis, and the use of paired sample t-tests to examine pre- and posttest change to measure learner outcomes. The NOSM CAST was transposed to create a retrospective pre–posttest survey with single-point-in-time scoring.

RESULTS

Respondents included five cohorts of first-year undergraduate medical students, with 305 of 320 participating (response rate of 95.3%). The validated survey subscales included knowledge, confidence/preparedness, attitudes, intentions for advocacy, antidiscrimination, and self-reflective practice, measured using 36 scale items. Cronbach's alpha showed good to excellent internal consistency for the scales (α range = 0.82–0.91). Composite reliability values were acceptable. The pre–posttest analysis showed statistically significant increases on four scales: knowledge [t(254) = 15.10, P < .001], confidence/preparedness [t(254) = 15.85, P < .001], intentions for advocacy [t(251) = 3.32, P = .001], and self-reflective practice [t(254) = 8.04, P < .001]. The largest mean increases were for knowledge (d = 1.07) and confidence/preparedness (d = 1.15).

CONCLUSIONS

The NOSM CAST tracks student progress in Indigenous health curricula. NOSM U's classroom and immersion-based Indigenous health curriculum enhanced students’ self-reported preparedness for culturally safe care. NOSM CAST implemented together with an assessment of Indigenous patient experiences with the same learners constitutes a rigorous evaluation approach to Indigenous health curricula.

Keywords

Indigenous health cultural safety cultural competence curriculum evaluation tool Indigenous curriculum validation social accountability

Introduction

In the context of the United Nations Declaration on the Rights of Indigenous Peoples¹ (UNDRIP) and the Canadian Truth and Reconciliation (TRC) Calls to Action,² policy directions have prioritized reducing access barriers to culturally congruent health care as an essential strategy to move toward the restoration of Indigenous Peoples’ health and wellness.³ Culturally competent and culturally safe care are internationally recognized as quality indicators that reduce healthcare barriers, improve provider-patient communication, patient-centered care, patient satisfaction, and, eventually, health outcomes experienced by Indigenous Peoples.⁴ Conversely, the lack of culturally competent health systems and culturally safe providers can negatively affect Indigenous patients’ health through overt and unconscious biases in diagnostics and treatment.⁵ Further, Indigenous Peoples may avoid accessing care altogether if interpersonal experiences with health service providers lack cultural safety.⁶

In Canada, the Association of Faculties of Medicine of Canada (AFMC) and the Indigenous Physicians Association of Canada (IPAC) released a Joint Commitment to Action on Indigenous Health (JCAIH) Report in 2019.⁷ Subsequently, Health Canada funded the National Consortium for Indigenous Medical Education (NCIME) whose six working groups are focused on developing Indigenous cultural safety and antiracism curriculum frameworks, as well as recommendations regarding admissions, recruitment and support, and physician well-being.⁸ Furthermore, the Committee on Accreditation of Canadian Medical Schools (CACMS) standard 7.6 addresses cultural competence and healthcare disparities.⁹ In order to improve the cultural competence of healthcare providers so that they can provide culturally safe care, these two concepts require succinct definitions within health education. It is important to note that these concepts are still actively debated in the literature with varied interpretations between countries.^10–14 We have found the following definitions helpful in guiding the development of Indigenous health curricula:

Cultural competence is the mastery of measurable skills, knowledge, attitudes, and behaviors in which practitioners become self-aware of their own culture in providing quality care to diverse populations.¹⁵ Potential limitations of cultural competence in medical education is the danger of essentializing Indigenous people or reducing cultural concerns in the clinic to the mastery of a few technical skills. To counteract this constraint, the AFMC has advocated for an emphasis on skill and attitudinal competency along with provider self-reflection to prepare students to engage in culturally safe care.^7,12 Cultural safety extends beyond cultural competence in that self-awareness requires active engagement and reflection of the practitioner in cross-cultural settings that interrogate personal biases, assumptions, and inherent power imbalances between the healthcare provider and the recipient of care^13,16 and “underpin health inequalities/disparities.”¹⁷ A second key concept in cultural safety is that its quality must be evaluated from the perspective of the patient.¹⁸

In order to create culturally safe healthcare systems required to reduce barriers to optimal health for Indigenous Peoples, healthcare providers require thoughtful Indigenous health curricula to support the development of the foundational skills and knowledge to provide care that supports health equity.¹⁹ This preparatory training should help providers strengthen their therapeutic relationships with Indigenous patients, increase Indigenous patient satisfaction, and ultimately improve clinical outcomes. The need for cultural competency training is recognized in nursing and medical associations in many countries with settler colonial histories, including Australia, Canada, New Zealand, and the United States.¹⁸ As a result, nursing, medical, and allied healthcare professional schools in these countries have begun to redress health inequities by implementing Indigenous health in their curricula.^12,20–23 Undergraduate education presents an ideal opportunity to teach skills underlying culturally safe care^10,24 but there is also a need to offer this through continuing professional education.^25,26

NOSM University

The Northern Ontario School of Medicine (NOSM U) was established in 2002 with an explicit social accountability mandate focused on improving the health of the people and communities in Northern Ontario, including Indigenous Peoples and communities.^27,28 Initially, it served as the faculty of medicine for Lakehead and Laurentian Universities and then in 2021 became a standalone University as a result of the NOSM University Act. In this context, as well as concurrent broader global developments in medical education, NOSM U developed an Indigenous health curriculum in consultation with Indigenous communities that has been described in detail elsewhere.^27–36 Briefly, it consists of lectures on a spectrum of topics related to Indigenous Peoples’ health including diversity of culture, history, colonialism, legislation, health policies, Indigenous health services, traditional medicine, and mental health and child welfare. In addition, relevant medical and clinical science curriculum is taught. Some of these topics are explored in small and large in-class groups during the first year of medical school. The first year culminates with a major curriculum commitment to Indigenous health, the world's only mandatory 4-week placement in Northern and rural Indigenous communities. It includes community-guided cultural and clinical experiences, and reflective tutorials with experienced faculty to help students make sense of the cultural, experiential, and health systems learning and the relevance of this curriculum to their future medical practice, and in particular, relevance to providing culturally safe care to Indigenous Peoples. The need to enhance cultural competency objectives with a focus on cultural safety and the inclusion of community led curriculum was realized early on; our 2005 pilot showed that without this intentionality student learning could be limited.^33,35

Evaluating Indigenous health curricula

While the UNDRIP¹ and TRC policy statements (eg TRC Call to Action 24 concerning medical education)² have been adopted by educational institutions and Indigenous health curricula implemented, the research evaluating these curricula is mainly limited to learner satisfaction data. Rigorous research assessing the outcomes of cultural competence training and its translation into culturally safe care is still largely lacking.^26,37–40 In a systematic review, Clifford et al.⁴¹ concluded that

There is a lack of evidence from rigorous evaluations on the effectiveness of interventions for improving cultural competency in health care for Indigenous Peoples. Future evaluations should employ more rigorous study designs and extend their measurement of outcomes beyond those relating to health professionals, to those relating to the health of Indigenous Peoples.

Similarly, based on a systematic review of the literature, Hardy et al.³⁹ also called for more research because currently, there is

limited evidence regarding the effectiveness of specific content and approaches to cultural safety training on improving non-Indigenous health professionals’ knowledge of and skills to deliver quality, non-discriminatory care to Indigenous patients.

Clearly, research that measures the impact of cultural competency curriculum on learners’ progression in the relevant skills for culturally safe care is needed. Furthermore, evaluating skills for culturally safe care should include an evaluation of learners by Indigenous patients. However, methodological challenges make it difficult to measure patient perspectives, especially during undergraduate training. We propose that the first step toward rigorous evaluations of the Indigenous health curriculum requires foundational research to establish validated outcome measures and to implement benchmarking to track progress against long-term learning goals. Although some of this work has been started in midwifery education,⁴² our review of the literature indicated a lack of appropriate, validated scales to evaluate medical education of Indigenous health curricula.

Exacerbating this knowledge gap is the current lack of a sophisticated understanding of the incremental learning levels that should be attained to meet the learning outcome of provider preparedness for culturally safe care. In addition, there is little research evidence of teaching strategies that support students in acquiring these skills. Medical educators need a better understanding of the specific aspects of Indigenous health curricula that enhance the knowledge, skills, and professional behavior of students, healthcare providers, and physicians as they prepare to become culturally safe providers.^19,43 We argue that discourse about the effectiveness of Indigenous health curricula is currently almost exclusively based on anecdotes and personal perceptions. Validated instruments are therefore urgently needed to evaluate teaching strategies and learning outcomes related to Indigenous health curricula.⁴⁴

We seek to address these knowledge gaps by developing, validating, and applying such an instrument through multiyear research with medical students at NOSM U in Canada. NOSM U is an ideal site for this research because the School designed its undergraduate medical curriculum to focus explicitly on Indigenous health as part of its social accountability mandate.²⁹ NOSM U's Indigenous health curriculum was designed to prepare medical students to address health disparities through longitudinal, knowledge-based, reflective, experiential, and immersive learning about Indigenous cultures, communities, health and health services, and culturally safe care.^30–36

To measure the effectiveness of the curriculum, our team of researchers first created the Northern Ontario School of Medicine Cultural Competency and Safety Tool (NOSM CAST), designed to measure self-reported progression in students’ readiness to provide culturally safe care to Indigenous patients over eight months of participating in the NOSM U integrated Indigenous health curriculum. In this article, we (a) describe the development process of learning constructs that prepare students for culturally safe care, (b) outline the validation process of an instrument designed to measure these constructs, (c) present the validated instrument, and (d) report Indigenous health learning outcomes for five cohorts of NOSM U students.

Methods

We set out to develop a validated learner self-report survey instrument to measure the effectiveness of the Indigenous health curriculum in preparing students for culturally safe care. Our reporting follows the Standards for Quality Improvement Reporting Excellence for Education (SQUIRE-EDU) which was developed to enhance the quality of health professional education research (Appendix 1 in the online supplemental materials).⁴⁵

Situating our team

Our team is composed of Indigenous and non-Indigenous faculty and staff who have led and supported the development and implementation of the Indigenous health curriculum at NOSM and then NOSM U over the past 20 years. Past and present roles include: Pilot (GH) and Module Coordinator of the Indigenous cultural immersion placement module (GH, MM, LM, DM); Chair of the Northern and Rural Health Course (KJ, MM); Chair of the Personal and Professional Aspects of Medical Practice Course (GH); Chair of the Social and Population Health Course (GH, MM, DU); Module Content Coordinators for one or more of these courses for the Indigenous Placement Module (GH, MM, LM, DM); Director of the Centre for Rural and Northern Health Research (WW, DU); AMS-NOSM U Hannah Chair in the History of Indigenous Health and Indigenous Traditional Medicine (DM); Indigenous Community Coordinator (SS); Director of Indigenous Affairs (DM, SS); and Founding Dean of NOSM U (RS). All team members have engaged and collaborated with Indigenous community members and Indigenous healthcare providers in formal and informal ways to shape the Indigenous health curriculum at NOSM U. Three coauthors are also members of Indigenous communities who host students in NOSM U's month-long Indigenous cultural immersion curriculum (DM, LM, SS).

Overview of the research approach

During Phase 1 of the research, we drafted an instrument to assess cultural safety preparedness learning constructs, populated in part with items adapted from the published literature. Item generation and content validity of a draft instrument to measure learner preparedness to provide culturally safe care to Indigenous Peoples were also established. In Phase 2, we conducted principal component analysis (PCA) of the survey items using pretest data from five cohorts of medical learners to assess the internal structure of the instrument. PCA is an established exploratory technique used in test development for examining relationships among variables; the purpose is to identify dimensions or components from a larger pool of items.⁴⁶ We used the method to determine viable subscales for inclusion in our tool.

We also generated additional indices of reliability and validity, including pre- and posttest survey results to measure learner outcomes of the Indigenous health curriculum at NOSM U. In Phase 3, we transposed the pre- and posttest instrument to create a retrospective pre–posttest tool to simplify future data collection to a single-point-in-time scoring, to support realistic evaluation processes in everyday teaching environments.⁴⁷

Phase 1: survey development

Current theoretical competency frameworks and policy statements on cultural competence and cultural safety were reviewed^12,21,48 alongside NOSM U's Indigenous Community Gathering reports.^49,50 Based on this review and the research team's collective experience in Indigenous health education, we developed a conceptual framework of enablers of provider preparedness for culturally safe care. Preparedness for culturally safe care was conceptualized based on earlier work by Ramsden,¹¹ Kurtz,¹⁸ the AFMC,⁷ and NCIME⁸ which combined the attainment of relevant competencies with the process of reflection to understand biases and power imbalances between the patient and the provider. The following conceptual domains comprised the framework of enablers of preparedness to provide culturally safe care: (1) knowledge/awareness of Indigenous Peoples’ health and its determinants; (2) confidence/perceived preparedness to provide care for Indigenous Peoples; (3) attitude toward providing care for Indigenous Peoples; (4) intentions for advocacy for Indigenous health issues; (5) anti-discrimination attitudes with respect to Indigenous Peoples; and (6) engagement in self-reflective practice related to culturally safe care (see Figure 1).

Figure 1.

Conceptual framework of enablers of readiness for culturally safe care.

Scale items from published scales on cultural awareness, sensitivity, competency, and safety were reviewed and appropriate items selected for adaptation to the Indigenous context. The search identified nine published scales that could provide adapted or verbatim scale items for the NOSM CAST (Appendix 2 in the online supplemental materials). Only one scale of cultural safety was identified, however it was aimed at the recipients of care rather than providers of care.¹⁰ Therefore, questions geared to identify provider preparedness for culturally safe care for Indigenous Peoples were created by cross-referencing learning objectives for Indigenous health curricula by Indigenous physician and nursing associations.^12,21 Items to assess attitudes of antidiscrimination were adapted from Gawronski's discrimination scale.⁵¹ Finally, items assessing our constructs of provider self-reflection and reflective practice and intentions for advocacy were developed de novo.

A total of 146 potential survey items were identified for inclusion into the NOSM CAST, mapped to our conceptual domains, and revised or removed if they were redundant or questionably relevant. Each item was assigned a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree, and the research team then again reviewed the tool for content validity. Eleven items were reverse coded to minimize response bias. Using a Delphi approach modified for Indigenous research,⁵² the team members revised the items iteratively until a consensus on the content was reached, and the measure was reviewed again in its entirety by the research team for relevance and clarity. The content review and revision process resulted in the inclusion of 46 items in the survey. The resulting scale items and their conceptual origins in the literature are presented in Appendix 2 in the online supplemental materials.

Using the 46-item tool, NOSM CAST data were collected for five cohorts of undergraduate medical students to measure learner outcomes of the Indigenous health curriculum delivered in their first year at NOSM U. Pretest data were collected at the beginning of the first academic year and the posttest data were collected approximately eight months later at the end of the same year. During the eight months, the students experienced a variety of Indigenous health learning opportunities, including lectures and small group learning, led by NOSM U faculty. Opportunities included learning from Indigenous faculty, patients, and knowledge keepers (described elsewhere), and culminated with a 4-week Indigenous community health immersion placement within a 7-week module focusing on Indigenous health.^29–36

Phase 2: survey validation

The draft NOSM CAST survey was validated in the second phase of the study. Inclusion criteria was enrollment as a student in the first year of the NOSM U UME program at the time of participation in the survey. The survey was administered to the 2013–2014 cohort of 64 NOSM U first-year undergraduate medical education (UME) students, and again with the next four consecutive cohorts of incoming students: the 2014–2015, 2015–2016, 2016–2017, and 2017–2018 cohorts of 64 students, respectively. The purpose of Phase 2 was to validate the instrument against the conceptual domains identified in Phase 1. We conducted statistical analyses to test (a) the internal structure of the survey using principal components analysis (PCA); (b) the internal consistency (Cronbach's α) and composite reliability (CR) of each scale; (c) item-total correlations; and (d) pre–posttest change using paired samples t-test with Eta square for effect size. In addition, descriptive statistics for the items and scales (means, standard deviations, skewness, and kurtosis) in the preliminary and final surveys were examined.

Guidelines suggest various minimum sample size estimates for the development of psychometric tools using PCA.⁵³ For example, Comrey and Lee⁵⁴ have recommended sample sizes of 50 as very poor, 100 as poor, 200 as fair, 300 as good, 500 as very good, and 1000 as excellent. In general, minimum required sample size has focused on the number of variables, the number of factors, the number of variables per factor, and the magnitude of communalities (with higher communalities, >0.40; factor loadings, >0.35; and more variables/per factor suggesting increased adequacy and interpretability). However, simulation analyses have shown that as variable to factor ratios and levels of communality increase, required sample sizes decrease.⁵⁵ We sought to adopt a balanced approach in the development of our instrument, and based our validation process (PCA, reliability and pre–post testing) on a minimum sample size of 300. The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was also examined.⁵³

Phase 3: NOSM CAST transposed as a retrospective pre–posttest tool

After validating the NOSM CAST, we transposed the content into a retrospective pre–posttest tool.⁴⁷ The retrospective pre–posttest tool was not used for data collection in this study.

Results

In this section we describe the five cohorts of medical student participants, followed by the survey validation analyses, including the PCA analyses of the NOSM CAST survey items, scale and item analyses, and a pre–posttest examination of NOSM CAST scales and items pre- and postimmersion in the first-year Indigenous health curriculum in the medical education program at NOSM U. We also provide the NOSM CAST retrospective pre–posttest instrument.

Participants

Participants included 305 of 320 (response rate of 95.3%) first-year undergraduate medical education students. Approximately two-thirds of the sample were female (63%), and one-third (37%) were male. The sociodemographic profile of the five student cohorts from the 2013–2014 to 2017–2018 academic years is presented in Table 1. The average age of participants was 25 years. Most students resided in northern Ontario for more than seven years (89–95%), and, on average, approximately 8% of students resided in rural or remote areas. Indigenous students comprised between 5% and 9% % of each cohort, whereas about 17% self-identified as Francophone.

Table 1.

Sociodemographic characteristics for the undergraduate student cohorts (academic years 2013–2014 to 2017–2018).

SOCIODEMOGRAPHIC	2013–2014	2014–2015	2015–2016	2016–2017	2017–2018	TOTAL
Eligible students	64	64	64	64	64	320
Student participants	63	64	64	58	56	305
Age (years)	25	25	25	26	25	25.2
Gender—female (%)	70	59	56	73	58	63.2
Gender—male (%)	30	41	44	27	42	36.8
Self-identified indigenous (%)	9	5	8	8	7	7.4
Self-identified Francophone (%)	20	19	20	12	14	17
Northern Ontario (%)	95	92	89	89	92	91.4
Rural and remote Canada (%)	5	8	9	9	8	7.8

NOSM CAST survey validation

Principal component analysis (PCA)

PCA of the 46 items was performed on the data from the 305 undergraduate students using IBM SPSS 29.0.⁵⁶ Initial PCA with an oblique rotated solution (Promax with Kaiser Normalization) was specified. Components extracted were assessed at Eigenvalues > 1.0, and factor loadings with a cutoff of 0.35. The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy (0.906) indicated that the data were suitable for PCA. Likewise, Bartlett's Test of Sphericity (<0.001) suggested sufficient correlation between the items to proceed. The initial analysis resulted in nine components with eigenvalues greater than 1.0. Examination showed that two components failed to retain sufficient items (factor loadings < 0.35). However, further specification of seven components still showed an inadequate number of retained items for two components. When the number of components was fixed at five for extraction, an acceptable number of items per component were achieved. The five components accounted for 50.3% of the total variance.

Thirty-seven items were retained; the factor loadings and communalities are illustrated for these items in Table 2. Items 1 and 34 had low factor loadings (<0.35) and poor communalities (<0.40). Items 3, 8, 9, 20, 22, 23, and 28 also had communality extraction scores below 0.40; this suggests relatively little variance explained by the component structure for these items. Overall, the obtained component structure corresponded to the proposed conceptual domains, with one main exception. The items associated with the confidence/perceived preparedness domain loaded on the same component (Component 1) as the knowledge/awareness items. In addition, one item from the attitude domain (item 26), and two items from the self-reflection domain (items 31 and 46), loaded on this component. Item 26 was related to one's comfort within Indigenous communities, and item 46 concerned one's understanding of how to integrate traditional approaches into the care of Indigenous patients. These items reflected confidence in, and perceived preparedness regarding the provision of care to Indigenous Peoples. Item 31 included learning to enhance one's skills to provide effective care, which aligned with the knowledge/awareness domain. Component 1 thus comprised 13 items that reflected two subscales: (1) knowledge/awareness of Indigenous Peoples (6 items), and (2) confidence/perceived preparedness in providing care to Indigenous Peoples (7 items).

Table 2.

Rotated component matrix of NOSM CAST items (academic years 2013–2014 to 2017–2018).

ITEM #	ITEM	DOMAIN	COMPONENT					COMMUNALITY
ITEM #	ITEM	DOMAIN	1	2	3	4	5	COMMUNALITY
Knowledge/awareness of Indigenous Peoples’ health and its determinants
I-1	I have an understanding of the barriers faced by Indigenous Peoples that contribute to health inequities	Knowledge	–	–	–	–	–	0.367
I-14	I have an understanding of the diverse cultural beliefs of the Indigenous Peoples in our region	Knowledge	0.622	–	–	–	–	0.506
I-18	I am able to describe the strengths of the Indigenous communities in my region	Knowledge	0.743	–	–	–	–	0.526
I-19	I have an understanding of Indigenous community contacts and local support structures in Indigenous communities	Knowledge	0.864	–	–	–	–	0.654
I-23	Health effects of the residential school system are propagated through several generations	Knowledge	–	–	–	–	–	0.374
I-31^a	I realize I have more to learn in order to enhance my ability to be a good physician for Indigenous Peoples	Knowledge	0.501	–	–	–	–	0.455
I-33	I have an understanding of colonization and its historical impact, including historical treatment of Indigenous Peoples by the Canadian Government	Knowledge	0.438	–	–	–	–	0.460
I-43	I have an understanding of the resilience and strength of Indigenous Peoples	Knowledge	0.417	–	–	–	–	0.540
Confidence/perceived preparedness to provide care for Indigenous Peoples
I-5	I feel confident in my ability to manage my nonverbal behavior and body language in a way that fits with the expectations and norms of Indigenous patients	Confidence	0.534	–	–	–	–	0.432
I-10	I feel confident in my ability to recognize the importance of humor to Indigenous cultures	Confidence	0.586	–	–	–	–	0.504
I-17	I feel confident in my ability to appropriately inquire whether a patient is taking traditional herbs or medicines	Confidence	0.644	–	–	–	–	0.448
I-20	I feel confident in my ability to be nonjudgmental	Confidence	–	–	–	–	–	0.395
I-26	I feel comfortable within Indigenous communities	Confidence	0.614	–	–	–	–	0.449
I-28	I have enough knowledge that I could disagree with a clinical preceptor who holds a discriminatory view toward Indigenous patients	Confidence	–	–	–	–	–	0.375
I-29	I feel prepared to provide culturally safe care to Indigenous Peoples	Confidence	0.689	–	–	–	–	0.541
I-44	I have an understanding of the Indigenous community healthcare team and how to involve them in the care of patients	Confidence	0.845	–	–	–	–	0.665
I-46	I have an understanding of how to facilitate integration of traditional approaches to healing into the care of Indigenous patients who desire it	Confidence	0.743	–	–	–	–	0.524
Attitude toward providing care for Indigenous Peoples
I-2	Establishing a trusting relationship with Indigenous patients is important to providing good care	Attitude	–	–	–	0.570	–	0.437
I-4	I am open to learning from Elders and traditional healers	Attitude	–	–	–	0.703	–	0.653
I-11	Culturally influenced spirituality and religious beliefs are important aspects of a patient's decisions around their health	Attitude	–	–	–	0.564	–	0.451
I-16	I feel that I can learn from Indigenous patients	Attitude	–	–	–	0.483	–	0.488
I-22^a	When I am surrounded by culturally diverse individuals, I feel that my own beliefs and values are being threatened	Attitude	–	–	–	–	–	0.300
I-25	Understanding a patient's cultural and social background will help me provide better care as a clinician	Attitude	–	–	–	0.632	–	0.588
I-32	Learning about alternative/non-Western medicine and traditional healing practices is an important part of medical training	Attitude	–	–	–	0.538	–	0.429
Intentions for advocacy for Indigenous health issues
I-8	I believe that physicians should use their position and status to redress existing health inequalities in Indigenous populations	Advocacy	–	–	–	–	–	0.256
I-27	After medical school, I will continue to be involved in Indigenous communities	Advocacy	–	0.379	–	–	–	0.483
I-37	I would support an initiative or outreach directed at Indigenous patients in their community to improve Indigenous health	Advocacy	–	0.818	–	–	–	0.609
I-38	I would raise public awareness about an Indigenous health or social issue by discussing it with family and friends	Advocacy	–	0.862	–	–	–	0.717
I-39	I would collaborate with other healthcare providers and agencies to solve a health problem in an Indigenous community	Advocacy	–	0.808	–	–	–	0.731
I-40	As a physician I would be willing to write a letter, sign a petition or participate in another form of public advocacy or lobbying to support an Indigenous health or social issue	Advocacy	–	0.747	–	–	–	0.642
I-41	As a physician, I would raise awareness among patients and colleagues about a pressing issue affecting Indigenous Peoples through informal discussions, a seminar, or other forum	Advocacy	–	0.838	–	–	–	0.680
I-42	I would consider advocating at a political level for Indigenous health	Advocacy	–	0.763	–	–	–	0.562
Antidiscrimination attitudes with respect to Indigenous Peoples
I-3^a	Indigenous Peoples would be best served by policies that integrate them fully into Canadian culture and society	Antiracism	–	–	–	–	–	0.264
I-6^a	A physician can be just as effective without learning about a patient's cultural perspective	Antiracism	–	–	0.470	–	–	0.452
I-9^a	There may be a few exceptions, but in general, Indigenous Peoples are pretty much alike	Antiracism	–	–	–	–	–	0.392
I-12^a	Discrimination against Indigenous Peoples is no longer a big problem in Canada	Antiracism	–	–	0.744	–	–	0.520
I-21^a	Society has reached a point where Indigenous and non-Indigenous Peoples have equal opportunities	Antiracism	–	–	0.693	–	–	0.584
I-30^a	Creating government policies that correct for inequalities among Indigenous Peoples is no longer necessary	Antiracism	–	–	0.704	–	–	0.549
I-34^a	Indigenous Peoples have the same desire to raise a healthy family as most other Canadians	Antiracism	–	–	–	–	–	0.269
I-35^a	The benefits Indigenous Peoples receive give them an unfair advantage compared to other Canadians	Antiracism	–	–	0.768	–	–	0.601
I-36^a	Indigenous Peoples need to put their past behind them	Antiracism	–	–	0.681	–	–	0.528
Engagement in self-reflective practice related to culturally safe care
I-7	I have reflected on potential power imbalances between myself and others that may be related to my status as a medical student and future physician	Reflection	–	–	–	–	0.648	0.503
I-13	I examine my own biases related to culture that may influence my behavior as a service provider	Reflection	–	–	–	–	0.801	0.617
I-15	I have reflected on how my own cultural identity can impact the care I provide	Reflection	–	–	–	–	0.711	0.532
I-24	I reflect on and examine my own cultural background, biases, and prejudices that may influence my behavior	Reflection	–	–	–	–	0.808	0.644
I-45	I understand my own biases and preconceived notions about Indigenous Peoples	Reflection	–	–	–	–	0.633	0.466
Total Initial Eigenvalues			12.04	5.64	2.06	1.80	1.63
Percentage of Variance			26.17	12.26	4.47	3.91	3.55
Cumulative Variance			26.17	38.43	42.89	46.80	50.35

Items removed from the scale are gray shaded (9 items). Factor loadings below 0.35 are suppressed.

Items reverse coded so that strongly agree = 1, strongly disagree = 5.

Component 2 was defined by 7 items assessing the concept of advocacy for Indigenous health issues, including raising awareness, and supporting public initiatives and outreach activities around Indigenous health. Item 27 (attitude domain) also loaded on this component and reflected continued involvement in Indigenous communities upon completion of one's medical training. Component 3 loaded with 6 items associated with antidiscrimination with respect to Indigenous Peoples, including item 6 (attitude domain), which assesses one's perspective on providing care to Indigenous Peoples without learning about a patient's culture. Component 4 was loaded with 6 items on the concept of attitudes, including the importance of trust and understanding cultural, spiritual, and religious beliefs in one's decision-making for Indigenous patients. Last, the 5 items loading on Component 5 were associated with one's self-reflection and awareness of their cultural identity and biases as a care provider. The component matrix of NOSM CAST items with factor loadings is shown in Table 2.

Scale and item analyses

Results for the NOSM CAST scale and item analyses are presented in Appendix 3 in the online supplemental materials, for the 37 items remaining after the exploratory PCR analysis. Cronbach's alpha showed good to excellent internal consistency for each of the scales (ranging from α = 0.82–0.91), and the CR values were acceptable (knowledge scale = 0.78; preparedness scale = 0.85; attitude scale = 0.76; advocacy scale = 0.90; antidiscrimination scale = 0.84; self-reflective practice scale = 0.84). The Cronbach alpha for the advocacy scale was >0.90, and CR, 0.90, which indicates some redundancy for the items on this scale.⁵⁷ The item-total correlations for the items on this scale were somewhat high (Appendix 3 in the online supplemental materials). However, the decision was made to retain all advocacy items as each captured distinctive educational objectives, including advocacy for public awareness of Indigenous health and social issues. The item-total correlations for the remaining items across each of the scales were within the acceptable range (ie 0.30–0.70), with the exception of the correlation for item 31 (Pearson r = 0.15). This item was subsequently removed from the knowledge scale, due to the low item-total correlation for this item.

Pre–posttest change—immersion in the NOSM U Indigenous health curriculum

The combined average pre- and posttest scores for the NOSM CAST subscales are presented in Table 3 for the 36-item measure. There were statistically significant increases pre- to postcurricular immersion in learner knowledge of Indigenous People's health [t(254) = 15.10, P < .001], preparedness to provide care for Indigenous Peoples [t(254) = 15.85, P < .001], intentions for advocacy for Indigenous health issues [t(251) = 3.32, P = .001], and engagement in self-reflective practice related to culturally safe care [t(254) = 8.04, P < .001]. The largest mean increases were for knowledge [M change = 0.73, 95% CI (0.64, 0.83), d = 1.07] and preparedness to provide clinical care (M change = 0.76, 95% CI (0.66, 0.85), d = 1.15] to Indigenous Peoples. All item responses significantly increased on these scales (see Appendix 4 in the online supplemental materials for detailed item statistics). Results showed the largest effect sizes for items covering knowledge of local Indigenous community contacts and community support structures (d = 1.21); understanding of the Indigenous community healthcare team and how to involve them in the care of patients (d = 1.11); knowledge of the strengths of regional Indigenous communities (d = 1.0); and understanding how to integrate traditional approaches to healing in the care of Indigenous patients (d = 0.97).

Table 3.

NOSM CAST average subscale scores and paired sample statistics.

DOMAIN	PRETEST MEAN (SD)	POSTTEST MEAN (SD)	95% CONFIDENCE INTERVAL OF THE DIFFERENCE		PAIRED t-TEST
DOMAIN	PRETEST MEAN (SD)	POSTTEST MEAN (SD)	LOWER	UPPER	t VALUE	SIG (TWO-TAILED)	EFFECT SIZE d
Knowledge	3.32 (0.75)	4.05 (0.62)	0.64	0.83	15.10	0.001*	1.07
Confidence/perceived preparedness	3.35 (0.73)	4.11 (0.58)	0.66	0.85	15.85	0.001*	1.15
Attitude	4.54 (0.44)	4.54 (0.52)	−0.05	0.05	.074	0.94	0.00
Intention for advocacy	4.09 (0.66)	4.23 (0.80)	0.06	0.23	3.31	0.001*	0.20
Antidiscrimination	4.32 (0.64)	4.30 (0.63)	−0.08	0.04	−0.68	0.50	0.03
Self-reflection	3.86 (0.56)	4.17 (0.59)	0.24	0.39	8.04	0.001*	0.54

N = 255.

*Significant at P < .001, two-tailed.

The pre–posttest change for the self-reflective practice scale was also relatively large [M change = 0.31, 95% CI (0.24, 0.39), d = 0.54], with the largest average increases observed for item 45 (understanding one's own biases and preconceived notions about Indigenous Peoples) [M change = 0.34, 95% CI (0.22, 0.45), d = 0.44)] and item 7 (reflective practice related to power imbalances) [M change = 0.33, 95% CI (0.21, 0.45), d = 0.38]. However, change for the advocacy scale was more modest [M change = 0.14, 95% CI (0.06, 0.23), d = 0.20], with item 37 having the largest effect size among individual items, and which assesses advocacy for Indigenous health or social issues with family and friends (d = 0.23). Average pre–posttest scores for the attitude and antidiscrimination scales showed only small, nonstatistically significant changes (Table 3; Appendix 4 in the online supplemental materials).

Retrospective pre–posttest instrument

Implementing a pre- and posttest tool requires research time and resources. It may be unrealistic to implement these in everyday teaching environments.⁴⁷ Therefore, we transposed the validated scales into a retrospective posttest tool that requires only a single testing time for use in future studies and ongoing program evaluation of Indigenous health curricula in a variety of training sessions. The instrument is provided in Appendix 5 in the online supplemental materials.

Discussion

An important emergent area of scholarship in health education is research on the effectiveness of social accountability-related education such as Indigenous health curricula in order to enable health educators to engage in continuous quality improvement to ensure that curricula are meeting the intended outcomes.⁵⁸ While there has been a flurry of development of Indigenous health curricula as a strategy to improve Indigenous patient care, little has been done to establish outcome measures for these curricula and to evaluate their effectiveness. While ultimately these measures should include Indigenous patient data such as perceived cultural safety and Indigenous patient health outcomes, our research has created a rigorous first step in this direction by creating a validated instrument to measure outcomes of Indigenous health curriculum in health education that tracks changes in students’ preparedness for culturally safe care.

Survey development and validation

We found that psychometric testing generally supported the domain constructs that the content experts developed; however, several items had to be removed or moved to a different domain based on the results of the PCA and item analyses.

Based on the PCA, nine items were removed due to poor factor loadings and communalities. Two items related to learners’ understanding of the barriers that lead to health inequities (I-1 “I have an understanding of the barriers faced by Indigenous Peoples that contribute to health inequities”) and the multigenerational effects of colonial violence (I-23 “Health effects of the residential school system are propagated through several generations”). It is possible that the understanding of these items require knowledge of chronic illness and comorbidity care which may be too advanced for students at this early stage in medicinal education. Item I-20 (“I feel confident in my ability to be nonjudgmental”) was also removed from the instrument. Anticipated was that this item would have a stronger relationship with the knowledge/confidence domains/component, but it did not contribute to the factor structure. This may indicate that the idea of nonjudgment has a weak or complex relationship with the constructs set forth in the NOSM CAST. It may also suggest that judgment does not align with confidence in one's ability around engaging in traditional healing practices and providing culturally safe care.

Likewise, the item designed to measure learners’ ability to disagree with a preceptor (I-28 “I have enough knowledge that I could disagree with a clinical preceptor who holds a discriminatory view toward Indigenous patients”) was removed. In retrospect, this item requires the confidence to disagree with a preceptor, which is often impeded by feelings of vulnerability as learners (such as the first-year students in this case) and the fear of risking poor evaluations instead of having an expected relationship with knowledge or preparedness to deliver culturally safe care. A recent Canadian study on sexual harassment of medical learners underscored their inherent fear of challenging preceptors in the medical education system due to the power imbalance in this relationship.³⁵ Further, the item I-22 (“When I am surrounded by culturally diverse individuals, I feel that my own beliefs and values are being threatened”) was removed; the item appears to require a high level of awareness of personal triggers when confronted with unfamiliar cultures and belief systems.

It is interesting to note that the one item assessing advocacy removed from the scale is the only item that is a general statement about the responsibility of the medical profession concerning advocacy (I-8 “I believe that physicians should use their position and status to redress existing health inequalities in Indigenous populations”). All other statements are focused on learners’ willingness to contribute to advocacy personally. It appears that the general responsibility may be a separate construct which contrasts with the students’ interest in becoming an advocate. While the internal consistency reliability are high for this scale, and therefore highly related, we believe the items are not redundant as they cover diverse areas of advocacy in healthcare, including at the interpersonal, organizational, and health systems level.⁵⁹

The antidiscrimination domain was perhaps the most challenging to measure, and three items were removed due to low communality extraction scores. These items included: (I-9) “There may be a few exceptions, but in general, Indigenous Peoples are pretty much alike,” (I-3) “Indigenous Peoples would be best served by policies that integrate them fully into Canadian culture and society,” and (I-34) “Indigenous Peoples have the same desire to raise a healthy family as most other Canadians.” This is an emerging field of inquiry of a complex construct and more work needs to be done to better understand the underlying concepts of discrimination and racism, both from the perspective of experiencing racism as well as implicit attitudes. That the items were poorly performing suggests lack of clarity in the wording presented to respondents, and/or variations in the interpretation of these items. To reduce acquiescence bias, these items were reverse coded, but reverse coding may affect psychometric estimates.^60,61 However, we sought to minimize the influence of coding method by reverse scoring all of the items in this construct instead of using mixed response option coding.⁶²

Results of the item analyses yielded a poor relationship between one knowledge item [(I-31) “I realize I have more to learn in order to enhance my ability to be a good physician for Indigenous Peoples”] and its respective knowledge scale. The item-total correlation for this item was r = .15; thus, the decision by the research team to remove this question. Although this item was included in the NOSM CAST tool as an indicator of knowledge of Indigenous People's health, the results suggest that it measures other underlying concepts, or nuances of knowledge acquisition that are not measured by the other items in the scale. It is also possible that social desirability affected the results; this item warrants further examination, eg using a retrospective pre–posttest design or the pre–posttransposed version of the NOSM CAST.

Effectiveness of the NOSM U Indigenous health curriculum in fostering readiness for culturally safe care among medical students

The NOSM U Indigenous health curriculum has been described in detail elsewhere.^27–36 Applying the NOSM CAST tool to five cohorts of first-year medical students, there is clear evidence of statistically significant growth in knowledge and awareness, confidence to provide care to Indigenous patients, intentions to advocate and reflective practice, all of which are key prerequisites for culturally safe care.

Knowledge/awareness of Indigenous Peoples’ health and its determinants

An increase in knowledge and awareness of Indigenous Peoples is likely connected to curriculum content focused on foundational information concerning the impact of historical, colonial, political, economic, and socio-cultural dynamics on the health of Indigenous Peoples, as well as basic knowledge of the diverse cultures and communities. This content is taught didactically as well as through Indigenous patient stories and the cultural immersion experience.

Confidence/perceived preparedness to provide care for Indigenous Peoples

Confidence and preparedness increases are particularly important as we, as educators, often observe a level of hesitancy among many non-Indigenous learners to engage with Indigenous patients, often out of fear of embarrassment, and fears of inadvertently offending Indigenous Peoples in conversation and, for some students, shame of the settler colonial history.^31,32,34 Our research with Indigenous patient actors has also shown that some Indigenous students may lack confidence with Indigenous patients because they perceive themselves to lack cultural knowledge and may worry about judgment by peers and patients.³²

Intentions for advocacy for Indigenous health issues

Advocacy for Indigenous health issues in curriculum requires a coordinated approach between institutional practices and the educational curricula.¹⁹ Increases in advocacy for social justice of marginalized populations, including raising awareness, and supporting public initiatives and outreach activities are known to result from cultural immersion experiences.¹⁹ Involving health humanities aspects of education such as the incorporation of meaningful interaction with elders, significant consideration of the history of health and medicine, and exposure to alternative representations of knowledge and self-reflection contributes to emotional growth, especially in empathy and compassion,⁶³ in part arising in an iterative process from relevant cues in an immersion self-study project.^29–36 The NOSM U Indigenous health curriculum addresses the TRC Call to Action 24 including the requirement that all students study “…the history and legacy of residential schools, the United Nations Declaration on the Rights of Indigenous Peoples, Treaties and Aboriginal rights, and Indigenous teachings and practices.”² In addition, the curriculum provides students with “skills-based training in intercultural competency, conflict resolution, human rights and anti-racism” as described in the TRC Call to Action 24.² Perhaps most importantly, students are able to observe health inequities and reflect on the need for advocacy during their month-long cultural immersion experience.

Antidiscrimination

The mean pretest score for this subscale was very high even at the pretest stage (4.5 out of 5) and showed no statistically significant change. This finding indicates that incoming NOSM U students, tested within weeks of beginning medical studies, hold perceptions that are aligned with positive attitudes toward Indigenous patients, and nondiscrimination attitudes, and it would be difficult to increase these scores even further. The high average pretest score may be a result of NOSM U attracting and preselecting students who identify with the values of social accountability through the admissions process, although social desirability bias may also be an important factor in this high score especially since students had recently studied and passed a socially accountable admissions process.

Pre- and posttest show only statistically insignificant variations and we can conclude that these personal convictions and commitment to antiracism and antidiscrimination appear to remain stable, while knowledge, confidence, and interest in advocacy increase. It is possible however that this construct may not only be affected by a social desirability bias, but also a response-shift bias, meaning students may be idealistically overestimating their antidiscriminatory perspectives at the pretest. Some researchers have argued that a retrospective pre–posttest may be a better measure when the internal frame of reference of a construct has the potential to change pre- and posttest. In these cases, a retrospective pre–posttest can reduce the effect of the response-shift bias.⁴⁷ We are very interested in testing this subscale further with the retrospective pre–posttest version of NOSM CAST and analyzing differences in this measure when the two different approaches are applied.

Engagement in self-reflective practice related to culturally safe care

Increase in one's self-reflection and awareness of one's cultural identity and biases as a healthcare provider is a key prerequisite for culturally safe care. The curriculum creates space during weekly small group reflective sessions in preparation for, during, and after the Indigenous community placement. The placement culminates in a research and reflection project that students present in their placement community as well as to faculty and peers once they return to campus. At NOSM U, these intentional curricular spaces for reflection are a cornerstone for integration of knowledge, gaining of confidence, and interest in advocacy and this is evident in the increased use of reflection as indicated in this subscale.

Implications

Cultural competence embodies abstract values and social constructs, embedded within the affective domain of learning that challenge many medical learners. Kumagai and Lypson⁶⁴ argue that when humanistic values are incorporated into medical education, “[the orientation] is directed towards fostering critical self-awareness, acquiring an understanding of social issues (both interpersonal and societal) in medicine, and developing an individual approach to reflective, ethical practice.” The enactment of culturally competent and culturally safe care requires complex growth in emotions and attitudes that are difficult to teach directly compared with clinical and medical sciences where the objective is to teach a foundation of knowledge to be applied in practice.⁶⁴

We argue that an added challenge of teaching preparedness to provide culturally safe care is that learning outcomes are difficult to pinpoint and evaluate. To advance medical education scholarship in this area, we developed a conceptual framework of learning outcomes that assess changes in cultural competence and readiness to provide culturally competent care to Indigenous patients among medical learners who undertake specialized cultural safety training. Our study demonstrates that the NOSM CAST is a pre–posttest tool that measures the changes in the levels of cultural competence and readiness to provide culturally safe care among prelicensure medical learners in response to exposure to the Indigenous health curriculum. To our knowledge, this is the first validated tool of its kind.

NOSM U is known for its social accountability mandate which may be reflected in admissions process preselecting for students who are, on average, highly committed to health equity as evidenced in their high scores in antidiscrimination and positive attitudes toward Indigenous Peoples in the pretest. The high average antidiscrimination scores did not increase any further, but were maintained, which is an important finding for the NOSM U Indigenous health curriculum as some cultural curricula have been shown to be counterproductive^13,65–69 which was not the case here.

The results also show that students can attain measurable learning goals such as skills, behaviors, and attitudes that prepare them for culturally safe care when exposed to the comprehensive, integrated Indigenous health curriculum at NOSM U. It is important to emphasize that this measure is not a substitute for the patients’ perspective on how safe they feel with a medical learner. At NOSM U, Indigenous community staff at each site gather information about how safe and respected community members felt when interacting with the student. Staff then complete a score sheet to evaluate perceived cultural safety in their interactions from the perspective of the Indigenous community members. This is an important aspect of the evaluation of the Indigenous health curriculum and complements the NOSM CAST which uses a self-reported measure of progression in students’ readiness for culturally safe care.

Creation of the NOSM CAST retrospective pre–posttest

The methodology of a pre- and posttest is difficult and time-consuming to implement in educational institutions. We believe that a retrospective pre–posttest tool can be very useful for evaluating Indigenous health curricula as it is more efficient to implement. Therefore, we have transposed the tool as a retrospective pre–posttool that can be administered at the end of any specific educational component for use in regular evaluation cycles (Appendix 5 in the online supplemental materials). In this application, learners score changes in their perceived knowledge, skills, and practices retrospectively after the teaching has taken place. This tool should help to reduce students’ bias of overestimating their competence before the educational intervention and account for shifts in personal frames of reference as students progress through a demanding and life-altering medical curriculum.⁴⁷ This version of the tool should undergo further validation research in the future.

Either version of the tool provides the first real opportunity to assess the effectiveness of different approaches to Indigenous health education. The tool can potentially contribute to health education knowledge globally in the area of evidence-based understanding of the impact of cultural safety teaching in health professional education specific to Indigenous health. As medical schools and other health professions invest in Indigenous health curricula to support the health-related goals of UNDRIP and the TRC, it is essential to monitor the effectiveness of Indigenous health curricula.

Limitations

The assessment of the provider's attainment of skills and understanding deemed to be underlying cultural safety is only an intermediate evaluation of the ability to provide culturally safe care. In this study, our scope was limited to assessing the learner's readiness to provide culturally safe care. While the content validity of the NOSM CAST is strong from the perspective of preparing students for culturally safe care, further research should be conducted to test the tools aligned with Indigenous patient perspectives about the safety of the care provided by the students at different stages of their learning as students and clinical providers.

Another limitation is that the tool was developed for and only tested in one Canadian medical school with an overtly strong commitment to preparing students to provide culturally safe care. Therefore, adjustment of terminology may be required for use of NOSM CAST in other countries. However, adding to the strength of our methodology is that the tool was tested on learners who participate in school-wide mandatory cultural training (ie students did not self-select into cultural streams) with a 95.3% response rate.

Conclusion

Teaching cultural competence and safety in medicine is one pathway that is required to realize Indigenous health equity and is encouraged by medical school accreditation. As we teach about culture and colonialism as determinants of health in medicine in response to recommendations by UNDRIP and TRC, it is extremely important to monitor if these cultural competency training initiatives help to achieve the desired outcomes. The PCA of the NOSM CAST tool demonstrated that a five-component model performed well in measuring “preparedness for culturally safe care” outcomes among first-year undergraduate medical students. The validation process should be extended to the NOSM CAST retrospective pre–posttest tool in future studies because its simplified data collection process makes it a practical tool to use at the end of any level of an Indigenous health curriculum.

Under its social accountability mandate and through community engagement, NOSM U includes the world's only mandatory integrated Indigenous health curriculum that includes a cultural immersion experience for each first-year learner. The month-long immersion experience is the main component of the first-year Indigenous health curriculum. Applying the NOSM CAST to evaluate the Indigenous health curriculum shows that NOSM learners progress significantly in acquiring skills for culturally safe care for Indigenous Peoples during their first year of medical school. The NOSM CAST results underscore the importance of the Indigenous immersion curriculum at NOSM U in supporting Indigenous health learning goals. Based on our results, we encourage consideration of mandatory extended stay Indigenous community placements at all medical schools with settler colonial histories. This enables all students and not just those who select an elective opportunity to augment classroom learning about Indigenous health in a meaningful and effective manner in the community. As part of rigorous evaluation of Indigenous health curricula, we advocate for the use of NOSM CAST in conjunction with an assessment of Indigenous community and patient experiences of safety with the same learners.

Supplemental Material

sj-docx-1-mde-10.1177_23821205241286292 - Supplemental material for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument

Supplemental material, sj-docx-1-mde-10.1177_23821205241286292 for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument by Marion Maar, Diana Urajnik, Geoffrey L. Hudson, Darrel Manitowabi, Lorrilee McGregor, Sam Senecal, Roger Strasser, Wayne Warry and Kristen Jacklin in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-2-mde-10.1177_23821205241286292 - Supplemental material for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument

Supplemental material, sj-docx-2-mde-10.1177_23821205241286292 for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument by Marion Maar, Diana Urajnik, Geoffrey L. Hudson, Darrel Manitowabi, Lorrilee McGregor, Sam Senecal, Roger Strasser, Wayne Warry and Kristen Jacklin in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-3-mde-10.1177_23821205241286292 - Supplemental material for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument

Supplemental material, sj-docx-3-mde-10.1177_23821205241286292 for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument by Marion Maar, Diana Urajnik, Geoffrey L. Hudson, Darrel Manitowabi, Lorrilee McGregor, Sam Senecal, Roger Strasser, Wayne Warry and Kristen Jacklin in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-4-mde-10.1177_23821205241286292 - Supplemental material for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument

Supplemental material, sj-docx-4-mde-10.1177_23821205241286292 for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument by Marion Maar, Diana Urajnik, Geoffrey L. Hudson, Darrel Manitowabi, Lorrilee McGregor, Sam Senecal, Roger Strasser, Wayne Warry and Kristen Jacklin in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-5-mde-10.1177_23821205241286292 - Supplemental material for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument

Supplemental material, sj-docx-5-mde-10.1177_23821205241286292 for Evaluating the Effectiveness of Indigenous Health Curricula: Validation and Application of the NOSM CAST Instrument by Marion Maar, Diana Urajnik, Geoffrey L. Hudson, Darrel Manitowabi, Lorrilee McGregor, Sam Senecal, Roger Strasser, Wayne Warry and Kristen Jacklin in Journal of Medical Education and Curricular Development

Footnotes

Acknowledgments

We would like to thank the Indigenous community partners of NOSM who provided cultural immersion placements for NOSM medical students. Indigenous community partners since 2005 have included Fort William First Nation; Kitchenuhmaykoosib Inninuwug (Big Trout Lake); Deer Lake; Fort Frances Tribal Health Authority; Northwest Bay First Nation; Fort Frances Tribal Health Authority; Lac La Croix First Nation; Couchiching First Nation; Naotkamegwanning First Nation (Whitefish Bay); Sandy Lake; Lac Seul First Nation; Constance Lake; Eagle Lake; Wasaaygiizhik Nanandawe’iyewigamig (Kenora Area Health Access Centre); Fort Frances Gizhewaadiziwin Health Center (Couchiching First Nation); Thunder Bay (Metis Nation of Ontario); Kingfisher Lake; Muskrat Dam; Nibinamik (Summer Beaver); Eabamet Lake (Fort Hope); Fort Severn; Atikameksheng Anishnawbek (Whitefish Lake); Attawapiskat First Nation; Brunswick House First Nation; Fort Albany First Nation; Garden River First Nation; Mattagami First Nation; M’Chigeeng First Nation; Metis Nation of Ontario—Timmins; Mississauga First Nation; Mnaamodzawin Health Services; Moose Cree First Nation; Nipissing First Nation; Ojibways of Batchewana First Nation; Ojibways of Pic River First Nation; Sagamok Anishnawbek; Serpent River First Nation; Temagami First Nation; Whitefish River First Nation; and Wikwemikong Unceded First Nation. We also thank Drs. Doris Mitchell and Robert Pastre for providing advice on the study. We would like to thank Lisa Boesch for her support in preparing this manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article. This study received funding from the NOSM Medical Education Research Grant fund (KJ and MM).

Author's contributions

RS made substantial contributions to the early conception and design of the work, and to revisions of the text for important intellectual content. GLH made substantial contributions to the early conception and design of the work, and to revisions of the text for important intellectual content, especially related to the development and implementation of the curriculum. WW provided substantive and critical advice on the design of the research, the conceptual development of the knowledge domains, questions de novo, and the early sorting of principal components analysis outcomes. KJ and MM served as the co-PIs on the research grant, co-designed the research project, including conceptual design and survey development, and made substantial contributions to early drafts, writing, and revisions of the text for important intellectual content. DU led the data cleaning, statistical analysis, and writing of the statistical sections of this paper. DU, SS, LM, and DM all made substantial contributions to the paper.

Ethics

Research ethics approval was provided by Laurentian (REB #2013-06-14) and Lakehead (REB #045 13-14) Universities.

Consent

Written informed consent was not obtained from the students in an effort to keep participation anonymous. Instead, consent was implied by completing and returning the survey.

ORCID iDs

Marion Maar

Kristen Jacklin

Supplemental material

Supplemental material for this paper is available online.

References

United Nations. United Nations Declaration on the Rights of Indigenous Peoples . 2008. Accessed March 2, 2020. https://www.un.org/esa/socdev/unpfii/documents/DRIPS_en.pdf

Truth and Reconciliation Commission of Canada. Truth and Reconciliation Commission of Canada: Calls to Action . 2015. Accessed July 24, 2023. https://www2.gov.bc.ca/assets/gov/british-columbians-our-governments/indigenous-people/aboriginal-peoples-documents/calls_to_action_english2.pdf

National Collaborating Centre for Aboriginal Health. Access to Health Services as a Social Determinant of First Nations, Inuit and Metis Health . 2011. Accessed April 16, 2019. http://www.nccah-ccnsa.ca/docs/fact%20sheets/social%20determinates/Access%20to%20Health%20Services_Eng%202010.pdf

Goode

Dunne

Bronheim

SM.

The Evidence Base for Cultural and Linguistic Competency in Health Care| Commonwealth Fund. 2006. Accessed April 16, 2019. https://www.commonwealthfund.org/publications/fund-reports/2006/oct/evidence-base-cultural-and-linguistic-competency-health-care

Kitching

Firestone

Schei

, et al. Unmet health needs and discrimination by healthcare providers among an indigenous population in Toronto, Canada. Can J Public Health. 2020;111(1):40-49. DOI:https://doi.org/10.17269/s41997-019-00242-z

McDermott

. Can we educate out of racism? Med J Aust. 2012;197(1):15. DOI:https://doi.org/10.5694/mja12.10936

Association of Faculties of Medicine of Canada. Joint Commitment to Action on Indigenous Health . 2019. Accessed February 28, 2024. https://www.afmc.ca/wp-content/uploads/2022/10/AFMC_Position_Paper_JCAIH_EN.pdf

The National Consortium for Indigenous Medical Education. Brief: Improving Cultural Safety in Curriculum. Accessed February 28, 2024. https://ncime.ca/wp-content/uploads/2023/05/Brief-Curriculum.pdf

Committee on Accreditation of Canadian Medical Schools. CACMS Standards and Elements: Standards for Accreditation of Medical Education Programs Leading to the M.D. Degree. Published online 2022. Accessed February 28, 2024. https://cacms-cafmc.ca/wp-content/uploads/2022/12/CACMS-Standards-and-Elements-AY-2023-2024.pdf

10.

Browne

Varcoe

Smye

Reimer-Kirkham

Lynam

Wong

. Cultural safety and the challenges of translating critically oriented knowledge in practice. Nurs Philos. 2009;10(3):167-179. DOI:https://doi.org/10.1111/j.1466-769X.2009.00406.x

11.

Ramsden

Cultural safety and nursing education in Aotearoa and Te Waipounamu. Victoria University of Wellington, 2002. https://www.croakey.org/wp-content/uploads/2017/08/RAMSDEN-I-Cultural-Safety_Full.pdf

12.

Indigenous Physicians Association of Canada, The Association of Faculties of Medicine of Canada. First Nations, Inuit, Métis Health Core Competencies: A Curriculum Framework for Undergraduate Medical Education. Published online 2009. Accessed February 16, 2024. https://www.afmc.ca/wp-content/uploads/2022/10/IPAC-AFMC_Core_Competencies_EN.pdf

13.

Curtis

Jones

Tipene-Leach

, et al. Why cultural safety rather than cultural competency is required to achieve health equity: a literature review and recommended definition. Int J Equity Health. 2019;18(1):174. DOI:https://doi.org/10.1186/s12939-019-1082-3

14.

Kaphle

Hungerford

Blanchard

Doyle

Ryan

Cleary

. Cultural safety or cultural competence: how can we address inequities in culturally diverse groups? Issues Ment Health Nurs. 2022;43(7):698-702. DOI:https://doi.org/10.1080/01612840.2021.1998849

15.

Hart-Wasekeesikaw

Gregory

DM.

Cultural competence and cultural safety in first nations, Inuit and Métis nursing education: an integrated review of the literature. Aboriginal Nurses Association of Canada, 2009. Accessed April 16, 2019. https://opus.uleth.ca/items/5abe12e3-fc4d-4124-b791-4906f7eebdef

16.

Kurtz

deSales

Nyberg

Moar

. Social justice and health equity: urban aboriginal women’s actions for health reform. Int J Health Wellness Soc. 2014;3(4):13-26. DOI:https://doi.org/10.18848/2156-8960/CGP/v03i04/41081

17.

Smye

Josewski

Kendall

Cultural safety: an overview. First Nations, Inuit and Métis Advisory Committee Mental Health Commission of Canada, 2010, p. 31. https://mdsc.ca/documents/Publications/CULTURAL%20SAFETY%20AN%20OVERVIEW%20(draft%20mar%202010).pdf

18.

Kurtz

DLM

Janke

Vinek

Wells

Hutchinson

Froste

. Health sciences cultural safety education in Australia, Canada, New Zealand, and the United States: a literature review. Int J Med Educ. 2018;9:271-285. DOI:https://doi.org/10.5116/ijme.5bc7.21e2

19.

Jones

Crowshoe

Reid

, et al. Educating for indigenous health equity: an international consensus statement. Acad Med. 2019;94(4):512-519. DOI:https://doi.org/10.1097/ACM.0000000000002476

20.

Dick

Duncan

Gillie

, et al. Cultural safety: Module 1: Peoples’ experiences of colonization. Accessed March 19, 2024. https://web.archive.org/web/20220627154439/https://web2.uvcs.uvic.ca/courses/csafety/mod1/

21.

Aboriginal Nurses Association of Canada, Canadian Association of Schools of Nursing, Canadian Nurses Association. Cultural competence and cultural safety in nursing education: a framework for first nations, Inuit and Métis nursing. Published 2009. Accessed February 16, 2024. https://indigenousnurses.ca/resources/publications/cultural-competence-and-cultural-safety-nursing-education-framework-first

22.

Australian Government Department of Health and Aged Care. The aboriginal and Torres strait islander health curriculum framework. Published online 2014. https://www.health.gov.au/sites/default/files/documents/2020/12/aboriginal-and-torres-strait-islander-health-curriculum-framework.pdf

23.

Godwin

Hyett

McMahon

, et al. Methods and processes for first nations health curriculum development for nursing, medicine, dentistry and allied health entry-level programs: a scoping review. SAGE Open. 2023;13(2):1-19. DOI:https://doi.org/10.1177/21582440231163834

24.

Cooper

Beach

Johnson

Inui

. Delving below the surface. Understanding how race and ethnicity influence relationships in health care. J Gen Intern Med. 2006;21(Suppl 1):S21-S27, DOI:https://doi.org/10.1111/j.1525-1497.2006.00305.x

25.

Gray

McPherson

. Cultural safety and professional practice in occupational therapy: a New Zealand perspective. Aust Occup Ther J. 2005;52(1):34-42. DOI:https://doi.org/10.1111/j.1440-1630.2004.00433.x

26.

Crowshoe

Han

Calam

, et al. Impacts of educating for equity workshop on addressing social barriers of type 2 diabetes with indigenous patients. J Contin Educ Health Prof. 2018;38(1):49-59. DOI:https://doi.org/10.1097/CEH.0000000000000188

27.

Strasser

Lanphear

McCready

Topps

Hunt

Matte

. Canada’s new medical school: the Northern Ontario School of Medicine: social accountability through distributed community engaged learning. Acad Med. 2009;84(10):1459-1464. DOI:https://doi.org/10.1097/ACM.0b013e3181b6c5d7

28.

Strasser

Hogenbirk

Minore

, et al. Transforming health professional education through social accountability: Canada’s Northern Ontario School of Medicine. Med Teach. 2013;35(6):490-496. DOI:https://doi.org/10.3109/0142159X.2013.774334

29.

Hudson

Hunt

The Northern Ontario School of Medicine and social accountability. In Making of the Northern Ontario School of Medicine: a case study in the history of medical education. McGill-Queen’s University Press, 2009, pp. 163-168. Accessed April 16, 2019. https://www.jstor.org/stable/j.ctt8052n

30.

Jacklin

Strasser

Peltier

. From the community to the classroom: the Aboriginal health curriculum at the Northern Ontario School of Medicine. Can J Rural Med. 2014;19(4):143-150.

31.

Bessette

Reade

McGregor

Berti

Naokwegijig

Maar

. Culturally safe practices in the co-creation of medical education curriculum with indigenous animators: outcomes from an indigenous learning circle. J Med Educ Curric Dev. 2023;10:1-12. DOI:https://doi.org/10.1177/23821205231219430

32.

Maar

McGregor

Desjardins

Delaney

Bessette

Reade

. Teaching culturally safe care in simulated cultural communication scenarios during the COVID-19 pandemic: virtual visits with indigenous animators. J Med Educ Curric Dev. 2022;9:23821205221091034. DOI:https://doi.org/10.1177/23821205221091034

33.

Hudson

Maar

. The making of the world’s only medical school mandatory placement in indigenous communities: Northern Ontario School of Medicine (NOSM). In: Gavrus

Lamb

(Eds), Transforming medical education: historical case studies of teaching, learning, and belonging in medicine. McGill-Queen’s University Press; 2022:252-277.

34.

Maar

Bessette

McGregor

Lovelace

Reade

. Co-creating simulated cultural communication scenarios with indigenous animators: an evaluation of innovative clinical cultural safety curriculum. J Med Educ Curric Dev. 2020;7:2382120520980488. DOI:https://doi.org/10.1177/2382120520980488

35.

Hudson

Maar

. Faculty analysis of distributed medical education in Northern Canadian Aboriginal communities. Rural Remote Health. 2014;14(4):2664.

36.

Strasser

Hogenbirk

Jacklin

, et al. Community engagement: a central feature of NOSM’s socially accountable distributed medical education. Can Med Educ J. 2018;9(1):e33-e43.

37.

Truong

Paradies

Priest

. Interventions to improve cultural competency in healthcare: a systematic review of reviews. BMC Health Serv Res. 2014;14(1):99. DOI:https://doi.org/10.1186/1472-6963-14-99

38.

Buhagiar

Liu

, et al. A pilot study to assess the impact of Aboriginal and Torres Strait Islander cultural humility webinars on Australian medical school students. BMC Med Educ. 2023;23(1):626. DOI:https://doi.org/10.1186/s12909-023-04612-7

39.

Hardy

Filipenko

Smylie

Ziegler

Smylie

. Systematic review of indigenous cultural safety training interventions for healthcare professionals in Australia, Canada, New Zealand and the United States. BMJ Open. 2023;13(10):e073320. DOI:https://doi.org/10.1136/bmjopen-2023-073320

40.

Browne

Varcoe

Ward

. San’yas indigenous cultural safety training as an educational intervention: promoting anti-racism and equity in health systems, policies, and practices. The International Indigenous Policy Journal. 2021;12(3):1-26. DOI:https://doi.org/10.18584/iipj.2021.12.3.8204

41.

Clifford

McCalman

Bainbridge

Tsey

. Interventions to improve cultural competency in health care for indigenous peoples of Australia, New Zealand, Canada and the USA: a systematic review. Int J Qual Health Care. 2015;27(2):89-98. DOI:https://doi.org/10.1093/intqhc/mzv010

42.

West

Wrigley

Mills

Taylor

Rowland

Creedy

. Development of a First Peoples-led cultural capability measurement tool: a pilot study with midwifery students. Women Birth. 2017;30(3):236-244. DOI:https://doi.org/10.1016/j.wombi.2017.01.004

43.

Green

Callands

Radcliffe

Luebbe

Klonoff

. Clinical psychology students’ perceptions of diversity training: a study of exposure and satisfaction. J Clin Psychol. 2009;65(10):1056-1070. DOI:https://doi.org/10.1002/jclp.20605

44.

Chun

MBJ

. Pitfalls to avoid when introducing a cultural competency training initiative: pitfalls to avoid in cultural competency training. Med Educ. 2010;44(6):613-620. DOI:https://doi.org/10.1111/j.1365-2923.2010.03635.x

45.

Ogrinc

Armstrong

Dolansky

Singh

Davies

. SQUIRE-EDU (Standards for QUality Improvement Reporting Excellence in Education): publication guidelines for educational improvement. Acad Med. 2019;94(10):1461-1470. DOI:https://doi.org/10.1097/ACM.0000000000002750

46.

Härdle

Simar

Applied Multivariate Statistical Analysis. Springer International Publishing; 2019, DOI:https://doi.org/10.1007/978-3-030-26006-4

47.

Thomas

Wells

Baumann

, et al. Comparing traditional versus retrospective pre-/post-assessment in an interdisciplinary leadership training program. Matern Child Health J. 2019;23(2):191-200. DOI:https://doi.org/10.1007/s10995-018-2615-x

48.

Smylie

Lessard

Bailey

, et al. A guide for health professionals working with aboriginal peoples: executive summary. J Obstet Gynaecol Can. 2000;100(December):1-6.

49.

Northern Ontario Medical School. Report of the NOMS aboriginal workshop “follow your dreams” . 2003. Accessed September 13, 2021. https://www.nosm.ca/wp-content/uploads/2018/06/2003_06_03_FollowYourDreams_en.pdf

50.

Northern Ontario School of Medicine. Report of the third NOSM aboriginal workshop “Living the Vision” . 2011, p. 48. https://www.nosm.ca/wp-content/uploads/2018/06/2011-Living-the-Vision-Report.pdf

51.

Gawronski

Peters

Brochu

Strack

. Understanding the relations between different forms of racial prejudice: a cognitive consistency perspective. Pers Soc Psychol Bull. 2008;34(5):648-665. DOI:https://doi.org/10.1177/0146167207313729

52.

Maar

Seymour

Sanderson

Boesch

. Reaching agreement for an Aboriginal e-health research agenda: the Aboriginal Telehealth Knowledge Circle consensus method. Rural Remote Health. 2010;10(1):1299.

53.

Meyers

Gamst

Guarino

. Applied multivariate research: design and interpretation. Sage Publications; 2016.

54.

Comrey

Lee

HB.

A first course in factor analysis, 2nd ed. Lawrence Erlbaum Associates, Inc, 1992, vol. xii, p. 430.

55.

Mundfrom

Shaw

. Minimum sample size recommendations for conducting factor analyses. International Journal of Testing. 2005;5(2):159-168. DOI:https://doi.org/10.1207/s15327574ijt0502_4

56.

IBM Corp. IBM SPSS Statistics for Windows. Published online Released 2022.

57.

Raharjanti

Wiguna

Purwadianto

, et al. Translation, validity and reliability of decision style scale in forensic psychiatric setting in Indonesia. Heliyon. 2022;8(7):e09810. DOI:https://doi.org/10.1016/j.heliyon.2022.e09810

58.

Eva

. Broadening the debate about quality in medical education research. Med Educ. 2009;43(4):294-296. DOI:https://doi.org/10.1111/j.1365-2923.2009.03342.x

59.

Hubinette

Dobson

Scott

Sherbino

. Health advocacy. Med Teach. 2017;39(2):128-135. DOI:https://doi.org/10.1080/0142159X.2017.1245853

60.

İlhan

Güler

Teker

Ergenekon

. The effects of reverse items on psychometric properties and respondents’ scale scores according to different item reversal strategies. International Journal of Assessment Tools in Education. 2024;11(1):20-38. DOI:https://doi.org/10.21449/ijate.1345549

61.

Vigil-Colet

Navarro-González

Morales-Vives

. To reverse or to not reverse Likert-type items: that is the question. Psicothema. 2020;32(1):108-114. DOI:https://doi.org/10.7334/psicothema2019.286

62.

García-Fernández

Postigo

Cuesta

González-Nuevo

Menéndez-Aller

García-Cueto

. To be direct or not: reversing Likert response format items. Span J Psychol. 2022;25:e24. DOI:https://doi.org/10.1017/SJP.2022.20

63.

de Leeuw

Larstone

Fell

, et al. Educating medical students’ “hearts and minds”: a humanities-informed cultural immersion program in indigenous experiential community learning. International Journal of Indigenous Health. 2021;16(1):87-107. DOI:https://doi.org/10.32799/ijih.v16i1.33078

64.

Kumagai

Lypson

. Beyond cultural competence: critical consciousness, social justice, and multicultural education. Acad Med. 2009;84(6):782-787. DOI:https://doi.org/10.1097/ACM.0b013e3181a42398

65.

Plaisime

Jipguep-Akhtar

Belcher

HME

. ‘White People are the default’: a qualitative analysis of medical trainees’ perceptions of cultural competency, medical culture, and racial bias. SSM - Qualitative Research in Health. 2023;4(December 2023):1-12. DOI:https://doi.org/10.1016/j.ssmqr.2023.100312

66.

Wendt

Gone

. Rethinking cultural competence: insights from indigenous community treatment settings. Transcult Psychiatry. 2012;49(2):206-222. DOI:https://doi.org/10.1177/1363461511425622

67.

Muaygil

. From paternalistic to patronizing: how cultural competence can be ethically problematic. HEC Forum : An Interdisciplinary Journal on Hospitals’ Ethical and Legal Issues. 2018;30(1):13-29. DOI:https://doi.org/10.1007/s10730-017-9336-1

68.

Shepherd

. Cultural awareness workshops: limitations and practical consequences. BMC Med Educ. 2019;19(1):14. DOI:https://doi.org/10.1186/s12909-018-1450-5

69.

Berger

Miller

. Health disparities, systemic racism, and failures of cultural competence: authors? Response to commentaries. Am J Bioeth. 2021;21(9):1-3. DOI:https://doi.org/10.1080/15265161.2021.1956636

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.03 MB

0.07 MB