Abstract
The model minority stereotype (MMS) has had a direct and significant impact on Asian Americans and their internal processes, like other stereotypes attributed to a group of people (e.g., stereotype threat; Steele & Aronson, 1995). In this article, we describe the development and validation of a measure named the Model Minority Stereotype-Response Scale (MMS-RS), which is designed to assess the responses Asian Americans experience when faced with the MMS to unveil how the stereotype impacts them. Exploratory and confirmatory factor analyses (CFAs) supported a two-factor structure of the MMS-RS, namely negative responses to and positive endorsement of the MMS. Alongside the reliability estimates, the evidence for construct validity demonstrated that the MMS-RS has good psychometric properties. Interestingly, both negative reaction to and the positive endorsement of the MMS were found to coexist. Overall, the MMS-RS is a useful instrument for scholars who are interested in researching, intervening, and advocating for Asian Americans’ experiences with the MMS.
This article described the development of a scale to measure Asian Americans’ responses to the model minority stereotype (MMS), a racial stereotype that frames Asian Americans as a group that is immune from the oppressive systemic barriers in the United States and that they can gain success more easily than other racial minority groups. The findings from this study suggest that the Model Minority Stereotype-Response Scale (MMS-RS) is a reliable and valid instrument to assess Asian Americans’ responses to the MMS and that it has useful potential for research, practice, and advocacy purposes.Significance of the Scholarship to the Public
Introduction
Anti-Asian discrimination in the United States has a long history, in which beliefs about Asian Americans have been conceptualized and promulgated differently across time by individuals and systems alike, to varying outcomes, that work to solidify the view of Asian Americans as “the others.” Various state-sanctioned discriminatory acts and legal decisions (e.g., the Paige Act and the Chinese Exclusion Act) were passed for the purpose of excluding Asian Americans from the U.S. mainstream society by imposing the perpetual foreigner stereotype. Then, the negative depictions on Asian Americans morphed drastically in the 1960s. Specifically, the concept of Asian Americans’ being a model minority was created to promote a rhetorical narrative that compared the experiences of Asian Americans to other minority groups, ultimately justifying and upholding the systemic oppression against non-Asian American minority populations (Suzuki, 2002). Such projections onto Asian Americans as the “successful” minority group positioned them against other racial and ethnic minority groups, thus negatively impacting the Asian American community as well as the society as a whole (Ho & Jackson, 2001). Scholars across multiple fields have since examined this narrative and coined the inaccurate generalizations that often came from the narrative as the model minority stereotype (MMS; e.g., Sakamoto et al., 2012; Shen et al., 2011; Yoo et al., 2010; Zhou & Bankston, 2020).
Although the terms model minority myth and MMS have been used interchangeably in the literature, we chose to use the term MMS for a specific reason. According to the Cambridge dictionary definition, a myth is “an ancient story or set of stories, especially explaining the early history of a group of people or about natural events and facts” (Cambridge University Press & Assessment, n.d.-a), whereas a stereotype is “a set idea that people have about what someone or something is like, especially an idea that is wrong” (Cambridge University Press & Assessment, n.d.-b). Thus, a stereotype is an inaccurate idea that people hold against a certain group of people that has a negative impact on the group (e.g., racial stereotype); in contrast, a myth implies that the idea about a certain group is neutral. Thus, in this study, we chose to use the term MMS rather than model minority myth when referring to the model minority trope imposed on Asian Americans, given its harm on Asian Americans.
The MMS is a racial stereotype of Asian Americans that promotes the idea that Asian Americans are an inherently exceptional minority group and that they do not face significant barriers for success in the United States. Asian Americans, therefore, are believed to be able to achieve greater educational and career outcomes as compared to other racial and ethnic groups (Fong, 2008). Given its positive connotation, the MMS has been promoted widely by the mass media (Taylor & Stern, 1997), the U.S. government (Immigration and Nationality Act of 1965), and in individual interactions in various settings. As a result, the MMS has led to the neglect of Asian Americans’ needs and challenges in the United States. For instance in the research realm, studies on Asian Americans are disproportionately underfunded and underrepresented due to the overarching belief that the MMS extends to greater levels of health, including mental health, among Asian Americans (Cheng et al., 2017; J. Y. Kim et al., 2021). This false belief stigmatizes those who struggle with mental illness and creates additional barriers to seeking help. Thus, studies that further unveil how MMS impacts Asian Americans’ various life aspects are warranted. As the MMS not only shapes the perception of Asian Americans in a stereotypical way, it also directly impacts Asian American individuals themselves and their internal psychological processes, similar to stereotypes that affect other minority populations (e.g., the stereotype threat among African Americans; Steele & Aronson, 1995). To better understand these internal psychological processes among Asian Americans, the measurement of the intrapersonal responses elicited by the MMS may help to clearly depict how the MMS exerts its effects on a range of life issues that Asian Americans face.
Definition of Responses to Model Minority Scale
Although specific response components can be detailed from interviews, media outlets, and study results, the responses to the MMS can be broadly and intuitively conceptualized as the reactions individuals have when faced with the MMS. Responses to the MMS depart and are distinct from the concept of internalized MMS—beliefs in the racialized stereotype of Asian Americans—although being in agreement with the MMS could be a part of a response that overlaps with internalized MMS.
The MMS, taken at its face value, may initially appear as a benign or even favorable belief due to its outward tenets of hard work, success, and achieving the “American dream,” and thus may appear to engender predominantly positive reactions among those faced with or who believe in the stereotype. However, the effects of the stereotype, as is true of any stereotype, may quickly reveal themselves as detrimental to those most intimately connected to the belief, often to varying degrees of harm (e.g., Gupta et al., 2011; Suh et al., 2023; Yoo et al., 2015), and likely lead to negative responses to the MMS.
Individual interviews, media portrayals, and empirical studies on the MMS have well documented the essentially two-fold responses elicited from the MMS: positive and negative (Choi & Lim, 2015; McGee et al., 2017). Positive reactions to the MMS include “a sense of pride for the relative ‘model’ status as high achievers over other minority groups” (Choi & Lim, 2015, p. 178) and a “feel[ing] like it’s a head start [to be perceived as a model]” (McGee et al., 2017, p. 12). Studies documenting the negative reactions to the MMS encompass Asian Americans having a sense of self-blame and frustration from failing to meet the MMS image (Choi & Lim, 2015). Studies also found Asian Americans experiencing a sense of MMS-rooted dismissal of their efforts by others after their achievements. One study documented an Asian American responding, “No [not because I am Asian but], I worked hard for it.” (McGee et al., 2017, p. 12).
Collectively, positive responses to the MMS can be primarily defined as feeling a sense of pride for the MMS to the point of having a sense of superiority over other racial groups. In contrast, negative responses to the MMS comprise elements such as stressful experiences due to the pressures to meet the MMS image while facing the societal acts of minimizing the challenges and dismissing the efforts to achieve. Based on this literature, we expected to observe these two dimensions of responses to the MMS emerge from this study, one positive and one negative.
Additionally, we conducted individual interviews with Asian American college students to further identify the specific components of each dimension of the MMS. Specifically, grounded in the tripartite model of attitude (Eagly & Chaiken, 1993), positive and negative responses were conceptualized to be comprised of three domains: cognitive, behavioral, and affective. The attitude model was deemed best to formulate specific elements of the responses because attitude is “a psychological tendency expressed by evaluating a particular entity with some degree of favor or disfavor” (Eagly & Chaiken, 1993, p. 1). In other words, the scale created from this current study aimed to assess responses (i.e., expressed psychological tendency) to the MMS (i.e., entity), which can be positive or negative (i.e., evaluated in some degree of favor or disfavor). Thus, the interview questions were generated and conducted accordingly (see the Item Generation section below).
Differentiating the Response to the Model Minority Stereotype from Internalization of the Model Minority Stereotype
To investigate the MMS, particularly how Asian Americans are affected by the stereotype, it is critical to further clarify the construct of response to the MMS in comparison to the internalization of the MMS. Prior studies on the MMS solely attended to internalized MMS and how it impairs performance and causes mental distress. Findings were inconsistent in that some studies found the MMS negatively affecting Asian Americans’ mental health (Suh et al., 2023; Yoo et al., 2010), while other studies found a positive association between the MMS and outwardly favorable traits (Atkin et al., 2018; Kiang et al., 2016). Studies attempting to explicate this inconsistency showed that depending on one’s achievement status (e.g., grade point average [GPA]; Yoo et al., 2015), the MMS either validated their success (i.e., students with high GPA) or created cognitive dissonance (i.e., students with low GPA) and in turn influenced their mental health in positive or negative ways, respectively. Another study attended to the context and showed how the MMS could either have a positive or negative influence on mental health outcomes (Atkin et al., 2018). The results showed that internalized MMS created a sense of pressure among students attending Asian-dominant schools due to the contextual factor of having competition against other high-achieving Asian Americans (Atkin et al., 2018). Hence, the context moderated the relationship between the MMS and mental distress. Although these findings are interesting, an important limitation is that only the internalization of the MMS was assessed whereas the outward response reactions to the MMS were not directly measured. In other words, the internalization of the MMS is distinct from one’s responses to the MMS; however, internalized MMS was assumed to also represent the responses in prior studies. Importantly, this limitation could be related to the inconsistent study findings regarding the relationship between the MMS and health outcomes.
To further illustrate the point that internalized MMS is not equivalent to the way individuals respond to the MMS, individuals who may not believe in the MMS (i.e., low internalization) may have strong adverse reactions to the MMS. But the low scores on internalized MMS could lead researchers to wrongly believe the scores represent a lack of adverse reaction to the MMS. This possibility warrants measuring the responses to the MMS separately from internalized MMS. However, scales commonly used in the past only assessed internalized MMS (i.e., a higher score indicates more endorsement of the stereotype), but not the responses to the MMS. In addition, study findings that portray the MMS as being beneficial may likely stem from how items in the measures of internalized MMS were written, in which the items assess individuals’ confidence in certain domains such as in the academic and career realms. Examples of this type of items are “Asian Americans are more likely to be good at math and science” and “Asian Americans have stronger work ethics,” both of which are from the Internalization of Model Minority Myth Measurement (IM-4; Yoo et al., 2015). It is also likely that these types of items function as a primer to boost one’s self-esteem given their wording. Another instrument of internalized MMS, the Internalization of Asian American Stereotypes Scale (IAASS; Shen et al., 2011), assesses the expanded stereotypical image of Asian Americans by measuring both model minority tropes and perpetual foreigner stereotype (“As an Asian American, I would face more difficulty in a career that requires a lot of reading, writing, and verbal communication in English”). However, similar to IM-4, IAASS assesses the internalization of the racialized stereotype against Asian Americans and does not assess Asian Americans’ reactions to the MMS. In sum, although prior MMS instruments are valuable in assessing the internalized MMS, they do not directly measure the elements representing how Asian Americans respond to the MMS. It is likely that the responses to the MMS have more direct relations with Asian American mental health than the level of MMS internalization itself. To our knowledge, no instrument to date has operationalized and assessed Asian Americans’ responses when experiencing the MMS and doing so could help reveal the impact of the MMS on various mental health outcomes.
To further support the importance of focusing on responses to the MMS, it is noteworthy to point out that a majority of studies on stereotypes and their threats employed experimental designs to elicit responses from the stereotypes in order to investigate how the evoked responses affect the variables of interest (e.g., Cheryan & Bodenhausen, 2000; Schmader & Johns, 2003). For instance, Cheryan and Bodenhausen (2000) conducted an experimental study with Asian American women college students and found that they showed impaired concentration when primed with the MMS. Based on the findings from this and other experimental studies, a model was proposed to illuminate a comprehensive process of stereotype that focused on the elicitation of responses rather than assessing the level of internalized MMS and how these responses consequently impacted performance (Schmader et al., 2008). Together, the trend on stereotype studies highlights that the responses to stereotypes are the critical component to focus on, especially when explaining the impact of stereotypes on outcome variables such as mental distress. Experimental design, thus, may be effective in showing the link between the experience of stereotypes, one’s responses to the stereotypes, and the resultant outcome; however, instruments that can direclty measure the responses to MMS (rather than inferring from proxies such as a change in the performance) are needed. Additionally, such a tool would allow experimental studies to measure the responses to the MMS in a consistent manner and enable tracking of any changes in responses to the intervention. Thus, the present study aimed to develop an instrument that directly measures responses to the MMS in order to promote studies to better uncover how the MMS is related to various life experiences among Asian Americans.
Obtaining Evidence of Validity
After creating the instrument and naming it the Model Minority Stereotype-Response Scale (MMS-RS; see the Method section), we aimed to evaluate the psychometric properties of the scale’s scores by obtaining evidence of internal and temporal reliability and criterion-related and incremental validity. Based on the conceptual definitions and prior study results, each of the two dimensions (i.e., subscales) was expected to reflect a particular valence of response to the MMS and to vary in its direction in the association with mental distress indicators (e.g., depressive symptoms) and perception of discrimination against Asian Americans. Specifically, in light of the findings that imply the protective role of the MMS against mental distress (e.g., Thompson & Kiang, 2010), a negative relationship was expected between positive response to the MMS and indicators that reflect mental distress. Additionally, as the positive components of the MMS likely relate to less awareness of societal inequalities (Tran & Curtin, 2017), positive response to the MMS was expected to have an inverse relationship with perceived discrimination. Similarly, we expected a positive association between the subscale that constitutes negative components of MMS-RS and scores on mental distress and perceived discrimination. This expectation was based on the research indicating the harm of the MMS (e.g., Suh et al., 2023) and findings on the negative relationship between structural awareness—particularly racial inequity—and belief in the MMS (Tran & Curtin, 2017).
Additionally, because perceived discrimination is frequently researched as the major risk factor regarding Asian Americans’ mental distress in prior studies (Cho et al., 2021), we investigated the evidence of incremental validity in which the MMS-RS would explain the variance on mental distress after accounting for the relationship between perceived discrimination and mental distress. The negative component of the MMS-RS was expected to be positively associated with depressive symptoms above and beyond the effect of perceived discrimination because although both assess reactions to stressful racial mistreatments, perceived discrimination assesses the discrimination that racial minorities face. In contrast, the MMS-RS measures the reactions to Asian-targeted racial stereotyping. By examining the evidence of incremental validity specifically on the negative component of the MMS-RS, the unique threat associated with Asian-specific stereotyping—that is different from racial discrimination—can be recognized, assessed, and intervened. As for the positive component of the MMS-RS, given the studies documenting its adverse association with mental distress, the subscale scores were expected to be inversely associated with depressive symptoms above and beyond the relationship between perceived discrimination and depressive symptoms.
Current Study
What follows in the Method sections are descriptions of Study 1 and Study 2. In Study 1, we report the process and results of measurement development, including item development, content review, and an exploratory factor analysis (EFA) to evaluate the scale structure. Based on the literature review on responses to the MMS, two factors (positive and negative reactions toward the MMS) were expected. Additionally, we performed a confirmatory factor analysis (CFA) and additional analyses to obtain evidence of validity. In support of criterion-related validity analyses, a positive association was expected between the negative responses to the MMS factor, depressive symptoms—an indicator of mental distress—and Asian American targeted racial discrimination. In contrast, associations were expected to be in a negative direction or not significant between the positive responses to the MMS factor and mental distress and perceived discrimination. As for incremental validity, each MMS-RS subscale was expected to significantly explain the variance on depressive symptoms above and beyond the effect of perceived discrimination. Depending on the valence of the response, the direction of association with depressive symptoms was assumed to vary—positive association between negative aspects of the MMS-RS and mental distress indicator and negative association between positive aspects of the MMS-RS and depressive symptoms. In Study 2, a new sample was recruited to investigate the test-retest reliability of the newly developed MMS-RS scores.
Study 1
Participants
The data were collected from students attending a university located in the southern region of the United States via an undergraduate research participation pool. In addition, Prolific, a crowdsourcing platform for research, was used to obtain responses from college and noncollege community adults. Inclusion of a nonstudent community adult sample through Prolific was deemed beneficial for a better generalizability of the results. Self-identified Asian Americans over 18 years old were eligible to participate. A total of 419 participants were recruited, of which six cases were removed for failing to pass the validity items. The sample consisted of 286 (69.2%) students, 126 nonstudents (30.5%), and one did not report their status (0.2%). Among the full sample, 220 identified as men (53.3%), 184 identified as women (44.6%), three respondents identified as nonbinary/nonconforming (0.7%), and six preferred not to respond (1.5%). The median age was 22 years (M = 26.1, SD = 8.8, range = 18–70). The majority self-identified as middle class (n = 262, 63.4%), followed by working class (n = 127, 30.8%), upper class (n = 13, 3.1%) lower class (n = 10, 2.4%), and one did not report their social class (n = 1, 0.2%). Various ethnic groups were represented in the sample: Chinese (n = 102, 24.7%), Vietnamese (n = 60, 14.5%), Indian (n = 56, 13.6%), Korean (n = 48, 11.6%), Multi-ethnic (n = 48, 11.6%), Filipino (n = 29, 7.0%), Pakistani (n = 13, 3.1%), Taiwanese (n = 11, 2.7%), Bangladeshi (n = 10, 2.4%), Japanese (n = 10, 2.4%), Hmong (n = 3, 0.7%), Laotian (n = 3, 0.7%), Burmese (n = 2, 0.5%), Cambodian (n = 2, 0.5%), Indonesian (n = 1, 0.2%), Mongolian (n = 1, 0.2%), Thai (n = 1, 0.2%), and 13 did not report their ethnicity (3.1%).
Procedure
Item Generation
A pool of items was generated during the pilot phase by a team consisting of a counseling psychology professor (Asian woman) and five graduate students in counseling and psychology programs (five women; two Asian, one Latinx, and two White). Prior to item generation, the team reviewed the literature describing Asian Americans’ experiences and reactions to the MMS. Additionally, a semi-structured interview was conducted with five Asian American college students (four women and one man; one first-year, two sophomores, and two juniors; three first-generation and two second-generation since immigration students; age range was 18–21 years). Participants were recruited through another study and expressed interest in partaking in an interview to share their experiences with the MMS. The students were given $20 gift cards for their participation. Interview prompts were developed to ask for responses on three domains (cognitive, behavioral, and affective). The following sample questions were asked to introduce the concept of the MMS: “Do you experience MMS?” and “How often would you say you experience MMS?” These questions were followed by questions about the specific responses that are formed and how they cope with the responses: “What is your response and/or how do you cope with the MMS?” and “Do you change your behavior in response to MMS?” The interview was conducted for approximately an hour per individual, in which the respondents were asked about their cognitive, behavioral, and affective responses when they encounter the MMS. Employing the thematic analysis approach (Braun & Clarke, 2006), which provides the framework to identify the recurring theme or meaning in the data, the research team identified themes from the interview data. Critical qualitative study findings and quotes relevant to the responses to the MMS were also highlighted. Based on prevalent themes from the data, 22 initial items were generated and reviewed for content and comprehensiveness of the items and wording by the team. As expected, items broadly represented two dimensions, positive and negative responses to the MMS, which aligned with the literature. Lastly, although completely discarding the effects of preconceptions about the topic of interest is not possible, we tried to minimize the effect by bracketing, in which team members documented and reflected on their biases and assumptions about the MMS.
Data Collection
After the initial draft of the MMS-RS was prepared, participants were asked to complete an online survey of the MMS-RS and other scales. Participation took less than 15 minutes. Participants who were recruited through the undergraduate research pool received nominal credit, and the respondents from Prolific were compensated based on a $14.52/hour rate. Approval was secured with the Institutional Review Board, and the study was conducted in compliance with the approved protocol.
Measurements
In addition to the MMS-RS, the following two instruments were administered to examine the possible relationships among responses to the MMS and experiences with discrimination and presence of depressive symptoms.
Discrimination Experiences
The Everyday Discrimination Scale (EDS; Williams et al., 1997) was used to assess the participants’ experiences of daily race-based discrimination in their lives (Williams et al., 1997). Specifically, the instructions ask, “Sometimes people feel as though they are treated differently than others because of their race (i.e., Asian). What are your experiences?” The nine-item measure includes items such as “You are treated with less respect than other people,” and “You are called names or insulted.” Participants were asked to respond on a 4-point Likert-type scale from 1 (never) to 4 (often), with higher scores indicating greater perceived discrimination. Based on a comprehensive national data on Asian Americans, Gee et al. (2007) reported a Cronbach’s alpha of .91. The Cronbach’s alpha of the EDS score in the present study was .87. Evidence for construct validity was provided in the adequate goodness-of-fit indices from a CFA based on data from various racial/ethnic groups (Kim & Kl, 2014).
Depressive Symptoms
Participants’ depressive symptoms were assessed with the Center for Epidemiologic Studies Depression Scale (CES-D; Radloff, 1977). On a 4-point Likert-type scale, participants were asked to indicate how often they experienced each item in the past week 0 (rarely or none of the time) to 3 (most or all of the time). Sample items from the CES-D include, “I felt everything I did was an effort,” and “I thought my life had been a failure.” Studies on 20-item CES-D reported that the four positively-stated items may not appropriately assess depressive symptoms among Asian populations, as individuals from Asian cultures are less likely to endorse positively framed items based on cultural values (e.g., emotional restraint is valued in Asian culture; Jang et al., 2010; Zhang et al., 2015). The CES-D without the reversely worded items performed better in accurately diagnosing Asian populations with depression (Jang et al., 2010; Zhang et al., 2015). Thus, the 16-item CES-D was used in this study to measure depressive symptoms. A prior study with an Asian sample reported the internal consistency of the 16-item CES-D as .91 (Zhang et al., 2015). In the current study, Cronbach’s alpha was .92.
Results
Initial Factor Structure and Reliability
To split the overall sample, the random selection procedure in SPSS was used. The first half of the sample (n = 194) was used to explore the initial structure of the MMS-RS based on an EFA. Missing data was insubstantial (less than 2%) and was found completely at random (Little’s missing completely at random [MCAR] test: χ2 = 114.72, df = 126, p = .76). Thus, expectation-maximization imputation was employed for missing value replacement. Findings from parallel analysis (O’connor, 2000) indicated potentially three factors could be extracted from the data. As binary reactions to the MMS (i.e., positive and negative reactions) were conceptualized in this study, two-factor and three-factor structures were examined.
Factor Loadings and Communalities for Retained MMS-RS Items (Study 1)
Note. EFA = factor loadings from the exploratory factor analysis result; NRS = Negative Response to Stress; PRE = Positive Response and Endorsement; h2 = communality; CFA = standardized factor loadings from the confirmatory factor analysis result.
The two factors each accounted for 35.37% and 18.19% of the total variance (53.56%). Each factor was labeled based on item content and related literature: Negative Response to Stress (NRS) and Positive Response and Endorsement (PRE). The reliability of each subscale score estimated with Cronbach’s alpha was .87 and .78, respectively. Additionally, NRS and PRE were significantly correlated with a small effect size (r = .17, p = .014).
Evidence for Construct Validity
Model Fit Indices for Confirmatory Factor Analyses (n = 219)
Note. CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = Root-Mean-Square Error of Approximation; CI = Confidence Interval; SRMR = Standardized Root-Mean-Square Residual.
Among the three comparison models, the one-factor unidimensional model showed poor fit, with CFI = .701, TLI = .630, RMSEA = .126, and SRMR = .130. For the uncorrelated two-factor model, the fit indices suggested it to be marginally acceptable, with CFI = .946, TLI = .933, RMSEA = .054, and SRMR = .080. For the bi-factor model, the fit indices suggested an acceptable fit, with CFI = .997, TLI = .996, RMSEA = .014, and SRMR = .03. These indices suggested that the bifactor model is superior to the correlated two-factor model. However, given the inherent mathematical advantage held by bifactor models which lead to erroneous adoption of the bifactor model, if merely attending to the model fit indices (Morgan et al., 2015), ancillary bifactor analysis was conducted. Specific ancillary bifactor analyses were calculated to determine the dimensionality of the instrument in order to further examine the potential acceptability of the bifactor model (Dueber, 2017). First, the explained common variance (ECV) was calculated to determine the proportions of the common variance accounted for separately by the general and the specific factors. The results showed that the ECV for the general factor was .29, indicating that 29% of the common variance across the 13 items can be attributed to the general factor. In contrast, the ECV for each subscale was considerably larger at .64 for NRS and .83 for PRE, indicating that 64% of the common variance across the eight items for NRS and 83% of the common variance across the five items for PRE can be attributed to these factors. Second, the percentage of uncontaminated correlations (PUC) was calculated to estimate the unidimensionality of the scale and the resulting value was .51, which is significantly lower than what would be expected if the unidimensional factor suggested by the bifactor model was valid. In sum, both the ECV and PUC estimates were considerably lower than the recommended guideline (>.70; Reise et al., 2013) to support the unidimensionality of the overall instrument that was suggested by the bifactor model. Rather, the findings from these ancillary bifactor analysis supported the acceptability of the alternate correlated two-factor structure of the MMS-RS.
Overall, the results of ancillary bifactor analysis, the minimal correlation between the two factors that argue against a general factor, the lack of theoretical support for a general factor suggested by the bifactor model, and the nature of the two sets of items representing the factors, the correlated two factor-model was accepted as the one that best represents the latent structure of the MMS-RS items. Based on this conclusion, the construct validity tests were performed on the NRS and PRE subscales.
Mean, Standard Deviation, and Bivariate Correlation of Study Variables
Note. NRS = Negative Response to Stress; PRE = Positive Response and Endorsement; CESD = Center for Epidemiological Studies-Depression; EDS = Everyday Discrimination Scale. *p < .05 **p < .01 ***p < .001.
Next, incremental validity was examined with the hierarchical multiple regression model. Specifically, the NRS score was entered to predict depressive symptoms after controlling for the effects of demographic variables shown to be related to depressive symptoms (i.e., age, educational standing, self-reported social class; Fletcher, 2010; Geldsetzer et al., 2019; Hoebel et al., 2017). As described in the hypothesis section, perceived discrimination was entered as another covariate. As expected, the NRS variable significantly explained the variance on depressive symptoms in Step 2 after the variance accounted for by the covariates were controlled in the first step: b = .33, SE = .10, β = .20; ΔR2 = .04, ΔF(1, 210) = 11.06, p < .001. This result demonstrated evidence of the incremental validity of the NRS subscale score. However, the finding was not replicated with the PRE subscale score. When the PRE score was entered in the second step of the regression model to predict depressive symptoms, the PRE score did not significantly explain depressive symptoms above and beyond the variance of the covariates: b = −.31, SE = .16, β = −.12, ΔR2 = .01, ΔF(1, 210) = 3.85, p = .051.
Study 2
Participants and Procedures
The data to test the stability of the MMS-RS were collected from students attending a Southern university via an undergraduate research participation pool. The sample consisted of 52 Asian American college students who completed the MMS-RS twice at 2- to 8-week intervals. The time frame was specifically chosen to avoid the potential carryover effect—first-time exposure to the items could influence the subsequent response—and to prevent substantial changes in the construct if it ranged outside of 2 months (Chmielewski & Watson, 2009).
Most participants were women (n = 33, 63.5%) and 36.5% of the sample identified as men (n = 19). As for the educational status, most of the participants were Juniors (n = 25, 48.1%), followed by Seniors (n = 18, 34.6%), Sophomores (n = 8, 15.4%), and Freshman (n = 1; 1.9%). Regarding social class, the majority of the participants identified as working class (n = 28, 53.8%) and middle class (n = 22, 42.3%), lower class (n = 1; 1.9%), and upper class (n = 1; 1.9%). The ethnicity distribution was as follows: Vietnamese (n = 7, 13.5%), followed by Chinese (n = 3, 5.8%), Filipino (n = 2, 3.8%), Korean (n = 4, 7.7%), Bangladeshi (n = 2, 3.8%), Indian (n = 2, 3.8%), Multiethnic (n = 8, 15.4%), Pakistani (n = 2, 3.8%), Taiwanese (n = 1, 1.9%), and Thai (n = 1, 1.9%). Others reported as Asian (n = 18, 34.6%) or did not report their ethnicity (n = 2, 3.8%).
Results
Test-retest reliability indices on the 13-item MMS-RS were calculated. Given the repeated nature of the test-retest measurements, intra-class correlation (ICC) was used to assess the temporal stability (Koo & Li, 2016). The ICC coefficients for NRS and PRE were .73 and .78, respectively, indicating that the MMS-RS subscales demonstrated good test-retest reliability (Koo & Li, 2016).
Discussion
The aim of the present study was to develop the MMS-RS, an instrument that measures Asian Americans’ reactions to the MMS. An additional aim was to investigate the psychometric properties of the MMS-RS scores including their initial reliability and validity. The multidimensionality of the MMS-RS was supported in which the correlated two-factor structure (with a small positive correlation between the factors) was deemed to best represent the construct of “Asian Americans’ responses to MMS” based on a combination of statistical findings and theoretical consistency; thus, the use of the subscales as separate measures is recommended. Additionally, across two studies, MMS-RS scores demonstrated strong psychometric properties that suggest that the measure is a reliable and valid tool for research, practice, and advocacy work. Below, we further explicate the study findings, implications, and limitations.
As noted above, the theoretical literature and statistical findings supported the acceptance of the correlated two-factor structure of the MMS-RS. Thus, the MMS-RS consists of two dimensions, representing Asian Americans’ negative response to stress and their positive response and endorsement of the MMS. The negative dimension represents adverse affective reactions to the MMS such as a sense of pressure due to the high and unrealistic expectations imposed on Asian Americans, as well as cognitive and behavioral responses from overlooking the challenge that Asian Americans face as a minority group. The positive dimension, on the other hand, represents a sense of positivity and endorsement of the MMS ranging from feeling proud to a sense of superiority for the racial affiliation. One thing to note is, similar to how scales on acculturation and enculturation are treated (e.g., Yoon et al., 2020), the multidimensionality of a measure does not imply that an individual will or can hold only one type of response and absent other responses. Considering conflicting study findings on the impact of the MMS on a range of outcomes (e.g., Kiang et al., 2016; Suh et al., 2023; Yoo et al., 2010), as well as the qualitative research findings from Asian Americans reporting mixed responses to the stereotype (e.g., Choi & Lim, 2015), it seems reasonable to assume that Asian Americans simultaneously experience both reactions to the MMS to a certain degree. Additionally, statistical findings such as the correlated factor structure being supported and the small yet significant positive correlation between the two factors substantiate the argument that the two factors may be slightly interdependent with both positive or negative responses being simultaneously present.
This composition of two seemingly opposing reactions in the slightly positive valence direction observed in the correlational result illustrates the complex and conflicting responses to the MMS that align with the findings from qualitative research with Asian Americans. We believe that the two-factor MMS-RS structure accurately reflects the experiences and responses of Asian Americans when faced with the MMS. Specifically, it appears that for Asian Americans both the negative and positive responses to the MMS can occur and coexist simultaneously without one negating the other. The complex nature of the MMS that suggests both positive and negative views of Asian Americans portray Asian Americans as being able to hold on to these views in a way that is cognitively and emotionally dissonant yet psychologically acceptable and manageable. Although more research is needed, perhaps this type of cognitive flexibility allows Asian Americans to function effectively when facing situations that evoke not only the MMS but other potentially damaging stereotypes.
Results also provided additional evidence of validity, specifically the evidence for criterion-related validity and incremental validity. The negative response to stress dimension was positively associated with depressive symptoms and perceived discrimination, corresponding to the body of literature that illuminates the harm of the MMS on mental health. Additionally, regression analyses provided evidence of incremental validity; the NRS scale score significantly explained the depressive symptoms even after the variance of depressive symptoms was explained by the score on perceived discrimination along with several demographic variables (e.g., subjective social class). Furthermore, the finding from examining NRS score’s incremental validity shows that negative reactions to the MMS pose a different and separate threat from that of perceived discrimination on Asian Americans’ mental health. Thus, the effects of the MMS should be considered when intervening and assessing Asian Americans’ depressive symptoms, particularly when the symptoms are related to their experiences with racial and cultural oppression. Similar results were not replicated with the PRE score in which PRE did not significantly relate to either depressive symptoms or perceived discrimination. However, the relationship between PRE and depressive symptoms, above and beyond the effects of the covariates (e.g., demographic variables, perceived discrimination), was close to a .05 significance level. Thus, additional research should be conducted with a larger sample size as future findings may show the potentially protective role of PRE against depressive symptoms. To support this tentative hypothesis, prior studies that suggested the positive or protective role of the MMS against mental distress among Asian Americans (e.g., Kiang et al., 2016) likely assessed the positive reactions to and endorsement of the MMS.
Lastly, evidence of internal consistency and test-retest reliability for the MMS-RS was obtained, which exhibited high degree of reliability among the items within each subscale and their reliability across the variation of time. This finding adds to the strong psychometric properties of the MMS-RS scores.
Limitations
Notwithstanding the strengths, this study is not without its limitations. First, the sample was mostly college students from the Southern region of the United States, so the factor structure and psychometric properties may differ depending on the sample characteristics. Notably, attitudes toward multiculturalism, specifically toward one’s race (i.e., Asian), may vary depending on the region within the United States, and the responses to these items may differ accordingly. Similarly, the sample included self-identified Asian Americans, while the experiences around the MMS may look different according to intersecting identities (e.g., multiracial). Relatedly, the sample was mostly comprised of East and Southeast Asian Americans and the evidence of reliability and validity the MMS-RS scores may not be generalizable to Asian Americans who are not from these groups. Second, the test-retest reliability coefficient was computed using a sample comprised only of college students and thus temporal reliability with a nonstudent community sample should be conducted in a future study. Lastly, as with any results from self-report measures, the findings from this study may be prone to response bias.
Implications for Practice, Advocacy, Education, Training, and Research
Practice
The study findings offer several critical implications for practice. Given the pernicious nature of the MMS in the Asian American community, assessing Asian American clients’ spoken or unspoken responses to the MMS by using the MMS-RS may proffer important clinical information. Per the current study results, Asian American clients who report depressive symptoms could be experiencing strong, negative reactions to the MMS, in which assessment on their negative responses to the MMS may aid the conceptualization of the case. Additionally, development of an effective intervention could be followed to disconnect the relationship between the negative responses to the MMS and depressive symptoms. For instance, counselors could help clients have a better understanding of their depressive symptoms, particularly in relation to their reactions to the racial and cultural oppression towards Asian Americans (i.e., reactions to the MMS). Counselors could further help clients to channel their reactions to the MMS by adopting critical consciousness-based actions (Diemer et al., 2016), practice acknowledging the MMS, normalize their reactions to the injustice targeting Asian Americans, externalize the harmful effects from the MMS, and consider joining in the critical actions against the MMS.
Advocacy
The MMS-RS could be utilized to facilitate the exploration of reactions to the MMS, particularly negative responses, and help to give voice to Asian Americans’ concerns and challenges caused by the MMS that have often been overlooked. In addition, the dissemination of the present study findings could galvanize discussions around the harm of the MMS (i.e., negative response to the stress of the MMS) and the conversations around unconscious and implicit endorsement of the MMS (i.e., positive response and endorsement of the MMS). Furthermore, we showcased that two seemingly opposing factors are, in fact, positively related, which helps us to move away from the notion that the MMS is either endorsed or not; an individual may hold both valences of reactions to the MMS regardless of one’s endorsement and internalization of the MMS. Such a finding may shed light on the prior mixed results among studies in which the impact of the MMS on various outcome variables was unclear. Additionally, findings may foster in-depth processing within the Asian American community regarding unconscious endorsement of racialized stereotypes manifested by the MMS—particularly the positive response and endorsement. Lastly, due to the systemic oppression that shares the same vein with White supremacy, Asian Americans’ MMS issue is largely invisible. The present study’s findings could be introduced and utilized in settings that promote advocacy to tackle White supremacy. For instance, outreach at various settings could introduce the MMS-RS and current study results to unmask the truth of the MMS and show that the MMS is a fabricated portrayal of Asian Americans that is used to suppress other racial and ethnic minority groups and uphold White supremacy, which directly and indirectly harms Asian Americans and all racial and ethnic minority groups. Overall, the results of this study can draw attention to the importance of confronting the pernicious nature of the MMS on Asian Americans.
Education and Training
We suggest the study findings be incorporated within the education and training context with the purpose of raising awareness among all involved parties—providers and recipients of training and mentoring. For example, the MMS-RS could be introduced and employed to help instructors and supervisors learn about how they may unintentionally stereotype Asian Americans, causing potential harm. Also, findings could be incorporated throughout the courses, without limiting it to diversity-related classes, to better understand Asian Americans’ challenges and raise awareness of this issue among students of diverse backgrounds.
Research
One of the strengths of this study is developing a tool to directly assess Asian Americans’ reactions to the MMS and investigating its impact on a range of mental distress indicators. In line with the trends in stereotype threat research, responses evoked from the stereotype takes importance as it readily relates to internal processes, such as mental distress, that one may experience as a result of the stereotype more so than one’s belief of the stereotype. Thus, the MMS-RS may help clarify the precise mechanism by which the MMS is related to various outcome variables, which may be of interest for both research and clinical purposes.
Additionally, some studies suggest that the MMS may function differently in various life aspects depending on the context (Atkin et al., 2018) or individuals’ alignment with the stereotype (Yoo et al., 2010). The MMS-RS enables us to explore if the valence of reactions to the MMS help to explain and clarify the relationship between the MMS and a range of outcome variables in addition to the contextual and individual factors. Furthermore, given that the NRS and PRE scores are slightly and positively related, a cluster analysis or latent profile analysis with the two subscales may help to illuminate the association between the subscales. For instance, findings from a cluster analysis or a latent profile analysis may identify groups of individuals with similar association patterns between the subscales and potentially how each cluster or association pattern relates (e.g., high in NRS-high in PRE or high in NRS-low in PRE) to variables of interest (e.g., mental distress, academic/career satisfaction).
Lastly, given the interest in this study was to explore whether the responses to the MMS would be related to discrimination experiences and depressive symptoms and the limited space in the questionnaire, measures of internalized MMS were not included in the present study. Hence, future study is warranted to explore the level of distinctiveness between responses to the MMS (via the use of the MMS-RS) and internalized MMS (e.g., IM-4, IAASS) in order to better understand the similarity and differences between these two constructs.
