Can a visual values-affirmation intervention improve test scores of students in areas affected by crisis?

Abstract

Values-affirmation (VA) exercises, which direct people’s attention to aspects of their lives that they value and broaden their sense of self, have been shown to improve performance in many populations, particularly those who worry that doing poorly will feed into negative stereotypes of the ethnic or other social groups they belong to. Most studies of VA have examined its benefits in highly literate, economically stable, English-speaking populations and have used written exercises. We conducted a randomized controlled trial of a visual VA exercise in an understudied population: marginalized Arabic-speaking students (mostly Syrians) living in a context (Lebanon) affected by conflict. Before taking final exams for a program to improve basic Arabic and English literacy skills and math proficiency, the participants, ages 14-24 years, made a drawing that represented a value important to them. This visual VA exercise improved performance on the Arabic test, particularly among the Syrians, suggesting that, at least for the Arabic test, it reduced anxiety related to stereotyping, allowing students to relax enough to demonstrate their true ability. If replicated, our findings would suggest that schools could use such exercises to improve the value of test scores for guiding decisions about next steps in the education of marginalized students in a context affected by conflict.

Testing is increasingly used to rank students, schools, and countries’ educational systems, ¹ and students know that poor results can have distressing repercussions, such as the humiliation of having to repeat a grade. ² The pressure to succeed is particularly stressful for students who are marginalized—that is, deprived of social acceptance or resources available to others—as is often the case for ethnic minorities and migrants. Marginalized students often fear that their poor performance will confirm negative stereotypes about their group, such as that group members are less intelligent than nonmarginalized students are. ³ This anxiety, in turn, can impair performance and amplify existing achievement gaps between marginalized and nonmarginalized students,^4,5 an effect that feeds concern that educational systems are replicating inequalities rather than addressing them.^6-10

Brief social-psychological interventions that target students’ thoughts, feelings, and beliefs offer a promising way to reduce achievement gaps in school. Values-affirmation (VA) interventions, in particular, have accumulated a substantial evidence base^11-14 and have been applied increasingly in recent years to improve students’ performance and reduce gaps in performance between subsets of students.^15-19 These interventions prompt students to bring to mind a value that is important to them, such as having positive relationships with family members or friends or a commitment to religion.

So far, VA interventions have been studied predominantly in highly literate populations, and little is known about their effectiveness in other groups, such as marginalized students living in settings affected by conflict. Yet interventions for these students are sorely needed. They often have poorer educational outcomes than marginalized students in more stable communities do.^20-22 Furthermore, some have expressed concern that individuals in such populations may be more susceptible to embracing extremism, ²³ although the evidence for this is mixed.

We have conducted a randomized controlled trial in Lebanon to directly assess whether a VA intervention could improve the test scores of marginalized students living in a setting affected by the neighboring Syrian conflict and internal crises. Our study addressed this question by creating an Arabic version of a VA exercise and assessing its effect on a population consisting mostly of adolescent and adult refugees from Syria who enrolled in a program in Lebanon that teaches basic Arabic and English literacy and numeracy. Our VA intervention involved having students express their values through drawing rather than in writing, because the literacy level of the students in the program was low. We found that this brief intervention could improve test scores in this population and was more effective for the Syrian students than for others in the program.

Past Research Supporting VA

The classic example of a VA intervention that had a powerful effect on academic performance was tested by Geoffrey L. Cohen and his colleagues, who delivered it to seventh-grade students in the United States attending schools that were in middle- to lower-middle-class areas and that had roughly equal numbers of students of European and African descent.^11,12 In two randomized controlled trials, participating students were assigned to either treatment or control groups early in the school year. Students assigned to treatment groups were given a list of values and asked to select their most important personal value (in the first study) or their two or three most important values (in the second study); the participants then wrote a paragraph about why the value or values were important to them. Students in the control group for each study were also given the list of values, but they were asked to select their least important value or values from the list, and they wrote a paragraph about why the value or values might be important to someone else.

At the end of the school year, the researchers found an improvement in the grades of the African American students in the treatment groups. ¹¹ In the first study, the intervention increased the mean grade point average (GPA) of these pupils by 0.26 on the four-point scale; in the second study, it increased the GPA by 0.34 points. Both results were statistically significant.

A follow-up study including students from those two studies and a similar third study conducted with a later group of students found that the effect lasted through the end of the following year, with a gain of 0.33 GPA points. ¹² Other studies have demonstrated that VA interventions can increase the probability of college enrollment among African American students seven to nine years later. ¹⁴

VA interventions have been shown to improve academic outcomes for other stereotyped social groups, too. For instance, they have improved academic outcomes and increased enrollment in college readiness courses among Latinx middle schoolers, ¹⁴ reduced the gender gap between the grades of men and women in a college physics course, ¹³ and enhanced performance on tests of mathematical concepts among undergraduate students in a psychology statistics course attended mostly by women. ²⁴

Recent studies have replicated the benefits of VA interventions at scale, providing evidence of the validity and robustness of VA interventions in varying settings, albeit with effects that were smaller than those seen in earlier studies. In the United States, a VA intervention delivered to seventh-grade students across an 11-school district led to improved cumulative GPAs among racial and ethnic minority students. ²⁵ In the United Kingdom, a study with tenth and eleventh graders across 29 secondary schools found modest effects that were sustained for a year.^26,27

Although the many findings supporting the ability of VA interventions to improve academic outcomes are encouraging, the findings are not equally positive in all contexts, ²⁸ and several questions remain. For instance, the smaller effect found in the large-scale studies raises the issues of how, when, and for whom VA interventions are most appropriate.^28-30 In addition, the research to date is marked by three gaps in the populations that have been studied, which may limit the generalizability of the findings to other populations.

first, most studies have been conducted in high-income, English-speaking countries. A search on “values(-) OR self(-) affirmation” in all databases of the Web of Science (https://www.webofscience.com/wos) on April 18, 2021, yielded 1,531 studies, of which 60.9% were in the United States (44.4%), England (10.9%), or Canada (5.6%).

Second, and related to the first point, the populations targeted by most VA studies live in contexts marked by relative stability, even though it is students who are experiencing displacement, military or other conflicts, economic insecurity, or fear for their own or their family’s safety who are at heightened risk of low educational attainment and achievement.^21,22,31,32

Third, the interventions almost always require study participants to make their affirmations in writing and therefore can only be deployed among populations with high levels of literacy. Indeed, a written intervention with a population that feels it has a low level of literacy may back-fire, as it makes such a perceived shortcoming more salient. This risk is not merely theoretical: Studies in which negative stereotypes apart from literacy have been primed (that is, brought to mind) have been shown to negatively affect performance among groups as diverse as African Americans, ³³ student athletes, ³⁴ older people, ³⁵ and immigrants. ⁵

Evidence of generalizability is important in light of both the much-publicized concerns over the replicability of many psychological studies ³⁶ and the need to better understand the factors that moderate the effectiveness of affirmation interventions.^28,29

Despite the predominance of written VA interventions with students, there is reason to suspect that variations that do not rely on writing can be effective. For instance, Crystal C. Hall and her colleagues have found evidence that a verbal self-affirmation exercise used by adults at a soup kitchen improved executive control and fluid intelligence (that is, logical reasoning). ³⁷ Moreover, some work has shown that drawing exercises can positively affect emotional states,^38-40 and other research indicates that images can have framing effects (that is, they can define a situation in a way that alters attitudes or behaviors) and priming effects that may be more powerful than those of text alone.^41-43 Finally, a rich literature amassed over more than half a century has examined how individuals’ expectations and values affect their visual perceptions ⁴⁴ and suggests that some social and cultural groups may be particularly responsive to visual imagery.^45,46

Research Into the Mechanisms Underlying How VA Works

Research into the psychological phenomenon of stereotype threat probably helps explain why VA interventions can improve test scores in populations subjected to stereotyping. Stereotype threat occurs when individuals want to perform well on a task, such as a test, but worry that a subpar showing will confirm negative stereotypes of a group that shares their race, ethnicity, nationality, gender, age, or some other aspect of their iden-tity. ⁴⁷ This worry, in turn, can increase anxiety and distract the individuals from concentrating fully on the task at hand. For example, women taking a mathematics examination may, in addition to feeling routine anxiety over a test, experience stereotype threat that increases this anxiety, ⁴⁸ because they fear a poor performance will lend credence to the stereotype that women are less skilled than men at mathematics.

The degree to which an individual’s performance on a task is impaired because of stereotype threat will be moderated by such factors as whether and how much the task brings the stereotypes to mind (that is, is priming) and how much they feel that their identity is linked to their performance on that type of task. Individuals who care a great deal about performing well in a given domain are more likely to experience the effects of stereotype threat.^33,49-51 For example, studies have found that when confronted with the stereotype that men perform better than women in math, women who felt that their math proficiency was an important part of their identity were more likely to underperform on math-related tasks then were women who identified less with being adept in that domain.^51,52

The effects of priming and of the importance to the individual of doing well on a task were shown in groundbreaking work on stereotype threat by Claude M. Steele and Joshua Aronson, ³³ who demonstrated that merely making a racial stereotype salient could impair the performance of Black students relative to White students. Many other studies have found similar results. For instance, scholars have shown the negative effects of stereotype threat on academic performance in racial and ethnic minority groups in general and on other groups in specific domains, such as women in STEM (science, technology, engineering, and mathematics) fields.^33,47,53-56 Hall and her colleagues ³⁷ have also shown that stereotype threat related to living in poverty can affect cognitive performance and decision-making in a nonacademic context.

Extrapolation from past research gives reason to believe that refugees and adults students may experience stereotype threat.^57,58 In the case of refugees, displaced individuals are often viewed in terms of what they lack, which can amplify stereotype threat. ⁵⁹ In the case of adults, most of the discourse and materials related to education focus on children. When programs do address adult learners, those programs are framed as being something other than formal education, which reinforces the perceptions among educators and students that adult learners are atypical and have fallen behind others in their age group. Fear of being stereotyped in this way may be particularly strong for adult students who feel that they need to take classes in basic literacy or numeracy.^21,22,60

How might VA interventions reduce stereotype threat? Existing research indicates that affirming what one values expands one’s perspective on the things that make up one’s identity and thus reduces the negative effects of stereotype threat and leads to better outcomes. In other words, broadening individuals’ perspective of themselves may enable them to recall that other people’s stereotypes do not define them, to trust in their own abilities, and to ease their worry about the consequences of failure.

If this process is a mechanism of change, the effects of a VA would be observed immediately. Accordingly, the timely use of a VA activity just before academic testing may improve the performance of students who experience stereotyping that they fear will be confirmed by the test. If a VA exercise can ease anxiety related to stereotype threat enough to help reveal a student’s true knowledge, it would make testing a more accurate guide to whether stereotyped students need remedial help or are ready to progress and what the next steps in their education should be.

Study Rationale & Predictions

Our study addresses the research gaps described earlier in this article—namely, the lack of research in populations other than English speakers from high-income countries who live in relatively stable conditions and are highly literate. It provides a model for how VA interventions could be deployed in schools among marginalized, low-literacy populations who endure living conditions made unstable by conflict. As we have mentioned, students in these populations and the education systems responsible for educating them are among the populations and contexts most in need of effective, evidence-informed practices to close the achievement and attainment gaps, especially with respect to literacy.^20,61

We conducted the study in multiple public academic centers in Lebanon, where most people speak Arabic as their primary language. Lebanon has the world’s highest concentration of refugees, ⁶² and there are nearly as many refugee students as Lebanese students in the public schools. ⁶³ Most Lebanese students attend private schools; the 31% who attend public schools represent the lower socioeconomic segment of society. ⁶⁴ The students in this study included adolescents and young adults enrolled in a program intended to remediate gaps in their education.

To the best of our knowledge, this is the first randomized evaluation of a VA intervention in Lebanon specifically and among the Levantine and Gulf countries more generally. It is also the first to include refugees as the majority of participants and thus to contribute to the nascent literature on improving educational outcomes for displaced learners in fragile and conflict-affected contexts. Having fled their homeland, Syrian refugees in Lebanon now find themselves in a country that is itself affected by crises: hosting a large number of refugees (particularly from Syria), ⁶⁵ recovering from conflicts with Israel ⁶⁶ and the deadly 2020 explosion of ammonium nitrate in Beirut, ⁶⁷ and contending with deepening economic and political problems. ⁶⁸ Although all of the students in the academic centers we studied are marginalized to some extent—carrying the stigma of being in the low-income stratum of society and needing remedial education—the Syrians might be considered doubly marginalized in that they are foreigners and do not have the same economic, political, or educational rights as people from Lebanon do.

An additional contribution of this study is that it enabled us to evaluate a visual VA intervention we designed for use in groups whose members’ literacy level is low.

The mix of participants in our study allowed us to explore the different effects of our VA intervention in multiple groups whose members commonly experience stereotype threat, including not only refugees but also adult students and females.

specifically, our study explored the following research questions:

• Can a visual VA intervention lead to an improvement in the test performance of marginalized students with low literacy?

• Will the refugees in our study benefit significantly more from the intervention than the nonrefugees do?

• Will the adult learners in our study benefit significantly more from the intervention than the minors do?

• Will the female learners in our study benefit significantly more from the intervention than the male learners do?

Subject Selection & Study Design

Our sample consisted of individuals aged 14-24 years who were enrolled in an accelerated basic literacy and numeracy program offered at various sites in Lebanon. Those 18 years of age or older are considered adults. The program consisted of two 96-hour cycles (about eight weeks per cycle) covering Arabic literacy, English literacy, mathematics, and information and communication technology skills. The program was open to disadvantaged individuals of all nationalities, but the majority had to be Syrian. Only individuals who had been out of school for at least two years were allowed to enroll.

All students in the program were eligible to participate in the study. Participation was voluntary. Students were given a generic description of the study as a “drawing task” during the consent process. A total of 150 out of the 248 students enrolled in seven learning centers agreed to participate. (See Figure 1 for learning centers’ locations and enrollments.) More than half of the participants were female (56.7%); approximately half were adults (46.7%); and almost all of them were Syrian (90.7%), whereas the rest were Lebanese or Palestinian. After they completed the task and their exams, the participants were debriefed and received further details about the intervention and study hypotheses.

Figure 1.

Map of the academic centers & the number of students enrolled in each center

Contrary to previous studies, some involving two or more VA interventions throughout the academic year, this study assessed one activity completed on the day of the final exams. As a result, it offers some insight into whether a one-time intervention can be effective.

Participants were randomized on the day of the intervention using a predetermined randomization sequence applied to consenting students who lined up at each center to receive their activity and room assignment. At each center, half of the participants were assigned to the treatment group and half to the control group.

We applied a randomization approach known as a stratified or block strategy to account for systematic differences that might have existed in the populations across centers.

In total, 77 participants were in the treatment group and 73 were in the control group. In both groups, about half the participants were adults and half were female; their baseline Arabic, English, and math scores were equivalent; and about 90% of both groups were Syrian. (See Table S1 in the Supplemental Material for more details.)

Participants in the treatment group were assigned to one room, where they received the VA treatment; those in the control group were assigned to another room and given a placebo activity not expected to affect test performance. The random assignment was completed under the supervision of one or more research assistants to ensure that the sequence was followed and that only individuals with consent forms took part in the study.

After being assigned to their room, participants in the treatment group received two sheets of paper with instructions written in Arabic. The first sheet included a list of 10 values (such as “friendship,” “honesty,” and “courage”), presented in writing and graphically, along with written instructions telling participants to rank these values. The second sheet contained instructions and a blank space in which participants could illustrate why their top value was important to them. Participants were informed that the quality of the drawing did not matter. A trained research assistant also read the instructions out loud and explained the task to make sure everyone understood it (See Figure 2 for the written instructions and the list of values).

Figure 2.

Instructions & list of values presented to participants

Participants assigned to the control group ranked the same 10 values according to what they thought others (not what they themselves) valued. They were then asked to draw something from their morning routine. The rest of the procedure matched that used for the treatment group.

The use of a placebo is standard practice in experimental studies of VA.^28,69 Thus, our use of a placebo control facilitated comparability with previous studies and helped reduce the possibility of our findings being driven by the Hawthorne effect (that is, people modifying their behavior in response to being observed) or simply by the act of drawing. (See note A for a further discussion of placebo use in VA studies.)

The intervention and the placebo activities each lasted 20 minutes. Afterward, all participants were invited to sit for their final exams with the rest of the students in the academic center.

Measures

Placement Test Scores

All students enrolled in the academic centers, whether or not they participated in the study, completed a placement test to assess their baseline Arabic, English, and math skills. The Arabic and math tests were scored on a scale of 0 to 100; the English test was scored on a scale of 0 to 25.

Final Exam Scores

All students, whether or not they participated in the study, completed final exams to assess their performance at the end of the program. The Arabic and math tests were scored on a scale of 0 to 100; the English test was scored on a scale of 0 to 40.

The raw placement and final exam scores of the study participants were retrieved from the centers’ administrative records. All test scores were standardized as the percentage of correct answers out of the maximum possible correct answers, to account for the different grading scales used on the placement and exit tests and on the three subject-matter tests. Data regarding the participants’ gender, nationality, age, academic center, and grade level were also collected.

Analytic Methodology

The primary analyses were preregistered before the trial was conducted. ⁷⁰ The analyses assessed the effect of the VA task on the performance of the participants in the treatment group relative to the performance of the participants in the control group.

Our primary analyses separately examined the effect of the intervention on the test scores for each academic subject and assessed the change in performance relative to the baseline for each participant.

In addition to the primary analyses, we conducted exploratory moderation analyses, which investigated differences in the effect of the intervention on subgroups in our study population—namely, whether the effect of the intervention differed in Syrians versus non-Syrians, adults versus adolescents, and females versus males. (For more details about our statistical methodology, see the Supplemental Material.)

Results

Figure 3 summarizes the results of the primary analyses. Participants in the treatment group scored 6.9 percentage points more on their Arabic test score compared with the participants in the control group, after baseline scores were taken into account (0.069, 95% CI [0.001, 0.137]); this is the equivalent of an effect size of 0.27 standard deviations. (See note B for a discussion of the statistical terms used in this article.) This effect remained when we accounted for the potential influence of age, sex, and nationality (0.070, 95% CI [0.001, 0.139]). However, the intervention had no effect on English or math scores. (See Table S2 in the Supplemental Material for more details.)

Figure 3.

Effect of the values-affirmation intervention on the proportion of correct answers on the Arabic, English, & math final exams

The moderation analyses showed that Syrian students benefited more from the intervention than did non-Syrian students with respect to the Arabic test (see Figure 4; for fuller details, also see Table S3 in the Supplemental Material).

Figure 4.

Effect of the intervention on the Arabic test scores of Syrians versus non-Syrians

“Our study demonstrates the effectiveness of a VA exercise for a new population and context: mostly refugees attending school in a setting affected by crisis.”

When we compared the percentages of correct answers on the Arabic test out of the total possible scores and accounted for variances in baseline scores, we found that the Syrian participants scored 25.6 percentage points higher than the non-Syrian participants did (0.256, 95% CI [0.001, 0.511]). The intervention did not improve Arabic test scores for females versus males or for adults versus youths. We also found no significant influence of nationality or age group on the intervention’s lack of effect on math and English test results, nor did we find a significant influence of gender on the math results. We did, however, find that male participants benefited from the intervention more than female participants did when it came to the English test (0.158, 95% CI [0.031, 0.286]). Although only tentative implications can be drawn, this finding may suggest that the intervention would be an effective way to support male students taking English language tests.

Discussion

In a study of primarily Syrian adolescent and adult students enrolled in a basic literacy and numeracy program in Lebanon, we found that a brief, low-cost, visual VA intervention improved study participants’ performance on the final exam for the Arabic course. This effect is statistically and substantively important despite null results for study participants’ performance on math and English tests. The effect size—0.27 standard deviations—is larger than the aggregate effect size (a Hedge’s g value of 0.15) reported in a recent meta-analysis examining VA interventions for identity-threatened students. ²⁸

Moreover, the effect size is almost twice the average effect size for VA interventions. ²⁸ For a sense of scale, consider that in the United States, 0.20 standard deviations is approximately equivalent to the gains in reading between grades 9 and 10. Meanwhile, the difference between student grades in a weak school versus an average school in the United States has been found to generally be between 0.20 and 0.40 standard deviations. ⁷¹

The effect size we found may be a conservative estimate of the intervention’s effect, because it may have been dampened by several factors. These factors include delivery of the intervention by researchers, the use of the schools’ existing exams rather than exams designed specifically to align with previous research, and the immediacy of our follow-up. ²⁸ Although the mechanisms behind these dampening effects, which have been found in previous VA studies, are unclear, the following logic could be explored in future research and practice: If students’ regular teachers deliver the intervention, students may take the exercise more seriously or find it more meaningful. If the academic outcomes were based on researcher-designed exams, the exams could provide more precise estimates of effects and address more competencies than is possible when existing exams are used. Finally, longer follow-ups in previous studies have been associated with larger effect sizes, even with relatively light-touch VA studies. Conceivably, the difference in outcomes between the treatment and control groups might also have been diminished if the exercise we gave to the control group participants ended up reducing their test anxiety or expanding their conception of self—such as by prompting them to spend a few minutes thinking about something other than the examination and to engage in a relaxing drawing activity. ⁴⁰

Our study expands the literature on VA interventions in three important ways. first, it demonstrates the effectiveness of a VA exercise for a new population and context: mostly refugees attending school in a setting affected by crisis. This is an important contribution, given the lack of robust evidence concerning which interventions can improve the performance of refugee students ⁶¹ and the concentration of previous VA studies geographically in the United States and demographically among nonrefugee racial minorities and women.^{11,12,14,15,72} The lack of gender-based differences in math scores in response to the VA intervention in our study—in contrast to findings in other studies—suggests that such gender-based effects may be influenced by whether gender-related stereotyping is prevalent in the academic contexts in which the intervention is used. ⁷³ The moderation-analysis finding that the male students benefited from the intervention more than the female students did with respect to the English exam may reflect the tendency of female learners in Lebanon to outperform male learners in academic subjects. ⁷⁴

Second, we show that traditional text-based VA exercises can be replaced by a visual exercise, such as a drawing task, and still be successful. This finding extends the applicability of the technique to low-literacy learners and, potentially, early-grade learners. Being able to use the VA technique with these populations would fill a need, given that low-literacy learners are likely to be stigmatized and are in great need of supportive interventions, especially when they live in areas affected directly or indirectly by conflict.

Third, we provide insight into possible mechanisms of effect for VA interventions. We show that a one-time VA intervention can have an immediate effect on test scores when implemented prior to an examination. This result lends support to the possibility that one of the underlying mechanisms is an expanded sense of self-worth, because an elevation in self-esteem can occur quickly. Moreover, another explanation for improved performance after an intervention—that the student learned new information before an examination—would not be possible, owing to a lack of time for this learning to occur^28,30,75

Arguably, VA is likely to broaden students’ sense of self, stimulating a feeling of integrity and pride across expanded domains that may otherwise be constricted by stereotype threat. Put differently, engaging in VA likely reminds people that stereotype threat is not all that defines the self, which then minimizes the effect of the threat.

“We show that traditional text-based VA exercises can be replaced by a visual exercise, such as a drawing task, and still be successful. “

Future research should examine whether this mechanism is at work by directly analyzing a student’s sense of self-worth and stereotype threat before and after a VA intervention. We would also like to see longitudinal research with refugees to determine the intervention’s long-term effects in this population.

That our primary analyses revealed an effect on Arabic test scores but not on English or math scores could have several explanations. One has to do with the probability that the students identified more with their results on the Arabic test. Because Arabic is the native language in the region and is linked to cultural and religious values (that is, the students felt high domain identification with the Arabic language), showing oneself to have low proficiency on an Arabic test could be more threatening to an individual’s sense of self than would displaying low proficiency in math and/or English—areas that mean less to the students’ sense of self (that is, for which they feel low domain identification). Hence, the affirmation of values may have been most effective against anxiety and stereotype threat regarding the Arabic text. Previous research has shown that proficiency in one’s native language enhances positive feelings about one’s ethnic identity; ⁷⁶ the importance of language skills for ethnic identity may be amplified for learners in this region, given the historical importance of the Arabic language in the Mashriq region, which includes Lebanon and Syria. ⁷⁷

Another explanation for the intervention’s effect only on Arabic test scores is suggested by differences in the baseline test scores between Syrians and non-Syrians. The Syrian students who benefited from the intervention on their Arabic test had higher baseline levels of performance in math and English compared with those of the non-Syrian students. A VA exercise often raises test scores less when students who engage in the exercise have smaller pre-achievement gaps to start with. ²⁸

“We would like to see longitudinal research with refugees to determine the intervention’s long-term effects in this population.”

Last, the self-affirmation intervention may have been more salient and thus more influential during the Arabic portion of the examination because the intervention itself was in Arabic. Future studies should investigate the relationship between the language of the VA intervention and the language of the examination.

Although our study design and our reliance on administrative data did not allow us to directly test whether a reduction in stereotype threat was a mechanism that enabled better test performance, the greater effect of the intervention on the Arabic test scores of Syrians (as was found by the moderation analyses) provides some suggestive evidence. The finding may indicate that the intervention raised the Syrians’ pride in their Arabic identity, an identity linked to the Arabic language, and this rise in pride may have counteracted the anxiety stemming from the stereotyping they experienced as refugees. This mechanism is speculative and would need to be tested in future research; however, our study generates some preliminary evidence that this route of inquiry may be productive.

What are this study’s additional implications for policy and future research? Although our study was small and policy implications should be made cautiously, the results highlight the promise of delivering a brief visual VA intervention to students from low-literacy populations in crisis-affected contexts prior to their taking exams testing the students’ knowledge of their dominant language. As we note in the sidebar Policy & Research Implications, the intervention’s greater effect on Syrian participants than on non-Syrian participants also suggests that the intervention could be effective for refugees elsewhere, especially when schooling is disrupted and students need to take a placement test to determine the most appropriate level at which to reenter school.

However, organizations implementing VA interventions should be careful not to risk further stigmatizing a subset of students by publicly limiting the intervention to those individuals. Besides, although VA interventions are designed to be effective with marginalized students, the cumulative evidence does not indicate that they are harmful for the more dominant groups in the same schools.

This study has direct implications only for improving the Arabic test scores of Arabic-speaking students who are in a catch-up program in a crisis context. We encourage organizations working with these and similar populations to extend our findings by combining this visual VA intervention with rigorous research on its effectiveness and investigating how, when, and for whom visual VA interventions are most appropriate. When implementing our intervention in the future, we recommend adjusting it to be delivered by teachers or trainers instead of researchers, as evidence suggests that this approach is more effective.^28,78,79 Having the intervention implemented by teachers or trainers would also help to assess its potential effect outside of the experimental context, because those people would likely be the implementers if a visual VA exercise became part of a school’s or learning center’s standard practices.

As we noted earlier, our study also suggests that Arabic students who engage in a visual VA exercise immediately before taking a test assessing Arabic literacy will enhance their test scores not by learning something new but by reducing their anxiety and stereotype threat or by expanding their sense of self enough to enable them to perform closer to their actual level of proficiency than would otherwise have been the case. After all, they would not have had time to learn something relevant to the test between doing the exercise and starting the test. Longitudinal studies would be needed to determine whether a visual VA intervention can also improve learning trajectories over time, as has been shown in studies of other VA interventions in other populations.^12,14,26

We would also like to see replication studies that assess the robustness of the current findings at scale and that are well powered to detect differences in the effects of treatment in various subgroups. Ideally, studies would also investigate how the content of students’ drawings relates to the observed effects; recent work shows that the content of essays written in traditional VA exercises can somewhat predict who will benefit most from the intervention. ⁸⁰

We hope that our study encourages more researchers and practitioners to explore low-cost social-psychological interventions to support the educational progress of learners with low literacy residing in conflict-affected areas and that it will encourage research into using VA interventions to support low-literacy students more broadly. Such interventions have the potential to benefit millions of learners in fragile countries around the world.

Supplemental Material

sj-pdf-1-bsx-10.1177_23794607231192720 – Supplemental material for Can a visual values-affirmation intervention improve test scores of students in areas affected by crisis?

Supplemental material, sj-pdf-1-bsx-10.1177_23794607231192720 for Can a visual values-affirmation intervention improve test scores of students in areas affected by crisis? by Daniel D. Shephard, Ali Osseiran and Fadi Makki in Behavioral Science & Policy