Sage Journals: Discover world-class research

Abstract

Growth mindset interventions delivered to mathematics students have resulted in improved mathematics performance in numerous countries in the global north. South African high school students consistently underperform on international measures of mathematics achievement. This study investigates whether a culturally adapted growth mindset intervention delivered on WhatsApp can make a significant difference to mathematics performance in South Africa. In addition, it investigates whether WhatsApp delivery is feasible and acceptable to high school participants. The intervention employed a combination of YouTube videos, text information and facilitated WhatsApp conversations and combined simplified information on neural plasticity, learning strategies, and growth mindset messaging. The intervention utilised a field-based, quasi-experimental design with 301 students, 160 in the experimental condition. Mixed-methods data were collected, including mathematics performance and mathematics mindsets tracked over 18 months and user engagement and feedback in the intervention. In mixed-effects linear regression, students receiving the intervention showed a significant change in their mathematics performance over time, controlling for baseline differences between the schools. WhatsApp was a well-accepted medium for delivering this intervention. This study adds to the understanding of behaviour change and the use of WhatsApp to deliver behaviour change interventions. The results underscore the importance of teaching study strategies alongside growth mindset content to equip students with the appropriate skills to act on their changed beliefs.

Keywords

Behaviour change high school intervention mathematics mindset South Africa WhatsApp

Introduction

Mindset theory is a theory about personal response to challenges and setbacks (Yeager & Dweck, 2020), based on lay beliefs about human attributes such as mathematics ability or intelligence (French, 2016). Mathematics mindset is measured on a continuum between fixed and growth beliefs. Fixed beliefs are beliefs that mathematics ability is predetermined and mostly unchangeable (Bernecker & Job, 2019). For people holding fixed beliefs, the response to challenges might include avoidance or helplessness. Growth beliefs are beliefs that mathematics ability can grow, change, and improve (Yeager & Dweck, 2020). The responses to challenges in this belief system might include perseverance, asking for help, and looking for strategies. There is fluidity in the mindset continuum for individuals across both time and contexts (Yeager & Dweck, 2020).

South Africa consistently underperforms in mathematics on both local (Laurence, 2024) and international measures such as the 4-yearly Trends in International Mathematics and Science Study (TIMSS) (Letaba, 2017). The underperformance is a cumulative problem with learning losses compounding for students as they progress through the school system. Poor mathematics performance is not uniform but falls loosely along racial and poverty lines, with the poorest and most disadvantaged students producing the lowest results (Spaull & Kotze, 2015). Mindset beliefs are sensitive to brief, cost-effective interventions that can be delivered at scale (Yeager & Dweck, 2012). A growth mindset intervention could give a small but significant boost to mathematics performance for the most vulnerable students.

In developing an intervention for mathematics mindsets, mathematics anxiety also needs consideration. The physical and emotional responses to challenges, including cognitive (worry thoughts) and affective (nervousness and tension) responses, are interpreted by students and likely mediated via their mindset theories. With a fixed mindset, these responses are interpreted to mean that the task is threatening and should be avoided, or that the task is too hard and pursuing it will be hopeless (Dweck, 2000; Oyserman, 2015). With a growth mindset, these responses are interpreted as functional and adaptive. The stress is present to promote activity (Jamieson et al., 2012). Students with a growth mindset believe that the task is worthwhile because it is hard.

Findings from the Program for International Student Assessment (PISA) (Foley et al., 2017) showed that countries with generally higher mathematics anxiety performed worse than countries with lower mathematics anxiety. Mathematics anxiety correlates negatively with scores on mathematics tests. Ashcraft and Kirk (2001) suggest that people with higher levels of math anxiety are more likely to avoid activities and situations that involve mathematics.

There are antecedent variables related to risk and resilience for young people from underprivileged backgrounds (Ebersöhn, 2017). Poverty has an impact on access to resources in the family, community, and school (Spaull, 2015). Poorer communities in South Africa are exposed to higher levels of violence, and this impacts on stress levels (Milam et al., 2010). Compared with other countries, South Africa has one of the strongest correlations between students’ home backgrounds and academic performance, accounting for roughly 60% of the learning gap between rich and poor (Shepherd, 2016). For these reasons, ‘Study Milieu’ is included as a control variable in the analysis. Study Milieu is a subtest on the South African developed ‘Study Orientation to Mathematics’ test and refers to the study environment at home and school (Maree et al., 1998).

Social Networking Services interventions

Social Networking Services (SNSs) have been used to promote health behaviour, targeting issues such as diet compliance for diabetes, heart disease, and food allergies, and supporting participants to quit smoking (Phua, 2013), lose weight (Turner-McGrievy & Tate, 2013), and promote sexual health (Bull et al., 2012). To a lesser extent, they have been used for mental health interventions targeting depression, anxiety, and psychosis symptoms (Alvarez-Jimenez et al., 2013; Rice et al., 2018), and mental health literacy (Li et al., 2018).

The roll-out and upscaling of SNS interventions, campaigns, and support are critical in contexts where there is an undersupply of affordable psychological and mental health services. SNS interventions have excellent potential to scale, so even a very small effect at a local level will have a noticeable impact when scaled up. SNS interventions also have the advantage of being accessible anywhere where the internet is available, thereby offering services to people living in remote places who may ordinarily miss out.

There are good early indicators that SNSs can be effective in changing behaviour. A meta-analysis completed by Yang (2017) found these mediating factors impacting on the effectiveness of health interventions on SNSs: topic of the intervention, study design, and participant engagement level. A discussion on these three points follows.

The topic of the intervention must be relevant to the participants. Healthy groups of people were less likely to participate in a health intervention than groups who recognised that they were at risk (Yang, 2017). In a preliminary study, Morse (2022) established that high school students in South Africa are interested in discussing mathematics achievement and that they do hold mathematics mindset beliefs.

The design of an intervention is critical in effecting behaviour change. A gap exists between intentions and actions (Wendel, 2020). Good intentions and the sincere desire to change a behaviour are not enough. They need to be followed through with the application of specific strategies that will lead to a change in outcomes. In a mindset intervention, growth mindset messaging should be followed through with learning and studying strategies as well as assistance in reinterpreting difficulty as a positive sign of growth.

Engagement is a key component to the success of such interventions. Engagement can be understood through various theoretical frameworks. Social support theory postulates that social support prompts participants to encourage one another (S. Cohen & Wills, 1985). Social comparison theory postulates that comparison of self against others prompts participation (Livingston et al., 2013). Programmes built on SNSs that encouraged engagement and personalised messaging were far more successful than broadscale messaging with no interactions (Yang, 2017).

Theoretical model of change

How does change happen for fixed mathematics mindset students? In this model, the vehicle of change is the SNS, WhatsApp. The mechanisms of change illustrated in Figure 1 are information, integration, social support, and social currency.

Figure 1.

Theory of change for a WhatsApp-based mathematics mindset intervention.

Growth mindset interventions typically deliver information in the form of simple neurobiology lessons that explain neuroplasticity and establish this as a basis on which to expect that mathematics ability can change (Bui et al., 2023). In previous interventions, this is combined with integration tasks that encourage participants to personally apply the information, such as asking the student to write a letter to help a younger learner (G. L. Cohen et al., 2006). Social support provides reinforcement for trying new strategies. Finally, social currency refers to the number of times the promoted ideas get mentioned both on the WhatsApp group and off the group, as reported by students and teachers. The more social currency the ideas get, the more likely it is that the group will implement them (Wendel, 2020).

The combined mechanisms of change lead to a reinterpretation of mathematics challenges. Feelings of stress or anxiety are not indicators that the challenge should be avoided. Rather, the emotional and physiological signs of stress are interpreted to mean that the body and mind are ready for a challenge. This is like how we might understand prerace physiological arousal. This new understanding builds confidence to try new strategies with the expectation that the mathematics challenge can be overcome (adapted from Oyserman, 2015).

Finally, new thinking and new strategies become habitual. Over time, this leads to improvement or stabilisation of declining mathematics grades, particularly for underperforming students.

To the author’s knowledge, this is the first brief social psychology intervention to be offered via WhatsApp and the first growth mindset adaptation to WhatsApp, globally. Given the socio-educational context that South Africa finds itself in, it is unsurprising that much of the previous research focus has been on the deficits in the system, on teachers, on students’ families and communities, and on the students themselves (for example, Spaull & Kotze, 2015). A small body of research focused on resilience and achievement against the odds is emerging, though (for example, Wills & Hofmeyr, 2019). Still, very little of this is specific to mathematics performance or high school students. While there has been local interest in the possibilities of growth mindset interventions boosting performance (for example, Porter et al., 2020), application has sometimes been ad hoc.

This research is important in the South African education landscape because it focuses on the potential to acquire beliefs and behaviours that promote performance in mathematics. While poverty accounts for a large portion of the variance in mathematics performance (Shepherd, 2016), it is very difficult to rectify poverty, while beliefs and behaviours may be malleable.

Hypotheses

1. Null (Ho1): Mathematics mindsets will have no significant correlation with mathematics grades.

Alternate (Ha1): Mathematics mindsets will positively correlate with mathematics grades.

2. Null (Ho2): Mathematics mindsets will have no significant correlation with mathematics anxiety.

Alternate (Ha2): Mathematics mindsets will correlate with lower mathematics anxiety.

3. Null (Ho3): Participants in the experimental group will not show significantly greater change in growth mindset scores than the control group.

Alternate (Ha3): Participants will show greater change towards growth mindset scores in the experimental group than the control group.

4. Null (Ho4): There will be no significant correlation between participation in the intervention WhatsApp chat group and changes in growth mindset scores among participants in the experimental group.

Alternate (Ha4): There will be a positive significant correlation between participation in the intervention WhatsApp chat group and changes in growth mindset scores among participants in the experimental group.

5. Null (Ho5): Participants in the experimental group will have no significant improvement in mathematics grades compared with the control group.

Alternate (Ha5): Participants will show greater improvement in mathematics grades in the experimental group than the control group.

Method

Students from Grades 8 and 9 in two Cape Town high schools were invited to participate in the study. These grades were selected as they are the only high school grades in which students do a uniform mathematics curriculum. One school was allocated to the intervention and the other to a passive control condition. At the intervention school, 164 students agreed to participate. At the control school, 146 students agreed to participate, resulting in a total sample of 310. Five participants from the control school with mathematics grades <25 at all three grade collection times were excluded as preliminary focus groups (Morse, 2022) showed that these students were unable to express coherent beliefs about mathematics achievement due to their general lack of participation in the subject. Four Grade 9 students left the intervention school before the intervention, and they were removed from the data set resulting in a final sample of 301.

At the intervention school, there were nine classes, and the four class teachers also participated. The nine classes were divided among four mentors who self-selected to assist with the research. The schools were purposively selected, based on their similar geographical, demographic, socioeconomic, and performance profiles. The intention in selecting two similar schools was to attempt to reduce the effect of variation between schools. The two schools are situated 6 km from each other in the Southern Suburbs of Cape Town. School performance and fees are shown in Table 1. Both schools are racially mixed, but neither has any White students.

Table 1.

Participant school profiles.

	Intervention (School 1)	Control
Fees per annum (2021)	R10 000	R10 600
Matriculation Bachelor Pass (2020)	86%	80.8%

Design

This study was a field-based quasi-experiment with one experimental group and one passive control group. The study used mixed-methods, integrating observational and quantitative data to promote a multi-dimensional understanding of the impact of a mathematics mindset intervention. The intervention was delivered on existing mathematics class WhatsApp groups that included teachers.¹

Quantitative data collected included student mathematics grades at three time points over approximately 18 months – baseline, midline, and endline. The timeline is described in Table 2: baseline psychometric testing, including a South African mindset scale – Thinking About Mathematics (TAM, Table 3) (Morse, 2022) and two subtests from the Study Orientation to Mathematics (SOM) (Maree et al., 1998). The SOM is distributed by JVR Psychometrics, and permission to use the subtests rather than the full assessment was given by the author. The SOM subtest – Study Milieu– elicits answers about the home and school environment and was used as a proxy for socio-economic status. The SOM subtest – Mathematics Anxiety – elicits answers about physical, cognitive, and behavioural signs of anxiety in mathematics contexts.² These data were collected using Survey Monkey forms on computers in the school computer laboratories, supervised by the researcher. JVR Psychometrics provided the Survey Monkey form with SOM branding and the scoring template with formulas embedded in Excel. The TAM was also scored with formulas embedded in Excel. Endline psychometric testing repeated the TAM to assess changes in mindset. Due to COVID-19 restrictions, the data were collected via Survey Monkey accessed from students’ own personal devices.

Table 2.

Quantitative data collection.

School + grade	Baseline math	Baseline TAM + SOM	Midline math	Intervention	Endline TAM	Endline math
Intervention Grade 8	March 2020 (Grade 8, Term 1, report)	May–June 2020	Nov 2020	May 2021	June 2021	June 2021
Intervention Grade 9	December 2019 (End Grade 8 examination)	May–June 2020	Nov 2020	May 2021	June 2021	June 2021
Control Grade 8	March 2020 (Grade 8, Term 1 report)	May–June 2020	Nov 2020		June 2021	August 2021
Control Grade 9	December 2019 (End Grade 8 examination)	May–June 2020	Nov 2020		June 2021	August 2021

TAM = Thinking About Maths; SOM = Study Orientation to Mathematics.

Table 3.

Thinking about maths–mindset assessment.

Item
1	I can learn new things but I don’t have the ability to change how smart I am at maths.
2	With enough time and effort I could significantly improve my maths.
3	I don’t think I can change how clever I am at maths.
4	I believe I have the ability to increase how maths smart I am, over time.

The intervention followed the theory of change process outlined in Figure 1. The mechanisms of change were information delivered via WhatsApp and YouTube. Integration of the information occurred via WhatsApp chats. Social support was provided through group mentors who initiated regular topic-based conversations. Finally, social currency was generated through chats both in the WhatsApp group and outside the group, as observed by teachers. The intervention was delivered over a period of 4 weeks in May 2021.

The programme was called ‘Maths Brain Tricks’.³ On four mornings per week, Monday to Thursday, a short message was delivered. In the evenings, students were invited to discuss the message. Each week followed this pattern:

Day 1 – A short story or poem introducing the key message for the week.

Day 2 – A YouTube clip link and introduction to the clip.

Day 3 – A learning strategy related to the key message for the week.

Day 4 – An integration challenge or task that students were invited to participate in.

The focus for each week was as follows:

Week 1: Your brain is like a muscle. The more you work it, the stronger it gets.

Week 2: There’s no such thing as smart or not smart at maths.

Week 3: Your history with maths does not determine your future with maths.

Week 4: Maths anxiety doesn’t mean you’re bad at maths.

These topic choices and the accompanying YouTube clips were previously piloted to ensure cultural and contextual relevance (Morse, 2022). The full programme is available in supplementary materials.

As process indicators, student WhatsApp interactions were collected along with initial survey feedback about their experience of the group. Process indicators gave information about engagement and uptake of the ideas introduced in the WhatsApp lesson. They were important in understanding barriers and incentives to change. Students additionally reported in the feedback surveys which of the YouTube clips they had watched.

Four young men⁴ with experience facilitating programmes for high school students volunteered to be group mentors. Two 2-hr training sessions were provided on Zoom. In addition, daily support and feedback were provided to the mentors via WhatsApp. At the endline, mentors participated in a 1-hr debriefing session and completed a survey about their experiences.

Four teachers were responsible for the nine classes. Teachers were asked to show the YouTube clip to each of their classes weekly. Teachers were also asked to complete a survey about their experiences and to report on which clips were shown to their classes. A summary of data collection about the intervention is presented in Table 4.

Table 4.

Intervention process data collected.

Teachers	Post-intervention survey – participation experiences, number of clips shown per class, helpfulness rating
Students	Post-intervention survey – participation experiences, number of clips watched, helpfulness ratings, favourite messages
Mentors	Post-intervention survey – participation experience, helpfulness rating
WhatsApp Groups	Number of participants who read messages, per dayNumber of interactions and students interacting on challenge days.Conversation content

Data analysis plan

The data analysis plan for the quantitative data was registered on Open Science Framework before the intervention commenced. All statistical analysis was completed in Stata. Power analysis was calculated assuming an α = .05, a target power of .9, and an effect size of .2. This effect size is in line with previous findings (Blackwell et al., 2007; Claro et al., 2016; Kizilcec & Goldfarb, 2019). A minimum of 212 participants were required for a one-tailed correlation using Fisher’s z test, 266 participants for a one-way analysis of variance (ANOVA), and 54 participants for mixed-effects linear regression.

Correlations between baseline mathematics grades and mindset, and mathematics grades and mathematics anxiety for individual students were calculated using Pearson’s R. There were several measures of participation by school class group. These included teacher-reported number of video clips watched in class, teacher-reported class discussion on intervention content, number of individuals in each class who contributed to WhatsApp conversations, and the length of these conversations. Group-level correlations between WhatsApp group participation (number of interactions during discussion times) and change in mathematics mindset between baseline and endline were calculated using one-way ANOVA.

The mixed-effects linear model regressed mathematics performance on two fixed effects (intervention: control versus experimental school, and time: pretest and posttest) and treated participants within conditions as a random effect, by-participant intercepts. Mathematics mindset, study milieu, and gender were entered as covariates in the model. Anomalous grades were identified by checking for within unit mark discrepancies of more than 20, followed up with verbal confirmation from the school. Initially, complete case analysis was performed for the 128 participants with complete data sets.

There was a substantial amount of missing data for mindset at endline due to difficulties of collecting the psychometric data during COVID-19 teaching restrictions. Given the level of missing data, a sensitivity analysis was performed. Missing values were imputed in Stata 15 using multivariate normal regression and predictive mean matching, and the resulting model compared with the complete case results.

Ethics

Ethics clearance (PSY2018-066) was granted by the Ethics Review Committee of the Department of Psychology, University of Cape Town. The Western Cape Department of Education granted research permission (20181121-8975), and school principals also gave permission. Permission forms were sent to parents, and both parents and students were asked to give written consent and were additionally reminded of their right to withdraw consent at each phase of data collection.

Results

Context

The intervention took place during a term that was disrupted by continuing COVID-19 school attendance restrictions. The intervention school implemented a rotational attendance system. Classes were split in two, and learners attended 50% of the time, completing work at home on other days. This puts an extraordinary burden on teachers to try and deliver content with shortened teaching time. The school term ended early due to a peak in infections. This disrupted the regular examination/assessment period. While the baseline psychometric data were collected in person prior to the COVID outbreak, the endline data had to be collected remotely, an inelegant process.

YouTube clips

Exposure to YouTube video clips varied from zero to four clips watched in class, one per week, over the 4 weeks, with a median of four. There was no significant correlation between the number of video clips watched in class and change in mathematics performance or change in mindset scores. However, students who watched videos in class engaged less frequently on the WhatsApp chat, r = –.35, p < .01, 95% confidence interval (CI) = [–0.50, –0.20]. This suggests that students regulated their own intervention ‘dose’.

WhatsApp group engagement

Overall, most daily messages were read by most students (92%) with a slight decline in Week 2 (86%) due to a public holiday. Engagement was measured by the percentage of the class commenting in the group. This was highest in Week 1, at 22%, and stabilised at 7% in the remaining weeks. There was no significant relationship between mathematics performance and the number of unique engagements, the breadth (to and fro) of engagements, or the combination of both engagement measures and the number of video clips watched in class.

Mentor support

Thirty-nine students in the intervention condition returned their feedback forms. Of these, 96% said that they felt supported by their mentor. Mentors developed strategies to engage students, including planning a time for WhatsApp conversation, using emoticons, having a good starter question, summarising at the end, not making right or wrong judgements on student ideas, using language that the students relate to, and sharing from their own experience. These strategies were identified by mentors collaboratively during daily check-in sessions, mid-intervention training, and final training debriefing.

Student engagement with intervention themes

Students offered diverse opinions on the themes and debated ideas. They were allowed to disagree. The following quotes are offered as examples of this. This first student expresses a fixed mindset but then begins to consider growth mindset strategies:

Sir we just have to accept that some people are born smart. They don’t even try. Its natural but I think if you’re lazy it does actually affect you. Even if ‘you’re born smart’ I still believe you should be trying harder to improve yourself. They should look at such things they might be a downfall.

In support of growth mindset beliefs, a different student said,

It’s like riding a bicycle, the more you fall the more you learn, same as the brain the more you fail your brain tries to build itself to become stronger and to not suffer from the same mistake.

This Grade 9 learner made comments on the role of the school system in developing fixed mindsets:

Because of the systems schools in place, kids that need more time to really understand things get left behind because the ‘syllabus needs to continue’. If you don’t get (it) in one go, hard luck. Kids find themselves in a classroom not understanding anything because (with) mathematics you (need) to understand what was taught before so you can understand what is being taught in the present. Then they end up being made to feel dumb and that’s one of the reasons a lot of kids drop out of school.

Student engagement with strategies

Reflections on strategy emerged from the content of each week, sometimes directly related to the intervention content but more often spontaneously generated. Here, a student discusses the strategy of practicing:

Algebraic equations of exponents; I practiced several times and finally got the hang of it, When the test came it just looked different to what I was practicing so I just stuck to (the process) I knew.

Another student discusses the strategy of slowing down and thinking:

Well because I did not remember, I skipped the question and answered the other questions. As time went by eventually, I remembered. I was so relieved.

It was evident from the engagements that students did not always understand the strategies or may not have agreed with them. Overall, mentors agreed or strongly agreed that the students who participated actively in the chats enjoyed the intervention.

Hypotheses 1 & 2 – correlations with mindset scores

The correlation between baseline mathematics performance and growth mindset was significant and within the expected range, based on previous studies (Morse, 2022). Mindset correlated with performance at r = .25, p < .01, 95% CI = [0.13, 0.37]. As this study is the second correlational study using the established mindset tool, Thinking About Maths, these results established reliability for this tool. Mindset also correlated with lower mathematics anxiety at r = .24, p < .01, 95% CI = [0.14, 0.34]. The first and second alternate hypotheses can be accepted. Two hundred seventy-seven complete correlations were available at baseline, meeting the requirements for statistical significance.

Hypotheses 3 & 4 – change in mindset scores

In the intervention school, mindset scores shifted towards a growth mindset from a mean of 3.85, SD = 0.79, to a mean of 4.26, SD = 0.57 between baseline and endline. However, mindset scores also shifted in the control school from a mean of 2.14, SD = 0.66 to a mean of 4.37, SD = 0.5.

A one-way ANOVA was conducted between schools for baseline and endline mindset means. While there was a significant difference between schools at baseline, F(1, 1) = 362.25, p = .001, there was no significant difference between the two schools at endline F(1, 1) = 2.05, p = .15, tentatively leading to acceptance of the third null hypothesis.

One-way ANOVA for change in mindset scores between classes also showed no significant difference, tentatively leading to acceptance of the fourth null hypothesis. This was true even when class interactions were considered. The results need to be interpreted with caution due to the level of missing mindset data at endline.

Hypothesis 5 – change in mathematics performance

Mathematics scores for the baseline, midline, and endline measurement points are shown in Figure 2. In both schools, there was a significant decline in average scores between baseline and endline, 18 months later. For Grades 8 and 9 at the intervention school, the large drop in mathematics grades occurred from baseline (pre-COVID-19) to midline (3 months into school closure). Subsequently, Grade 8 grades appeared to stabilise at the new, lower mean, although standard deviations continued to increase. In Grade 9, there was mark recovery at the endline. At the control school, Grade 8 stands out as there is no decline in grades; rather, there is a slight upward trajectory and decreasing standard deviation. Grade 9, however, shows the same pattern seen at the intervention school with a drop in grades and increased standard deviation.

Figure 2.

Mathematics marks at baseline, midline, and endline.

There were 133 complete cases (due to missing endline TAM data), and requirements for statistical power were met. A mixed-effects linear regression model was built to test whether mathematics performance changed over time and school (intervention), with an interaction between time and school to specifically test the intervention. Gender, mindset, and study milieu were added as covariates. The model was fit using restricted maximum likelihood estimation (REML), and the results showed a significant overall model, χ²(5) = 94.74, p < .001. The model showed a significant interaction effect between school and time indicating that the intervention may have made a difference to mathematics performance, b = –5.3, SE(b) = 1.8, 95% CI = [–8.88, –1.67], p < .01 and providing support for accepting the fifth alternate hypothesis. The negative coefficient indicates that the control school had worse mathematics performance than the intervention school when all aspects of the model were taken into consideration.

There was a small amount of missing mathematics data and a more substantial amount of missing mindset data (see Table 5). There was no significant difference between students’ mathematics scores at midline and whether they submitted endline psychometric data in a one-way ANOVA. Also, there was no significant difference in mathematics trajectories and submission of endline psychometric data in one-way ANOVA.

Table 5.

Missing data by school.

Data & time	School 1n = 160	School 2n = 141	Total missing	Total completen = 301
Math baseline	1	3	4	297
Math mid-point	1	3	5	296
Math endline	0	8	8	293
Mindset baseline	1	29	30	271
Mindset endline	76	87	163	138

A sensitivity analysis was performed to confirm the complete case regression modelling. Missing mathematics performance scores were imputed in Stata using multivariate normal regression and predictive mean matching. The mixed-effects linear regression using the mean of five imputed models had very similar means and standard deviations to the original data, and the betas were in the same direction, see Table 6. This adds weight to the accuracy of the findings.

Table 6.

Coefficients and standard error for the mixed-model linear regression based on the original data set, compared to that based on the imputed data set.

Variable	Original		Imputed
Variable	b	SE	b	SE
Time	–1.96	0.89	–2.87	1.32
School	–11.21	3.00	–9.43	3.04
School × Time interaction	–5.40	1.83	–3.92	2.45
Gender	–0.18	2.14	–0.43	1.82
Study Milieu	0.58	0.17	0.57	0.51
Mindset	0.73	1.21	2.21	1.37

In both the full-case model and the model with imputed values, study milieu was significant with similar values at b = 0.57, SE(b) = 0.51, 95% CI = [0.28, 0.94], p < .01 (model with imputations). Mathematics mindset was significant in the multiple-imputation model at b = 2.21, SE(b) = 1.06, 95% CI = [0.10, 4.32], p < .05, but not in the full-case model. This is likely due to the large amount of missing data in the full-case model.

Discussion

WhatsApp intervention

WhatsApp as a medium of intervention delivery attracted student attention as indicated by almost all messages being read. Students felt comfortable sharing and debating ideas via this medium, as indicated by the interactions within the groups. The content of the engagements on WhatsApp shows that the key intervention messages were understood by some students, and those students were able to integrate learning into their own situation. Students also used the WhatsApp platform to troubleshoot problems, covering both strategies from the intervention and introducing new strategy content.

When students engaged on WhatsApp, their engagements showed that they did not always understand the lessons or strategies or may have understood but still disagreed. Mentors found ways to support group engagement, including setting a time for the conversation, having a good opening question, using emoticons and teen-friendly language, using examples from their own lives, and summarising at the end.

The mindset ideas had some social currency measured by student reports of in-class and outside class discussions and observed participation in the chats. All four YouTube clips posted on WhatsApp were mentioned by students as among their favourite messages, indicating that the medium of delivery was acceptable to teenagers and that each clip added subjective value to the intervention.

There was no relationship between the WhatsApp group interaction level, number of videos watched, or teacher self-rating of participation in class conversations and change in mindset or mathematics scores, leading to the rejection of the first hypothesis. There was good evidence that students self-regulated dose by watching missed class videos in their own time and by participating more on the WhatsApp groups if there was less interaction in class time. This has interesting implications for stipulated doses in experimental interventions. If dose is driven by participant choice to engage with available material, then possibly the length and depth of intervention set by the researcher are of less importance.

Correlations with mindset scores

Stronger mathematics grades significantly correlated with growth mindset at baseline, as expected. In addition, lower mathematics anxiety significantly correlated with mindset scores. This supports the inclusion of the Week 4 content in the intervention to target mathematics anxiety. In addition, it provides evidence for the incorporation of mathematics anxiety into the theoretical model explaining mathematics performance, mindsets, and behaviour change and represents an important development in this field.

Change in mindset scores

At both schools, mindset scores significantly improved from baseline to endline, 18 months later, despite expectations that improvements would only occur at the intervention school. There are several possible explanations for this. First, the baseline had additional items that were not repeated at the endline.⁵ The additional questions at baseline may have assisted in disguising the mindset questions and reducing demand characteristics.

A second explanation may arise from the unique period, coinciding with COVID-19 school closures and at-home learning. It is possible that schools attempted to compensate for lack of teaching time by encouraging learners to work hard and have a positive mindset towards their work. Some online learning sites, which were more frequently used during COVID-19, promote a growth mindset. Khan Academy⁶ is one such example popular in South Africa. Therefore, given the unique teaching circumstances, the control school may not have been entirely passive.

Change in mathematics performance

There was a general mathematics performance decline across the 18 months of the study. The intervention school cohort was compared with a previous cohort from the same school, prior to COVID-19. The performance decline was similar indicating that the decline between Grade 8 and 9 is not due to COVID-19.

In both the complete case analysis and the sensitivity analysis based on imputed data, a mixed-linear effects model demonstrated a significant interaction effect between school, time (baseline, midline, endline), and mathematics scores. Even after controlling for study milieu, the imputed model showed a significant relationship between mathematics mindset and scores.

Students at both schools improved their mindset scores over the course of the study and improved mindset is related to relatively better mathematics performance; however, the intervention school still showed a performance advantage over the control school, considering baseline, midline, and endline results; hence, the fifth alternative hypothesis was supported, pending a repeat study under normal teaching conditions. Notwithstanding the weaknesses of the intervention study already noted, the relationship between mindset and mathematics performance has been established by both international research (Aronson et al., 2002; Bettinger et al., 2018; Blackwell et al., 2007; Paunesku et al., 2015) and in a South African correlational study (Morse, 2022). Hence, we can be reasonably certain that the relationship between mindset and performance in this model is an accurate representation. However, the intervention school still had an additional boost over the control school. One or more of the following ideas might explain the interaction effect.

Students at the intervention school were not only introduced to mindset ideas, but they were also introduced to specific strategies to implement these ideas. These strategy lessons were reinforced by mentors in WhatsApp conversations. Mindset proponents are clear that believing is not enough. Belief needs to be followed through with action (Dweck, 2013). Bettinger et al. (2018), in their Norwegian adaptation of the PERT mindset programme, added measures of perseverance and willingness to try more difficult questions. This was of particular concern in South Africa where students are often given the message ‘you can be anything’ without the necessary tools and resources to realise their aspirations (Morse, 2019). The specific teaching of strategies and the opportunity to integrate these ideas through discussion account for the boost in performance at the intervention school compared with the control school, despite similar improvements in mindset scores.⁷

Individual change in mathematics interactions is not only explained by a growth mindset. Change begins with establishing psychological relevance, and seeing growth mindset statements as consistent with current and future identity. This influences an individual to interpret difficulty as positive, thereby creating an opportunity for learning and growth. Finally, the individual must take actions to optimise learning by developing strategies and habits that support learning, such as taking notes in class. The intervention did not just teach growth mindset beliefs but offered students an opportunity to integrate these ideas through group conversation, developing psychological relevance. This was followed through by the teaching of new ways to interpret difficulty based on an understanding of neural development and reinterpretation of physiological arousal. Finally, strategies were explicitly taught through the intervention content. The WhatsApp chat platform gave students additional opportunities to discuss strategies with each other. Hence, although students at both schools improved in their mindset scores, students who experienced the intervention (mindset + reinterpretation of difficulty + strategies) had an advantage over students who did not have the intervention, and this is evident in their mathematics grades.

There are additional possible reasons why the intervention school may have improved beyond the control school. First, the social support offered by the mentors at the intervention school may also have been influential in boosting mathematics performance. Future iterations of this study should add a control group receiving non-specific support via WhatsApp. Second, the baseline mathematics grades in the control school were lower than those in the intervention school. The two schools were selected because they are similar in racial profile, socioeconomic standard, and academic matriculation mathematics performance. This should have reduced the chance of selection bias. Although baseline grades were controlled for in the multivariate analysis, there still may be selection bias with stronger performing students possibly more likely to maintain or improve grades and weaker performing students possibly more likely to decline.

A careful redesign could help pinpoint the critical mechanism(s) of change in the intervention model.

Limitations

While these results seem positive, the quasi-experimental, field-based model of the study introduces uncertainties. The whole school intervention and whole school control design was chosen to minimise intervention leakage and to make use of the existing class WhatsApp groups. However, while the design should have strengthened the outcome, the unfortunate overlap of COVID-19-related school closedowns makes the reliable measurement of the dependent variable between schools less certain. Repetition of the intervention after COVID-19 teaching instabilities have passed will help establish its reliability.

In addition, COVID-19 impacted the design of the collection of psychometric data which were self-reported but facilitated by the researcher in-person at baseline and fully remote at endline. The endline collection problems led to more than half of the endline mindset scores being missing. While they appeared to be missing at random, the large amount of missing data limits this study.

Emerging research indicates that mathematics mindset interventions may be enhanced by teacher education and changes to classroom strategy (Boaler et al., 2021; Bui et al., 2023). While teachers were not direct participants in this study, they were exposed to the same intervention as the students, and some reported discussing the concepts in the classroom. Future research should incorporate teacher-targeted messaging to facilitate changes to teaching methods and the way that teachers frame mathematics challenges.

There are no prior examples of the SOM subtests being used apart from the full assessment, and it is possible that administering two subtests in isolation may have affected the overall validity of the measures. However, the correlation between mathematics mindsets and mathematics anxiety was in the expected range, indicating that the construct was not affected by the modification.

Finally, there were too many uncontrolled variables to adequately estimate mindset dose, and there was a substantial amount of missing endline mindset data, so the rejection of the third and fourth hypotheses should be interpreted with caution. For example, some students who missed the video clip in class watched it in their own time. Some students may have read all the messages and thought about them; others might have clicked on the messages but not read them. In addition, there is uncertainty about the strength of the dose created by social currency. On average, students said that the ideas were talked about sometimes, both in class and outside class, indicating some social currency. Students who were engaged in more conversations about mindset effectively increased their own intervention dose, but there was no way to identify who those students were.

Conclusion

The intervention delivered on WhatsApp was well accepted by students who actively engaged with the mindset messages and strategies. Mindset scores improved for both intervention and control school participants, but the intervention school still showed an advantage over the control school. Apart from mindset messaging, the intervention also taught students to reframe difficulty as worthwhile rather than something to avoid. In addition, the intervention explicitly taught strategies to apply mindset concepts. It is likely that reframing and/or strategies gave the intervention school an advantage over the control school. This emphasises the importance of teaching follow-through actions and not just mindset beliefs alone.

Supplemental Material

sj-docx-1-sap-10.1177_00812463241306251 – Supplemental material for Delivering a South African mathematics mindset intervention via Social Networking Service (WhatsApp)

Supplemental material, sj-docx-1-sap-10.1177_00812463241306251 for Delivering a South African mathematics mindset intervention via Social Networking Service (WhatsApp) by Katherine Morse in South African Journal of Psychology

Footnotes

Data availability statement

This study was pre-registered on Open Science Framework and the quantitative data from this study is available there.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethics statement

Ethics clearance was granted by the University of Cape Town, Department of Psychology, Ethics Review Committee (PSY2018-066). The Western Cape Department of Education granted research permission (20181121-8975) and school principals also gave permission to research. Both parents and students gave written consent.

ORCID iD

Katherine Morse

Supplemental material

Supplemental material for this article is available online.

Notes

References

Alvarez-Jimenez

Bendall

Lederman

Wadley

Chinnery

Vargas

. . .Gleeson

J. F.

(2013). On the HORYZON: Moderated online social therapy for long-term recovery in first episode psychosis. Schizophrenia Research, 143(1), 143–149.

Aronson

Fried

C. B.

Good

(2002). Reducing the effects of stereotype threat on African American college students by shaping theories of intelligence. Journal of Experimental Social Psychology, 38(2), 113–125.

Ashcraft

M. H.

Kirk

E. P.

(2001). The relationships among working memory, math anxiety, and performance. Journal of Experimental Psychology: General, 130(2), 224–237.

Bernecker

Job

(2019). Mindset theory. In Sassenberg

Vliek

M. L. W.

(Eds.), Social psychology in action: Evidence-based interventions from theory to practice (pp. 179–191). Springer.

Bettinger

Ludvigsen

Rege

Solli

I. F.

Yeager

(2018). Increasing perseverance in math: Evidence from a field experiment in Norway. Journal of Economic Behavior & Organization, 146, 1–15.

Blackwell

L. S.

Trzesniewski

K. H.

Dweck

C. S

. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246–263.

Boaler

Dieckmann

J. A.

LaMar

Leshin

Selbach-Allen

Pérez-Núñez

(2021). The transformative impact of a mathematical mindset experience taught at scale. Frontiers in Education, 6, Article 784393.

Bui

Pongsakdi

McMullen

Lehtinen

Hannula-Sormunen

M. M.

(2023). A systematic review of mindset interventions in mathematics classrooms: What works and what does not? Educational Research Review, 40, 100554.

Bull

S. S.

Levine

D. K.

Black

S. R.

Schmiege

S. J.

Santelli

(2012). Social media–delivered sexual health intervention: A cluster randomized controlled trial. American Journal of Preventive Medicine, 43(5), 467–474.

10.

Claro

Paunesku

Dweck

C. S

. (2016). Growth mindset tempers the effects of poverty on academic achievement. Proceedings of the National Academy of Sciences, 113(31), 8664–8668.

11.

Cohen

G. L.

Garcia

Apfel

Master

(2006). Reducing the racial achievement gap: A social-psychological intervention. Science, 313(5791), 1307–1310.

12.

Cohen

Wills

T. A.

(1985). Stress, social support, and the buffering hypothesis. Psychological Bulletin, 98(2), 310–357.

13.

Dweck

C. S.

(2000). Self-theories: Their role in motivation, personality, and development. Psychology press.

14.

Dweck

C. S.

(2013). Brainology. https://www.mindsetworks.com/programs/brainology-for-home

15.

Ebersöhn

(2017). A resilience, health and well-being lens for education and poverty. South African Journal of Education, 37(1), 1–9.

16.

Foley

A. E.

Herts

J. B.

Borgonovi

Guerriero

Levine

S. C.

Beilock

S. L.

(2017). The math anxiety-performance link: A global phenomenon. Current Directions in Psychological Science, 26(1), 52–58.

17.

French

R. P.

II (2016). The fuzziness of mindsets: Divergent conceptualizations and characterizations of mindset theory and praxis. International Journal of Organizational Analysis, 24, 673–691.

18.

Jamieson

J. P.

Nock

M. K.

Mendes

W. B.

(2012). Mind over matter: Reappraising arousal improves cardiovascular and cognitive responses to stress. Journal of Experimental Psychology: General, 141(3), 417–422.

19.

Kizilcec

R. F.

Goldfarb

. (2019, June). Growth mindset predicts student achievement and behavior in mobile learning. In Proceedings of the Sixth (2019) ACM Conference on Learning@ Scale (pp. 1–10).

20.

Laurence

(2024). Matric results improve, but maths needs urgent attention. https://techcentral.co.za/matric-results-improve-maths-attention/238240/

21.

Letaba

(2017). South African performance on the trends in international mathematics and science study. The National Advisory Council on Innovation. https://www.naci.org.za/

22.

T. M.

Chau

Wong

P. W.

Lai

E. S.

Yip

P. S.

(2013). Evaluation of a web-based social network electronic game in enhancing mental health literacy for young people. Journal of Medical Internet Research, 15(5), Article e80.

23.

Livingston

J. D.

Tugwell

Korf-Uzan

Cianfrone

Coniglio

(2013). Evaluation of a campaign to improve awareness and attitudes of young people towards mental health issues. Social Psychiatry and Psychiatric Epidemiology, 48(6), 965–973.

24.

Maree

J. G.

Claassen

N. C. W.

Prinsloo

W. B. J.

(1998). Development of a study orientation questionnaire in mathematics. South African Journal of Psychology, 28(2), 101–106.

25.

Milam

A. J.

Furr-Holden

C. D. M.

Leaf

P. J.

(2010). Perceived school and neighborhood safety, neighborhood violence and academic achievement in urban school children. The Urban Review, 42(5), 458–467.

26.

Morse

(2019). ‘You can be anything’–Career guidance messages and achievement expectations among Cape Town teenagers. South African Journal of Education, 39, S1–S10.

27.

Morse

(2022). Closing the mathematics achievement gap: Exploring the applicability of growth mindset in South Africa [Doctoral thesis]. University of Cape Town.

28.

Oyserman

(2015). Identity-based motivation. In Scott

Kosslyn

(Eds.), Emerging trends in the social and behavioral sciences: An interdisciplinary, searchable, and linkable resource (pp. 1–11). John Wiley & Sons.

29.

Paunesku

Walton

G. M.

Romero

Smith

E. N.

Yeager

D. S.

Dweck

C. S.

(2015). Mind-set interventions are a scalable treatment for academic underachievement. Psychological Science, 26(6), 784–793.

30.

Phua

(2013). Participating in health issue-specific social networking sites to quit smoking: How does online social interconnectedness influence smoking cessation self-efficacy? Journal of Communication, 63(5), 933–952.

31.

Porter

Martinus

Ross

Cyster

C. F.

Trzesniewski

(2020). Changing learner beliefs in South African townships: An evaluation of a growth mindset intervention. Social Psychological and Personality Science, 11(7), 991–998.

32.

Rice

Gleeson

Davey

Hetrick

Parker

Lederman

Alvarez-Jimenez

(2018). Moderated online social therapy for depression relapse prevention in young people: Pilot study of a ‘next generation’ online intervention. Early Intervention in Psychiatry, 12(4), 613–625.

33.

Shepherd

D. L.

(2016). The open door of learning–Access restricted: School effectiveness and efficiency across the South African education system [Doctoral dissertation]. Stellenbosch University.

34.

Spaull

. (2015). Schooling in South Africa: How low-quality education becomes a poverty trap. South African Child Gauge, 12, 34–41.

35.

Spaull

Kotze

(2015). Starting behind and staying behind in South Africa: The case of insurmountable learning deficits in mathematics. International Journal of Educational Development, 41, 3–24.

36.

Turner-McGrievy

G. M.

Tate

D. F.

(2013). Weight loss social support in 140 characters or less: Use of an online social network in a remotely delivered weight loss intervention. Translational Behavioral Medicine, 3(3), 287–294.

37.

Wendel

(2020). Designing for behavior change: Applying psychology and behavioral economics. O’Reilly Media.

38.

Wills

Hofmeyr

(2019). Academic resilience in challenging contexts: Evidence from township and rural primary schools in South Africa. International Journal of Educational Research, 98, 192–205.

39.

Yang

(2017). Are social networking sites making health behavior change interventions more effective? A meta-analytic review. Journal of Health Communication, 22(3), 223–233.

40.

Yeager

D. S.

Dweck

C. S.

(2012). Mindsets that promote resilience: When students believe that personal characteristics can be developed. Educational Psychologist, 47(4), 302–314.

41.

Yeager

D. S.

Dweck

C. S.

(2020). What can be learned from growth mindset controversies? American Psychologist, 75(9), 1269–1284.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB