Abstract
High-ability students living in regional, remote, or rural areas of Australia face numerous barriers in accessing quality science, technology, engineering, and mathematics (STEM) education to meet their learning needs. However, there is limited research in how to overcome these barriers and support the development of rural high-ability students within the STEM domain. This small-scale study explores a group of high-ability rural secondary school learners and the impact of their engagement in an authentic research mentor program (ARMP). Using a mixed-method convergent parallel design, qualitative and quantitative data were collected over 3 years from 32 high-ability students in Year 10 of a rural Australian school. Rural high-ability students identified the value of the program in terms of equipping them with transferable science knowledge and skills, as well as social communication and problem-solving skills. Quantitative data analysis supported these qualitative findings. High-ability student participation in the ARMP, as measured by an independent science assessment, significantly enhanced the high-ability students’ science knowledge, understanding, and skills, as well as their problem-solving skills, and communication skills, as compared with a control group. Effect sizes for these measures were large ranging from 0.81 to 1.57. This ARMP addresses the inequities faced by rural high-ability students, providing them with exposure to authentic STEM education and research under the guidance of an academic mentor. Importantly, this research highlights the positive impact of ARMP’s on the development of rural high-ability students’ social and emotional skills; key skills needed for the development of talent in STEM.
Plain Language Summary
High-ability students living in rural areas of Australia face a range of barriers in accessing quality science education. However, there is limited research in how to overcome these barriers. This study explores a group of high-ability rural secondary school students and the impact of their engagement in a science mentoring program. This research took place in a rural Australian school and is based on student statements and academic data collected from 32 high-ability Year 10 students. Rural high-ability students identified the value of the program in terms of equipping them with science knowledge and skills, as well as social communication and problem-solving skills. Participation in the science mentoring program resulted in a significant enhancement of high-ability students’ academic skills including their science knowledge and science skills, as well as their problem-solving skills, and communication skills. This science mentoring program addresses the inequities faced by rural high-ability students, providing them with exposure to authentic science education. Importantly, this study shows the positive impact of the program on the development of rural high-ability students’ social and emotional skills; key skills needed for the development of talent in science.
Keywords
High-ability students represent the world’s future leaders, problem-solvers, and innovators, with a large portion of these talented students expected to work within science, technology, engineering, and mathematics (STEM) industries (Watters, 2021). Given that STEM innovations are key drivers of the world economy, and potential solutions to the issues currently facing humanity, high-ability students’ participation in STEM education and their transition into STEM careers represent a crucial area of education. However, Australia, like many nations, is experiencing challenges in engaging students in STEM (Freeman et al., 2015). Over the last two decades, Australian students’ performance has been declining in international STEM assessments, including the Program for International Student Assessment (PISA) and Trends in International Mathematics and Science Study (TIMSS). Alarmingly, there has been a decrease in the percentage of high-performing high school students in both mathematics and science since 2003 (De Bortoli et al., 2023; Thomson et al., 2020). These results suggest that Australia’s highest achieving students, who have the potential to solve society’s most complex and technical problems (Watters, 2021), are not receiving sufficient STEM support. This problem is even more exaggerated for high-ability students living in regional, rural, and remote (RRR) areas (Azano & Callahan, 2021; Bannister-Tyrrell & Wood, 2021; Plunkett, 2018).
Internationally, there is an educational STEM gap between urban and RRR schools, and Australia is no exception (Saw & Agger, 2021; Sullivan et al., 2018). RRR secondary schools face a wide variety of interconnected issues that potentially hamper high-ability students’ performance in STEM subjects, their views of science, and their opportunities to pursue STEM careers. These issues include geographical distance, recruiting and retaining qualified STEM teachers, access to professional development, lack of STEM educational resources and instructional materials, and high-quality STEM learning opportunities (Australian Institute for Teaching and School Leadership [AITSL], 2023; Fraser et al., 2019; Lyons, 2008; Sullivan et al., 2018; Timms et al., 2018). While there are selective high schools for academically gifted children in Australia, they are typically located in metropolitan areas, with RRR options requiring students to board away from their families and communities.
An authentic research mentoring program (ARMP) represents a way to support high-ability students within their own RRR school and community. In ARMPs, students engage in hands-on research, and are mentored by, and collaborate with, university academics to conduct open-inquiry research to produce authentic scientific outputs, such as scientific manuscripts (Puslednik & Brennan, 2020). These programs can be embedded into the school structure in a way that allows high-ability students to engage with authentic scientific research to develop solutions to original problems. The learning needs of high-ability STEM students often differ from those of their peers (Ireland et al., 2021; Watters, 2018). ARMPs have the potential to address these needs due to their real-world focus and high-quality STEM learning. This immersive pedagogy allows high-ability students to engage in the curriculum at a more meaningful level, while providing them with opportunity to transfer their learning to a real-world context (Puslednik & Brennan, 2020). In ARMPs, teachers support high-ability learners in the learning activities. The highly interactive nature of the ARMP provides teachers with multiple professional learning opportunities as academic mentors role model how scientific knowledge and skills are developed (Puslednik & Brennan, 2020).
Our research examines the impact of an ARMP in a rural Australian school. The ARMP engaged three cohorts of high-ability students in scientific research that examined radiologists’ performance in breast cancer detection. This small-scale study examined the academic and socio-emotional benefits of the program for high-ability rural students and makes an important contribution to understanding how to improve high-ability RRR students’ engagement with STEM.
Background Literature
High-Ability Rural STEM Students
Like other minority groups, RRR students are under-identified and underrepresented within gifted education in Australia, as well as globally (Jung et al., 2022; Plunkett, 2018; Rasheed, 2020). Giftedness has been defined by a range of scholars over time. Renzulli’s (1976) Three Ring Conception of Giftedness is a developmental approach that proposes that gifted behaviors emerge when the following three characteristics (three rings) are all present and cooperating: above average ability, creativity, and task commitment. Gagné’s (2018) differentiated model of giftedness and talent clearly distinguishes between gifts (natural abilities) and talents (systematic development of gifts). Talent is developed from the combination of natural ability and chance being acted upon by both intrapersonal and environmental catalysts. Giftedness may occur in the intellectual, creative, socio-affective, and/or sensorimotor domains, and talent falls into the fields of academics, arts, business, leisure, social action, sports, and/or technology. Davidson and Sternberg (1984) identify several characteristics of giftedness in terms of cognitive processes, information processing, development of cognitive abilities, and skills. However, the translation of these definitions into practice is not always evident at the school level. Another important concept in gifted education that underpins the three definitions described above is the talent development framework (Olszewski-Kubilius & Thomson, 2015). This approach to gifted education focuses on providing opportunities to students who may have the potential to perform at those advanced levels. Within this framework, social and emotional skills are viewed as skills that can be learned and are an important component for students in achieving their full potential.
A range of tools are needed to identify RRR gifted students given their diverse experiences, but many RRR teachers lack confidence in identifying gifted students. Teachers do not feel supported by their schools’ identification policies and are concerned that these policies may miss identifying some students. Typically, initial teacher education only briefly addresses gifted education, and as a result, only a limited number of teachers feel comfortable teaching gifted students (Bannister-Tyrrell & Wood, 2021; Horsley & Moeed, 2021).
In addition, teacher recruitment issues, high staff turnover and out-of-field teaching in RRR schools result in a compounded disadvantage that can place rural gifted students at risk (Callahan & Azano, 2019; Halsey, 2017). Research highlights that while Australian science teachers are aware of the needs of high-ability students, and endeavor to cater to their learning needs despite the systemic constraints they face, this approach is not enough to fill the disparity between what high-ability students need and what teachers provide (Horsley & Moeed, 2021; Ireland et al., 2021). Designing differentiated STEM learning for a small number of students can be challenging without a strong professional network to draw upon and without access to appropriate gifted education professional learning (Bannister-Tyrrell & Wood, 2021). Geographical distance and financial constraints can limit access to role models, enrichment opportunities, networking opportunities with like-minded peers, and extracurricular activities that provide educational support (Halsey, 2017).
Technology can mitigate some of these issues for both teachers and students, depending on the remoteness of the school (Callahan & Azano, 2019). Smaller RRR communities also provide a range of benefits for gifted learners, including providing unique learning opportunities that develop students’ academic as well as social and emotional skills. Gifted students have more leadership opportunities in RRR contexts, which can develop their social and emotional skills and sense of self. Smaller classes help to develop a strong sense of belonging and increase the opportunity for stronger student–teacher relationships and more personalized learning. Creative timetable structuring can also create extension opportunities for high-ability students (Hammack et al., 2023; Morris et al., 2021; Murphy, 2022). Greater connections to the local community and natural environments may result in the opportunity to work with community members on authentic projects that allow high-ability students to participate in real-world science and develop a diverse range of skills with local outcomes (Avery, 2013; Bhaduri et al., 2022). Importantly, school leaders who actively promote STEM education, empower STEM teachers, and leverage community capital in RRR schools achieve better student STEM outcomes (Fraser et al., 2019; Morris et al., 2021; Murphy, 2023).
Authentic Scientific Research for High-Ability Students
Authentic research and inquiry science is an important pedagogical approach for high-ability STEM students, who are often underchallenged academically, especially within RRR schools (Ireland et al., 2021). Engaging students in authentic scientific research typically occurs at university, although there are examples of scientific research being embedded into high schools (Ahmad et al., 2021). The limited Australasian studies examining authentic inquiry research for high-ability RRR school students suggest that there is an academic and socio-emotional benefit for students (Horsley & Moeed, 2021; Puslednik & Brennan, 2020; Watters, 2021; Webber et al., 2020). Using authentic research science experiences to teach RRR high-ability students provides them with the opportunity to be challenged by complex science concepts and engage in in-depth investigations that sustain their curiosity and motivation, resulting in greater engagement in learning, production of higher quality work, and increased academic performance (Puslednik & Brennan, 2020; Webber et al., 2020).
Quality STEM education, such as authentic scientific research, can foster high-ability students’ social and emotional skills, which are crucial for academic success (Ozkan & Kettler, 2022). These skills, such as communication, collaboration, creative and complex problem-solving, critical thinking, and adaptability, are considered crucial STEM workforce skills and have been identified as key candidate criteria within STEM job advertisements (Bybee, 2013; Jang, 2016; Rios et al., 2020). Research has shown that working on authentic research science projects alongside experts develops high-ability students’ sense of autonomy (Morris et al., 2021; Riley et al., 2017). Learning that meaningfully engages high-ability students with appropriate levels of challenge, collaboration, and community interaction has the potential to positively support students’ social and emotional growth (Smith, 2017). However, authentic scientific research opportunities for high-ability STEM students typically are implemented as extracurricular activities, for example, summer camps (Ahmad et al., 2021; Kim, 2021; Wu et al., 2019). These extracurricular activities can be challenging for high-ability RRR STEM students to access due to geographic isolation and economic disadvantage (Bannister-Tyrrell & Wood, 2021).
Mentoring in STEM
Any STEM program aimed at providing authentic intellectual and academic growth for high-ability students should ensure students have access to specialist teachers and/or experts within the field to maximize student engagement (Ireland et al., 2021; Rogers, 2019; Watters, 2021). Role modeling and mentoring has been identified as a crucial factor in developing high-ability STEM students’ academic sense of belonging, with real-world internships being highlighted as one way to achieve this (Shin et al., 2016; Wu et al., 2019). However, given the challenges faced by high-ability RRR STEM students, participation in internships at university laboratories and other research institutions may not be possible due to the lack of, and distance from, these facilities in RRR Australia (Ihrig et al., 2018). Given the strong community relationships within RRR settings, a mentoring program, using local resources and experts, to support high-ability STEM students is a more workable approach (Avery, 2013; Azano & Callahan, 2021).
The importance of mentoring for high-ability students has long been recognized and is well documented (Tan et al., 2019). Mentoring is an effective pedagogical practice that not only supports the academic development of high-ability students, but also critically addresses their social and emotional needs. Gifted students have been characterized as curious and creative problem-solvers who seek perfection. They typically have a mature sense of social justice and high levels of empathy (Peterson, 2009; Rinn, 2024). Although it should be noted that high-ability students are a heterogeneous group, it cannot be assumed that they are all autonomous, self-regulated learners with high levels of motivation, and well-developed time-management skills (Watters, 2021). Therefore, optimal learning environments for high-ability students should be characterized by high expectations and challenging learning tasks combined with motivational and emotional support (Wright-Scott, 2018). Mentors provide disciplinary and intellectual support, build collaborations, help students to develop ownership, and encourage effective hands-on communication. Importantly, mentorships develop personal growth through the provision of psychological and emotional support and role modeling (Byars-Winston & Dahlberg, 2019).
While STEM mentorships vary in their aims and implementation, several characteristics need to be addressed when designing mentoring programs for rural high-ability students. These characteristics are discussed below.
Program Length and Frequency of Contact
Mentor programs can vary in length; however, short-term programs can limit programs’ impact (Kim, 2021; Little et al., 2010). The longer the length of a program, the more likely the program will be integrated into the school structure, which is crucial for program productivity (Ahmad et al., 2021; Kettler & Taliaferro, 2022). Longer programs, which are combined with regular mentor-high-ability student contact, allow for the establishment of quality mentoring relationships, which provides academic and socio-emotional support (Du Bois et al., 2011). In addition, longer programs potentially allow for high-ability students to engage in STEM work outside of the school, thereby building diverse skills in students. This approach helps students to apply their learning from school as well as develop their problem-solving skills and sense of professionalism within an authentic context (Ahmad et al., 2021).
Focus on Real-World Issues
High-ability students’ potential can be nurtured and grown through the development of mentor programs that focus on real-world issues (Maker & Wearne, 2020; Watters, 2021; Wu et al., 2019). By applying authentic STEM research methods and production, students develop a deeper awareness of the profession and the integrated and complex nature of real-world STEM issues (Kettler & Taliaferro, 2022). Providing high-ability students with the opportunity to work on ill-defined problems appeals to their sense of curiosity and social justice and develops their problem-solving, critical thinking, and creativity skills (Kim, 2021; Ozkan & Kettler, 2022; Watters, 2021). Within RRR areas, place-based learning pedagogies provide strong connections to community, and local issues can potentially form the basis of the mentor programs (Avery, 2013; Azano & Callahan, 2021). Furthermore, this authentic approach can result in novel knowledge that benefits the scientific and wider community (Brooks et al., 2011).
Explore Concepts in Depth, Breadth, and Complexity
High-ability students tend to be more precocious in subject matter content and have a desire to understand concepts at a deeper level than is typically required by most curriculums (Watters, 2021). Therefore, it is critical that mentor programs engage students in learning that extends either beyond the curriculum and/or beyond the students’ age-based learning year. Extension programs that increase the depth, breadth, and complexity of learning are the evidence-based practice to support high-ability students’ learning and motivation (Kettler & Taliaferro, 2022). The role of the mentor is critical in this aspect, as they have the knowledge and expertise to support the disciplinary and intellectual demands of high-ability students and potentially supplement the STEM knowledges in RRR communities (Tan et al., 2019). Novel and interesting learning tasks allow high-ability students to be exposed to new knowledge. This exposure challenges and deepens their understanding of concepts and provides students with the opportunity to develop their continuous learning skills (Mullet et al., 2018).
Provide Students With Autonomy
High-ability learners are sensitive to academic autonomy and value the opportunity to engage in activities that allow for some level of academic freedom (Mullet et al., 2018). STEM mentor programs need to create learning environments whereby high-ability students can actively decide the focus and/or direction. Utilizing an authentic research inquiry approach as the basis of the program will naturally allow for these opportunities and hence support students’ sense of autonomy and self-direction and develop their personal accountability and professionalism skills (Shoemaker et al., 2016) A crucial aspect to be incorporated into these opportunities for autonomy is support from the mentor. The mentor must balance the need to validate the high-ability students’ choices and support their feelings of competence with timely, appropriate, and actionable feedback on their decisions (Larose & Tarabulsy, 2014). Students’ ability to apply and to respond to this feedback effectively will help to develop their mastery of learning and adaptability skills (Watters, 2021).
Incorporate Opportunities for Collaboration
Mentor programs that allow for a high level of collaboration between a variety of stakeholders, including students, academics of varying level, teachers, and the wider community provides students with the experience of an authentic STEM workplace (Brooks et al., 2011; Otterstetter et al., 2011; Petersen & Chan, 2020). This approach helps high-ability students understand what authentic collaboration looks and feels like, as compared with classroom group work that can be problematic for high-ability students (Watters, 2021). Authentic collaboration is also important for students’ social development and their career choice and trajectory (Ahmad et al., 2021). High-ability students enjoy solving ill-defined and challenging problems in the company of like-minded peers and with the support of mentors, who are experts in their fields, and open-minded teachers (Kim, 2021; Watters, 2021). Genuine collaboration, where students contribute in a meaningful and substantive way to a team project, supports the development of high-ability students’ social intelligence skills and ability to collaborate in diverse teams (Otterstetter et al., 2011; Shoemaker et al., 2016).
A Strong Mentor–Mentee Relationship
Underlying all the above characteristics is the quality of the mentor–mentee relationship. This relationship is a crucial characteristic that can influence the learning outcomes associated with mentor programs (Tan et al., 2019). Mentors need to provide intellectual and socio-emotional support for high-ability students while role modeling their profession. As such, they need to have expert disciplinary knowledge as well as be approachable, available to share ideas, and provide clear guidance and feedback to high-ability students. Employing a competency-based inclusive mentoring practice allows for a trusting collaborative learning relationship to develop, as mentors and high-ability students’ work toward mutual goals (Byars-Winston & Dahlberg, 2019). Within the rural context, mentors also need to be able to work collaboratively with teachers and within the confines of rural schools’ policies and structures. Mentoring structures can differ and can include mentoring dyads, triads, collectives, or group mentoring, and even mentoring networks. These structures can be either face to face, online, or a mixed model of face to face and online (Byars-Winston & Dahlberg, 2019; Stoeger et al., 2019).
Drawing upon the above characteristics and the talent development framework (Olszewski-Kubilius & Thomson, 2015) we designed a formal academic mentoring program for rural high-ability science students. Our program allows rural high-ability students to authentically engage in scientific research under the guidance of an academic mentor, the larger academic team, and their classroom teacher. Critically, this program allows students’ talents to be grown through learning experiences that resemble the work that scientists engage in. Mentoring used in this study is therefore defined as an immersive pedagogy conducted with the guidance of a research scientist that allows students to engage in authentic scientific research. We developed a model for the ARMP grounded in Bronfenbrenner’s (1979) ecological lens and the theoretical model of the mentoring process (Rhodes, 2005) and draws upon the socio-motivational mentoring model (Larose & Tarabulsy, 2014). Using this model, the following research questions were investigated:
Theoretical Framework
The theoretical framework for this research is grounded in Bronfenbrenner’s (1979) ecological systems theory. The theory proposes that a persons’ development is influenced by a series of interconnected nested systems and relationships that radiate outwards from the individual. The immediate surrounds represent the closest relationships, while the broader societal and cultural influences are further away from the individual and therefore have less immediate impact on a persons’ development. Using Bronfenbrenner’s (1979) ecological lens, as shown by the concentric circles in the mentoring relationship box on the left-hand side of Figure 1, high-ability students’ academic, and social and emotional skills are affected by the surrounding environment with the academic mentor being the most influential within the social ecosystem as shown by the concentric circles. The academic mentor is a professorial scientist who works at a major metropolitan university but lives rurally (see roles in the appendix). The classroom teacher serves in an interconnecting role between the high-ability students, the mentor, the academic team, and the classroom learning environment. The academic team, composed of three post-doctoral researchers from the same university, supports the students in their learning experiences outside of the school on their visits to the major metropolitan university and during the international experiment (see roles in the appendix). The university and school community are the exosystem (Bronfenbrenner, 1979), whereby the research being undertaken by this institution affects the subject matter in which the high-ability students engage. The school community and associated leaders support the culture of high expectations, and this builds relationships between the educational institutions (Figure 1).
Authentic Research Mentor Program Model Based on the Theoretical Model of the Mentoring Process (Rhodes, 2005), the Mentoring Socio-Motivational Model (Larose & Tarabulsy, 2014) and Grounded in Bronfenbrenner’s (1979) Ecological Systems Theory.
The socio-motivational mentoring model (Larose & Tarabulsy, 2014), listed as bullet points in the mentoring relationship box on the left-hand side of Figure 1, draws on self-determination theory (SDT). This theory focuses on how cultural and social factors can facilitate people’s motivation, well-being, and performance quality. In SDT, the conditions of autonomy, competence, and relatedness are thought to foster high-quality motivation and engagement, which results in enhanced performance, persistence, and creativity (Ryan & Deci, 2000). Therefore, the socio-motivational mentoring model suggests that specific mentor behaviors can foster high-ability students’ motivation, and the model discusses four sets of mentor behaviors that foster a productive relationship. These behaviors include structure and clear objectives, open and respectful engagement, autonomy support to validate high-ability student choices, and competence support to increase high-ability students’ feelings of competence (Larose & Tarabulsy, 2014; Figure 1).
The theoretical model of the mentoring process (Rhodes, 2005) posits that a mentoring relationship is characterized by empathy, mutuality, and trust, and that the mentoring affects high-ability students via the processes of the development of social and emotional skills, and improving academic development. These interrelated processes act in concert with each other over the time of the mentoring relationship. The effectiveness of these processes is influenced by the quality and longevity of the relationship between the mentor and high-ability students (Rhodes, 2005). In accordance with this model, we designed the immersive learning activities of the program with the aim to improve high-ability students’ STEM competencies, knowledge, and skills, and hence their academic development, and social and emotional skills. Jang’s (2016) STEM competencies, knowledges, and skills, as shown under the social and emotional development box on the right-hand side of Figure 1, have four categories: science knowledge and skills, problem-solving skills, social communication skills, and time, resource, and knowledge management skills. Within each of these STEM competencies, knowledges, and skills, a series of critical STEM skills have been identified by Rios et al. (2020). A mixed-methods convergent parallel design was implemented. We used quantitative and qualitative data to measure impact of the ARMP on high-ability students’ social and emotional development and their academic growth in science knowledge and skills (Figure 1).
Method
A mixed-method convergent parallel design was employed in this study. Quantitative and qualitative data were collected at the same stage of the research process, analyzed separately, and synthesized, compared and interpreted during the last phase of the research (Creswell & Clark, 2017). Thus, the quantitative and qualitative data had an equal interpretation weight. This triangulation of using multiple data sources focused on the same phenomenon allows for a comprehensive and deeper understanding of the impact of the ARMP on the development of high-ability students as well as providing the study with a greater level of trustworthiness, validity, and integrity (Stahl & King, 2020).
Context
This mixed-method convergent parallel study was undertaken in a rural New South Wales (NSW) Australian school in a town with a population of less than 15,000 people (Australian Bureau of Statistics, 2023). The school had an Index of Community Socio-educational Advantage (ICSEA) of 1,036 and approximately 800 students at the time of the study. The ICSEA is a measure of socio-educational advantage that is calculated for each Australian school. The median value is 1,000 with a standard deviation of 100. ICSEA values range from 500 to 1,300 with lower scores representing schools with more socio-educational disadvantage. Students from lower socio-economic communities and families on average report lower levels of school engagement and academic performance, and socio-economic disadvantage increases the further the geographical distance from cities and metropolitan areas (Council of Australian Governments, 2008). More than 90% of students at the school speak only English and the dominant student cultural background is Caucasian Australian (Australian Curriculum, Assessment, and Reporting Authority, 2024). We refer to the participants in this study as high-ability students from here on in. The small-scale study was conducted in one school, over 3 years, with three separate cohorts of high-ability students.
Participants
Nineteen female and 13 male Year 10 secondary students who were identified as high-ability students within the academic domain of science participated in the ARMP. We adhere to Taber and Akpan’s (2017) definition of high-ability students as those students who, with appropriate support, can achieve exceptionally high levels of achievement in some or all components of the science curriculum, or are able to engage in science tasks at a level well above Year 10.
High-ability students were identified using multiple measures as outlined in NSW (2024a) education policy and as is common in academic extension programs in Australia (Fitzgerald et al., 2019). Identification took place after the Year 9 semester 1 reporting period and the identification tools included achieving a Grade A performance in Year 9 science and a Grade A or B in Year 9 mathematics, and Year 9 English (NSW Education Standards Authority [NESA], 2018). According to NSW curriculum policy, Grade A students demonstrate extensive content knowledge and a very high level of skill competence, including applying skills to new situations. Grade B students have a thorough understanding of content knowledge and a high level of skill competence and can apply these skills to most situations (NESA, 2018). Teacher nomination was also used for the identification of students who did not meet the above criteria due to high-ability students not always being high performers (NSW, 2024a). We acknowledge the literature that highlights the subjectivity and potential bias associated with teacher nomination of high-ability students (Golle et al., 2023; Hodges et al., 2018). However, teacher nomination in our study was not used to create a screening pool of students to be further evaluated via additional performance-based tests. Instead, teacher nomination was used to identify students who did not meet the performance-based criteria but may demonstrate potential in science inquiry behavioral characteristics, such as curiosity, enthusiasm, and interest, as well as an ability to clearly interpret data (Renzulli et al., 2009). Parental consent, as per NSW (2024a) policy, was sought prior to students enrolling in the program.
Over 3 years, 32 students participated in the ARMP. The first cohort was composed of nine students (five female and four male), the second cohort 11 students (eight female and three male), and the third cohort 12 students (six female and six male). Two high-ability students identified as Wiradjuri First Nation Australians. In addition, one high-ability student identified as Filipino, one as Lebanese, and one as Venezuelan. This percentage of student diversity is representative of the school’s overall population. No other demographic data of the high-ability students were collected.
A control group for each high-ability cohort was developed from within the same academic cohort to examine the impact of the ARMP. Using a quasi-experimental approach with a between-subjects design, we compared a high-ability student group and a control group for each cohort. Students were not randomly assigned to the groups, although a pre- and post-test design was employed to help control for confounding variables between the groups (Denny et al., 2023). The control group students were retrospectively assigned to the control group using the independent state-wide Year 8 NSW (2024b) Department of Education Validation of Assessment for Learning and Individual Development (VALID) science assessment. This independent assessment examines students’ science skills and knowledge at the end of Years 8 and 10 in NSW allowing student growth to be tracked. A control group for each cohort was determined via a retrospective comparative analysis of the entire cohort’s previous Year 8 scores in the VALID assessment. Initially, the high-ability students’ overall scores for the Year 8 VALID were examined; their scores ranged from levels 4 to 6 for each cohort (Table 1). Other students in the same cohort, who recorded an overall Year 8 score in levels 4 to 6 were identified for use as a control group when analyzing the Year 10 high-ability students’ scores post-intervention (Table 1). This control group was chosen to be as academically similar as possible to the intervention group, with both groups experiencing similar science teaching conditions throughout Year 9. Results of a one-way analysis of variance (ANOVA) pre-test showed that there was no significant difference between the high-ability and control groups’ Year 8 VALID overall science scores (Table 1), suggesting that they are academically similar groups.
Year 8 Overall Science Assessment Scores of High-Ability Student Group and Control Group for Three Consecutive Cohorts.
Note. M = mean; N = number of students; SD = standard deviation.
The ANOVA test compares the means of multiple groups. In this study, the means of the VALID overall science for the three intervention and control cohorts were compared (Zar, 2009). The primary assumptions of an ANOVA were tested using a Shapiro–Wilk test to examine the normality of the data and the normality of the residuals, and a Bartlett’s test was used to examine the homogeneity of variance (Zar, 2009). The data did not violate any of these assumptions. A Tukey’s multiple comparison post hoc test examined the significance between high-ability and control groups for each cohort (Zar, 2009). A p-value of less than or equal to .05 was considered significant. GraphPad PRISM software was used for all statistical comparisons. Comparisons between the high-ability group and control group across the three cohorts showed no significant difference for their Year 8 VALID overall science scores (Table 1).
Fifty-one students were allocated to cohort control groups, the first control cohort with 16 students (eight female and eight male), the second control cohort with 22 students (seven female and 15 male), and the third control cohort with 12 students (seven female and five male). Two students identified as Wiradjuri First Nation Australians and one student identified as from Venezuelan heritage. This percentage of diversity is representative of the school’s overall population. No other demographic data of the control group were collected.
ARMP Design
The ARMP was designed to improve and extend the quality of rural high-ability students’ science knowledge and skills, and to develop students’ social and emotional skills within a collaborative scientific research context. Five different groups of people were involved in the mentor program, each with a different function (see roles in the appendix). The mentor for the program and the academic team were selected based on their experience with rural education, their excellence in teaching, and the accessibility of the research topic to high school students. Selection also allowed for a diversity of genders and cultural backgrounds to be represented (see selection in the appendix). The mentor and academic team worked at a major metropolitan university located approximately 300 km from the school, although the mentor lived rurally. The same mentor and academic team worked with all three cohorts of high-ability students. The program was not designed to be didactic, but the small number of students per cohort ensured a close relationship between the mentor and high-ability students. The inclusion of an academic team also lowered the ratio of students to scientists. The length of the program, one whole school year, ensured a greater tie strength between the mentor and high-ability students. The ARMP was implemented as an elective subject and was scheduled 100 min per week within the school timetable; approximately 1 hr per week was face-to-face contact time with the mentor and/or academic team (see learning activities in the appendix).
Training was conducted for the mentors, the academic team, and teachers prior to the beginning of the ARMP. This 5-hr induction included orientation to the program objectives, learning activities, contact expectations, child safe policies, and familiarization with the school facilities. Given the regular monitoring throughout the ARMP additional training was not provided (see training in the appendix). The ARMP for each cohort and the mentor relationship concluded at the end of the school year after 11 months of mentoring. High-ability students were able to leave the ARMP at any point in time, although no students left prior to the completion of the ARMP. At the end of the year, high-ability students completed an evaluation of the program and a self-assessment of the growth of their scientific skills. The mentor, teacher, and academic team also evaluated the program with parents also being invited to provide feedback to the supervisor (see monitoring in the appendix).
ARMP Implementation
To achieve the objectives of the program, high-ability students engaged in authentic scientific research within the context of breast cancer detection, specifically examining the efficacy of international radiologists in detecting breast cancers on x-ray mammograms (Trieu et al., 2021). The identified scientific outputs of the program each year were first, a manuscript to be submitted for publication in an international scientific journal, and second, the design, implementation, and facilitation of an international scientific experiment to collect data firsthand (see Figure 1 and scientific research outputs in the appendix). This international experiment examined the efficacy of Vietnamese radiologists in detecting breast cancers on mammograms (University and Government IRB approvals: 2019/HE000013, ETH2023-0380; 503/2019/YTC-HD3). The learning activities of the ARMP were designed around these two scientific outputs, with the science skills and knowledge high-ability students needed to undertake this research being backwards mapped from these outputs.
Learning activities were typically delivered as 1-hr tutorials or masterclasses on the school grounds by the mentor, except when students visited the university and conducted the international experiment (Table 2). At the beginning of the school year, high-ability students were introduced to the mentor who then delivered tutorials and masterclasses. The masterclasses initially developed students’ scientific knowledge of medical imaging and breast mammography, followed by more intensive and hands-on tutorials addressing statistical data analysis. The students learnt how to apply appropriate statistical tests of confidence and use statistical programs (Table 2).
Summary of the Learning Activities of Mentee Students in the Authentic Research Mentor Program.
Note. Cohort 3 did not travel internationally to collect firsthand data due to COVID-19 pandemic travel restrictions.
Facilitated by mentor. b Facilitated by academic team. c Facilitated by classroom teacher.
Following this, the students visited the university for 2 days to meet the academic team, attend science academic seminars conducted by the academic team and become familiar with the BREAST software (P. C. Brennan et al., 2014) to be used in the experiment (Table 2). Upon return to school, the students were familiarized with the scientific data previously collected by the academic team or the previous cohort of the ARMP students. From these data, the students, in small groups, developed independent areas of inquiry to examine. After developing their hypotheses, the students applied their statistical knowledge and began analyzing the data, with the guidance of the mentor and academic team (Table 2). In classes without the mentor and/or academic team, teachers provided guidance and support associated with the learning activities (Table 2). Once the data had been analyzed, students began to construct the scientific manuscript in sections, while receiving tutorials on scientific writing from the mentor. Thus, the students were provided with continued feedback on their writing to ensure continual learning and motivation.
Toward the end of the academic year, high-ability students prepared to undertake an international experiment. They became familiar with the software that would collect the data and developed protocols for the collection of valid and reliable data. Students traveled internationally to set up and undertake the experiment (see Table 2 and scientific research outputs in the appendix). At the end of the academic year, students finalized the scientific manuscript for submission to an international scientific journal. These manuscripts were subsequently published (A. Brennan et al., 2020; Caspar et al., 2021; Jackson et al., 2019).
Instruments and Data Analysis
Quantitative Data
To determine the impact of the ARMP on high-ability students’ academic performance and growth, the NSW Department of Education VALID science assessment (NSW, 2024b) was compared with a control group of students. This independently designed and scored assessment examines students’ science skills and knowledge at the end of Years 8 and 10 allowing student growth to be tracked. The assessment was composed of extended response tasks, short responses, and multiple-choice items, thus allowing assessment of students’ higher-order thinking and deeper understanding of scientific concepts. English (2020) describes how the NSW Department of Education establishes the validity and reliability of the VALID test items. Students’ scientific achievements are described against syllabus standards (NESA, 2018) and mapped to five performance areas: (1) overall science, (2) problem-solving and communication, (3) planning, designing, and conducting experiments, (4) knowledge and understanding of science, and (5) extended response (NSW, 2024b). These measures were mapped back to Jang’s (2016) STEM competencies, knowledges, and skills as identified in the theoretical framework (Figure 1). Overall science, planning, designing, and conducting experiments, and knowledge and understanding of science were grouped under Jang’s (2016) science knowledges and skills. Problem-solving and communication were attributed to Jang’s (2016) problem-solving skills and extended response was grouped under Jang’s (2016) social communication skills.
All students participated in this assessment during the third quarter of the school year with results released at the end of the academic year. We used the VALID assessment scores for Year 8 and 10 students at the school from 2016 and 2020. The rationale to utilize this assessment to evaluate students’ academic skills and growth was due to its independence, as well as the ability to be able to determine growth in the science learning domain (NSW, 2024b) as compared with a control group of students. Individual scores were recorded into the five performance areas as described above.
For each performance measure, a one-way ANOVA was performed. This statistical test compares the means of multiple groups, and in this study, the means of three cohorts were compared (Zar, 2009). The primary assumptions of an ANOVA were tested using a Shapiro–Wilk test to examine the normality of the data and the normality of the residuals, and a Bartlett’s test was used to examine the homogeneity of variance (Zar, 2009). No performance measure violated the assumptions of the ANOVA. A Tukey’s multiple comparison post hoc test examined the significance between high-ability and control groups for each cohort, as well as between high-ability groups across cohorts and between control groups across cohorts (Zar, 2009). A p-value of less than or equal to .05 was considered significant. GraphPad PRISM software was used for all statistical comparisons. Comparisons between the high-ability groups across the three cohorts showed no significant difference for each of the five measures. The same pattern was observed for the control groups across the cohorts, justifying the pooling of the five performance scores into two groups for analysis of academic growth and effect size of each measure (i.e., high-ability students and control students). The effect size was calculated for each of the five measures of performance according to Lakens (2013). Hedge’s g was used as a measure of effect size due to the small sample size.
To test whether participation in the ARMP had a significant impact on students’ academic growth, two separate two-tailed paired t-tests were performed. This statistical test compares the means of two groups, and the two-tailed option examines both whether there is a significant increase or significant decrease in growth (Zar, 2009). The paired t-test assumes a normal distribution in the growth scores between Years 8 and 10. We tested this assumption using a Shapiro–Wilk test (Zar, 2009). The growth data for both the high-ability students and the control group did not violate the assumptions of the paired t-test.
The first t-test was composed of only high-ability students, and the second t-test was composed of only control students. For each paired t-test, students’ academic growth was measured using their Years 8 and 10 VALID overall science scores. For each student, their Year 8 score was paired with their Year 10 score. A paired t-test was then used to determine whether there was significant academic growth first in the high-ability group and second in the control group. To further test whether there was a significant difference between the two groups’ level of academic growth, we compared the confidence intervals from the t-tests for high-ability group and control group. If the 95% confidence intervals from the two separate t-tests overlap, the level of academic growth between the high-ability students and the control group is not significantly different. However, if the 95% confidence intervals from the two separate t-tests do not overlap, the level of academic growth between the high-ability students and the control group is considered significant.
Qualitative Data
To identify if any of the students experienced growth in their academic and/or social and emotional skills, the students completed an anonymous online survey at the end of the program that asked the open-ended question of “what did you value the most from being part of the program?” These qualitative data were collected in the first 2 weeks of December for the years 2018, 2019, and 2020. Of the 32 students who participated in the ARMP, 30 completed the survey. A deductive content analysis of each students’ response was performed using 11 critical STEM skills as identified in our theoretical framework. Deductive content analyses are used to test existing ideas in new contexts (Krippendorff, 2019). In this research, we used a priori coded STEM skills to validate or refute that the ARMP supported the development of these skills in rural high-ability students. In addition, a number of these critical STEM skills aligned with the skills examined by the quantitative data, thus allowing for data triangulation (Creswell & Clark, 2017).
To implicitly code the students’ open-ended responses, units of analysis were identified in the text. These units were phrases rather than singular words or whole sentences. The units were then condensed to represent a shortened version of the same text while still conveying the essential message. These condensations were then deductively coded using first-order codes based on 11 critical STEM skills (Rios et al., 2020). Definitions of these STEM skills and examples of high-ability students’ statements are shown in Table 3. To increase the validity of the qualitative data additional student quotes are included in the “Results” section using pseudonyms. The first-order codes were then grouped into STEM knowledges, competencies, and skills following Jang (2016) as shown in Table 3. A total of 342 units of analysis were identified across the three cohorts, with 306 units being coded into Rios et al.’s (2020) 11 critical STEM skills. The average number of codes per high-ability student was 10, although the range in codes per student was 2 to 22. Cohort 1 had the smaller range of 2 to 10 codes per student and Cohort 2 had the largest range of 4 to 22 codes per student. The percentage frequency of each STEM skill was calculated for each cohort and for the high-ability students as a group. Rios et al.’s (2020) skills of written communication, oral communication, and communication skills were collapsed into one category titled communication due to some ambiguity associated with the units of analysis. For example, “the ability to explain a concept in detail” could be coded into either written communication, oral communication, or communication skills. In addition, the skills of critical thinking and problem-solving were also collapsed into the one skill of problem-solving, due to problem-solving skills requiring the application of critical thinking skills as per the definition “demonstrating the ability to apply critical thinking skills to solve problems by generating, evaluating, and implementing solutions” (Rios et al., 2020, p. 82).
Categorization, Definition, and Examples of 11 Critical STEM Skills Valued by Rural High-Ability Students Who Participated in the Authentic Research Mentor Program.
To determine the reliability of the categorization and assess the coding trustworthiness, 15% of the units of analysis were coded again by the first author to measure intra-rater reliability, and by an independent researcher to measure inter-rater reliability. The independent researcher was provided with the units of analysis, the condensations and the 11 critical STEM skills and their definitions and the process of coding was explained to the researcher. Cohens’ kappa (κ) was calculated as a measure of reliability. Intra-rater reliability and inter-rater reliability were calculated at κ = .85 and κ = .83, respectively, which is regarded as “almost perfect” (Landis & Koch, 1977, p. 165).
Positionality Statement and Trustworthiness
The first author of this study, a science education academic, was the supervisor throughout the implementation of the ARMP. The second author is a professorial scientist who co-designed the research outcomes with the first author. The first author led the data collection and analysis processes, and both authors contributed to interpreting findings and the implications of the study. There are numerous ways in which our positions might influence the research process. Both authors hold positions of power relative to the high-ability students who participated in the ARMP and the students’ responses to the open-ended question about the ARMP may have influenced their answers. To ameliorate this issue, the survey was anonymous and administered by the classroom teacher. Both authors have positive professional experiences in conducting scientific research. This perspective, combined with the closeness to the research in terms of the design and administration, shapes not only the design of this research but also potentially the interpretation of the qualitative research results. A deductive content analysis approach, which relies on pre-defined codes and categories, was employed to help reduce researcher bias via limiting the interpretation and hence create a more objective analysis (Krippendorff, 2019).
Trustworthiness, as described by Lincoln and Guba (1985), of our research was strengthened by the implementation of the following measures. Credibility was demonstrated using data triangulation whereby both quantitative and qualitative data were used to examine the impact of the ARMP. Reflexivity, and being aware of one’s own biases, ensured the application of processes to maintain a more objective position throughout the data collection, analysis, and interpretation phases of the research. The articulation of the sampling methods used, and participant selection, allows for the determination of transferability to other RRR schools. Documentation of the research process and analyses allows for transparency. The inclusion of high-ability students’ quotes demonstrates confirmability due to the accuracy between the participants’ written statements and the interpretation of that data.
Results
High-Ability Student Mentor Relationship Quality
Across the three cohorts, 87% (28 individuals) of the high-ability students acknowledged in their open-ended statements that they valued the relationship that developed with the academic mentor. The equality of the student–mentor relationship where students were “treated less like students and more like equals by the mentor” (Blair, Cohort 2) was valued by the high-ability students as was the cooperative nature of the relationship where students “were working with the mentor to achieve a goal” (Ali, Cohort 1). A strong sense of trust and support was established between the students and the mentor as expressed by Casey (Cohort 3) “I felt like we knew the mentor on a personal level. We could ask him anything, no matter what it was, even if it sounded dumb. He would always answer all our questions.” Students also valued the mentors’ on-going support that was provided to produce the scientific research outputs as exemplified by Carter (Cohort 2) who stated, “writing the manuscript with the mentors’ guidance.” In addition, four student responses were recorded that described positive experiences with the academic team and two responses referenced the value of the teachers as part of the ARMP.
Science Knowledge and Skills
Analysis of the mean VALID science scores demonstrates that the high-ability students, who participated in the ARMP had greater science learning gains than the control group for each cohort (Tables 4 and 5). Post-test analysis of the high-ability students shows that across all cohorts the high-ability students had significantly higher overall science VALID scores than the control group (Table 4). The high-ability students also had significantly higher scores than the control group in the areas of knowledge and understanding of science, and in planning, designing, and conducting experiments (Table 4). A large effect size for each of the above performance measures was observed. Hedges’ g score ranged from 0.96 to 1.57 for the three measures (Table 6). Thus, for a randomly selected pair of individuals, the chance that the performance score of the high-ability student is higher than the score of a control student is 81% or above, depending on the performance measure selected (Table 6).
Year 10 Science Assessment Scores of High-Ability Student Group and Control Group for Three Consecutive Cohorts.
Note. Cohort 1 data published in the work of Puslednik and Brennan (2020). M = mean; N = number of students; SD = standard deviation.
Mean Overall Science Performance and Growth for Mentee and Control Groups.
Note. M = mean; N = number of students; CI = confidence interval; SD = standard deviation.
Effect Size for Each Performance Measure for the Authentic Research Mentor Program.
Based on the analysis of Year 8 and Year 10 overall science scores, 89% of the high-ability students had above expected growth, whereas 47% of students in the control group had above expected growth. The average growth of the high-ability group from Year 8 to Year 10 was 9.55 (SD = 8.20), whereas for the control group, the average growth was 2.54 (SD = 6.09). The high-ability and control groups both showed a significant increase in their scores across the 2-year period (p < .001 and p = .006, respectively). However, the high-ability group had a significantly higher level of growth as demonstrated by the non-overlapping 95% confidence intervals of the high-ability group and the control group (Table 5). Only one high-ability student had no growth, whereas 34% of the control group had no growth.
This result of increased growth in high-ability students’ science knowledge and skills shows congruence with the qualitative data. The content analysis of high-ability students’ responses shows the science knowledge and skill of continuous learning was identified as the most valuable skill. Continuous learning accounted for 23% of all coded responses from the high-ability students (Figure 2). The students valued this opportunity to acquire new STEM skills and knowledges as Ashley (Cohort 3) stated they valued that the program allowed them “to experience a greater demand for knowledge.” These results support the consistently significant difference between the control and high-ability group across three cohorts in the performance measure of overall science, scientific knowledge and understanding and planning, designing, and conducting experiments (Table 4). Each cohort of high-ability students identified continuous learning within the top 3 most valued STEM skills, although only Cohort 2 identified it is as the most valued skill (Figure 3).
Content Analysis of Student Responses Coded Into Critical STEM skills and Grouped Into STEM Competencies, Knowledges, and Skills.
Variation Between Cohorts in the Content Analysis of Student Responses Coded Into Critical STEM Skills.
Two areas were consistently identified by the students as important new areas of learning: statistical data analysis and writing scientific manuscripts. Students valued the opportunity to develop new knowledge and skills in the areas of inferential statistical data analysis as they identified that the program allowed them to “learn lots of new concepts and information” and that the “program helped with my statistical thinking” (Amos, Cohort 1). The skills of learning how to write scientific manuscripts was also an area that the students appreciated as they “learnt valuable lessons on how to write scientific papers” (Cameron, Cohort 2). The statement by Charlie (Cohort 1), “I developed new skills in areas where I previously had very little experience. These valuable lessons have benefited other subjects I do,” exemplifies that the rural high-ability students were able to reflect on their growth in learning and identify that they could apply these new skills and knowledges to other areas of learning at school.
Problem-Solving Skills
The quantitative measure of problem-solving from the VALID assessment shows the high-ability students across all three cohorts had significantly higher scores in problem-solving and communication (Table 4). Further support for this result is provided by the large effect size of 1.43 for this measure (Table 6) as well as by the qualitative data of the students’ responses. The high-ability students identified problem-solving skills as the second most valuable STEM skill, with 16% of responses being coded as problem-solving (Figure 2). The students enjoyed the challenge and extended learning associated with the ARMP. They identified that the program environment helped them to develop their problem-solving skills as Bailey (Cohort 2) stated, “I found it also challenged me and encouraged me to problem solve and really gain an understanding of the topic.” Importantly, this challenge was perceived as a positive aspect of the program as Harper (Cohort 3) describes “It was a good challenge because it is extending your abilities and problem-solving skills.” Students also felt supported in this challenge by the mentor and the academic, as they were able to “do the experiments alongside scientists” (Frankie, Cohort 3).
Each cohort of high-ability students identified problem-solving within the top 3 most valued STEM skills, although only Cohort 1 identified it is as the most valued skill (Figure 3). The additional problem-solving skills of adaptability and creativity also accounted for 4% and 2% of the high-ability students’ responses, respectively (Figure 2). Interestingly, only students from the third cohort identified creative skills as applied to writing a scientific manuscript while only Cohorts 2 and 3 valued adaptability (Figure 3).
Social Communication Skills
Social communication skills, as recorded via the qualitative data, represented the most valued of all four STEM competencies, knowledges, and skills by the high-ability students. Combined social communication skills represented a total of 43% of students’ responses (Figure 2). Collaboration was the most highly valued social communication skill identified by the high-ability students, representing 16% of all responses (Figure 2). While all three cohorts identified collaboration within the top three STEM skills, only Cohort 3 identified it as the most valuable skill (Figure 3). Even though collaboration was highly valued by the students, it was a difficult skill to develop as Jordan (Cohort 3) stated, “working collaboratively as a group was important, it was challenging, but it was an important skill to develop.” Certain aspects of collaboration were identified in the students’ statements. Many students highlighted the valuable experience of being engaged in collaborative scientific research. This is exemplified by the statement from Blake (Cohort 1), “getting to work as part of a team collaboratively conducting real-world scientific research with an impact was great.” Importantly, the high-ability students were able to distinguish and point out how this scientific collaboration differed from group work within the regular classroom. Alex (Cohort 3) provides insight into how this collaboration felt compared with group work at school, “in group projects at school sometimes people aren’t interested in the topic and workloads become heavier on one person. The workload was shared a lot more evenly as compared to working in groups at school.” Collaboration was not only evident between the students and the mentor but was also present between the high-ability students themselves. Rowan stated that, “it has also been valuable for me to work consistently in a collaborative group, this has helped develop my teamwork skills and I have learnt from each of my peers,” demonstrating the importance of peer-to-peer collaboration in the ARMP.
Communication skill was the fourth most valued STEM skill by the high-ability students, accounting for 12% of all coded responses (Figure 2). Across the cohorts’ communication was consistently valued, being in the top 4 STEM skills for all three cohorts. For Cohort 2, it was identified as the third most valued skill. Cohorts 1 and 3 identified it as the fourth most valuable skill (Figure 3). This is consistent with the quantitative results where high-ability students scored significantly higher than control students in the VALID assessment extended response measure for two of the three cohorts (Table 4). In the VALID assessment, students are required to write explanations of scientific concepts and so that the extended response score represents a measure of their written communication skills. The large effect size of 0.81 for the extended response measure also supports the difference between high-ability students and the control group for this measure (Table 6). Students’ comments that unambiguously referred to valuing the development in writing scientifically accounted for 74% of the communication coded responses. Writing a scientific manuscript was a challenging task for the high-ability students which required students to adapt their writing style. This is exemplified by Dakota (Cohort 2) who stated, “I had to condense my writing, it wasn’t easy, but I got help from the academic and from the rest of the group.” This statement also highlights the important role of the mentor in helping the students to develop these written communication skills. The statement from Quinn (Cohort 1), “the ability to work with a mentor to learn how to write a scientific paper was also greatly valued and a skill I will carry,” demonstrates that despite the challenge of writing scientifically, the high-ability students could see the value in developing this skill. These statements also show how the mentor helped students to develop this skill.
The other social communication skills that students valued included cultural sensitivity, social intelligence, and service orientation. These STEM skills represented 6%, 5%, and 4% of students’ coded responses, respectively (Figure 2). Cultural sensitivity was more highly valued in Cohorts 1 and 2 than in Cohort 3 (Figure 3). Although Cohort 3 did not travel internationally due to COVID-19 travel restrictions. Luca’s (Cohort 1) statement supports this finding, “I really valued meeting lots of different professional academics, the opportunity to travel, and expand my knowledge of other people and their cultures.” The sense of service orientation was only identified as a valuable skill by Cohort 2 (Figure 3). Remi’s statement from Cohort 2, “I loved that this group made a small difference and gave Vietnamese radiologists new opportunities to learn,” demonstrates that the students understood the wider social implications of the scientific research.
Time, Resource, and Knowledge Management Skills
Only qualitative data were collected to examine students’ development of time, resource, and knowledge management skills. This group of skills accounted for 11% of the high-ability students’ coded responses (Figure 2). High-ability students identified professionalism and self-direction as the STEM skills they valued as part of the ARMP. Professionalism accounted for 6% of the coded responses and self-direction accounting for 5% (Figure 2). For each cohort, the students initially felt challenged by program, but as they continued to work together throughout the year, they began to feel more comfortable with the new knowledge and skills they were learning. This is exemplified by the statement from Miles (Cohort 3), “at the start it was all a bit daunting, there was a lot to take in, but as you keep working it gets easier.” Only Cohorts 2 and 3 identified professionalism as a valuable STEM skill, while all three cohorts identified self-direction as an important STEM skill (Figure 3).
STEM Career Knowledge
Five students identified that the program provided them with insights into the career of scientists. Students in the program were exposed to a range of career options, as Lowen (Cohort 1) stated, “it allowed me the opportunity to open my mind to career opportunities.” Students were also provided with “a much broader and more significant understanding of science” (Everest, Cohort 3) and the work of a scientist.
Discussion
The performance of high-ability STEM students in Australia has not only declined over the last 15 years, but the proportion of students achieving higher outcomes in science has also declined (De Bortoli et al., 2023). Within 1 year, our ARMP has impacted the learning of high-ability students who have participated in this program. Thus, this approach has the potential to address the challenges faced by high-ability students within rural Australia. Based on both quantitative and qualitative data, three separate cohorts of rural high-ability students have consistently demonstrated significantly greater growth in their academic science knowledge and skills than the control group, in addition to the development in their social and emotional skills that are critical to STEM success.
While the data show growth in rural high-ability students’ science knowledge and skills and their problem-solving skills, social communication skills were the most valued STEM skill identified by the high-ability rural students. Within the social communication skills, collaboration was coded as the most valuable STEM skill and across all cohorts was within the top 3 most valuable STEM skills. The students’ statements acknowledged the challenges of true collaboration and also demonstrated the students’ realization of the importance of developing strong collaboration skills. Similar studies with high school students highlight that engagement in authentic scientific inquiry can foster the development of collaboration skills (Flowers et al., 2016; Kim, 2021; Leuenberger et al., 2019; Otterstetter et al., 2011; Sadler et al., 2010; Shoemaker et al., 2016; Watters, 2021). More specifically, students developed a strong sense of the importance of collaboration in science (Charney et al., 2007; Hay & Barab, 2001; Richmond & Kurth, 1999). The high-ability students’ statements also identified how this collaboration differed from group work in the regular classroom. These statements are supported by previous research which highlights the problems of group work for high-ability students (Watters, 2021) and provides further evidence for ensuring high-ability students work with like-minded peers when engaging in an ARMP. The length of the ARMP could have supported students’ development of their collaboration skills. Challenges faced by true collaboration can take time to be resolved, and the length of the ARMP could potentially allow these challenges to be worked through and solutions reached. Therefore, providing the high-ability students with a positive collaborative experience from their participation in the ARMP.
Within the academic domain, rural high-ability students have shown significantly greater growth in their science knowledge and skills as exemplified by their significantly higher VALID assessment scores in the measure of overall science, knowledge and understanding, and planning, designing, and conducting experiments as compared with a control group. This trend is further supported by the high-ability students identifying continuous learning as one of the most valued critical STEM skills. Their comments highlighted the importance of the learning activities that addressed data analysis and interpretation and scientific writing. There is congruence in the students’ comments coded as continuous learning and communication being identified as the second most valuable social communication skill. There was some variation here between the cohorts, with Cohort 1 recording half as many written communication responses as Cohorts 2 and 3. This result is also reflected in the VALID assessment extended response measure whereby only Cohort 1 recorded a non-significant difference from the control group. Even though there was a large effect size for extended response, these findings suggest that there was potentially some growth in the mentor and their mentoring ability associated with how to teach scientific writing over the 3 years of the ARMP. Interestingly, mentors have identified that one of the major advantages of mentoring is their own professional development, thus acknowledging their own learning in such programs (Shoemaker et al., 2016; Tan et al., 2019). Research in this area of mentor growth within mentor programs is potentially a rich area of research that will allow for a deeper understanding of ARMPs and the impact of the program beyond that of the mentees.
The benefits of high-ability students engaged in an ARMP goes beyond achievement and growth in science assessments; students improved in problem-solving, creativity, and adaptability as evidenced by students’ coded responses and the high proportion of total responses represented as problem-solving skills in our study. These findings are further supported by the VALID assessment scores for which problem-solving was significantly greater than the control group across all cohorts with a large effect size. Cultivating high-ability students in STEM requires not only the development of domain specific knowledge but also the development of problem-solving skills and the ability to apply these domains to novel situations (Ozkan & Kettler, 2022). Programs, such as our ARMP, that are designed to address real-world issues facilitate improvement in creative problem-solving skills and increase high-ability students’ adaptability and motivation toward challenging tasks with the added benefit of students devising authentic and innovative solutions (Maker & Wearne, 2020; Morris et al., 2021; Wu et al., 2019). Using real-world issues as the focus of an ARMP also addresses high-ability students’ perfectionist tendencies, as they enjoy the challenge of complex tasks that go beyond the curriculum and develop skills that can be applied in other areas (Watters, 2021). In addition, engaging high-ability students in real-world problems with a social justice aspect can also appeal to the students’ strong sense of equity and develop their sense of service orientation (Morris et al., 2021) as was observed with the second cohort in this ARMP. Therefore, programs, such as our ARMP, should be viewed as a multifactor model that addresses a wide variety of skills simultaneously across a range of academic and social–emotional domains.
Our research provides a deeper understanding of the social and emotional experiences of rural high-ability students who participate in an ARMP. These findings are important for teachers and researchers who are designing programs to address the needs of this group of students. All three cohorts identified continuous learning, problem-solving, collaboration, and communication as their top 4 STEM skills that were developed throughout the program. These findings suggest that these skills are not routinely addressed, or are not challenging enough, for these rural high-ability students in the regular classroom. The inclusion of learning activities in the classroom that address these skills could help support rural high-ability students’ growth in the regular classroom. Interestingly, these skills are congruent with the most important STEM career skills advertised for and needed for the STEM workplace success (Jang, 2016; Rios et al., 2020). Thus, our research highlights the importance of ARMPs not only in exposing rural high-ability students to STEM careers, but also in preparing high-ability students for STEM careers.
These results support the idea that talent development programs need to provide high-ability students with access to insider knowledge via professional mentors. These mentors can provide students with knowledge about career activities and provide a deeper understanding of their day-to-day work activities. Such knowledge is particularly pertinent to disadvantaged groups, such as RRR high-ability students (Subotnik et al., 2023). A more fine-grained analysis of high-ability students’ responses shows there is variation observed between what STEM skills each cohort valued and at the individual level based on the range of codes per student. This finding challenges the assumption that all high-ability students have the same experience, growth, and development when participating in an ARMP and adds further evidence to the heterogeneous social and emotional skills of high-ability students (Watters, 2021). Future research should allow for a more nuanced examination of which skills matter for which groups and/or individuals. For example, examining the experiences of marginalized groups, such as rural high-ability young women, could provide information to inform the future design of ARMPs. Two students did not complete the anonymous survey, and it would be interesting to explore whether their experience would report disconfirming evidence.
The role of the mentor has been crucial in supporting the academic and social and emotional growth of the high-ability students in this ARMP. Over 80% of the students mentioned this relationship in their open-ended responses, all of which were positive. These responses reflect the importance of the relationship to the students and the quality of support the mentor provided. The students’ comments also highlight the high level of trust they felt working with the mentor. The high-ability students in our research identified the sense of vulnerability associated with asking questions. Indeed, high-ability students can experience barriers to asking questions or seeking help from teachers due to feelings of shyness or inadequateness (Horsley & Moeed, 2021). But the learning environment fostered by the mentor in the ARMP allowed students to take risks, and the high-ability students in our research felt comfortable asking the mentor questions, knowing that their questions would be answered.
Throughout the program, the mentor was able to recognize the social, emotional, and academic needs of the high-ability students and provide answers to all the students’ questions as well as provide constructive feedback throughout the authentic inquiry process (Tan et al., 2019). The design aspects of this ARMP, including the high frequency of mentor-high-ability student contact and the program being embedded into the school context, ensured the mentor was aware of students needs and was able to provide feedback when necessary. These findings suggest that the conditions for the socio-motivational mentoring model (Larose & Tarabulsy, 2005) and SDT (Ryan & Deci, 2000) were addressed. The mentor was able to foster high-quality motivation via the facilitation of autonomy and competence support in a challenging yet rewarding environment, which resulted in enhanced academic and social and emotional development for the high-ability students. Two students identified the value of the academic team to their learning, and only one mentioned the role of the teacher. This result supports the grounding of the ARMP in Bronfenbrenner’s (1979) ecological theory, whereby the academic mentor has the most influence on the rural high-ability students within the social ecosystem.
High-ability students within rural contexts face a range of barriers in accessing high-quality STEM education. Our research demonstrates that ARMPs can address some of these challenges and support both high-ability students and teachers within rural schools. Results of this research demonstrate that an ARMP, which is embedded into the school can deliver high-quality STEM education that addresses the academic and social and emotional needs of rural high-ability learners, as well as the issue of traveling long distances to access STEM opportunities. Rural schools typically are characterized by a wealth of social capital (Saw & Agger, 2021) and strong relationships within rural settings can afford the opportunity for local experts to engage in an ARMP as a mentor and address local issues. The implementation of an ARMP within RRR schools can also potentially ameliorate some of the issues associated with STEM teachers’ access to quality professional learning. Throughout the ARMP, academic mentors role model how science knowledge and skills are developed, thereby providing teachers with multiple professional learning opportunities from subject-matters experts. These professional learning opportunities could have important implications for supporting current STEM teachers as well as training out-of-field STEM teachers. It is estimated that 20% to 30% of Australia’s STEM teachers are teaching out of field, with this percentage being higher in RRR schools (AITSL, 2023). Therefore, via the ARMP, academic mentors can help support “out-of-field” teachers’ transition into the subject area. Future research should examine the professional growth of teacher involved in ARMPs.
Limitations
The aim of this work was to assess the impact of an ARMP on rural high-ability students’ science knowledge and skills, and their social and emotional skills. However, we acknowledge this research focuses on the short-term outcomes of the program. Further analysis of the long-term impact of this ARMP should include longitudinal research of students’ education pathways and choices. This research would be a powerful approach to assessing the durability and transferability of the knowledges and skills learnt throughout the ARMP. Expansion of this program into other schools would also address the limitation of the relatively small number of students participating and the single location of this program. In addition, the expansion of the program should include other academic mentors. The program in its current format has a strong reliance on the skills of one mentor. To fully investigate the generalizability of the results of this program, additional academic mentors need to be included in future research.
Each of the instruments used to measure the impact of the program are also not without their limitations. We acknowledge that the quasi-experimental design is not a true experimental method and cannot unequivocally rule out the influence of confounding variables on the results of our research. However, the pre-test and post-test design helps to control for confounding variables, allows for robust statistical analysis of the data, and is a common design used to examine interventions in educational research (Denny et al., 2023). In addition, the pre-test and post-test VALID assessment is a valid and reliable measure of students’ science understanding and is an independently designed and scored assessment (English, 2020). The authors acknowledge their involvement in the execution of this program and as such the potential for bias in the qualitative data. Numerous measures were employed to address these issues as outlined in the “Methods” section. However, the independence of the VALID science assessment and the high level of intra-rater reliability and inter-rater reliability for the content analysis demonstrate that bias has not impacted the results of this work. Additional measures could have been taken to increase the trustworthiness of the qualitative data, including member checking and taking of field notes (Stahl & King, 2020). The implementation of a mixed-method approach whereby more than one data type is used to establish findings, as well as the congruence of the findings, increases the credibility, reliability, and validity of the instruments and of the study overall (Creswell & Clark, 2017). Indeed, the array of measures used here shows a congruent pattern of increased students’ performance and growth across both the academic and social, and emotional domains.
Conclusion
ARMPs represent a power pedagogical approach to address the needs of rural high-ability STEM students. The combination of having academic mentors implements authentic inquiry-based research results not only in academic gains for rural high-ability students, but critically in social and emotional gains as well. Our research provides deeper insights into the social and emotional experiences of high-ability students and how we can help to develop these broad range of skills to support the development of talent in the STEM domain. Critically, this approach of embedding authentic research into schools enables rural high-ability students to engage in authentic science activities as practiced by scientists, and to directly observe the relevance of academic content while developing critical STEM social and emotional skills. Programs, such as this one, address the inequities faced by rural high-ability students and support their socio-emotional development. They also provide students with exposure to authentic STEM research which has the potential to substantially impact rural high-ability students’ post-school trajectory and participation in STEM careers in the longer term.
Footnotes
Appendix
Design Elements of the Authentic Research Mentor Program (ARMP).
| Design element | Description |
|---|---|
| Objective | The program had two main objectives for high-ability students in a regional Australian school: 1. To extend and improve the quality of a science knowledge and skills 2. To develop social and emotional skills |
| Roles | Mentor (professorial scientist): inducted and supported mentees in the epistemology of scientific research and delivered masterclasses, and tutorials associated with the learning content, as well as provided mentees with feedback throughout the manuscript production process and the design and implementation of the international experiment (Table 2). The mentor also explored with mentees the identity of being a scientist and co-designed the research outcomes and learning activities with the supervisor. The same mentor worked with all three cohorts of high-ability students Academic team (three post-doctoral scientists): provided academic and social support to mentees during excursions to the university and on the international experiment. The academic team shared their experience of what it is like to be a scientist and their journey to becoming a scientist. The same academic team worked with all three cohorts of the high-ability students Supervisor (science coordinator, first author): Designed the program, supervised the implementation of the program, and monitored the program with regular check-in points with the mentors, mentees, and teachers. Contacted parents prior to students’ enrollment and sought parental consent for students to participate in the program as per New South Wales (NSW, 2024a) policy and evaluated the program at the end of each academic year. The supervisor developed criteria for student selection, established the goals and expected outcomes of the ARMP, delineated mentor, academic team member and teachers, and roles, and created a schedule for contact between mentees and mentors. In collaboration with the mentor, the supervisor co-designed the research outcomes and learning activities Teacher (STEM teachers): One science and/or maths-trained teacher was allocated to each cohort, with each cohort having a different teacher. The role of the teacher was to support the mentees in undertaking the associated learning activities outlined in Table 2. The teacher attended all the masterclasses and tutorials that were facilitated by the mentor and academic team and traveled with the students to the university campus and on international experiments Mentees (high-ability students): participated in the ARMP, engaged in learning activities (Table 2) and produced scientific manuscripts and designed, and implemented an international science experiment |
| Selection | Mentor: was selected due to their experience of having grown up outside of an urban center and their understanding of living in the small regional community. Other selection criteria included recognized teaching excellence and a research program that addressed a broad issue that students could relate to Academic team: were also selected based on their experience of regional education as an adolescent. A diversity of genders and cultures was also a consideration in the academic team selection, this ensured that a diversity of experiences and ideas were represented in the academic team. The academic team was also required to hold a PhD in science Mentees: high-ability students were invited to participate in the program based above standard academic performance in independent state-wide Year 8 science assessment VALID (NSW, 2024b), Year 9 science, Year 9 mathematics, and Year 9 English (NESA, 2018). Teacher nomination was also used to identify students who did not meet this performance-based criteria but may demonstrate potential in science inquiry behavioral characteristics, such as curiosity, enthusiasm, and interest as well as an ability to clearly interpret data (Renzulli et al., 2009). Parental consent, as per NSW (2024a) policy was sought prior to students enrolling in the program |
| Tie strength | The mentoring relationship was not designed to be dyadic, however, the low number of students within the cohorts allowed for a closer mentoring relationship than is normally experienced by students in a traditional high school environment. The length of the program and the regularity of contact with the mentor mediated the tie strength of the mentor–mentee relationship |
| Relative seniority | The ARMP utilized mentors within a traditional seniority model, where the mentor, academic team, and teachers had significantly greater experience, influence, and achievement within the learning area of science relative to the mentees. This traditional structure also applied to the relationships between the mentor and the academic team and teachers |
| Time | The ARMP was carried out over the academic secondary school year with the total length being approximately 11 months. The program was implemented as an elective subject and was scheduled 100 min per week within the normal school timetable. Mentees had regular contact with the mentor and the academic team throughout the year as outlined in Table 2 and typically equated to 1 hr per week. Contact time differed during the visit to research laboratory and participation in an international experiment and was approximately 5 hr a day (Table 2) |
| Specialist resources and tools | Mentees were provided with access to specialist statistical software packages for data analysis and access to BREAST, a digital screen reading test-set of breast mammograms software used to assess the performance of radiologists (P. C. Brennan et al., 2014) in the international experiment at the end of the academic year |
| Training | A 5-hr training session was conducted with the mentor, academic team, and teachers prior to the beginning of the program. This orientation to the program included outlining the goals and objectives of the program, a discussion of the learning activities and the delivery of the activities, timetable for learning activities and the monitoring of the program, familiarization of the school and its facilities, policy associated with working with children, the expected outputs, and evaluation of the program and how raise issues if needed. This mentoring model did not incorporate on-going training |
| Policy | All mentors who were part of the program adhered to child-safety policies, including having the required current working with children checks. Mentors had to include the teacher in all email correspondences to mentees and were never to be alone with individual mentees. During excursions and international travel mentors, academic team, teachers, and mentees had to adhere to school-based excursion policies and risk assessments |
| Monitoring | The supervisor checked in with mentees, mentor, teacher, and the academic team twice a term, and on an individual basis, to determine the on-going effectiveness of the mentor program. Any comments or concerns from the mentees and/or teachers were passed onto the mentor and academic team, and any issues collaboratively solved with the supervisor, mentor, teacher, and/or academic team were appropriate. At the end of the year, mentees completed an evaluation of the program and a self-assessment of the growth of their scientific skills. The mentor, teacher, and academic team performed a self-reflection at the end of the academic year to identify areas of strength and areas of need for the program. Parents were also invited to provide feedback to the supervisor |
| Learning activities | Specific activities were designed to develop the high-ability students’ knowledge and understanding of science and scientific research, as well as their social and emotional skills (Rhodes, 2005). To develop students’ academic science knowledge, the scientific content and skills associated with the masterclasses, tutorials, and applied learning of the ARMP were beyond the demands of the curriculum being implemented within students’ regular science lessons, thus representing extension work for the high-ability students. The integration of mathematical statistical analysis aimed to develop students’ critical thinking and develop a sense of continuous learning. Allowing students’ choice in the direction of the data analysis aimed to develop students’ sense of autonomy, self-direction, and time management. The writing of the manuscripts was undertaken in small groups within the regular school setting and aimed to develop students’ collaboration and communication skills. The rationale of having students design and implement an international science experiment was to develop students’ skills of social intelligence, adaptability, cultural sensitivity, service orientation, and professionalism |
| Termination | The relationship concluded at the end of the school year after 11 months of mentoring. Mentees were free to leave the program at any point in time throughout the academic year without academic penalty. Two students over the 3 years, who were offered a place in the program declined to participate, however, no students once enrolled in left the program early |
| Scientific research outputs | To achieve the objectives of the program high-ability students engaged in authentic scientific research within the context of breast cancer detection. Breast cancer is the second most common cancer world-wide and is the most common cancer in women. Patient survival depends heavily on accurate detection of abnormal lesions via X-ray imaging of the breasts. However, radiologists’ diagnostic accuracy can vary anywhere from 50% to 80%, with higher accuracy being recorded in more developed countries (Trieu et al., 2021) The identified authentic scientific outputs of the program each year was a co-authored manuscript to be submitted for publication in an international scientific journal and the design, implementation, and facilitation of an international scientific experiment to collect firsthand data that were passed onto the cohort the next year and formed the basis of the next scientific manuscript The international experiment examined the efficacy of Vietnamese radiologists in detecting breast cancers on mammograms within their home countries. A test-set of breast mammograms was provided to Southeast Asian radiologists, via the BREAST software (P. C. Brennan et al., 2014). Radiologists then decided the absence, presence, and location of lesions, and the radiologist’s mammographic detection efficacy was determination at the end of the test-set. The experiment destination was part of a broader grant commitment from the academic team. The academic team contacted participants and host organizations, as well as organized the delivery of computing equipment and uploading of software to computers. The high-ability students were responsible for being familiar with the software, setting and packing up the computing equipment, establishing data collection protocols to ensure valid and accurate data would be collected, recording radiologist participation, introducing, and describing how to use the software to radiologists, and managing any problems that arose during the experiment. Upon returning to Australia the academic team extracted the data from the software and the high-ability students were responsible for cleaning up the data before passing it onto the next cohort Funding for students’ international travel was supported by the school, the broader school system, and local non-for-profit organizations. Families were asked to pay a nominal amount for the international travel, and it was emphasized that the school would financially support any students who were not in an economic position to pay this nominal amount |
Acknowledgements
The authors would like to acknowledge the anonymous reviewers whose comments and suggestions improved the manuscript, and the independent researcher who helped with the coding. They would also like to thank the school principal for their initial and on-going support of this program.
Ethical Considerations
Ethical approval for this study was obtained from the University of Southern Queensland, Human Research Ethics Committee (approval no. ETH2023-0380) on July 4, 2024, the University of Sydney, Human Research Ethics Committee (approval no. 2019/HE000013) on October 2, 2019, and the Hanoi University of Public Health, Ethics Committee (approval no. 503/2019/YTC-HD3) on December 12, 2019.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The BreastScreen Reader Assessment Strategy (BREAST) software, which was used in this research, was funded by the National Breast Cancer Foundation (IF-12- 02), Australian Department of Health (2018-2021), the Cancer Institute NSW (2018-2021), and the Sydney Southeast Asia Centre (2016-2019).
Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Open Science Disclosure Statement
The data analyzed in this study are available upon request for purposes of reproducing the results. The code or protocol used to generate the findings reported in the article is available upon request for purposes of reproducing the results or replicating the study. There are no other newly created, unique materials used to conduct the research.
Artificial Intelligence Use
The authors confirm that no generative AI tools were used in the development of this article.