Abstract
Authentic student-led inquiry and exposure to scientific research impact students’ science career choices. Given Australian students decline in STEM skills, knowledge of whether such programmes impact student learning is critical. This research examined the short-term impact of an authentic, hands-on research mentor programme on rural student’s science skills. Nine Year 10 students participated in a science academic research programme leading to scientific publications and students collecting of first-hand data from international experiments on a major world-wide health issue. The NSW Department of Education Year 10 VALID assessment scores of this intervention group were compared to a control group. Intervention students had significantly higher overall scores as well as significantly higher scores in 21st century skills. These results were supported by student’s self-assessment of their learning growth. Our study suggests authentic science research mentor programmes are pedagogically advantageous for Year 10 high achieving rural students. Educators’ willingness to embrace these innovative approaches has the potential to produce the next generation of scientists.
Keywords
Introduction
The decline in young people choosing careers in science, engineering and technology is of concern for many industrialised nations. Indeed, international studies, such as the Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study (TIMSS), indicate a lack of motivation to pursue science, technology, engineering and mathematics (STEM) related careers (Thomson et al., 2017), as well as a decline in science and mathematics skills (Organisation for Economic Cooperation and Development (OECD), 2019; Thomson et al., 2016). Due to declining interest in science careers, there has been a long-term decrease in the proportion of students selecting science and engineering-related subjects at the senior secondary school level (Kennedy et al., 2014; Marginson et al., 2013; Palmer et al., 2017; Tytler, 2007; Wagner-Luptacik, 2011). Educators have identified this lack of interest to be a pressing issue given the concern that Australia is continuing to fall behind other countries in science education (OECD, 2019). Recent PISA results highlight that Australian students are demonstrating a steady negative decline in Mathematics skills and an increasingly negative decline in science skills (OECD, 2019). However, exposing students to authentic scientific research with a strong focus on inquiry could help reverse this decline in career choice and skills. Numerous studies have demonstrated that involving secondary school students in authentic research programmes, that address real world issues and foster a deep understanding of science, results in students being more likely to pursue and maintain careers in science (Burgin et al., 2012; Kitchen et al., 2018; Roberts & Wessersug, 2009; Sadler et al., 2010; Sasson, 2019; Tai et al., 2017).
Authentic student research is defined as hands-on research in which students actively engage with original problems and attempt to find unknown answers or solutions to the problem. This research is often conducted with the guidance of a research mentor (Murray et al., 2016), whose role is to respond to students needs throughout the research project, provide technical and intellectual support, build collaborations, develop ownership and encourage effective research communication in a hands-on manner and often in one-to-one interactions with students (Shanahan et al., 2015). Whilst the benefits of authentic student research, such as engaging in open-ended inquiry, building connections across disciplines, and increased skills, have been recognised in research, inquiry-led scientific research typically is more commonly the domain of tertiary education than secondary education (Beier et al., 2019; Bleicher, 1996; Chinn & Malhotra, 2002; Hunter et al., 2007; Molhenrich et al., 2018; Sadler et al., 2010; Seymour et al., 2004). Thus, only students who pursue science majors at university level are exposed to authentic scientific research. However, secondary students enrolled in research programmes show significant growth in research abilities, and their understanding of the nature of science and science skills (Aydeniz et al., 2011; Burgin et al., 2012; Charney et al., 2007; Eales, 2014; Eales & Laksana, 2016; Molhenrich et al., 2018; Sadler et al., 2010).
There is wide agreement that secondary school students’ development of 21st century skills through hands-on STEM programmes is critical (for review see Australian Industry Group, 2015).However, such programmes require a significant level of reform across all education system levels – national, state and regional. The reform required for the successful delivery of these programmes can present a challenge for education systems (Ritz & Fan, 2015). Currently within Australia, there are a limited number of programmes which allow students to experience authentic, real-world scientific research (for exceptions see Davies, 2006; Fitzgerald et al., 2014; Gomez & Fitzgerald, 2017; Howitt et al., 2009; Tytler et al., 2011; Tytler & Nakos, 2003) and whilst there is a range of diverse community STEM partnerships across Australian schools, the impact of these programmes on student learning has not been formally quantified (Cripps Clark et al., 2014; Marginson et al., 2013). Within secondary schools, there is constant competition for time and resources across a range of key learning areas (Danaia et al., 2012). Therefore, having students engaged in a pedagogically-rich, hands-on STEM experience can be beyond the scope of the classroom teacher and the resources of the school. This may be particularly true for rural and remote schools.
Within a rural and regional context, there are a number of challenges schools face in terms of STEM education. Australian students from rural schools demonstrate lower performances in science and mathematics as measured by PISA and TIMSS; rural students show a significant decline in science performance as compared to their metropolitan peers (Thomson et al., 2016, 2017). Furthermore, rural students are also less likely to participate in senior secondary science courses (Murphy, 2020). The same pattern of achievement is observed in senior secondary science courses, with metropolitan students displaying this urban advantage of higher performance in science courses (Murphy, 2018). Interestingly, non-metropolitan students are less likely to have senior science courses provided by their school (Murphy, 2018), which could be attributed to the lack of specialist science teachers in regional Australia. Rural and remote schools consistently struggle to not only fill specialist science and mathematics positions, but also retain high-performing STEM teachers (Halsey, 2017; Handal et al., 2013). Thus, out-of-field teaching increases with distance from metropolitan locations, with early career teachers more likely to be teaching outside of their subject area (Weldon, 2016). Furthermore, access to professional development for rural and remote mathematics and science teachers can be difficult not only due to geographical distance but also due to differing professional development priorities at the personal, school, regional and state level (Tytler et al., 2011). Professional development opportunities are needed for rural teachers to redress gaps in STEM knowledge as well as support early career teachers in teaching new science subjects or curriculum areas (du Plessis et al., 2019; Halsey, 2017; Jenkins et al., 2011; Tytler et al., 2011). These factors combined with the suggested science teacher shortages, raise concerns about the quality of STEM instruction by teachers who are both new to teaching and unprepared in the subject area (Hobbs, 2020; Nixon et al., 2017; Weldon, 2015). Crucially, these staffing challenges in rural schools can impact students’ motivation and hence their achievement in STEM subjects. This has a negative flow on effect to students’ pathways to their STEM learning and careers (Hobbs, 2012; Office of the Chief Scientist, 2014).
The aim of this research was to examine the short-term impact of an authentic, hands-on research programme on student’s skills within a rural school context. During the course of this programme, students were mentored by and collaborated with university academics to conduct authentic, open-inquiry research over one academic school year (January to December) which included students participating in an international experiment and collecting data firsthand.
Methods
Authentic research mentor programme
A strategic partnership was developed with a major Australian university and a team of science academics. The goal of this programme was to strengthen student’s science skills in the areas of data analysis, experimentation, and scientific writing through current, hands-on research within the context of a major world-wide health issue, over the course of one academic year. This programme was a research project that would lead to scientific publications, in which students contributed to the design of and collection of first-hand data from experiments carried out by two international teams. The scientific outputs of the programme were identified as a manuscript to be submitted for publication in a scientific journal and the design, implementation and facilitation of a scientific experiment. This international experiment examined the efficacy of Australian and South East Asian radiologists in detecting breast cancers on mammograms within the countries they were working. Radiologists were provided with a test-set of breast mammograms and were required to decide the absence or presence of a lesion, and if present, the location of the lesion. At the end of the test-set, radiologists were given a sensitivity (true positives) score, a specificity (true negative) score, and lesion sensitivity score, as well as receiver operating characteristic area under the curve (ROC AUC) and jackknife free response operating characteristic figure of merit (JAFROC FOM) scores, which summarised their mammographic detection efficacy (for methods see Jackson et al., 2019).
Intervention programme
This programme was characterised by three main phases: programme design, programme implementation and programme evaluation.
Programme design.Prior to the implementation of the programme, the science coordinator and leader of the academic team developed criteria for student selection, the goals and expected outcomes for mentees and mentors, delineation of roles for the academic team members and secondary teachers, a schedule for contact between mentees and mentors, as well as determining how the programme was to be evaluated. Initial selection was based upon Year 9 science grades, with students having an extensive or A science grade (NESA, 2019c) being initially considered. In addition, students also needed to have a thorough or B grade in both Mathematics and English (NESA, 2019c) to be considered for the programme. A total of nine Year 10 students (5 female and 4 male) met these criteria and participated in the entirety programme. The ability of students’ families to afford international travel was not a requirement of the programme; it was emphasised to students that the school would financially support any of the students who were not in an economic position to pay for the international component of the programme.
The goals of the programme for mentees were to extend student’s science skills in the areas of statistical data analysis, experimentation and scientific writing, whilst the goals for mentors were to develop meaningful relationships with rural students and to demonstrate to students that careers in scientific research are accessible to all. The role of the academic team was to deliver one hour weekly in-school tutorials on statistical data analysis, scientific writing as well as experimental design within the context of an international experiment which would collect first-hand data. The classroom teacher supported this learning through the implementation of an adaptive science curriculum.
Two outcome measures of the programme were selected – the VALID 10 assessment, an established assessment of student learning in science areas, and a student self-assessment. The rationale to utilise the VALID 10 assessment to evaluate the success of the programme was due to its independence, as well as the ability to be able to determine growth in scientific skills and writing as compared to a control group of students. Student self-assessment was identified as a powerful evaluation tool to assess student growth in areas such as statistical data analysis and to provide qualitative student feedback on the programme. Further information on these measures is provided in the section on programme evaluation. Collaborative meetings, prior to the implementation of the programme, with secondary school science and mathematics coordinators ensured the academic team had a sound understanding of students’ background knowledge of scientific concepts, experimental design knowledge and mathematical concepts of statistics.
Programme implementation. A close mentor relationship between the intervention group and the science academics, including professors and post-doctoral researchers, was established with academics providing students with guidance and support as they participated in the research activities of the academic team. The programme was implemented according to the timeline summarised in Table 1.
Summary of the learning activities intervention students participated in throughout the academic year, showing the duration of the activity, type of activity and who facilitated the activity.
L: lecture; T: tutorial; HOL: hands on learning activity.
#Facilitated by academic leader.
^Facilitated by post-doctoral fellows of academic team.
@Facilitated by classroom teacher.
Term 1. Intervention students were introduced to the leader of the academic team and participated in a series of tutorials held at their school and led by the academic team leader. Tutorials were held each week for a period of one hour (see Table 1). Initially, students examined medical imagining techniques and breast cancer mammograms, followed by statistics tutorials in which students were introduced to descriptive and inferential statistics; learning how to apply appropriate statistical tests of confidence and use statistical programmes. Importantly, this highlighted for students the application of mathematics to scientific research, demonstrating the relationship between these two disciplines that are traditionally taught as separate bodies of knowledge at secondary school (NESA, 2019a, 2019b).
Terms 2 and 3. At the beginning of Term 2, intervention students visited the academic team’s research laboratory on the university campus to meet the team and attend academic seminars. During these two days students also accessed and familiarised themselves with the BREAST software (Brennan et al., 2019) to be used in the international experiment later in the year (see Table 1). Back at school students applied their statistical knowledge to analyse data previously collected by the academic team; students explored areas of personal interest within the data. Post-doctoral researchers provided video conferencing support throughout this phase. During Term 3, students attended face-to-face tutorials with the lead academic, which addressed how to present data for scientific manuscripts. By the end of Term 3 students had prepared a methodology and results section for a scientific manuscript (see Table 1).
Term 4. With guidance from the academic team, intervention students wrote a scientific manuscript based on the results section completed in Term 3. This manuscript was written in sections, and continued feedback was provided by the academic team to facilitate motivation and improve learning (Hattie & Timperley, 2007). Students also prepared for the international experiment, learning to use the software radiologists would be engaging with as well as developing protocols to ensure valid and accurate data would be collected. During the experiment the students set up equipment, managed any problems that arose and packed up equipment. Upon return to Australia students finalised the scientific manuscript (see Table 1).
Programme evaluation.The programme was evaluated at the end of the academic year using two different measures, the VALID science assessment and a student self-assessment survey as described below.
VALID science assessment. The New South Wales (NSW) Department of Education Validation of Assessment for Learning and Individual Development (VALID) programme provides online, multimedia assessments for the science key learning area at the end of Years 6, 8 and 10. The VALID assessment is a high quality, syllabus-based assessment focusing on the content and skills as described in the NSW science syllabus (NESA, 2019b). Year 10 students are tested on their knowledge and understanding of science, understanding and skills in the process of scientific investigation, ability to evaluate evidence, make judgments and think critically and ability to access information and communicate scientific ideas using a variety of strategies. The test contained extended response tasks, short responses and multiple choice items, thus there was an opportunity to assess higher order thinking and deeper understanding of a scientific concept. Student’s achievements are described against standards and are mapped to five main performance areas: (1) overall, (2) problem solving and communication, (3) planning, designing and conducting experiments, (4) knowledge and understanding of science, and (5) extended response. Students are provided with a score for each performance area as well as a level from 1 through to 6, with 6 being the highest level. This VALID assessment of students’ skills occurred during the second half of Term 3 in September, with results released late in Term 4, towards the end of the academic year. This study utilised the VALID assessment for Year 8 and 10 students at the school from 2016 and 2018, respectively, to compare with the results of the intervention group.
Student self-assessment survey. Students who were part of the intervention group completed a self-assessment of their growth in learning in terms of the following science skills: statistics, data analysis, presentation of data, scientific writing, experimental design, developing hypotheses, referencing and reading scientific literature. Students were asked to score their understanding and application of these skills prior to beginning of the programme and at the end of the programme. The scale ranged from 1 to 5, with 1 being not at all competent to 5 being very competent (see Online Appendix 1).The self-assessment was conducted towards the end of the academic year (prior to students receiving their VALID results).
Data analysis
VALID test scores, as provided by NSW Department of Education, for individual Year 10 students from the school for the academic year of 2018 were analysed. Individual scores were broken down into five main performance areas as described above. A control group, from within the same academic cohort as the intervention group, was developed via a comparative analysis of the entire cohort’s previous Year 8 VALID overall scores and levels. Intervention student’s overall scores for the Year 8 VALID assessment ranged from level 4 to level 6. Sixteen other students from this Year 8 cohort who recorded an overall score in these levels were identified for use as a control/comparison group when analysing the Year 10 scores post-intervention. The rationale to separate this group from the larger 2018 cohort was to assess more rigorously the effect of intervention compared to a group of students with a similar academic level. The average scores for the five main VALID performance areas for the intervention group were compared to those of the control group using t-tests. This was followed by a calculation of effect size according to Coe (2002). The students’ self-assessment survey data were analysed using a paired t-test for the application of each scientific skill to examine whether there was a difference between the scores prior to being involved in the programme and at the end of the research programme. GraphPad© PRISM software was used for all statistical comparisons, a pvalue of <0.05 was considered to be significant.
Results
Statistical comparison of the average VALID scores of the intervention students and the control students indicated that students who participated in the hands-on research mentor programme (the intervention group) had greater gains in their scientific learning (see Table 2). The intervention group recorded higher overall VALID scores (p < 0.05), as well as significantly higher scores in knowledge and understanding of science (p < 0.05), planning, designing and conducting experiments (p < 0.05), and problem solving and communication (p < 0.05, see Table 2).
Comparison of VALID scores between intervention group and control group.
n: number of students.
Mean values are shown, with standard deviations in brackets. Square brackets represent the percentage of control group students below the average of the intervention group.
*Significant difference (p < 0.05) from the intervention group for this VALID measure.
Effect size analysis shows that greater than 84% of the control group VALID scores would be below the average intervention group VALID score for their overall scores, as well as for the measures of problem solving and communicating, planning designing and conducting and knowledge and understanding (Table 2). Despite there being no significant difference between the control and intervention group for the extended response measure, 69% of the control group had a score lower than the average of the intervention group (Table 2).
The results of the VALID assessment were in line with the students’ own assessment of their understanding and application of science skills. Within each of the skill areas, the students identified overall mean growth, with the mean self-assessment scores at the beginning of the programme being significantly different (p < 0.05) from the mean self-assessment scores for all skills assessed at the end of the programme (Figure 1).

Change in students’ mean self-assessed application of scientific skills, from the beginning of programme compared to at the end of the research mentor programme. White bars represent the mean score at the beginning of the programme, grey bars represent the mean score at the end of the programme, with error bars indicating the standard deviation of the mean. Only changes that reached statistical significance (p < 0.05) are presented. The self-assessment scale ranged from 1 (not at all competent) to 5 (very competent).
Discussion
Within the period of one academic year, students who participated in an authentic student research programme demonstrated significantly greater growth in their science skills, including problem solving skills, communicating skills, and planning, designing and conducting skills compared with a control group. These high achieving students also experienced a significant change in their own evaluations of their scientific knowledge and skills. These results highlight the pedagogical worth of this authentic student research programme, adding strength to the argument for authentic science research at the secondary school level, especially for high achieving students. The enhanced development of science and 21st century skills prepares learners to be literate within the field of science, by allowing students to ‘do the work of a scientist, not just learn about what they do’. Authentic science research programmes within secondary schools are the exception rather than the rule; most students do not experience authentic science research until late in an undergraduate degree or beyond (Molhenrich et al., 2018; Sadler et al., 2010). However, authentic research at the secondary school level provides students not only with a greater opportunity to develop and practice complex mental processes and other 21st century skills but also provides a natural platform for integrating the sciences and mathematics within a real-world context. This style of integrated interdisciplinary education nurtures students to become big thinkers and creative problem solvers, and importantly emulates tertiary education reform (Bosh, 2018; Graham, 2018; Millar, 2016; Osborne & Dibben, 2017).
The timing of the implementation of an authentic science research programme at the secondary school could be critical to the success of the programme and students growth in learning. The recent introduction of a new science course in NSW, Science Extension, provides students with the opportunity to participate in authentic science (NESA, 2017); however, students are unable to study this course until their last year of secondary school. The advantage of an authentic science research programme during Year 10 is that it provides students with an engaging and well-supported learning environment without the added pressure of results determining university entry. Thus students have the freedom to develop their own questions or change the direction of the research as new and interesting results emerge. This low pressure environment allows students to pursue false ends, as well as fail and try again; not uncommon traits of scientific research that help students to develop resilience, tenacity and perseverance (Tytler, 2007). Furthermore, authentic science research programmes that include academic mentoring and visits to universities make Year 10 students aware of science at the university level, perhaps encouraging students to select appropriate senior courses and thereby expand their options (Alexander & Fraser, 2001). Research has suggested that students in Year 10 may be more receptive to interventions that aim to increase university aspirations than students at other stages (Fleming & Grace, 2014), further supporting Year 10 as a target stage for introducing secondary school authentic science research programmes.
Importantly, an authentic research programme with a strong central focus on academic mentoring allows students to develop relationships with academics and universities, thus helping students to aspire to higher education. This is especially critical in rural and remote schools within Australia, where a significantly lower percentage of rural school finishers pursue university study (Drummond et al., 2011). Having rural students develop positive and fruitful relationships with university academics will help rural students see higher education as a realistic option (Cayetano-Penman, 2010; Tytler et al., 2008). Having initial face-to-face contact with students was central to developing relationships between academic partners and students in this programme give the rural locality of the students and the limited access to science academics. Once this relationship had been established, video conferencing could be employed to facilitate learning; having a video conference aspect to this programme was essential given the geographical distance between the academic team and the intervention students. However, regular face-to-face contact was an important aspect of maintaining the continuity of relationships within this rural context. The shared experience of travelling to conduct the experiment alongside the academic researchers may have also helped to strengthen relationships between the academics and the students. University outreach programmes have identified that students show higher intentions to attend university when they have received academic mentoring (Curtis et al., 2012; Dabson et al., 2010). Therefore, for authentic research programmes to have a long-term impact on rural students, there should be a focus on developing relationships between the mentor and students.
Shortages of secondary science teachers willing to teach in Australian regional and rural schools have been evident for some time (Halsey, 2017; Handal et al., 2013; Jenkins et al., 2011; Tytler, 2007), with access to effective professional learning being a key area of concern for rural teachers (du Plessis et al., 2019; Halsey, 2017; Jenkins et al., 2011; McConaghy et al., 2006; Tytler et al., 2011). The establishment of a science academic research programme that includes a focus on academic mentoring allows university academics to serve as role models demonstrating how scientific knowledge is gathered, analysed and communicated whilst enhancing the teacher’s cognitive understanding and skills (Pegg et al., 2010; Rennie, 2012; Tytler et al., 2008, 2011). This is particularly pertinent to secondary science teaching where nearly 20% of science teachers are ‘out of field’ teaching (teaching a subject without specific training in that subject), with this proportion being even higher in rural and remote secondary schools (Tytler, 2007; Weldon, 2015, 2016).
A number of limitations of this study need to be acknowledged. It is noted that the data collected here represent only one small cohort of high performing students who participated in this particular programme. Still, the results of our study support the findings of similar authentic student research programmes (Burgin et al., 2012; Charney et al., 2007; Eales & Laksana, 2016; Hannum, 2016; Molhenrich et al., 2018; Sogo, 2016; Ward et al., 2016; Wasserman, 2016). Furthermore, the students’ self-assessment and VALID results show the same pattern of growth in science skills. Future research should include students with a broader range of abilities so the benefits of these programmes within a wider student body can be examined.
The scheduling of the tutorial topics and any evaluations of the authentic scientific research programme should be considered carefully in future studies. An unexpected result in the current study, in which there was no significant difference between the intervention and control groups’ scores on the extended response VALID measure may be attributed to the timing of the VALID assessment, which took place in Week 6 of Term 3. As indicated in Table 1, this was before the students in the intervention group participated in the scheduled tutorials on manuscript writing.
This research has focused on the short-term outcomes of our programme; however, the positive impact of similar programmes on careers in science has been identified by some studies (Kitchen et al., 2018; Roberts & Wessersug, 2009). Further evaluation of our programme should include longitudinal research of students’ education pathways and choices to determine the long-term impact. Given the continuation of this programme, the opportunity to examine student outcomes across Year 10 cohorts will be used to deepen the assessment of our programme. Timing of the introduction of the programme, Year 10 versus Year 12, could also be a potential research focus.
Conclusion
Given the ever increasing curriculum content and skills delivered by educators developing research programmes, allowing students to passionately and confidently pursue areas of interest whilst developing complex mental processes and skills is a logical way to produce the next generation of scientists. Such transformative research programmes, however, do not have to be restricted to the domain of science; there is the potential to develop similar programmes in other key learning areas. Whilst such programmes do not fit the mould of traditional teaching, educators need to embrace innovative practices to ensure we equip our students with the necessary skills to solve the multifaceted issues facing the modern world.
Supplemental Material
sj-pdf-1-aed-10.1177_0004944120919890 - Supplemental material for An Australian-based authentic science research programme transforms the 21st century learning of rural high school students
Supplemental material, sj-pdf-1-aed-10.1177_0004944120919890 for An Australian-based authentic science research programme transforms the 21st century learning of rural high school students by Louise Puslednik and Patrick C Brennan in Australian Journal of Education
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
