Abstract
This study explored students’ feelings during the Australian National Assessment Program – Literacy and Numeracy. It features student voice depicted as self-drawn images accompanied by words as they journeyed through four phases of the NAPLAN process: preparation, participation, completion and results. The 34 students in Years 3, 5 and 7 attended one of two Queensland primary schools with divergent approaches to National Assessment Program – Literacy and Numeracy. One-hundred and thirty-six images accompanied by 372 words were collected. School A allocated considerable time to test readiness and emphasised the importance of National Assessment Program – Literacy and Numeracy while School B engaged in minimal preparation and did not highlight National Assessment Program – Literacy and Numeracy as important. We explored similarities or differences in how students from each site reported their feelings. Our exploration reveals an overall trend of negative images and words during the preparation and participation phases followed by more positive depictions when the tests are complete and results are received. Furthermore, School A participants reported more negative feelings in the preparation and testing phases across all year levels.
Keywords
Introduction
In Australia, the 2008 introduction of the National Assessment Program for Literacy and Numeracy (NAPLAN) marked a national shift towards an accountability-driven education agenda (Lobascher, 2011; Polesel, Dulfer, & Turnbull, 2012) which has the potential to impact upon the cognitive challenge of learning and assessment practices, especially as schools and systems strive to reach benchmarks (Swain, 2014). Delivered by the Australian Curriculum, Assessment and Reporting Authority (ACARA), NAPLAN is undertaken by students in Years 3, 5, 7 and 9 simultaneously in all Australian schools annually and provides standardised external measurement of reading, writing, language conventions and numeracy. It is intended to be a measure of how individual students are performing at the time of the tests; whether or not young people are meeting literacy and numeracy benchmarks and how educational programs are performing (Dulfer, Polesel, & Rice, 2013). The comparative performance of schools and school systems is announced publicly and data for year levels and components of the tests are deeply scrutinised. This data is based on aggregated individual results, which are provided separately to students.
More recently, in addition to the aggregated data sharing as an outcome of the process, the body of research about the impact of NAPLAN on individual students’ well-being as participants of the process is increasing, though the lack of voice given to students continues to be a distinguishing feature (Howell, 2015). Furthermore, little research has been undertaken about the journey of participants over a NAPLAN cycle, that is, during the preparation, participation and completion of the assessment, and the period after when results of student performance are received; and explicitly from the perspective of students through their own visual depictions and accompanying words (Swain, Pendergast, & Cumming, 2018). The current study specifically addresses this gap in the research about NAPLAN and aims to discover how students perceive and react to NAPLAN preparation and testing.
NAPLAN as assessment of and for learning
The three main purposes of assessment are: assessment for learning, assessment as learning and assessment of learning (Earl & Giles, 2011). Assessment for learning, known as formative assessment, includes all tasks undertaken by teachers and students to provide information which informs future learning. Assessment as learning involves students taking ownership of their own learning by self-assessing to identify learning gaps as well as receiving and responding to feedback from their teachers. Assessment of learning, known as summative assessment, occurs at the completion of a learning cycle or task and can be school-based/internal or external. Each of these assessment frameworks has different target audiences, purposes and procedures. The expectation is that NAPLAN will serve multiple purposes, specifically summative for accountability reporting, and formative to guide teachers and students (ACARA, 2011). NAPLAN is also high stakes in that it involves interest beyond those who sit the tests.
NAPLAN and learning
In 2008, a public website, www.myschool.edu.au, was also introduced to provide reports on NAPLAN outcomes for every Australian school. Already considered high-stakes testing because of the links between NAPLAN results and government funding, the publication of school data increased the high stakes and consequentially changed the way NAPLAN was perceived by systems, schools and teachers (Swain, 2014). When assessment becomes high stakes it has impact for teachers and school administrators as well as students. This was confirmed in a 2013 survey of 8300 Australian teaching staff who reported that NAPLAN impacted their teaching style and content choices and led to a reduction in timetabling of other subjects in their schools (Dulfer et al., 2013), specifically in the learning areas of art, music and language. One-third of the surveyed teachers set more than seven practice tests prior to the NAPLAN testing period and 80% reported that test preparation added to an already overcrowded curriculum. They noted a reduction in face-to-face teaching time with one-half reporting that their pedagogy had changed to being more test-driven. Thirty-nine per cent reported they were teaching by rote and were administering weekly practice tests as a method of increasing NAPLAN results. Teachers also indicated that other consequences included negative impacts on students’ health and well-being, staff morale, school reputation and the capacity of schools to attract and retain students and staff. The survey found 90% of the teachers reported students feeling stressed prior to testing and students being sick, crying or experiencing sleepless nights (Dulfer et al., 2013).
Stress can lead to test anxiety and, as Broadfoot and Black (2004) argue, student attitudes to learning and the strategies they use to further their own learning may be affected by the way assessment is conducted. Furthermore, if anxiety affects test performance, it can be regarded as a source of invalidity distorting test scores. Hohensinn et al., (2011) similarly argue that students cannot perform to their best when they are feeling upset or anxious. Hence, the interpretation and use of test scores must be carefully considered particularly when item-level information used inappropriately can lead to misunderstanding of student capability (Messick, 1998). Hence, listening to and hearing student ‘voice’ regarding experiences of NAPLAN is paramount to confirm instrument validity and the validity of the data it generates.
Student voice
Research can be conducted using a range of theoretical and methodological approaches and providing voice in such a way as to privilege one over another. Privileging student voice means students are able to speak on their own behalf about their interests and beliefs (Cook-Sather, 2006; Mitra & Serriere, 2012) and to express their views and opinions, or their likes and dislikes.
In educational research, student voice is often absent or tokenistic (Holdsworth, 2000) typically reflecting a leader-centric structure of schools (Cook-Sather, 2006) where students are allowed mediated participation in decision making regarding particular matters such as toilets and tuckshops but are not provided a voice in important educational matters affecting their schooling (Gunter & Thomson, 2007; Voight, 2015). When student voice is privileged, the dominant power imbalances between adults and young people alter and students become more connected to school (Jenkins & Pell, 2006). Respected students feel valued in their classroom and are more able to develop quality relationships to create a mutually beneficial learning environment with their teachers (Cook-Sather, 2006; Mitra & Serriere, 2012).
Enabling genuine student voice in educational discourse adds real-world significance (Gunter & Thomson, 2007; McLeod, 2011). Logan and Skamp (2008) draw attention to the significance of listening to student voice and acknowledging what is said as important. They suggest that not to do so may result in a decline in positive student attitudes and connectedness with school. McLeod (2011) warns against ignoring the importance of listening and suggests that the opposite of speaking is listening; both are socially embedded practices which involve engaging in and with the perspectives of others. Privileging student voice, therefore, means remembering the significance of listening and hearing what is said, shifting the responsibility from the speaker to the listener (McLeod, 2011). Where there is a lack of teacher voice in educational reforms and the measure of success is gauged by student achievement on standardised tests, where not a student voice is heard (Cook-Sather, 2006), the effect is uncertain.
Repositioning students so that they can express their feelings about their educational experience opens up the curriculum and pedagogy to choices (Gunter & Thomson, 2005; Mitra & Serriere, 2012; Voight, 2015). This type of reform, involving a shift in power away from dominant adult decision making to collaborative student/adult problem solving, decision making and long-term goal setting, challenges education systems to realise a new approach (Gunter & Thomson, 2005). Enacting this change involves acceptance, courage, commitment and change. The driving force behind such change is to remove the discrepancy between what is said and what is done (Cook-Sather, 2006), by encouraging students to remain connected and engaged through involving them in the problem-solving and decision-making process while empowering them to accept personal ownership of their education. If it is accepted that students have legitimate fundamental rights, to ignore their voices is to violate their rights (Cook-Sather, 2006).
The use of student drawings is one way of providing students with voice about their experiences. There are many precedents for the use of this approach. Students’ drawings have been used as windows into the perceptions of the child for over a 100 years from a psychological perspective in order to study cognitive development and emotional concerns (Weber & Mitchell, 1995) – but rarely for educational use. Using student drawings in large-scale educational research has only developed in the last 30 years. Haney, Russell and Bebell (2004) started including students’ drawings in both their large-scale and classroom-based methodology in 1994 (Haney et al., 2004, p. 267). In more recent years, the use of student drawing methodology has become more common. Libarkin, Thomas and Ording (2015), Harris, Brown and Harnett (2014), McHatton, Shaunessy-Dedrick, Farmer, Ray and Bessette (2014) are just a few who have adopted this method of data collection as a way of documenting students’ perspectives. Cheng and Gilbert (2009) recommend that drawing be used as a means of corroborating and detailing student understanding and hence the combination of words and drawings together is a powerful way of validating student voice. Haney et al. (2004) warn that reading too much into students’ drawings can be problematic. Without consultation it is easy to misinterpret meaning behind individual drawings. Hence the importance of including descriptive words to accompany student drawings (Wilks, Pendergast, & Service, 1996). In this study, student voice as data is intended to enable students to share their feelings about NAPLAN such that their voice is added to the outcomes associated with NAPLAN, alongside the typical reporting of the data comparisons for systems and sectors.
Method
This study was conducted over a period of 11 months in 2009 when Year 7 was still included in Queensland primary schools. It captures the reported feelings of 34 students across years 3, 5 and 7 in two separate primary schools depicted in words and drawings as they journeyed through the four distinct phases of the annual NAPLAN process, namely preparation, participation, completion and after receiving their results. The aim of the study was to explore students reported feelings during the NAPLAN journey and to explore similarities and differences in students’ feelings in different school settings where the relative focus on NAPLAN differs.
School sites
Two south-east Queensland primary schools agreed to participate in the study. The selected sites employed different approaches to the implementation of NAPLAN but were geographically and economically alike. The researcher was familiar with both sites before the research was conducted. School A’s Year 3, 5 and 7 students engaged in intentional preparation focussed on NAPLAN including developing test readiness up to, whereas School B adopted a low-key approach to NAPLAN preparation and testing with no specific preparation.
Participant recruitment
School A is a Queensland State School, funded by state and federal governments. It is a co-educational, metropolitan, public state school, with an enrolment exceeding 700 students. School B is a Queensland Independent School, funded by a combination of private and government funding. It is a co-educational, metropolitan, independent school with a student enrolment under 300. The inclusion of schools from two alternative systems enabled an exploratory comparison of some of topic under review.
At School A, the principal selected students while at School B, all students were invited to participate by the principal. Participants were selected to represent students from all year levels in which NAPLAN was conducted in their schools, namely Years 3, 5 and 7 a mix of male and female students and a range of cognitive and achievement abilities.
No knowledge of particular students’ level of achievement was made available to the researchers. The study involved 34 students, 17 at School A and 17 at School B. Details of year levels are provided in Table 1. All students at School A and B participated in the study after returning their parental consent forms. All students have been allocated a pseudonym for ease of sharing the findings.
Student data population.
Data collection
Data collection was conducted away from the classroom. At School A, data were collected in a room adjacent to the school hall and at School B data were collected in the school library. The students and the researcher were the only people present at the time of data collection so that the students could speak freely about their drawing without being overheard.
Each student was provided with a piece of A4 paper and pencils at each of the four separate data collection instances. During each separate session, students were invited to draw a picture of how they were feeling in relation to the current stage of the NAPLAN process, namely during NAPLAN preparation, during NAPLAN testing, on completion of NAPLAN testing and upon receiving their NAPLAN results. This resulted in a total of 136 pictures being drawn (i.e. 17 students × 4 pictures × 2 schools) with 68 from each school.
In addition, students were invited to write three words to support and explain each picture. While some students did not write any words others wrote fewer than three words. No adjectives (e.g. ‘very’) were included in the word count. Also, where a student wrote something such as ‘It was very boring’ only the word ‘boring’ was considered. This resulted in 316 words being included in the analyses.
Spelling was not corrected as it was not considered an issue because the researcher was able to consult with each student one-on-one at the completion of each data collection session to discuss the students’ words and drawings. No further detail was added to the students’ work. However, the researcher was able to gain a better understanding of the meaning behind their illustrations.
Some parents had chosen not to share their child’s NAPLAN results with their child. This was explained to the researcher by the child while the child discussed their illustration with the researcher. The researchers were not privileged to see the students’ results but this was not of significance because the research focus was students’ reactions to and their perceptions of their NAPLAN results rather than their actual NAPLAN results.
Data analysis
Adopting the process of Haney and colleagues for student drawing analysis (Haney et al., 2004, pp. 249–256), the students’ illustrations were subjected to three stages of coding, namely emergent analytical coding followed by trait coding and finally holistic coding and holistic review.
Emergent analytical coding was used to identify features contained in the drawings. These features were developed into a checklist for the purpose of further analysis. Two independent researchers not involved in the study and the researcher responsible for data collection each viewed a sample of drawings and noted common features. These features were then combined, compared and condensed, and a draft coding sheet was developed. The draft coding sheet was used to view another sample of drawings. This time, the researchers noted the presence or absence of listed features such as detail, number of human figures, facial expression and inanimate objects. A final coding checklist was developed by comparing results with only features exhibiting a high level of agreement (Swain, Pendergast, & Cumming, 2018). Areas requiring further agreement or consensus were moderated collaboratively. The complete data set was then independently coded by each of the researchers.
Trait coding is an analysis of the extent to which the features occur in the drawings. The analysis in this study used tally marks as a simple method of tabulating the frequency of features. The tally mark method determined which categories of features could be discarded, thus condensing the coding scheme. Holistic coding and holistic review were combined to complete the analysis.
All drawings were coded on a 4–5 point scale based on the representation of a particular aspect, for example, a proportion. To ensure reliability, Haney et al. (2004) suggest features with less than 80 per cent agreement be discarded, however, the results indicated that a high level of inter-rater reliability existed between the three raters who coded all materials in this study, which allowed the figure to increase to 95 per cent agreement prior to results being discarded.
The words that accompanied the images were grouped according to their frequency, similarity in meaning and finally their valence – that is the combined effect of the image and words as being either a positive, neutral or negative emotional representation (Wilks, Pendergast, & Service, 1996). Drawings and words were collated first for each of the data collection instances in order to determine the valence of the images combined with the words. Then, for each year level in each site for each of the four phases, the word data were collated and presented as a valence analysis (for an illustration of this process, see Tables 2 and 3 for School A and Tables 4 and 5 for School B). The inclusion of the full data set in the analysis ensured that each student had a voice in presenting his or her experience during the NAPLAN process.
Combined image and words valence, School A.
Word frequency, School A, alphabetical.
Combined image and words valence, School B.
Word frequency, School B, alphabetical.
Results
The findings are presented by school and year level, keeping in mind that School A had a focus on test readiness and highlighted the importance of NAPLAN while School B engaged in minimal preparation and did not emphasise student performance in NAPLAN.
School A Year 3 student feelings
Five students, two female and three male, participated in School A, at Year 3 with Figure 1 showing the drawings across the four stages of the NAPLAN journey. When responding to the first prompt, namely students’ feelings during NAPLAN preparation, four of the five Year 3 students, Carol, David, Andy and Brian, drew pictures containing three faces. Each face displayed typical characteristics such as eyes, nose, mouth and sometimes hair and all four illustrations included faces with down-turned mouths. Expectation of the unknown is also demonstrated by Carol and Brian when drawing faces with surprised expressions represented by a widely opened mouth. Demi drew the face of an animal which was interpreted as depicting her disengagement with NAPLAN, as if she was not involved. This interpretation was supported by her choice of words ‘fine’, ‘yeh’ and ‘whatever’.
School A Year 3 student responses.
When asked to illustrate how they felt while participating in NAPLAN testing the same four students, drew bodiless figures, however, detail had increased to include tears and the line used to depict a down-turned mouth had increased in length, indicating increased sadness. Words chosen to explain and support their illustrations had intensified by inclusion of the adverb ‘very’, in front of each chosen word. Furthermore, descriptive verbs, such as ‘shaking’, were added, and were considered by the analysis team as indicative of increased fear.
Unlike Demi’s initial drawing where her illustration indicated she possibly had disengaged from NAPLAN, her second drawing included extensive detail and a number of significant features. Demi’s increased pressure on the writing instrument to increase darkness of objects to attract the viewer’s attention was considered significant. Demi seems fearful of participating in NAPLAN testing as her involvement would require a large amount of writing and a large amount of ‘bubbles’ to colour in, to answer questions which she would not answer correctly. Although others are participating in the tests, the testing situation prevents any interaction between. Demi considers herself as all alone; totally isolated.
There appeared to be a marked difference in School A’s Year 3 students’ illustrations once NAPLAN testing was completed. Figures had increased in size and detail. Other changes included more rounded eyes and smiles had replaced down-turned mouths, indicating happiness had replaced sadness and a shift from negative to positive valence emotions. Demi again supplied an extremely detailed picture, this time she was dancing. The smile on her face, the boom box, the musical notes and the highlighted word ‘dance’ are indicative of Demi’s happiness that the testing period was over.
The fourth and final prompt asked students how they were feeling about the return of their NAPLAN results. At this stage, some students had seen their results and others had not. Of the five students, all except Brian drew large smiling faces. Brian wrote the words ‘I’m dead’. His extreme happiness upon completion of NAPLAN testing had been replaced by his fear of failure and is confirmed by his choice of words.
School A Year 5 student feelings
Seven students, five female and two male, participated in School A, at Year 5 with Figure 2 showing the drawings across the four stages of the NAPLAN journey. Like students in Year 3, Callum used facial expression to portray his feelings and supported this with negative valence words. Christy and Mandy’s illustrations and supporting words were similar and portrayed negative valence. Jack and Jackie each drew large, sad faces. Sally drew three separate figures; each explained by a word. The students’ failure to include faces and mouths in their illustrations was interpreted as the students’ perception of having no voice regarding events occurring in relation to NAPLAN preparation. Abby’s drawing described NAPLAN preparation using a sad student sitting at a table with a test paper. ‘Ah! Upset and sad’, were written above the illustration.
School A Year 5 student responses.
For phase 2 participation, Callum’s response shifted to a facial expression which appeared to represent anger. His words also increased in severity from the preparation stage to the testing stage to include ‘steamed’. Christy and Mandy again drew themselves as stick figures; however, they now sat alone at tables. Detailed representation of the test booklet drawn by Abby was interpreted as representative of its significance in the illustration, it appeared to draw the viewer’s attention. Figures drawn by these two students although in a sitting position were generally unchanged; however, facial expressions were interpreted as representing increased unhappiness and words used to describe their pictures included, ‘nervous and worried’, and ‘failure, scared’ and ‘worried’.
Sally’s illustration was of one enlarged face. A squiggly horizontal line representing the mouth implies indifference. This illustration was not unlike those provided by Jackie, who showed little variance in illustrations submitted for each of the four prompts. Jack commenced drawing but scribbled out his first attempt. The paper provided room for a drawing of equal size, but Jack elected to draw a smaller face. This might be Jack’s perception of himself as insignificant and a decrease in self-confidence during the testing period. Abby’s illustration draws attention to the down-turned mouth by increasing pencil pressure to draw a dark, thick line.
Illustrations drawn by Year 5 students in response to test completion were collectively of more positive emotional valence with typically large up-turned mouths. Words used by all of these students in response to feelings experienced upon test completion supported their illustrations and told the same positive story.
At the time of the final meeting, some students had received their results and others had not. Christy’s drawing in response to the arrival of NAPLAN results included three numbered faces. The analysis team concluded that each face represented Christy’s level of confidence in her ability to achieve a successful outcome in each academic area; Maths, Reading and Language. Three remaining students provided illustrations representing happiness and hence a positive emotional valence in response to the NAPLAN results.
School A’s Year 7 student feelings
Five students, three female and two male, participated in School A, at Year 7 with Figure 3 showing the drawings across the four stages of the NAPLAN journey When responding to feelings experienced during NAPLAN preparation two Year 7 boys in School A, Jake and James, submitted illustrations of themselves sleeping. Speech bubbles provided a picture of a test paper displaying ‘A+’ results indicating their level of confidence. Year 7 female participants drew expressionless faces. Sharny and Xanthia included beads of sweat running down from their foreheads. Words used to accompany their illustrations included negative valence: ‘nervous, scared’ and ‘overwhelmed’. Rhianna drew a whole-bodied picture of herself portraying a closed-body position suggesting her lack of self-confidence. Sharny continued to draw the same face regardless of the prompt, however, an obvious changing characteristic, which raised contention amongst the analysis team, was the student’s hair.
School A’s Year 7 student responses.
Jake and James both had a changed attitude in response to the prompt related to NAPLAN testing. James portrayed himself as ‘scared, shocked’ and ‘sad’ in his illustration indicating decreased confidence which contrasted with his confidence leading up to the test. Over-sized furniture items, including the pencil placed on the table, contrasted with the diminished human figure, while the down-turned mouth attracted the viewer’s attention to the sadness expressed on the student’s face. Jake’s confidence was replaced with anger. Using upper-case letters the adjective provided was ‘UPSET’. Jake’s illustration included a large head perceived as indicating the amount of information stored within and emphasising the extent of his anger. Sharny’s tight curls in her second response portrayed an up-tight and frazzled student and it is noteworthy that Sharny does not have curly hair nor did she have her hair curled on this particular day, so the representation was related to being frazzled. Figures drawn by Rhianna show her illustrations to be recognisable as the same person, however, each revealed a different emotional state.
Illustrations provided by Xanthia included the same representation of self throughout; however, changes were apparent in each illustration. The sweat beads appeared again in the second illustration but were absent in the third and fourth illustrations. ‘OMG Naplan (Oh My God NAPLAN)’ encased in a speech bubble was included in her second drawing, submitted in response to feelings experienced during NAPLAN testing; however, a small smile and adjectives, ‘nervous and curious’, that she may be anxious about NAPLAN testing but her level of stress did not represent extreme fear.
In response to the completion and results received phase, the boys illustrated smiling faces. At completion, Sharny again illustrated herself with tightly curled hair perhaps representing tension. In contrast to her previous drawing, Rhianna’s large smile represented her feelings upon completion of NAPLAN testing. In addition, Rhianna drew her arms raised in a wide, open-bodied position, as opposed to the close-to-body position of the previous two pictures. Her final illustration is comparable to that which she drew in her initial one; however, a straight-legged body pose indicates a greater relaxation with her arms returned to a close-to-body position. Xanthia’s response to the completion of NAPLAN testing was demonstrated by a large, more obvious smile, arms were raised in celebration, and the words, ‘Yes it’s over’, encased in a speech bubble, represent sheer elation, a very positive emotional valence.
School A findings – Summary
In order to summarise the feelings depicted in the images and the words together for School A, the combined valence of the student voice is provided in Table 2. Valence refers to the combined effect of the image and words as being either a positive, neutral or negative emotional representation (Wilks, Pendergast, & Service, 1996).
This valence summary highlights the shift for all three year levels from a stronger negative valence during phases 1 and 2 – the preparation and participation stages – to a stronger positive valence at phases 3 and 4 – completion and after students receive their results.
The student voice data included up to three words students could use to represent the way they were feeling at each phase of the NAPLAN program. Table 3 provides a complete list of the words and how they have been grouped to reflect frequency and valence alignment according to year level and data collection instance. Some word may appear to be placed in the incorrect category; however, the explanation of the word contextualised by the student has determined placement, for example nervous excitement. This summary is based on individual words.
Across each of the four phases, the 17 students provided a total of 207 words, 47 for phase 1, 42 for phase 2, 45 for phase 3 and 33 for phase 4. The words provide additional insight into the way the students feel during the four phases of the NAPLAN journey and reflect the shift for all three-year levels from a stronger negative valence during the preparation and participation stages to a stronger positive valence at completion and after students receive their results. Students in all three year levels in School A employed more words with negative valence during phase 1 preparation, and in phase 2 sitting the test. This was particularly marked for the year 5 cohort. In phase 3, when the testing was complete and phase 4 when results were received, there is a strong shift to positive valence emotions across all three year levels.
School B’s Year 3 student feelings
Eight students, four female and four male, participated in School B Year with Figure 4 presenting the drawings across the four stages of the NAPLAN journey. In response to the initial prompt regarding students’ feelings about NAPLAN preparation, Peter drew a circle for his head, two dots for eyes, a single dot for a nose and a slightly up-turned line for a mouth. Peter included both ‘scared’ and ‘excited’. Adam included only a mouth. A significant alteration to the mouth from one illustration to the next was the shape and size. In response to NAPLAN preparation the mouth was non-specific. Three Year 3 students represented themselves using whole bodied characters with raised arms and large smiles and Kevin added two rows of teeth. The raised arms and smiling faces were supported by the positive valence words ‘great’ and ‘happy’, however, also included were the negative valence words ‘scared, nervous, frightened’ and ‘pressured’ which reflects both positive and negative feelings towards the test.
School B’s Year 3 student responses.
Anna, who reported loving school, indicated that she was looking forward to NAPLAN testing. Her illustrations across the four phases are consistent with raised arms, a smiling face and a figure jumping. In Mary’s illustration the words, ‘nervous, scared’ and ‘butterflies in her stomach’, aligned with a small smile. Ella and Bec drew three figures in their initial illustrations which may be a consequence of asking students to include three words. This was considered to be the case with the illustration provided by Ella, however, it was not considered to be the case with Bec who included three images in response to the preparation stage only. In this illustration she had placed herself in the foreground with an upward curved mouth and the word ‘excited’ written above her head, reflecting positive emotions. However, the same cannot be said for the other two students with untidy, flyaway hair, whereas her hair is drawn as neat and tidy. One figure has a squiggly line representing a mouth and the other a circle. The figure representing Bec appears calm in contrast to her peers suggesting a comparative representation of her feelings compared to others.
In contrast Ella drew three representations of herself. Each figure was explained by a word. The first character displayed a smile and appeared under the word ‘excited’. The second image, identifiable as the same person, appeared under the word ‘nervous.’ Ella increased pressure on the pencil to draw the mouth, no longer in the shape of a smile, and the larger circles used to represent the eyes, altered the facial expression from happy to nervous. The third image in Ella’s illustration included the word ‘scared’. The mouth changed shape and was represented by a circle. Seven of the eight students belonging to Year 3 indicated varying levels of happiness, although the words often indicated a nervous anxiety.
One noticeable difference in Peter’s phase 2 illustration was the straight line representing the mouth. This alteration changed the emotion from happy to expressionless and the words ‘Left Out’ were written above the picture. Sam and Kevin’s responses to the second phase included significant differences from their initial illustrations. Furniture had been included in the illustration submitted by Sam and the smiling face was replaced by an expression of fear. After considering the words selected to support and explain Sam’s picture ‘unreal, great’ and ‘happy’ the team re-analysed the illustration and further illustrations provided by the student, and concluded that the large circular mouth was not drawn to represent fear, but to express a larger smile. In contrast to Sam whose illustration had changed to demonstrate an increased level of excitement, Kevin’s second illustration indicated a decreased level of happiness. The head had increased in size and the body decreased, the arms once held high above the head had been replaced with a test booklet on one side and a pencil on the other. A further alteration was the mouth shape which transformed from a large smile displaying two rows of teeth to a straight mouth with only one row of teeth. The word ‘bored’ was written above the picture.
Illustrations in phase 2 by Mary indicated that her initial stress had ceased upon commencement of the testing period and the smiling face suggests happiness. The second illustration submitted by Anna was consistent with her initial illustration. The bright eyes and smile were supported by a raised arm directing the viewers’ attention to the speech bubble. Contained in the speech bubble was the expression ‘This Rules’. Her inclusion of words, ‘tense, confident’ and ‘excited’ completes the picture. Adam’s facial expression in the second illustration was altered by the absence of teeth and the slightly downwards curve of the mouth representing sadness. This suggestion of sadness was confirmed by Adam‘s choice of adjectives ‘bored’ and ‘unhappy’.
In Bec’s illustration in phase 2 her previously calm demeanour had disappeared as had her display of confidence. The words ‘strange, scared’ and ‘nervous’ were added. Ella’s illustration in response to NAPLAN testing included three images. Consistent with the previous illustration each image represented Ella’s self. ‘Excitement’ in the initial illustration was replaced by ‘happiness’ that testing had commenced. The ‘nervous’ image was replaced by feeling ‘great’ about NAPLAN testing and the ‘scared’ image had transposed to represent feeling ‘awesome’.
In response to the phase 3 – after completion of NAPLAN, Sam’s arms reverted to a raised, open position and the words ‘happy, great’ and ‘good’ suggest a positive emotional valence. The happy child in Kevin’s initial drawing re-appeared in his second drawing and included the words ‘glad, happy’ and ‘awesome’, alongside a very large smile. The facial expression in Mary’s third drawing represented the same emotion as depicted in response to the second prompt, however, the student added the word ‘relieved’ indicating that although happy to participate in the NAPLAN process she was pleased that it had concluded. Anna’s third illustration could be mistaken for a child displaying unhappiness with her perceived achievement. This misunderstanding may be emphasised by the inclusion of words ‘sad, disappointed’ and ‘impatient’. However, the thinking bubbles explain that Anna’s unhappiness is related to the two year wait until she is able to participate in NAPLAN again and the wait for the return of her results. The use of body language in the third illustration clearly indicates her sadness. The mouth appearing on the face of the third drawing submitted by Adam now up-turned displayed two rows of teeth slightly spread to accentuate the smile. Adjectives such as ‘so happy’ and ‘glad’ further reinforced the transformation from sadness to happiness.
In Bec’s third illustration, she included only a black outline of herself. The words ‘glad, worried’ and ‘empty’ appeared above the illustration.
Illustration three submitted by Ella although appearing in a different order contains the same configuration of images as those previously submitted, two faces and a whole bodied stick figure. The first image, is drawn using squiggly lines and is supported by the word, ‘wonky’, perceived as indicative of a child who has given everything to NAPLAN and feels she has nothing left, not even enough to stand straight. The expression appearing on the face of the second image indicates happiness and is supported by the word ‘happy’, however, the final image is indicative of a student ‘worried’ about NAPLAN achievement.
Happy smiling faces were not present in six of the final illustrations submitted by the Year 3 students to express how they were feeling after receiving their results. Mary drew a picture of a girl displaying a large down-turned mouth, supported by the words ‘bad, sad’ and ‘upset’. Sam replaced his smiles and his up-lifted arms with an expressionless face and arms as horizontal as the line representing her mouth. Words ‘okay, good’ and ‘bad’ explained the student’s decision to express neither happiness nor sadness. Sam had seen his results and perceived his achievement as neither ‘good’ nor ‘bad’ and therefore was neither ‘happy’ nor ‘sad’. Kevin illustrated his level of achievement in each of the three NAPLAN areas, as depicted in NAPLAN reports. Written in uppercase font the word ‘BAD’ appeared twice. In contrast, the word ‘good’ written in lowercase font only appeared once. In Anna’s phase 4 illustration in response to her NAPLAN results she indicates the return of happiness and this is reflected in her word choice of ‘joyful, proud’ and ‘on top of the world’.
Adam’s feelings were positive which is clearly displayed in his fourth illustration by an exaggerated smile supported by the words, ‘happy, so, happy, so, so, happy’. The final illustration submitted by Bec in response to delivery of NAPLAN results represents a child delighted with her achievement. The ‘WOW’ written in the text box indicates her surprise at her achievement, supported by words, ‘good’, Happy’ and ‘glad’. Ella appears forlorn in her final illustration. She had not viewed her results and in the final image the words chosen to explain her picture were ‘questions’ and ‘weird’. The image included three question marks as if the student was thinking about her unknown level of achievement in each of the NAPLAN components.
School B’s Year 5 student feelings
Five students, four female and one male, participated in School B, at Year 5 with Figure 5 showing the drawings across the four stages of the NAPLAN journey.
School B’s Year 5 student responses.
The students’ illustrations of their feelings during NAPLAN preparation indicated a variety of student reactions. Kim drew what appeared to be a happy child unaffected by NAPLAN preparation. The smiling face included in Rose’s illustration also indicated happiness, however, Rose also indicated that she was experiencing anxiety regarding the approaching testing period by including the word ‘scared’. Daisy also selected ‘scared’ along with ‘frightened’ and ‘nervous’ to explain her illustration. The tears streaming down the face and outstretched arms were perceived as indicating helplessness. The final two Year 5 students demonstrated angst towards NAPLAN preparation in their initial illustrations. Susan drew a figure vomiting, the speech bubble provided sound effects ‘blah’. Fred portrayed a clear message in his illustration which included a frown, a down-turned mouth, and thinking bubbles leading to the statement written using capitalisation, ‘I HATE NAPLAN!’ and a question, ‘What’s the point’?
A down-turned mouth indicated unhappiness in Kim’s second illustration. Her representation of herself sitting at a table with a test paper was surrounded by question marks. The words ‘kind of nervous, uptight’ and ‘sometimes bored’ explained the visual image. Rose’s illustration included a figure ‘running away’. This was the only picture representing happiness during the NAPLAN testing period because the child was running away. The figure standing behind a table with arms outstretched and ‘What do I do’? written in the speech bubble indicated that Daisy’s perception of helplessness remained during NAPLAN testing. Susan further elaborated on the boredom indicated previously by adding the words ‘annoyed, frustrated’ and ‘boredom’. Supporting her illustration of herself seated at a table holding a test paper was a trail of ‘Zs’ implying that Susan suffered boredom to such an extent that she had fallen asleep. Fred’s attitude towards NAPLAN did not improve with the onset of NAPLAN testing. He failed to supply an illustration and instead chose to write his message, ‘www.I AM BORD.com.au’. Feelings portrayed by Year 5 students in illustrations in response to NAPLAN completion emphasised happiness – up-reaching arms, happy faces, ‘Sunshine, Lollypops’ and ‘Rainbows’ all featured.
In response to feelings experienced at the time of the NAPLAN results delivery, question marks reappeared in Kim’s illustration. She had not seen her results and therefore was not able to assess her achievement. Having viewed her results Daisy indicated her feelings by illustrating her results as they appeared on the NAPLAN results sheet. The words ‘bad, bad’ and ‘okay’ were chosen to explain her perception of her achievement. Rose also drew a picture of her results, but they did not appear in NAPLAN results format. The large tick drawn alongside the results sheet was reflective of the words appearing above the image, ‘Good’ and ‘Happy’.
Susan had indicated throughout her illustrations that she perceived NAPLAN as boring. She continued to emphasise her lack of interest in her final illustration by including a large hand in the ‘STOP’ position. The hand, disproportionate to other items included in the illustration had been cleverly drawn to capture the viewers’ attention. Fred seemed pleased with his results and indicated this by drawing a trio of ‘A + s’ under which was written each component of the NAPLAN tests compartmentalised by the student as ‘English, Math, & Reading’.
School B’s Year 7 student feelings
Four students, three female and one male, participated in School B, at Year 7 with Figure 6 showing the drawings across the four stages of the NAPLAN journey. In response to prompts associated with the stages of the NAPLAN process, all Year 7 students displayed some level of uneasiness as well as some level of excitement. Jenna drew speech bubbles to represent her feelings for the first three prompts. In response to NAPLAN preparation a test booklet appeared in the speech bubble. Above the speech bubble ‘nervous, excited’ and ‘confused’ were written, suggesting that Jenna was experiencing anxiety and excited in relation to the approaching testing period. Louise and Hilda drew whole bodied figures with punctuation marks around their heads offering a visual representation of thought processes. Rob included three images in his initial illustration. The first image described as ‘strange’ featured squinty eyes, a down-turned mouth and a hand positioned across the cheek which offered a ‘strange’ appearance. Described as ‘worried’, the phase two image included eyes wide open, raised eyebrows and hands positioned across the mouth giving the facial features a ‘worried’ appearance. ‘Freaky’ was the adjective used to describe his image drawn after completing NAPLAN.
School B’s Year 7 student responses.
In phase 2, Jenna drew a cross and a tick in a speech bubble above which was written the word ‘confused’. Hilda gave the appearance of a decreased level of anxiety in her second illustration. She drew herself seated at a table and although not smiling the facial expression portrays a more relaxed appearance than in her previous illustration. Words selected by Hilda to support and explain her feelings were ‘relaxed’ and ‘confident’ confirming the emotion represented in the illustration. Louise also drew herself sitting at a table with a test paper. She too selected ‘confident’ to describe her feelings but also included ‘scared’. The emotion displayed on the face appears anxious. Two speech bubbles, one containing a ‘tick’ representing ‘confident’ and the other a ‘question mark’ indicating ‘scared’, appear in the illustration. In Rob’s response to the testing period the only feature considered significant was the exaggerated appearance of the oversized head as it is disproportionate to other images included. The enlarged head was interpreted as indicative of the students’ perception of the amount of information stored in their heads so as to complete the NAPLAN tests.
In the third artefact provided by Jenna written in the speech bubble was ‘naplan?! 100%’ above were the words ‘happy but worried’. This was perceived as meaning that Jenna was ‘happy’ that the testing period was completed but still experienced anxiety regarding her results. Self-portraits submitted by Louise and Hilda in response to NAPLAN completion portray happiness; however, there still appears a level of anxiety present in relation to their perceived success or failure. Louise illustrated herself smiling and at first glance appears happy, but the selection of words ‘relieved, excited’ and ‘nervous’, and the two speech bubbles, one containing a smiley face and the other a question mark, told a different story. A smiling face present in Hilda’s third illustration indicated happiness. Words ‘happy’ and ‘excited’ appeared above the illustration, however, the inclusion of the word ‘worried’ indicated that Hilda, although happy was still experiencing anxiety with regard to her achievement. The mail-box appearing on the left-hand-side and the speech bubbles encasing the repetition of the word ‘waiting’ and ‘but happy’ indicated that Hilda is waiting for her NAPLAN results. In response to the completion of the NAPLAN testing period Rob drew a stick figure with a smile and included ‘happy’ above the image.
Jenna demonstrated positive emotions with her success, however, she indicated through her selection of words that she was still ‘confused’ but ‘proud’ of her achievement. Louise and Hilda used exaggerated smiles to attract the viewers’ attention to the happy emotion portrayed in the pictures. In Rob’s final illustration regarding the delivery of NAPLAN results the stick figure re-appears. The inclusion of a thinking bubble encasing two further figures was perceived as a signal of the return of other students into Rob’s world. The appearance of the word ‘unknown’ explains that Rob had not yet viewed his results.
School B findings – Summary
In order to summarise the feelings represented in the images and the words together for School B, the combined valence of the student voice is provided in Table 4.
This valence summary highlights greater divergence of student experience in School B than in School A. Students in School B were more likely to be neutral during the preparation phase, and to represent the full range of valencies across the year levels during the participation phase. Upon completion there is almost universally a positive valence of student voice represented in the images and words. After receiving results there is a tendency towards positive and neutral valence.
The student voice data included up to three words students could use to represent the way they were feeling at each phase of the NAPLAN program. Table 5 provides a list of the words and how they have been grouped to reflect frequency and valence alignment for School B students.
Across each of the four phases, the 17 students provided a total of 165 words, 44 for phase 1, 41 for phase 2, 43 for phase 3 and 37 for phase 4, to express their feelings. The words provide additional insight into the way the students feel during the four phases of the NAPLAN journey. Students in all three year levels in School B employed words with positive valence almost as often as they used words with negative valence during phase 1 preparation. In phase 2 during the testing this pattern continued for year 3 however, year 5 students used words that shifted strongly to negative valence while year 7 students used more positive valence terms. In phase 4, when the testing was complete there remained a relatively even split in valence for the year 3 students, however the year 5 and 7 students employed positive emotional terms, especially the year 5 students. Phase 4 terms tended to be more positive for year 3 and 7 however year 5 had more negative terms being employed.
Discussion and implications
There continues to be a gap in research-based findings and discussions privileging students’ voice in relation to the implementation of NAPLAN testing, especially as it relates to their feelings and emotional response to the high stakes test regime and hence potentially the validity of the results in terms of formative and summative assessment (Messick, 1998). It is important to enable student voice in order to gain a better understanding of their experience (Cook-Sather, 2006) and for their perspectives to be recognised and valued (McLeod, 2011) and listening to and responding to students’ perceptions and reactions to NAPLAN is paramount to ensure the validity of the test results.
This study explored primary students’ feelings as they engage in the NAPLAN journey in two Queensland schools, expressed in their own voice. Engaging with students from the commencement of the NAPLAN preparation and concluding with the delivery of NAPLAN results enabled insight into their emotions as they engaged in the journey. This study did not set out to provide data that would represent all students’ experiences, but rather to feature the experiences of selected students in two different school settings.
In this study, School A and School B have adopted different approaches to the implementation of NAPLAN. School A’s Year 3, 5 and 7 students engage in NAPLAN focused curriculum and pedagogy up to and including the NAPLAN testing period, whereas School B has adopted a low-key approach to NAPLAN preparation and testing.
The combined valence of the image coding and the accompanying word summary presented in the findings for School A highlights the shift for each of the three year levels from a generally stronger negative valence during the preparation and participation stages to a stronger positive valence at completion and after students receive their results. The data from this study indicate that students at School A have developed predominantly negative perceptions of, and reactions to NAPLAN testing and preparation where the curriculum and teaching and learning experiences have been driven by efforts to optimise NAPLAN scores. The approach taken in School A reflects what the 90% of 8300 teachers in Dulfer et al. (2013) study reported as unintended consequences including negative impacts on students’ health and well-being and students feeling stressed prior to testing. Our study confirms – from the students themselves – that there is greater negative valence in their feelings during preparation reinforced through words such as: scared (9), nervous (8), worried (4) being the most frequent words used across years 3, 5 and 7. These, plus the largely negative words contribute to anxiety as the overall state the students experience during preparation.
In contrast, in School B, there is greater divergence of student experience. Students in School B were more likely to be neutral during the preparation phase, and to represent the full range of valencies across the year levels during the participation phase. Upon completion there is almost universally a positive valence of feelings represented in the images and words and after receiving results there is a tendency towards positive and neutral valence.
The differences in the responses by student in schools A and B highlight the range of ways schools respond to high-stakes testing regimes and the impact on students. Importantly, the journey for almost each of these individual students represents a roller coaster of emotions.
The aim of this research was to discover how students perceive and react to NAPLAN preparation and testing. This research opens a window that enables a view of two schools that have adopted different practices related to NAPLAN, and while these case studies are unique and cannot be used to generalise more widely, it can be speculated that there are other schools engaging in similar practices, and schools ranging everywhere along the spectrum in between and perhaps beyond. Hence, it is a reasonable assertion that NAPLAN implementation is likely to be having a significant effect on the feelings and emotions of students in our classrooms. In particular in the preparation and participation phases they are likely to have negative feelings and to feel anxious which has the probably effect of impacting on their test performance and potentially affecting the validity of the formative and summative test results. This study strongly points to the unique experience each student has in their journey through the NAPLAN process and without exception the 34 students’ images and words in this study depict a journey where they are experiencing a range of emotions.
Limitations and recommendations
Case study research conducted at two schools with different approaches to NAPLAN provided a significant quantity of comparative data. However, the scope of the study and the generalisability of its results are limited which highlights the need for more expansive studies, replicating the methods and theoretical framework in a diverse range of country-wide school contexts.
The inclusion of the Year 3 students in the study, who had no prior direct experience with NAPLAN, allowed analysis and comparison with students who had experienced NAPLAN testing previously. Tracking these students longitudinally would allow a more in-depth look at the relationship between prior NAPLAN experience and students’ perceptions and reactions to NAPLAN over time.
Although the data population included a cross-section of students, it was too difficult to focus on specific groups of students. Reducing the range of overall characters in the study and increasing the number of characters in a specific group would have enabled more in-depth analysis of a number of varieties of great interest in educated research. For example gender, culture, socio-economic status etc.
The insight gained from this study would be of assistance in modelling a state-wide or country-wide study investigating the links between NAPLAN implementation and students’ perceptions of and reactions to NAPLAN. A study with a larger data population but a narrower field, for example, 500 Year 7 students attending ten different schools across the country would allow more focussed analysis of the link between Year level (maturity) and a variety of factors. Further studies may have the opportunity to include variables not closely examined in this study, variables which may also impact students’ perceptions of and reactions to NAPLAN, such as, students who speak English as a second language, children from various family structures, Aboriginal and Torres Strait Islander students, children with diagnosed special needs, those students requiring learning support and/or behaviour adjustment, and those who are Gifted and Talented, just to name a few.
In this study the students were active participants engaging in spoken, written and illustrative dialogue. Each lens provided a rich opportunity to study educational life in schools from the perspectives of the students, what goes on behind the classroom door in the minds of the students. The most valuable lens, when seeking how students think and feel, is their natural form of symbolic expression, their drawings. The use of student drawings as a method of data collection provided a valuable vehicle to illuminate students’ perceptions of and reactions to NAPLAN.
Furthermore, this study has identified the need and recommends further research using this method of data collection into the implementation of NAPLAN and its impact on students.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.