Abstract
This paper investigates the implementation of a reading program designed for third grade elementary school classrooms in Austria. Using a mixed-methods approach, lesson types were identified, respective class compositions analyzed, and the effects on students’ reading gains examined. The results show that the lesson types seem to reflect learner group needs as students in the different lesson types profited similarly from the reading program. This supports the hypothesis that reading is a highly individualized process and leads to the conclusion that the instructional approach of the reading program investigated supports teachers in conducting reading lessons tailored to their students’ needs.
Keywords
Reading proficiency is not only a crucial part of education (Guthrie & Wigfield, 2000) it is also essential for successful participation in wider society (Cunningham & Stanovich, 1997; OECD, 2019; World Literacy Foundation, 2015). It is thus necessary that throughout the curriculum (Cain, 2012; Eskey, 2005; Scarborough, 2001), and from an early age on, clear emphasis be placed on enhancing students reading comprehension (RC). Given the increasing diversity of today’s classrooms in terms of, for example, initial reading skills, second language (L2) learners, special educational needs (SEN), and differences in background knowledge, a form of instruction capable of addressing a large variety of learner needs is required (Schuelka et al., 2019). Global developments are thus making it more important to devise instructional practices which incorporate the latest research findings. The development of suitable teaching materials and professional practice play an important role in successful implementation (Crawford & Torgesen, 2006; Hattie, 2008; 2012; Herzog-Punzenberger & Schnell, 2012; Moats, 2009a; 2009b). A growing body of research has examined different reading programs and evaluated how the implementation of program materials has affected student reading outcomes (Capin et al., 2018). However, in practice, such materials are not always implemented by the program developers themselves, but by teachers. Clearly, the instructional methods used by the latter are likely to have an impact on the implementation outcomes (Capin et al., 2018). The present paper thus investigates teacher influence on the implementation of a reading program. In particular, the paper focuses on the implementation of a specific reading program in Austria (LARS: Language and Reading Skills) in third grade classrooms with different classroom compositions and examines whether teachers’ methods respond to students’ reading abilities and thus influence student progress in reading competence. Hence, the following research questions are investigated: 1. Which types of instruction/lesson types can be identified in third grade classrooms when teachers implement the LARS program? 2. What is the relationship between classroom composition (in terms of respective shares of L2 German speakers and initial reading levels) and identified lesson type? 3. How do the different lesson types impact student progress in decoding and in reading comprehension (RC)?
Teaching and learning to read
For most children, the systematic acquisition of reading skills begins when they enter school. Within the first years of schooling, basic reading skills (including decoding, single word reading and sentence comprehension) are acquired. When this surface reading level is mastered, acquisition of deep level reading takes place allowing the learner to master increasingly complex comprehension processes. Toward the end of the primary school years, students should be able to interact with texts, connect various ideas, and draw inferences within a text. This final level is called reading for transfer, the mastery level in reading development (Fisher et al., 2016). Once students have reached this level of literacy, they successfully read to learn as they are able to make connections between and across texts (Chall, 1983; Duke & Carlisle, 2011; Fisher et al., 2016).
Reading comprehension (RC)—defined as the ability to understand the meaning of a written word, sentence, or text (Perfetti et al., 2005)—plays a central role in reaching mastery level and is closely linked to linguistic skills such as vocabulary acquisition, in particular when it comes to learning to read in a second language (August et al., 2005; August & Shanahan, 2006; Kieffer & Lesaux, 2012; Kieffer et al., 2016; Koda, 2005; National Reading Panel, 2000; Verhoeven, 2000; Verhoeven & Leeuwe, 2012). For L2 learners, vocabulary is a crucial component in deciphering text meaning, making text-based inferences, and in monitoring comprehension.
However, not only individual linguistic skills influence the development of reading skills. According to the Component Model of Reading (CMR; Aaron et al., 2008; Joshi & Aaron, 2011), both psychological (e.g., motivation, learning style) and ecological factors (e.g., classroom and home environment) also affect reading development. Thus, with respect to the ecological component in the CMR, teachers’ instructional practices as well as the teaching materials used, among other things, affect progress in reading development (Beerwinkle et al., 2018).
The significant impact of instruction on learning progress in general has been emphasized by Hattie (2009), and the impact of how reading is taught has been dealt with by numerous researchers (e.g., Capin et al., 2018; Dehaene, 2009; Snow & Juel, 2005). To date, the growing body of intervention research has not only investigated what works for which learner group but also how one may best obtain knowledge concerning what works in the classroom, with the emphasis often being placed on evidence-based reading instruction (Castles et al., 2018; Lovett, 2015; Treiman, 2018). Based on this body of research it is now well known that expanding children’s vocabulary and conveying structural and text-related comprehension strategies are the two main pillars in fostering RC (Fisher et al., 2016; Foorman & Torgesen, 2001; McNamara et al., 2007; Moats, 2009). Vocabulary is most effectively taught when direct and indirect learning situations are combined. While direct or explicit vocabulary instruction, which is known to enhance RC, includes the pre-instruction of word meanings, keyword exercises and repeated multiple readings (National Reading Panel, 2000), indirect vocabulary learning occurs when words are encountered in context, such as in conversations, read alouds or individual reading (Sénéchal et al., 2006).
Teaching reading strategies has also been shown to increase students’ RC, in particular, text comprehension (Guthrie et al., 1999; National Reading Panel, 2000), which is of particular relevance from later elementary grades on. Reading strategies are divided into pre-reading, while-reading and post-reading strategies. While good readers have broader background knowledge and richer vocabulary, and learn how to orchestrate these strategies by themselves, struggling readers require instruction that helps them develop these skills (Marzola, 2005). For example, the pre-reading strategy of activating background knowledge on a text topic in order to support RC is well acknowledged (Dole et al., 1991; McNamara et al., 2007; Tarchi, 2015). A different pre-reading strategy, the clarification of difficult or unknown words, that is, a vocabulary-focused strategy, has also been widely recognized as a strategy for supporting RC (Carlisle & Katz, 2005).
Another factor to consider is individual student needs and the tailoring of intervention measures. An increasing body of research has stressed the importance of differentiated instruction in order to allow for equal learning opportunites for all students (e.g., Ankrum et al., 2017; Fahey, 2000; Förster et al., 2018; Lewis & Batts, 2005; Pentimonti et al., 2017; Rock et al., 2008; Shaunessy-Dedrick et al., 2015; Subban & Round, 2015; Suprayogi et al., 2017) since there can be significant differences in reading levels in one and the same classroom (Firmender et al., 2013). Differentiation means modifying teaching and learning routines, curricula, teaching methods, resources, learning activities and/or student products in a way that allows students to learn at their own pace and with respect to their own “zone of proximal development” (Tomlinson, 2000; Tomlinson et al., 2003). This requires teachers to constantly monitor student progress (e.g., by using national assessment resports) and make informed decisions on how to design meaningful, evidence-based instruction (Tomlinson, 2000). This also entails offering some form of “scaffolding” to individual learners and learner groups. Scaffolding is a support technique to help students develop the skills needed in order to complete a particular RC task independently. For example, the teacher asks specific questions which serve to guide students’ problem solving (Fisher & Frey, 2014; Gibbons, 2015; Regan & Berkeley, 2012; Reynolds & Shannon, 2017; Van de Pol et al., 2010). It can thus be considered an important element of effective differentiated instruction (Reynolds & Shannon, 2017). In addition to instruction, teaching materials should also be developed based on evidence, and differentiated in order to cater for diverse learner needs (Lovett, 2015; Reid, 2012). Using differentiated texts adapted to student reading levels enables all students to work on the same topic, to receive the same basic information and to participate in an inclusive lesson (Altrichter et al., 2009; Paleczek et al., 2020). Thus, all students develop content area knowledge through while-reading and post-reading activities (e.g., discussions about the text) irrespective of their reading level.
Differentiated instruction with the LARS program
One reading program which is in line with current research findings is the language and reading program LARS (Paleczek et al., 2020; Seifert et al., 2014). It was developed with a specific focus on classrooms with a high number of L2 learners. The program was evaluated from 2012 to 2015 in second and third grade classrooms in Austria. The reading program LARS provides materials for the development of two key components of reading: (1) materials for vocabulary development and (2) materials for RC training. The materials are closely related to the content area subjects of the Austrian elementary school curriculum and thus allow content area teaching and literacy development to be combined. Furthermore, the close link to the curriculum was necessary since the materials were designed for implementation during regular German and/or social science lessons.
The study presented in this paper focuses on the LARS reading materials for Grade 3, placing emphasis on the second pillar, the materials for reading development. Information on the full scope of the program can be found in Paleczek et al. (2020) and Seifert et al. (2014). Information on vocabulary work in reading lessons according to the program can be found in Seifert et al. (2017).
Topics and number of expository and narrative texts in the LARS program.
Symbols and Percentile ranks according to the four reading levels.
Based on the reading level, the quantity (text length) and quality (complexity) of the reading texts differ. The lower the reading level, the shorter and linguistically easier the texts become. Text length was determined by taking the number of words read per minute by Grade 3 students (SLRT II: Moll & Landerl, 2010) into account. Based on this information, the appropriate text length for approximately 15 minutes of reading time was calculated for each reading level. Since reading speed increases, text length (number of words) also increases over the academic year. Linguistically, the texts differ in syntactic complexity (e.g., number of subordinate clauses and relative clauses) with level 4 employing the syntactically simplest texts. Additionally, the texts differ in terms of typeface, layout, and font size. For example, for level 4 readers, larger type is used, and each sentence is presented in a single line.
Each differentiated reading text is accompanied by several RC tasks. These tasks are designed to check for text comprehension. Once again, the comprehension tasks differ according to reading level in terms of quantity (e.g., sentence and word length, number of tasks and subtasks) and quality (e.g., structural complexity of sentences).
Implementation of the LARS material requires establishing a reciprocal relationship between vocabulary development, RC development and content area learning. While differentiation and RC tasks are taken care of by the LARS materials, other aspects such as establishing a meaningful connection to content area learning, allowing for familiarity with content area vocabulary, or reading instruction, all lie within the responsibility of the teacher planning the LARS lessons. Hence, teachers need specific knowledge and understanding of the LARS program if it is to be implemented successfully in their classrooms (Kulmhofer-Bommer, 2018). Roughly speaking, a LARS reading lesson should cover the following steps (Figure 1): Lesson framework for LARS reading lessons (adapted from Kulmhofer-Bommer, 2018).
A lesson with the LARS reading materials can be divided into four phases: the first and the last of these being teacher-guided, while in the second and the third phase, students work mostly independently with the materials provided by the program. All four phases are to be completed during regular class time. The initial phase (Phase 1) of the lesson addresses topic-relevant vocabulary and is also intended to activate respective background knowledge in the students by means of appropriate pre-reading activities (reading strategy prior to reading). In Phase 2 (the core phase, since it relies most on the LARS material), students usually work individually on the texts and the corresponding RC tasks for their reading level, providing the teacher with the opportunity to support individual students or groups of students with scaffolding practices (e.g., asking guiding questions to help a student with comprehension questions). The third phase in a LARS reading lesson concerns feedback and correction. This phase strengthens student responsibility for their own learning, with students engaging in self-correction of their responses to the RC tasks. The teacher can also help with correction. The final phase (Phase 4), the post reading phase, is, like the first phase, very teacher-guided. Activities such as summarizing, discussing text content, and making the connection between the text and content area learning explicit to the students, ensure that students have understood the text and have extracted the same information. This fosters content area learning and facilitates the development of content-relevant vocabulary through a process of revision.
Evaluations of the LARS program
In total, three studies were conducted to evaluate the program in Grade 2 and 3 classrooms, whereby two of the studies were used to evaluate the LARS program in Grade 2. Study 1 (academic year 2011/2012) was the pilot study with second grade students (n = 55). The program was implemented over a period of 3 months. In the pilot study, a member of the project team was responsible for program implementation. The project team assessed decoding, RC and language skills before and after the intervention and compared the results to a control group (n = 50 students receiving regular reading instruction with no particular focus on vocabulary and differentiation). Following the intervention, the LARS group showed higher gains in both decoding and RC compared to the control group (Schwab et al., 2014). In Study 2 (academic year 2012/2013), the LARS program was implemented in second grade (n = 159 students) and covered almost the whole academic year (October to May). In this study, teachers were largely responsible for implementing the program but received a lot of support from the project team. This meant that in the first 2 months, a project member introduced the material to the students and was primarily responsible for classroom implementation in order to ensure that similar conditions prevailed during implementation in all LARS classrooms. After 2 months, the implementation was handed over to the teachers, although the project member continued to visit the classes monthly to provide teacher support. Again, the research group assessed decoding, RC and language skills before and after the intervention and compared the results to a control group (n = 218). The results revealed higher gains for the LARS group in sentence comprehension (Seifert et al., 2015).
Following implementation in Grade 2, the program was then implemented and evaluated in Grade 3 classrooms. In Study 3 (academic year 2013/2014), the LARS program was implemented in third grade with a new cohort (n = 345 students). Implementation monitoring was reduced, and teachers were fully responsible for program implementation. In order to support teachers in getting started with the program, they received training and model lesson plans. Again, decoding, RC and language skills of the 345 students were assessed before and after the intervention and compared to a control group (n = 275). No significant effects could be identified for third grade students (Gasteiger-Klicpera et al., 2015). Various factors are likely to have contributed to this outcome. One explanation for the decline in intervention impact over the three studies is probably related to the decrease in the level of researcher monitoring of implementation. Lower monitoring could have led to lower implementation fidelity (i.e., the degree to which an intervention is delivered as intended) from Study 1 to 3 as teachers’ decision-making power increased. In the research field of intervention studies, implementation fidelity is given a crucial role when analyzing intervention outcomes (e.g., Carroll et al., 2007; Century et al., 2010). However, teacher knowledge of their students must not be underestimated and might lead to deviations in the implementation from the researchers’ plans. This provides the focal point for the present paper which takes data from classroom observations and relates them to the student performance data obtained during pre- and post-assessments. This is done in order to gain a better understanding of (1) what exactly teachers did in Study 3 when implementing the LARS program in their reading lessons, (2) whether the different uses were specifically adapted to the composition of the students in the class, and (3) whether different uses of the materials led to different outcomes.
Method
The present study employs a mixed-methods approach to investigating the implementation with respect to LARS reading materials. The research design uses qualitative data to describe the reading lessons conducted with the LARS reading materials, and longitudinal quantitative data to discuss students’ reading performance in relation to the qualitative data.
Context
Over a period of 8 months, all the children of the intervention group received two LARS lessons per week. The regular classroom teacher implemented the program after first having participated in specific teacher training introducing them to the program (pre-treatment) and after having observed a project member implement the program in one reading lesson at the beginning of the project. For the purpose of evaluating the implementation of the program, students’ reading abilities were tested before and after the implementation (quantitative data). To gain more insight into how teachers implemented the materials, we observed LARS reading lessons between March and May 2014 (qualitative data). By this point in time, the participating teachers had already had more than 4 months of experience in planning and conducting lessons with the LARS materials. Participation in the observation study was voluntary. Thus, for the present study, only the quantitative data obtained from those Grade 3 classrooms participating in the observation study were analyzed. Besides, quantitative data was only analyzed in relation to the qualitative data (for general intervention outcomes, see Seifert et al., 2015).
Participants
The 13 classrooms analyzed were located in six different elementary schools in the Austrian province of Styria. Sample selection was based on volunteer sampling (Cohen et al., 2007). This meant that no controls were exercised concerning teacher experience, training, or expertise in teaching reading or teaching L2 learners.
In total, 278 students (48.9% girls; 46.4% L2 learners) participated in the study. Their average age was 8.79 years (SD = 0.47).
Procedure
Data were collected at three points in time: The quantitative data on reading abilities was collected at the beginning of October 2013 (pre-test, Time 1) and at the end of June 2014 (post-test, Time 2), after the implementation of the material. To obtain the qualitative data, classroom observations took place between March and May 2014. Carrying out observations in the middle of the school year ensured that teachers had already developed a routine in implementing the LARS materials so that lack of knowledge or of familiarity could be ruled out as a source of data distortion. Furthermore, the observers explicitly requested to see a lesson that reflects the general use of the material over the year, aiming at getting an idea of the average implementation procedure in each classroom.
Each participating classroom was observed once for 50 minutes (one instructional hour) during the implementation of the LARS reading materials. A specifically developed observation protocol was used (see Instruments section for further detail). In total, 13 observations in 13 different classrooms took place. Since the aim was to see as many teachers as possible implement the program in as many different classrooms as possible, the decision was made to go into each classroom once instead of visiting one classroom several times.
The observations were conducted by two observers, with one observer (observer 1) acting as instrument developer. In order to ensure that both observers interpreted various categories and aspects in the same way, observer training was carried out prior to the observation period with videos of vocabulary and reading lessons using other programs. The observers first watched the videos independently and tried to apply the observation tool. Their experiences where then discussed, and the tool was improved to ensure both observers could work with it. Additionally, the first live observation of a reading lesson was carried out by both observers in the same classroom. Based on the data recorded in this first session, inter-observer agreement for the observation tool for reading lessons was calculated and identified as being moderate (к = 0.45; Landis & Koch, 1977). Most of the discrepancies occurred in the open-ended field notes. Here, observers recorded the tasks and activities of teachers and students, estimated the number of students on-task (5 categories; more than 75%, 75%, 50%, 25%, less than 25%) during the different lesson phases and the time for when different parts of the lesson started and ended. By excluding the notes concerning times and reducing the number of categories for students on-task to 3 (75% or more; 50%, 25% or less), it was possible to increase rater agreement to к = 0.57 (which still represents moderate agreement, Landis & Koch, 1977). In addition, to improve shared understanding of categories in future observation periods, discrepancies in the field notes regarding the tasks and activities of teachers and students were also discussed. Minor changes were then made to the open-ended field notes after retrospective discussion of the lesson. Following these adjustments, rater agreement level rose to substantial agreement (к = 0.75; Landis & Koch, 1977). During the observation study, Observer one observed five reading lessons, and Observer two observed nine reading lessons.
Instruments
At Time 1, overall cognitive abilities were assessed using the Cultural Fair Intelligence Test (Cattell et al., 1997). Retest reliability for the five subtests ranges from r tt = .65 to r tt = .86. Additionally, information on student first language and age was gathered with teacher questionnaires.
In order to assess decoding (at Time 1 and 2), the reading subtest of the Salzburg reading and writing test (SLRT II: Moll & Landerl, 2010) was used. This test is an individual reading test that assesses decoding using two subscales: the non-words subscale (assessing non-lexical decoding) and the words subscale (assessing lexical decoding). Non-words and words presented in a list are read out by the students in a period of 1 minute. The raw scores thus obtained represent the number of correctly read words or non-words per minute. According to the test manual, the retest reliability is high (r tt = .90–.98).
Reading comprehension (at Time 1 and 2) was assessed using the Reading Comprehension Test for First to Sixth Graders (ELFE 1–6: Lenhard & Schneider, 2006). The ELFE 1–6 assesses RC at the word, sentence, and text level. At word level, children need to choose one out of four nouns representing a given picture. Raw scores represent the number of correct matches out of 72 possible matches within 3 minutes. At sentence level, students choose one out of five words of different word classes as best fitting in a sentence context. Raw scores represent the number of correct matches out of 28 possible matches identified within 3 minutes. At text level, a short text of one to eight sentences is presented along with one or two comprehension questions. The children have to choose one out of four answer options related to the questions. Raw scores represent the number of correct answers out of a possible 20 items identified within 7 minutes. The internal consistency of the subscales word, sentence, and text level ranges from α = .92 to .97.
All tests were administered by the research team for the purpose of investigating the implementation of the LARS reading program and are thus not standard procedure when implementing the program.
Observational data was collected using an observation tool designed specifically for the purpose of gathering information on the implementation of the reading material (learning tasks and activities) and on other elements related to quality of instruction (e.g., classroom climate, classroom management, feedback: Pianta & Hamre, 2009; Pianta et al., 2008). This tool is based on a careful study of observation tools used in the context of observing literacy instruction, reading instruction and teaching non-native speakers the language of instruction (Haager et al., 2003; Henk et al., 2000; Hoffman et al., 2011; Pianta & Hamre, 2009; Vaughn & Briggs, 2003). The observation tool combines open-ended field notes with dichotomous single-choice items, multiple-choice items, and checklists (Kulmhofer-Bommer, 2018). The focus of this study was on lesson phases, teacher and student behavior, tasks and activities, and student time on-task. Since a detailed description of the tool is beyond the scope of the present paper, Table 3 only shows the aspects relevant for this paper covered by the observation tool (for a detailed description of the observation tool see Kulmhofer-Bommer, 2018). Overview of observed instructional aspects.
Figure 2 gives an overview of the instruments and the research design.
Overview of instruments and research design.
Data analysis
The observational data was analyzed by the instrument developer using thematic analysis (Braun & Clarke, 2006) and qualitative content analysis (Kuckartz, 2014). This was done by assigning codes to the data using the software Atlas.ti. The coding process was theory driven and highly iterative, leading to coding, recoding, and grouping of codes in relation to current research on effective reading and literacy instruction. It followed the six-step model as suggested by Braun and Clarke (2013). In total, 51 codes in 10 code groups were assigned to the whole data set. The codes were primarily assigned to the open field notes on teacher and student activities and tasks (e.g., direct introduction of topic, revision of vocabulary words, distribution of materials, matching words with part of speech), and to the information on the lesson phase as assigned by the observer (e.g., pre-reading phase, individual reading phase, feedback and correction phase, post-reading phase).
Activity patterns and instructional patterns with examples.
aOnly code groups relevant for the study at hand are shown.
In a subsequent step, the identified patterns were further analyzed with regard to the lesson phase they occurred in. Based on the LARS reading lesson framework (see Figure 1), four phases were identified: (1) a pre-reading phase, (2) an individual reading phase, (3) a feedback and correction phase and (4) a post-reading phase (see also Figure 1). Based on the assumption that the pre-reading phase shapes the whole lesson, the patterns found in this phase (e.g., language awareness activities, word meaning activities, topic-related activities) were then further analyzed in order to cluster the observed lessons into different lesson types.
The quantitative data set only included data from students in those classrooms where observations took place. In order to analyze for differences in the sample composition of the classrooms according to the identified lesson types, chi-square tests (in respect of L1 and L2 learner classroom composition) and a Kruskal–Wallis test (in respect of composition of the initial reading ability level) were conducted. To compare students’ achievements considering the observed lesson types, analyses of variance with repeated measures were conducted. As dependent variables, the achievement of the students (decoding: lexical and non-lexical decoding; RC: word, sentence, and text comprehension) were used as raw scores in the analyses. The two decoding subscales and the three RC subscales were analyzed with multivariate analyses. The lesson type was used as independent variable. The cognitive abilities (IQ-scores) were used as covariates in the analyses.
Results
Lesson types identified in third grade classrooms implementing the LARS program
Overall, the observational data showed that the LARS program was not fully implemented as expected. A pre-reading phase (Phase 1) followed by a phase of individual reading (Phase 2), followed by a phase of feedback and correction (Phase 3) was observed in all lessons and implemented as intended. In Phase 2, the students silently read specific texts matched to their individual reading level and completed the RC tasks. The teacher’s role as observer and facilitator was to support mainly those students who needed help. The support ranged from scaffolding the RC tasks to support in handling the materials. In Phase 3, teachers or students themselves corrected the RC tasks and teachers provided feedback. Phase 4 should then conclude the work with the LARS materials with activities such as whole-class discussions about the content of the texts. Post-reading activities (Phase 4), however, were only observed in 6 of 13 classrooms.
The observed lessons were then further analyzed to gain a better understanding of similarities and differences between the lessons. The analysis showed that the activity patterns in Phase 1 set the overall scene for the lessons. Therefore, based on the observed activity patterns in phase 1 the following three lesson types were identified (see Figure 3). Overview of the three lesson types.
Lesson Type 1 is characterized by a particular focus on activating students’ vocabulary knowledge during the opening phase by providing activities addressing word meaning and language awareness. After the individual silent seat work (in Phase 2), the lessons concluded with correction of and feedback on the RC tasks (self-correction and teacher-correction, Phase 3). No post-reading phase (Phase 4) was observed.
The following sample lesson sequence serves to illustrate Lesson Type 1: Teacher announces the LARS lesson. Before handing out the texts, however, 10 words that appear in the text are discussed together. First, the written words are linked to the corresponding pictures on the board. Then the articles of the nouns and the inflected form of the verbs are added, and individual children are allowed to form a sentence with the words. After 15 minutes, the reading-level matched texts are handed out to the children accordingly. The children read the text silently and work on the tasks while the teacher supports those children who need help. Afterward, when the children have finished, they can correct themselves using a solution sheet and ask the teacher again if necessary. Then, the students begin to work on individual tasks that have nothing to do with the LARS lesson.
Type 2 lessons (topic-focused lessons) were constructed around activities addressing the topic of the texts to be read (e.g., guessing the topic by title or looking at topic-related pictures) and aimed at activating students background knowledge. The individual reading phase (Phase 2) and feedback and correction phase (Phase 3) were followed by whole-class post-reading activities such as text-related discussion and games addressing text content and key vocabulary (Phase 4). Particularly, text-related discussions and games strengthened the link between reading and content area learning.
The following sample lesson sequence serves to illustrate Lesson Type 2: Teacher announces LARS lesson. First, she shows three pictures that are thematically related to the text and asks the children what they thought the text could be about. After the children had expressed their first ideas, the teacher specifically names the topic of the text. She also asks what the children already know about this topic, which leads to a brief whole group discussion. After this introduction, which lasts about 10 minutes, the teacher hands out the respective texts that match their ability level to the children. The students work independently and complete the tasks. The teacher supports those children who need a little more help. When the students have finished, they go to the teacher with their sheet and the teacher looks over it briefly. Then, in pairs or threes, they choose a game from a selection that consolidates the words used in the text (e.g., memory with the words at word or sentence level, word puzzle, board game with quiz cards on the words). Once all students have completed the RC tasks and corrected them, the lesson concludes with a brief discussion about text content (e.g., students report what they remembered from the text or what was knew information for them).
In Lesson Type 3, the only activity observed during the pre-reading phase (Phase 1) concerned classroom management tasks related to distribution of materials. Thus, after materials were distributed, teachers in this lesson type began directly with the individual reading phase (Phase 2) of the lesson. Consequently, students in these lessons had more time for individual reading (Phase 2). Hence, Lesson Type 3 focuses on individual reading. The lesson terminates with self- and teacher-correction as well as feedback but omits a concluding phase.
Figure 3 provides an overview of the three lesson types and illustrates similarities and differences between them.
In sum, the observations showed that in Type 1 and Type 3 lessons only three out of the four phases of the lesson framework for LARS reading lessons were implemented. Lesson Types 1 and 3 lacked the post-reading phase (Phase 4), which is the lesson phase ensuring content area learning for all students. Lesson Type 3 also lacked pre-reading activities (Phase 1), that is, activities needed to activate background knowledge and/or content-related vocabulary, and thus facilitate the RC process. Only Lesson Type 2 implemented the four phases as intended by the LARS framework for reading lessons (see Figure 1).
As far as the frequencies of the lesson types in the classrooms are concerned, Type 1 lessons (vocabulary-focused lessons) were identified in four classrooms (98 students, or 35.3%), Type 2 lessons (topic-focused lessons) in six classrooms (117 students, or 42.1%) and Type 3 lessons (reading-focused lessons) in three classrooms (63 students, or 22.7%).
Differences in class composition across the lesson types: proportions of L2 learners and reading levels
Information on the sample composition in terms of size, language (German as L2), and initial reading level at Time 1 in the three lesson types.
aPR = percentile rank.
bThese students were absent at Time 1, thus, reading levels were assigned by the teacher and were changed during intervention if deemed necessary.
In order to examine whether one of the individual reading level groups (Level 1 to Level 4) was more likely to appear in one of the three lesson types, a Kruskal–Wallis test was performed. No significant differences (X 2 (2266) = 4.33, p = .115) between the three lesson types could be identified. Thus, the children in the lesson types were equally balanced in terms of initial reading levels (cf. Table 5).
Effects on reading achievement depending on the lesson type
The MANCOVA with repeated measures (IQ was controlled) showed, in terms of decoding, a significant main effect for time (F(1253) = 14.89, p < .001), but no significant interaction between time and lesson type (F(2253) = 2.24, p = .108).
Means and standard deviations for Time 1 and Time 2 in reading abilities as well as cognitive abilities for Time 1 of the three lesson types.
aassessed as control variable only at Time 1; RC = reading comprehension.
Discussion and Implications
This study investigated the implementation of the reading program LARS using a mixed-methods approach. In particular, the instructional methods used by teachers, which are known to impact implementation outcomes (Capin et al., 2018), were first observed, then examined, and finally, related to quantitative data on students’ gains in decoding and RC after the implementation period of one school year. The results of the observation study (qualitative data) allowed three different ways of designing lessons with the LARS materials to be identified: (1) lessons with a focus on vocabulary, (2) lessons with a focus on the topic and (3) lessons with a focus on individual reading. Lesson Types 1 and 2 involved two important pre-reading strategies (clarification of unknown words: Carlisle & Katz, 2005, and activation of background knowledge: Dole et al., 1991; Tarchi, 2015). These strategies are widely acknowledged as key to fostering RC as they facilitate reader access to a text. Differences between the identified three lesson types were found, not only in terms of the different foci set by the teacher, but also in terms of the presence or omission of the four lesson phases foreseen in the LARS program, namely: (1) the pre-reading phase, (2) the individual reading phase, (3) the feedback and correction phase, and (4) the post-reading phase. As expected, since the LARS program explicitly provides materials for the second phase (individual reading phase), this phase was observed in all three lesson types. Here, students used one of the four differentiated reading materials, matched to their own specific reading level, thus allowing them to work individually with the texts and RC tasks. However, Lesson Type 1 and Lesson Type 3 terminate with feedback and correction of the RC tasks and lack post-reading activities and thus fail to provide opportunities for integrating newly encountered information with prior knowledge (Kintsch, 1998), organizing or synthesizing information learned from the text (McNamara et al., 2007), or constructing meaning through classroom discussions (Garas-York & Almasi, 2017). In effect, this means that inadequate use is made of texts as vehicles for fostering content area learning. Lesson Type 3 also skipped the pre-reading phase by directly going from distributing materials to individual reading. Only Lesson Type 2 (topic-focused lessons) implemented the four steps as intended in the LARS framework for reading lessons (see Figure 1 and Figure 3).
Taking this in conjunction with the information gained concerning class composition, it can be hypothesized that teachers adapt how they implement the material to the general ability level of their students. For instance, Lesson Type 2 was observed in classrooms with a high proportion of average and above average students (62.4%). For these students the topic was at the center of the lesson, thus allowing reading to be used as a vehicle for learning. Lesson Type 1, in contrast, was observed in classrooms with a high proportion of L2 learners (62.2%). The teachers seemed to have made a conscious decision with regard to their learners’ needs when focusing on revising and activating topic-related key vocabulary before students started to work on the reading texts. In these classrooms, students profit from pre-instructions of word meanings and repeated multiple readings of the words, as recommended by the National Reading Panel (2000). With Lesson Type 3, an emphasis on reading time was observed. The teachers who implemented the LARS materials in Lesson Type 3 seemed to have placed greater emphasis on increasing reading time since in their classrooms, a high proportion of average and below average readers was found (69.9%). This leads to the conclusion that a reading program providing differentiated reading materials and scientific support on how to group students according to skill level offers teachers the opportunity to focus more on what their students need in order to make the texts accessible to them. Earlier findings on reading instruction in Austria showed that teachers usually do not base their choice of instructional methods or programs on their knowledge about learners’ reading abilities and needs (Bachinger et al., 2019; Oesterbauer et al., 2020). However, in these studies, in contrast to the study at hand, teachers were not scientifically supported in their decision making by pre-testing students’ reading skills. Obviously, if teachers receive support by specialists and are provided with evidence-based materials, which was done in the present study, their instructional choices and lesson planning seem more in line with individual learner needs, showing their knowledge and awareness concerning individual student needs.
According to the data, the teachers observed were able to design lessons that, at least partly, fitted their students’ needs. As a result, the development curves for all three lesson types were similar, that is, irrespective of the lesson type, all students profited similarly from the material in terms of growth in reading abilities. In order to learn more about the LARS program, the extent to which improvements in other areas (e.g., vocabulary, content area learning) also vary depending on lesson set-up, is to be investigated in a follow-up project. Additionally, more research needs to be conducted regarding the process of lesson planning with the LARS program in order to better understand the interaction between teaching strategies, the needs of the students and learning outcomes.
Nonetheless, the results of this study illustrated the importance of equipping teachers with the tools and knowledge needed to enable them to provide effective reading instruction. This entails the use of differentiated reading materials allowing students of all reading levels to work simultaneously on the same topic, but also the understanding for evidence-based lesson planning (e.g., Förster et al., 2018). For evidence-based lesson planning, teachers, but also material developers need to acknowledge the importance of assessment as a supporting tool for learning processes. For the material developers it seems crucial to illustrate how a specific reading program is linked to—or rather its implementation is based on—national or international reading assessments so that teachers are supported in translating assessment results into classroom practice.
The results of the present study also emphasize the importance of (peer) observation—as research on lesson studies (e.g., Cunningham et al., 2015) has already shown—to increase teacher effectiveness when planning lessons (e.g., Willems et al., 2019). Additionally, the implementation of reading materials might require support by the respective researchers/program developers and training of the teachers who implement the program into their classrooms. Particularly, informative training sessions outlining how to plan lessons with and around a particular program seem essential in order to increase procedural fidelity of implementation which most likely impacts learning outcome (Durlak & DuPre, 2008; Gresham et al., 2000; Stahmer & Gist, 2001).
Furthermore, the results also illustrate that for reading programs such as LARS, a one-size-fits all approach will not be enough to address individual learner needs. It is thus more important for teachers to have evidence on students’ performance levels, an understanding for how effective reading instruction works and knowledge about what the material they want to use entails to enable a highly individualized form of implementation in response to teacher and student needs. This is particularly important when it comes to teaching diverse classrooms where L1 as well as L2 learners are taught together. Particularly for L2 learners, it is crucial to collect evidence about their strengths and weaknesses in order to best cater for their needs. By focusing on vocabulary acquisition as a prerequisite for RC, teachers can ease RC for L2 learners and might prevent a Mathew effect (Stanovich, 1986), which has often been reported in studies on reading abilities.
Limitations and Outlook
The study at hand clearly has specific limitations and results should therefore be interpreted with caution. First, the results of the observational data are based on direct classroom observations of only one reading lesson per classroom. If analytical accuracy is to be improved, then observations need to be carried out in more than one reading lesson per classroom, and the lessons should be videorecorded. No conclusions can be drawn whether the one observed lesson type is really typical for a teacher or not. It may even be the case that one teacher produced more than one lesson type over the course of the year. Additionally, teachers’ accounts and opinions regarding representativeness of the observed lessons for their teaching with the LARS materials are missing. They would have, however, added valuable data to be taken into consideration for analysis and discussion.
One also needs to remember that the LARS lessons were only implemented twice a week. No information on teacher reading activities or subject matter activities in other lessons was collected. In Austria, the national curriculum for German, published by the Austrian Federal Ministry of Education, BMBWF (BMBWF, 2003), calls for five German language lessons per week, whereby reading is one of the primary foci in Grade 3 classrooms. It is the principle decree of reading education in the upper elementary grades that reading instruction has to be cross-curricular, and is thus relevant for all subjects taught in school (BMBWF, 2013), making it necessary to combine reading instruction and content area teaching. Undoubtedly, these activities also influenced the development of student reading skills. Future studies need to take this into account and to examine the impact of all different reading activities in classrooms. This will help provide insight into the specific effects of various reading programs, and more importantly, it will also provide a clearer picture of which teacher-directed measures work best for which children.
Footnotes
Acknowledgments
We would like to thank the Styrian government (department of education) for supporting the research project financially. We are grateful to Susanne Schwab, Norbert Holzer, Kerstin Waldmüller, Petra Watko, Kathrin Ortner, Barbara Vormaier, Mike Trautschnig, and all the research interns for their assistance during the whole project, especially with respect to the preparation of the reading material and data collection. We also acknowledge the financial support by the University of Graz.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
