What Is Going on During Vocabulary Lessons?

Abstract

It has been 9 years since the Congressionally appointed National Reading Panel made recommendations for literacy instruction that comprise a five-component framework of phonological awareness, phonics, fluency, vocabulary, and comprehension. Vocabulary, a critical pillar of literacy, has reciprocal and correlational relationships with reading achievement. The researchers piloted an observational instrument to determine the methods and materials K-3 teachers use to teach vocabulary in today’s classrooms. This brief evaluates a vocabulary observation tool the researchers developed to gather information from early childhood classroom settings in the midsouth region of the United States. Understanding materials utilized in various contexts will enable practitioners and researchers to address the significant disparity between vocabulary “haves and have-nots.” An examination of the instrument was conducted (n = 18 raters at 3 ratings apiece for 45 trials) to determine reliability and validity of observations. Reliability was addressed via training with discussion and resolution of ratings from video of vocabulary instruction. Validity was analyzed via multidimensional scaling (MDS) to visually portray ratings along the dimensions of student or teacher control. From this data, we were able to determine the number of possible senses (auditory, visual, kinesthetic, smell, and taste) students used. Results indicated observer ratings (n = 45) clustered or separated material-type consistently indicating variance along both dimensions. The researchers are currently applying this piloted instrument in a large-scale study to depict teachers’ vocabulary material use. Understanding vocabulary materials and contexts of their use may lead to more effective vocabulary curriculum, instruction, and assessment.

Comprehension is the overall goal of reading and as such, researchers and practitioners wrestle with how to facilitate students’ routes to understanding what they have read. This is not an easy task given the complexities of reading where students are required to process a symbol system rapidly enough to identify and hold written words in memory as they glean meaning from connected text (Smith, 2004). Rasinski (2003) and Samuels (2002) similarly conveyed that reading comprehension requires fluency, the rate and degree of expression, to overcome an information “bottleneck” caused by the interference of a lack of word recognition and word meaning. Slower reading taxes the physiological limitations of short-term memory and this detracts from comprehension. For years, researchers have documented the correlational relationship of vocabulary knowledge to reading comprehension and to overall academic success (Baumann, Kame’enui, & Ash, 2003; Becker, 1977; Davis, 1942). Early word exposure varies across socioeconomic levels. Hart and Risley (1995) and Snow (1991) found that language development prior to entering school either empowers children with language or sends them off into the world with a word deficit that is hard to overcome.

Vocabulary Learning and Instruction: Contexts and Materials

When they enter this world, infants immediately begin to learn the words of their language through exposure. They initially take in entire streams of sound and later begin to distinguish words and later their individual phonemes (Lyon, 1995). Around 5 months of age, infants begin to communicate through babbling to those around them. Risley and Hart (2006) call talk between caretaker and child, “the social dance of language” where infants and children receive and learn words through chitchat, purposeful talk, storytelling, book reading, songs, conversation, and other forms of oral language that take place in the child’s environment. These early experiences are the first steps toward vocabulary building. Parents and caregivers not only engage their children through indirect exposure to words but also through purposeful direct vocabulary teaching where they model pronunciation and demonstrate the meaning of words with objects.

Objects in the child’s environment form a natural context that is integral to the early learning experience. For example, when caregivers give milk to the infant or young child, he or she gains a deep and lasting knowledge of the meaning of the word “milk” through aural, visual, and tactile sensory input. This process, that aligns words with their concrete referents, is repeated hundreds of times and results in the learning of 2,500 to 6,000 words before children even enter school (Wren, 2006; Beck & McKeown, 1991; Chall, 1996).

Vocabulary instruction in the formal institution of school is the next step for word learning. In school, an immediate purpose and the environmental contexts that supported earlier word learning are diminished or not present (Pinker, 1994). Systematic vocabulary instruction of targeted words stands in stark contrast to earlier word learning that arose naturally in multiple contexts in children’s home environments and was centered on their needs and interests. In Holmes and Canterbury’s (2012) review of core reading teacher’s manuals, they found that the recommended strategies for involving students in multiple contexts almost always were through multiple written contexts. Recommendations to continue the earlier concrete materials-based contexts to teach vocabulary were almost nonexistent. School-age children, who initially learned many of their words through interactions with concrete materials, now use flat two-dimensional materials, written words, and pictures to learn word meanings. We must clarify, however, that our search consisted primarily of scanning the research literature base addressing vocabulary acquisition.

Looking through the lens of cognitive psychology, the proximal and distal relationship between the materials and the students may play a vital role in the quality of the contexts for word learning (Dunn, Beach, & Kontos, 1994). Proximal materials are those in, or closest to, the students and their environment thus enabling them to engage with actual representations of words rather than abstract definitions and remote descriptions. Their authenticity makes it easier for students to move beyond simple word definitions to a deeper level where they can examine, analyze, and categorize attributes of the word’s meaning. Because context is critical to word meaning, the authors sought to explore the use of proximal contexts in K-3 classrooms where students can interact with materials to determine nuances of meaning through guided or discovery exploration of the word’s referent. According to Pearson, Hiebert, and Kamil (2007) “the more contexts in which a word is encountered, the greater the likelihood that its meaning will be acquired” (p. 290).

The need to document the materials teachers use to teach the meaning of words arose when Holmes noted through classroom observations that student teachers rarely used concrete materials to teach new vocabulary. A commonly taught lesson on the stages of the butterfly often contained only oral and written descriptions of words such as pupa and chrysalis. With just a little imagination and effort, these lessons could have included authentic materials, or at least their representations such as models, that would have given students a more in-depth and memorable experience with the words they were learning. To determine whether teachers in K-3 classrooms use authentic materials to teach vocabulary, the authors developed the Vocabulary Observation Instrument (VOI; see Figure 1.). This article reports on the results of a pilot study that confirmed the reliability and validity of VOI. From the data collected through the use of VOI, the researchers are looking to develop a theory of practice to lend specificity to the term multiple contexts by labeling the contexts according to their proximal and distal relationships to students. To test this instrument, 15 observers piloted it during three different vocabulary lessons (45 trials) to determine the ease of use and whether the categories were sufficient for gathering the intended data on material use. Observers were required to list the materials and how they were used in the lesson. Since the pilot study, this instrument has been used to collect data in a descriptive study in more than 500 K-3 classrooms. (See a description of these online lessons and the link to them in the following section.)

Figure 1.

Vocabulary Observation Instrument

Reliability and Validity of the VOI

Reliability and validity were assessed in the following manner: (a) training of raters via video analyses of three vocabulary lessons, (b) discussion and resolution of differences in ratings, (c) multidimensional scaling (MDS) of rater scores from Grade K-3 classrooms (n = 30).

Reliability via Observer Video Training With Discussion

The pilot lesson

Observer training utilized three video cases of vocabulary instruction to identify events in teaching episodes. The vocabulary lessons were conducted for three different grade levels and are accessible online at http://www.scoe.org/pub/htdocs/archer-videos.html. The three lessons were Kindergarten, 1st grade, and 2nd grade hyperlinks. Two methods were used to increase reliability of ratings: (a) insight-oriented analysis enabling accurate and descriptive entries on methods and (b) coding-oriented analysis to generate categories of materials and types by teachers and students (Castorr et al., 1990).

Collaborative viewing of three vocabulary instruction videos and follow-up discussion enabled the trainees to become confidently grounded in their assertions gleaned from the evidence. We categorized all of the instruments by video episode (three different grade level lessons) and then checked to see whether the participants recorded the same information from each of the videotaped lessons. In addition to discussing rater responses post observation of the three videos, the authors engaged the raters in discussion of different ratings of the same video episode to reconcile differences. We also further analyzed the ratings of the three video episodes to determine the interrater reliability of the instrument. Many have argued that reliability of rater judgments increases via discussion of rating differences (Mott, Etsler, & Drumgold, 2003) and also functions to increase rater competency for understanding the construct domain being observed.

For the first trial, we noted some discrepancies for two categories of materials: drama and word lists. For the purpose of this study, we defined drama as actions, facial expressions, and variations of voice that conveyed the meaning of the word. Raters were confusing the video episode component when the teacher provided a definition using affective speech characteristics such as exaggerated voice tone, facial expressions, and so on. Thus, the author-researchers provided a rationale, via discussion, for separating out teacher sharing of a word list with authentic dramatic activity. For an accurate documentation on the use of visuals, we requested that participants included a brief description of the word lists on overhead transparencies and charts so we would not miss any pictures and graphic organizers that accompanied the words. We also recommended a brief description of book use as well (certainly books may contain all kinds of visuals but we wanted to know when a book was used versus solely a chart or word list). In conclusion, all participants were in agreement on how to use the observation instrument. Furthermore, an analysis of the instruments used during 30 pilot lessons showed congruity in the documentation of data.

VOI Validity

The validity of an assessment is the degree to which it assesses what it purports to assess. The VOI seeks to enable observers to accurately record material use and context. In addition to the face validity of the instrument established through the genesis of its creation derived from the research literature, both experts in K-adult literacy instruction and research, construct validity was technically evaluated to support the instruments use in this study. According to Messick (1996), construct validity is a key component in the hierarchy of validity and is the degree to which an assessment measures an underlying construct, in this case vocabulary learning. To determine construct validity, MDS of vocabulary instruction observation ratings (n = 45) were applied to visually portray similarities and dissimilarities in the data set.

MDS detects meaningful underlying dimensions allowing the current researchers to explain observed similarities or dissimilarities between observation items. Sireci and Geisinger (1995) summarized the benefits for using MDS in establishing the construct validity of an assessment. The rationale being that an ordination or the tendency of like items to be similar in proximity and unlike items to be further away from each other provides evidence that the instrument is indeed discriminating material-type between defined construct dimensions, in this case “control” and “number of senses.” In other words, the dissimilarity of ratings along categories of “senses” and “control” lead us to believe that when “teacher control” is high and “number of senses” used is low, this indicates that the VOI is yielding important information about the tendencies occurring in a lesson. In this case, “active” students (student control) utilized more “senses” during the lesson. This merely indicates that the VOI data enable us to distinguish patterns and relationships occurring in the lessons. Consider the alternative: An observation instrument that yields all “similar” data without dissimilarity among observations along predefined categories might not be sensitive to the vocabulary lesson. Please note that MDS requires the researcher to define the dimensions and thus is only an initial step in establishing validity (Sireci & Geisinger, 1995).

Thus, the vocabulary observations might reveal evidence for material-use characteristics along “student control” if those item responses cluster in proximity and “teacher control” if item responses cluster with each other but distant to “student control.” Similarly, “high number of five senses” (auditory, visual, kinesthetic, smell, and taste) and “low number of senses” item responses might follow suit. Construct validity would be evident if the instrument discriminates material along the dimensions of control and senses. Should materials cluster in proximity along the dimensions, we would have evidence that observer judgments are indeed based on the key instructional elements of control and number of senses gleaned from the three video lessons (see Figure 2).

Figure 2.

Multidimensional scaling scatterplot of observer ratings across dimensions

Analysis and Discussion

Two key aspects of analyzing MDS ¹ are to look at clusters and dimensions: first, which materials group together and/or separate? Second, do the clusters scatterplot in clear dimensions? For the current pilot study, “word list,” “poster/chart,” and “projector” data points scattered in proximity to each other along the dimensions of “teacher control” and “low number of senses” indicating that observations consistently identified material-type along the dimensions. “Book” clustered in “teacher control” and “high number of senses” and “drama” clustered in “student control” and the midpoint between low and high “number of senses.” The dissimilarity between “drama” and the other materials provides evidence that the VOI observer ratings are indeed discriminating material-type and placing that material-type in a context of “number of senses” used and “student control.” “Drama” contained the highest number of senses used and thus the researchers expected to see dissimilarity with “drama” and the other materials (and similarity between “wordlist,” “poster/chart,” and “projector”). “Book” material-type clustered in “high number of senses” but it is important to note that this is due to the fact that this material-type was only identified by a few observers and thus contains an inflated “number of senses” value. It is safe to identify “book” as an outlier. Thus, “book” should not necessarily be viewed as being associated with a high number of senses used given the low number of “book” data points.

Conclusion

To assess the degree to which observers would be able to identify materials teachers use to teach vocabulary, it was necessary to conduct a pilot study of the vocabulary instrument VOI. Results showed that the instrument measured what it was intended to do. In other words, the data similarities and dissimilarities revealed by the MDS analyses provided information about how the materials distinguished themselves along dimensions (the categories defined by the researchers based on MDS coefficients revealed in the scatterplot) generated by observations. Observers reported that the instrument was simple and easy to use for collecting data. Participants in the pilot study were less confident during the first classroom observation, but gained the necessary confidence and expertise to rate the two subsequent classroom observations accurately. The authors found that the VOI provided the appropriate categories and space for the transcription of relevant data. In addition to documenting the materials teachers used, the instrument enabled observers to document the context teachers used with the materials. In the analysis of the data gathered during this study, the authors were able to compare the incidences of word learning in written contexts versus learning through environmental contexts where meaning is supported by visuals, representations, artifacts, and real-world materials.

There is evidence for more holistic incorporation of sensory-rich experiences to promote vocabulary development in early childhood classes (Moyeda, Gomez, & Flores, 2006). Understanding teacher material selection and the number of senses incorporated with specific materials will enable the researchers to develop a theory of vocabulary learning for early childhood educational settings. Accurately documenting the use of materials in K-3 classrooms will provide the researchers with essential information for understanding effective materials-rich methodologies early childhood educators can use to promote vocabulary learning given the ever-present constraints of time and resources. Through VOI, observers were able to document whether the parents’ and caregivers’ incorporation of sensory-rich experiences into vocabulary lessons continued in school, specifically Grades K-3. By expanding the word context to include concrete materials, the authors have added specificity to contextual learning.