Abstract
This study describes the development and evaluation of an assessment of phonological sensitivity (PS) designed to empower teachers to identify children’s phonological and phonemic awareness levels for meaningful instruction and to aid literacy researchers in advancing current understandings of the developmental continuum of PS skills. The Leveled Assessment of Phonological Sensitivity (LAPS) is based on specific causal, reciprocal, and correlational relationships to literacy skills, including reading. The assessment contains 11 levels of item types divided in two parts synthesizing linguistic complexity and cognitive operation based on the literature: (a) phonological awareness and (b) phonemic awareness for determining a child’s level of PS skill. Technical examination of the LAPS’s reliability and validity are presented, including developmental trends of students (n = 333) in Pre-K, K, and Grades 1 and 2. Internal consistency (α = .93) and split-half reliability (Guttman coefficient = .95) were high. Content validity is discussed based on the historical body of research addressing the PS continuum in comparison with the current construct. Developmental validity, determined via ANOVA, revealed LAPS scores discriminating grade level of participants. Confirmatory factor analysis with structural equation modeling revealed single-factor structure, indicating evidence for discriminant validity across LAPS item types along a single latent variable. All 11 paths contained high correlations (across 10 item types with α = .79) between a single factor with rhyme as the exception (α = .40). Findings support LAPS use for teacher identification of students’ PS and reinforce the PS hierarchy of task difficulty set forth from the body of research that dates back to 1976.
Keywords
Introduction
Little doubt exists that phonological processing ability is a prerequisite for early reading success. With the publication of the influential National Reading Panel Report in 2000 synthesizing research from the 1990s, attention rightfully focused on the most complex phonological skill, phonemic awareness (PM) and the corresponding positive correlations to phonics (Ball & Blachman, 1991; Bentin & Leshem, 1993; Blachman, Ball, Black, & Tangel, 1994; Griffith, 1991; Torgesen, Morgan, & Davis, 1992; Treiman & Baron, 1983). However, research is now pointing educators to the premise that phonological sensitivity (PS) is holistically important to early literacy success in English (Anthony, Williams, McDonald, & Francis, 2007), Spanish (Anthony et al., 2006), and Greek (Aidinis & Nunes, 2001). The phrase phonological sensitivity better illustrates the phonological processing abilities related to how children developmentally navigate and work with sound units. Stanovich’s (1992) earlier and Pufpaff’s (2009) recent advocacy of PS as a term to describe the two different components typical of phonological processing offers a dual conception for PS. The two latent variables that make up a single unidimensional PS construct are phonological awareness (PG) and phonemic awareness. Applied research (Bryant, MacLean, Bradley, & Crossland, 1990; Burgess & Lonigan, 1998; Lonigan, Burgess, & Anthony, 2000) and meta-analyses (Burgess, 2006; National Early Literacy Panel, 2008; Pufpaff, 2009) provide evidence that PS develops along a progression and that success at the beginning levels of PS predict success at the more advanced levels. Indeed, Carroll, Snowling, Hulme, and Stevenson (2003) and others (Blachman, 2000; Bryant et al., 1990; Byrne & Fielding-Barnsley, 1991, 1993) found that achievement in PG led to achievement in PM. In addition, PS acquired in a bidirectional manner (Burgess & Lonigan, 1998) generally following a developmental pattern of “shallow” or rudimentary word-unit sounds (do these two words rhyme?) to “deep” individual sounds (what sound do you hear at the end of h-a-t? Anthony et al., 2002). Webb, Schwanenflugel, and Kim (2004) similarly discerned the correlation between phonological skill and later phonemic skill, and their corresponding predictive relationship to alphabetic knowledge.
The National Early Literacy Panel (2008) concluded that PG (rhyme, syllables, onset-rime, phonemes) has a significant relationship to decoding and reading comprehension (with rhyme being the weakest link to decoding skills). As mentioned earlier, the National Reading Panel (2000) also concluded that PM (phoneme isolation, manipulation, and segmentation) was a predictor of early reading success as measured via children’s competency in decoding phonics. When children understand phonemes, their ability to understand phonics is greatly enhanced. In fact, Juel (1988) stated, “without phonemic awareness, phonics instruction is meaningless” (p. 410). Furthermore, the National Reading Panel (2000) clearly delineated the robust relationship between phonics and learning to read. It is with this developmental progression from a general or shallow PS to a more refined or deeper sense of phonemes to the more sophisticated ability to understand phonics (sound and symbol relationships) that children learn to negotiate the difficult task of decoding words.
While PS at the PG level is not directly linked to the phonics skills learned to decode words, PG is directly linked to PM skills that in turn are linked to phonics skills (Carroll et al., 2003). With this insight, researchers and educators are now advocating assessment and instruction in the full PS continuum to leverage the phonological ability of children to promote later, more advanced knowledge of PM, which leads to achievement in phonics. Once children become proficient and automatic word decoders, they become more able to focus on the task of reading comprehension—the ultimate goal of reading.
The plethora of phonological screens (Yopp, 1995), tests (Watkins & Edwards, 2004), and assessments (Torgesen & Bryant, 2004) developed by psychologists, psychometricians, educational psychologists, and communication disorders professionals tend to focus on the identification of PA deficits and corresponding need for intervention. Recently, researchers have identified the need to apply a developmental conceptualization of PS that dictates the need for tools to teach (and measure) PS (Anthony, Lonigan, Driscoll, Phillips, & Burgess, 2003; Cassady, Smith, & Putman, 2008). In this vein, the Leveled Assessment of Phonological Sensitivity (LAPS) was designed to assess the continuum of PS as advocated by Torgesen et al. (1992) and Pufpaff (2009).
PS as a Developmental Progression
Pufpaff (2009) suggested that “insufficient attention is being paid to the developmental nature of phonological sensitivity skills” (p. 679). Researchers (Anthony et al., 2002, 2003; Byrnes & Wasik, 2009; Carroll et al., 2003; Lonigan et al., 2000) have discussed the developmental nature of PS as a progression that begins with a child’s ability to analyze larger units of sound and progressively continues in difficulty until a child can analyze the smallest units of sound (phonemes). Byrnes and Wasik (2009) stated that “between the ages of 1 and 7, children seem to progress from the ability to recognize whole words to the ability to recognize progressively smaller portions of words (e.g., whole words to syllables, onsets and rimes, to eventually phonemes)” (p. 58). Generally, children are able to work with larger phonological units before smaller phonological units.
Currently, a developmental focus on PS is being advocated to better prepare children to develop the more difficult tasks involved in manipulating individual sounds or phonemes (Anthony et al., 2003; Carroll, Snowling, & Hulme, 2003; Cassady et al., 2008; Pufpaff, 2009). This focus stems from the research supporting PS as a unidimensional construct that develops along a continuum from less to more complex tasks. Once educators understand this continuum, they are more likely to assess children’s PS and use the assessment data to plan explicit instruction focusing on children’s developmental needs. In fact, the authors of the National Early Literacy Panel’s (2008) report stated that “what is more likely more important is that the assessment and instructional activities occur within a child’s development level along the developmental continuum” (p. 77). PS instruction is beneficial for most if not all children and is absolutely necessary for others who are at risk for experiencing reading difficulties. Without this very foundational knowledge of the sound structure of language, children will likely experience reading difficulties.
Since PS develops along a continuum, assessment devices should address all the levels within the construct of PS as described by researchers. These tools should be available to use for screening and progress monitoring along that developmental continuum; thus, the assessment needs multiple forms that discriminate among the different developmental levels so that individual student needs can be addressed via explicit and systematic instruction that is fun and engaging for students.
Translating Research Into Practice
Overview
For research to affect educator practice, reading researchers and experts are called on to make research available to educators in ways that they understand how the research should look in classroom practice. In the case of PS, it is imperative that educators understand the dynamics of PS related to the progression of skills. This is no easy task, as Phillips, Clancy-Menchetti, and Lonigan (2008) related:
One pragmatic implication of this continuum of phonological awareness along levels of linguistic complexity and cognitive operation is that at any given point in time, a classroom of preschool children will include children at numerous points along the continuum. (p. 2)
In addition, researchers must be able to adequately explain PS so that educators understand PS and are not confused by the plethora of terms used to describe PS. If PS is thoroughly understood, educators will be better able to recognize the different levels of PS and apply that knowledge in their classroom instruction and assessment. With a comprehensive understanding of PS, educational professionals can apply the research so they know what they are teaching, what they are measuring, why they are teaching it, and why they are measuring it in ways that effectively meet students’ instructional needs.
It is important that educators understand the development continuum through which young children proceed in acquiring phonological skills. This understanding will result in the use of assessment tools and techniques for documenting children’s PS progress and for identifying those children who are not progressing to the appropriate developmental level. By identifying those children who have problems and pinpointing the PS level at which those problems exist, educators can plan appropriate, focused instruction to support the PS development necessary for early reading success.
Despite the significant body of evidence supporting the notion that PS skills be taught and assessed, many early childhood educators lack the competencies to differentiate developmental PS instruction (Dickinson & Brady, 2005; Moats & Foorman, 2003; Zill & Resnick, 2006). Due to educators’ lack of understanding, a formative classroom PS assessment can anchor and focus educators within a usable framework echoes Ganske’s (1993) call for a developmental spelling assessment to assist educators in making sense of the developmental progression of children’s orthographic knowledge. One could argue that prior to Ganske’s (2000) articulation and system, Word Journeys, for enabling teachers to easily level the developmental spelling ability of all classroom students, a teacher would have to be exceedingly clever to know which level of orthographic knowledge for each student based on a comparison of developmental writing stage descriptions from research and the actual students’ writing samples in the classroom.
Formative Assessment
The idea of connecting a formative assessment “teaching tool” to technical characteristics usually reserved for norm-referenced measures is not new. To make research more practical for educators, researchers have attempted to bridge the divide between assessment designed to inform policy and assessment designed to improve educational practice so that formative assessments could serve the dual purpose of guiding learning and informing policy decisions for educational reform. In this vein, rigorous and convincing reliability and validity characteristics exist for collections of children’s narratives (Gearhart, Herman, Novak, & Wolf, 1995; Novak, Herman, & Gearhart, 1996), collections of narrative writing with media elements (Mott, Etsler, & Drumgold, 2003), developmental spelling (Ganske, 1999), and more recently with districtwide performance-based assessment connecting high-quality assessment for large-scale policy decision making (Niemi, Baker, & Sylvester, 2007). The above examples represent a mere sampling of assessments developed as part of, or influenced by, research from the Center for Research on Evaluations, Standards and Student Testing (CRESST) that advocate appropriate assessment use and development for improving teaching and learning.
Assessment for Guiding Instruction
To effectively plan instruction that pinpoints the appropriate PS level for children’s developmental progression, formative assessment or performance-based assessment tools are effective because they link teacher–student language to educational objectives. Formative assessments are not used to assign grades but are used to determine the child’s skill level for targeted, differentiated instruction. Once the assessment data is collected, educators can use the data to ascertain whether the child needs additional instruction on that particular skill or needs to move forward to more challenging levels of skill attainment. Thus, formative assessment tools and techniques assist educators in planning future instruction to match children’s developmental needs.
Salinger (2006) stated that classroom-based assessments should “accommodate the dynamic nature of young learners’ progression from preliteracy to literacy. The tests would be quick and easy to administer, score, and interpret; and data would have immediate utility to teachers” (p. 430). Also according to Salinger (2006), for teachers to use the data immediately, they would need to be trained to “give the tests and understand the results” (p. 430). Although classroom-based assessments may not typically contain the full gamut of psychometric properties, these assessments usually possess “face validity.” However, for the current assessment, the researchers seek to establish more rigorous validity characteristics along the hierarchy of validity with the corresponding psychometric data as the basis for promoting its use.
Instructional Sensitivity of an Instrument
According to Niemi, Wang, Steinberg, Baker, and Wang (2007), validation of an instrument is dependent on the degree to which it supports teacher “instructional sensitivity” along a construct domain in a given curriculum. The current instrument has been designed to address, in a fine-toothed manner, the continuum of PS addressed above. Accordingly, “when assessments . . . are intended to guide and improve instruction, the sensitivity of the assessment(s) to instruction is an essential piece of the evidence needed to validate them” (Niemi et al., 2007, p. 216). Niemi et al. argue that if schools are expected to improve along educational outcomes, the assessment tool must be aligned to the instructional objective. Believing that alignment in critical, the current assessment tool includes classroom teacher feedback as well as validity analyses. In addition, participant-teachers (n = 28) assisted in capturing the instructional sensitivity of the LAPS via their classroom use following the Niemi et al. “opportunity-to-learn” (OTL) study format enabling teachers to capture experiences of students encountering the assessment under investigation (p. 216).
Content Knowledge Related to PS
Pufpaff (2009) conducted a literature review that incorporates the last 30 years of research related to PS in an effort to translate the research base for educators with the knowledge they need to effectively address the phonological developmental needs of students in their classrooms. PS is defined as “encompassing both phonological and phonemic awareness” (Pufpaff, 2009, p. 679). Both of these terms refer to an individual’s ability to work with the sounds of the English language without print involvement. Within the research literature, the terms phonological awareness and phonemic awareness have been used incorrectly and often interchangeably. As a result, educators who seek to understand phonological development are often confused about the instruction and assessment techniques that are appropriate for their students. Due to the confusion related to the use of the terms, the International Reading Association (1998) issued a position statement that defined PG as encompassing “larger units of sound as well, such as syllables, onsets, and rimes” (p. 3). The International Reading Association also stated that “phonemic awareness refers to an understanding about the smallest units of sound that make up the speech stream: phonemes” (p. 3). More recent research (Anthony et al., 2003; Anthony et al., 2007; Byrnes & Wasik, 2009; Carroll et al., 2003; Lonigan, Burgess, & Anthony, 2000) has pointed reading researchers to the notion that PS actually exists along a developmental continuum that begins with PG (units of sound larger than individual phonemes) and progressively emerges to the more complex PM (individual phonemes). In translating this research into classroom practice, educators must understand this increasingly complex developmental progression of PS to better develop instructional and assessment practices to meet the needs of young learners sitting in classrooms.
PS Continuum Influences on Assessment, Instruction, and Intervention
Armed with a clearer understanding of the PS developmental continuum, appropriate assessment, instruction, and intervention practices can be developed and implemented. Before instruction can occur, educators must use assessment data to better serve students’ needs. Assessment tools should address all levels of the PS continuum for maximum benefit to teachers and students. Table 1 illustrates this synthesis: (a) LAPS item types capturing the full PS continuum in hierarchical order, (b) the relationship of LAPS item types to the body of research and PS assessments since 1976 as comprehensively addressed by Pufpaff (2009), and (c) the development and rationale of LAPS levels and rationalized differences with Pufpaff’s summary in consideration of the body of research.
Comparison of the Developmental Sequence of Phonological Sensitivity Skills
Note: LAPS = Leveled Assessment of Phonological Sensitivity; PS = phonological sensitivity.
Table 1 synthesizes the LAPS PS continuum by skill type with Pufpaff’s (2009) comprehensive compilation of PS skill levels integrating the body of PS research addressing the nature of the continuum (Fox & Routh, 1976; Goldstein, 1976; Helfgott, 1976; Lewkowicz & Low, 1979; Liberman, Shankweiler, Fischer,& Carter, 1974; Rosner & Simon, 1971; Seymour & Evans, 1994; Skjelfjord, 1976; Stahl & Murray, 1994; Stanovich, Cunningham, & Cramer, 1984; Vandervelden & Siegel, 1995; and Yopp, 1988).
LAPS: Moving Children From Lower Level to Higher Level PS Skill
PS as a Continuum: Developing Inventory Levels
In developing the LAPS (see Figure 1), Pufpaff’s (2009) synthesis of the research base provided insight into the different PS levels and the linguistic and cognitive complexity within the levels. A comparison of the PS levels assessed on LAPS and levels as defined by Pufpaff (2009) is described in Table 1. As documented in the Table 1, LAPS follows Pufpaff’s articulation of PS developmental levels except that LAPS collapses several of Pufpaff’s levels into one level for the purpose of making the tool more accessible for teachers during classroom assessment and instruction. For example, Pufpaff describes five levels of rhyming skills (rhyme detection to rhyme oddity), but the LAPS includes the one rhyming skill level (rhyme detection) that progressively becomes more difficult within the rhyme detection skill level. Another difference in LAPS and Pufpaff’s progression is that the LAPS includes alliteration. Many researchers (Bryant et al., 1990; Chard & Dickson, 1999; Moats & Tolman, 2009) include alliteration in the development continuum. In addition, LAPS uses developmental progression of rhyme, words in a sentence, syllables, and onset-rime, whereas, Pufpaff describes the progression from rhyme, syllable blending, words in a sentence, and syllables. However, others (Byrnes & Wasik, 2009; Moats & Tolman, 2009) explain the development progression that compares with the levels in the LAPS—words in a sentence, syllables, onset/rime progression. Key studies informing the structure of the LAPS within item-type characteristics were Chafouleas, VanAuken, and Dunham’s (2001) and Cassady et al. (2008) examining the effects of linguistic complexity, manipulation type, and the corresponding influence on difficulty. LAPS within items along each type or “level” move from easier to more difficult based on the Chafouleas et al. and Cassady et al. findings that suggested the easy to difficult structure of blending, segmenting, and deletion by initial, final, or medial sound.
LAPS sample items for “PgE” onset and rime
Technical Evaluation of LAPS
Participants
Because the relationship between PS and literacy skill, including reading, is bidirectional (Ehri, 1993; Perfetti, Beck, Bell, & Hughes, 1987; Wagner et al., 1997), we selected pre- and literate student participants (n = 333) across grade levels from Pre-K-2 from a public school district in the mid-South region of the United States. Typically, studies of PG address only Pre-K children; however, for the current study, a broader range was utilized. The student population comprised 100% of free lunch leading to vast differences in student exposure to literacy (Mississippi Department of Education, 2010). Students sequentially experienced LAPS Items 1 to 11 (see Figure 2) in later winter to early spring. Teachers (n = 27) and teacher aides (n = 16) were trained in a series of professional development workshops conducted by the researchers in using the LAPS as an assessment for guiding differentiated PS instruction.
LAPS conceptual framework
Establishing Procedures for Use and Assessment Procedure
The LAPS 11 item banks with 15 questions per each item type posed a challenge to assessors of very young children. To off-set frustration, for all age groups, we established a “discontinue rule” during the pilot phase of informally assessing children prior to the current study utilized by Webb et al. (2004) in their construct validation study of PG involving preschool-age children. When a child received a score of less than 8 of 15, assessment was stopped. The cut-rate of 8 was established by determining, in the pilot phase, that once an 8 or less was achieved, the subsequent scores on the rest of the items steadily decreased, and frustration, on the part of the children, was avoided. Trained teachers and teacher aides performed assessments from January to February in their classrooms, in a “quiet corner” throughout the day. Assessors explained to students, “I want to learn about how you can recognize sounds so that I can teach you more about sounds.” Assessors notated student answers in the provided scoring boxes adjacent to the prompt.
Results and Discussion of LAPS Reliability and Validity 1
Niemi et al. (2007) supported the notion that professional educators in schools need “high-quality learner-centered assessment.” As a component of the effort to determine LAPS qualities along the idea of “high quality,” we conducted in-depth analyses of the reliability and validity of the LAPS. In Messicks’ (1996) hierarchy of validity, that pinnacle of truth only theoretically possible where a test perfectly measures what it purports to measure, certain validity types nonetheless can provide convincing evidence for the validity characteristics of a test. Such validity evidence can inform teachers to appropriately interpret assessment results for instructional decision making. In addition to the prerequisite of reliability, examined via internal consistency and split-half reliability, the following validity types are addressed: (a) content, (b) developmental, and (c) discriminant validity. In addition, as a component of this discussion interwoven with the results, we closely address one aspect of Baker’s 2 (2001) four tensions inherent to testing and assessment, that being Tension 4: “individual accomplishment versus standardized attainment.” Although the authors desire the rigorous technical characteristics of a standardized measure, equal priority is given to the value for assessment results for informing instruction along the specific domain LAPS is designed to assess. Baker’s sensibility regarding the duality of assessment purposes, with the inherent assumption that assessment results can be utilized for informing policy while adding value to the teaching and learning environment, is a guiding premise for the technical evaluation of the LAPS.
Reliability: Internal Consistency
In an effort to determine the reliability of the LAPS, or the degree to which the assessment yields consistent results, Cronbach’s alpha coefficient was applied to all available (pairwise comparisons) completed forms (n = 122). Cronbach’s alpha measures the internal consistency of a measure and is frequently used for norm-referenced tests. The advantage for using this analysis is that it can be applied to scores collected at a single time at a single administration. Internal consistency estimates how well the set of items on the LAPS correlate with one another. Another way to view internal consistency is that it enables the researchers to see how closely the items are related to one another, a key indicator of reliability. Less variance across items provides insight into the reliability of the LAPS.
Cronbach’s alpha and Guttman’s split-half coefficients were high (α = .93 and .95), indicating convincing evidence for LAPS internal consistency across all completed forms (n = 122). LAPS internal consistency is higher than the level of Watkins’ and Edwards’ (2004) alpha coefficient of .89 for the Mountain Shadows Phonemic Awareness Scale (MS-PAS). In addition, LAPS measures are higher than the Test of Phonological Awareness–Second Edition: Plus (Torgesen & Bryant, 2004), which has an alpha coefficient of 0.8.
Content Validity
The LAPS, as previously addressed in earlier sections of this article, is designed based on the significant body of research addressing the importance of teaching children PS skills along both the phonological (deep, words and rhyming) and phonemic (shallow, individual sounds) levels of the continuum. In this vein, for the current study, the LAPS has been well documented. Thus, the content validity has been established via the genesis of the LAPS design explained in the comparison found in Table 1. The item types have been designed based on previously tested item types from tests, screens, and inventories since 1976. This analysis is addressed in earlier sections of this article (see Table 1). Two key studies, however, undergird the argument for the content validity of the LAPS: Torgesen, Wagner, Rashotte, Burgess, and Hecht (1997) and Pufpaff’s (2009) definition and articulation of PS skills and Pufpaff’s meta analysis of the body of PS item types studied over four decades. Table 1 provides the context for LAPS item type and rationale for how item types are leveled by difficulty along the PS continuum.
Developmental Validity
Given the assumption that older (Grade 2) children have more highly developed phonology than younger children (Pre-K), one might expect to see a significant difference between LAPS scores along grade levels Pre-K-2. Although chronological age does not always positively correlate with cognition, the general phonological developmental milestones have been established in numerous studies (Yopp, 1988, and most notably in Burgess & Lonigan, 1998). Two ANOVA tests were conducted to determine the developmental sensitivity of LAPS scores to the grade level of participants for (a) phonological “Pg” and (b) phonemic “Pm” item types. A significant difference between grade levels along Pg (A-E) and Pm (A-F) item types (see Table 1 and Figure 2 for identifying Pg and Pm type for P-label) would indicate developmental sensitivity of the LAPS.
The analyses were significant for all five Pg item types: PgA F(3, 324) = 8.23, p = .001; PgB F(3, 324) = 2.62, p = .05; PgC F(3, 324) = 4.99, p = .02; PgD F(3, 322) = 7.22, p = .001; and PgE F(3, 323) = 7.70, p = .001. Tukey HSD tests were used to follow-up these effects and were significant between Pre-K and all grade levels for PgA, PgC, and PgE but not PgB and PgD. For example for “PgA” items, Pre-K children scored lower than K (d = −2.08, SE = 0.62, p = .05); Grade 1 (d = −1.75, SE = 0.60, p = .02); and Grade 2 (d = −2.78, SE = 0.59, p = .001). In addition, Grade 1 children scored lower on PgA than Grade 2 children (d = −1.03, SE = 0.35, p = .02), but children in K did not score lower than Grade 1 children (d = 0.33, SE = 0.39, p = .82). The statistically significant mean differences (d) indicate that younger children scored lower than older children, providing evidence for the developmental sensitivity of LAPS Pg items between Pre-K and Grade 2 and all other grades, Pre-K and K, Grades 1 and 2, but not K and 1. It is important to note, however, that mean scores across all grade levels Pre-K to 1 for all LAPS items were higher for each grade level. For Grades 1 to 2, mean scores reached a plateau indicating that LAPS Pg items do not have developmental sensitivity above Grade 1.
The lack of discrimination between PgB (alliteration) and PgD (syllables) supports research indicating that (PgB) alliteration is closely related to rhyme and more difficult to discriminate (Bryant et al., 1990). The lack of discrimination between K and 1 could be explained by the time of year that the pilot study was conducted. Since K students would have more experience with PS by the middle of the second semester, those discriminatory factors could have been dissipated due to progression along the continuum.
Similar to the Pg item types, analyses were significant for all six Pm item types: PmA F(3, 324) = 6.70, p = .001; PmB F(3, 323) = 6.08, p = .001; PmC F(3, 324) = 7.39, p = .001; PmD F(3, 324) = 9.37, p = .001; PmE F(3, 321) = 7.19, p = .001; and PmF F(3, 324) = 9.55, p = .001. Tukey HSD tests were used to follow-up these effects and were significant between Pre-K across all grade levels and between K and 1 for PmC (d = −1.90, SE = 0.72, p = .04) and PmD (d = −3.56, SE = 0.83 p = .01). For example, for “PmA” items, Pre-K children scored lower than K (d = −4.09, SE = 1.2, p = .05); Grade 1 (d = −5.2, SE = 1.2, p = .001); and Grade 2 (d = −4.09, SE = 1.2, p = .05). The statistically significant mean differences (d) between Pre-K and Grades K-2 provide evidence that younger children scored lower than older children (thus the negative values in d), indicating the developmental sensitivity of LAPS Pm items between grades Pre-K and all other grades. It must be noted, however, that for “Pm” phonemic item types, with inherently more difficult linguistic complexity and cognitive operation levels, that Pre-K and K respondents received lower scores than Grade 1 and 2 students without significant results possibly due to the “discontinue rule” (see Webb et al., 2004), which effectively stopped assessment leaving Pre-K and K student response rates very low for Pm levels.
The above results support other research that global PS develops along a continuum with the global Pg developing prior to the more sophisticated Pm (Anthony et al., 2003, 2007; Byrnes & Wasik, 2009; Carroll et al., 2003; Lonigan et al., 2000). Due to the discontinue rule, students in Pre-K and K did not continue with the assessment, which certainly follows previous research findings that few preschool and kindergarten children are able to work at the phoneme level (Pufpaff, 2009).
Discriminant Validity
Convergent validity, the degree to which operations of a measure are related to operations from another measure, or even within a measure, determine the relatedness or degree that items converge indicating that what is being measured across the instrument is indeed one construct. The notion that the LAPS measures a unidimensional construct domain of PS would support previous research (most notably Torgesen & Burgess, 1992, and Pufpaff, 2009). Discriminant validity, the degree to which items on a test possess sensitivity to specific areas or subdomains (LAPS item types PgA to PmF) intended by the test designers would indicate, for example, the sensitivity of the instrument to discriminate between PG and PM. In addition, discriminant validity would provide evidence that LAPS item types are assessing subtypes within the continuum of PS. Both divergent and discriminant validity provide support for the construct validity of LAPS. Construct validity is the degree to which an assessment measures the construct (PS) that it is designed to measure.
Confirmatory factor analysis (CFA) with structural equation modeling (SEM) was utilized to determine the ability of three predefined factor models to fit an observed set of data. Specifically, how do LAPS levels (factor indicators) relate to latent factors of PS identified in the literature? CFA enables researchers to assess multiple models to address discriminant and convergent validity for a single or multiple factor model. The ability to test the significance of specific factor loading combinations (such as phonological and phonemic double factors; phonological, phonemic and onset-rime triple factors; or PS as a single factor) leads to the ability to frame the convergent and discriminant validity of the instrument. CFA was selected versus exploratory factor analyses due to the significant volume of research in PA addressing the developmental continuum, and test, screen, and assessment item characteristics since 1976. Armed with this well-established theoretical base, three CFA models were hypothesized to incrementally scrutinize LAPS factor indicators (item types/levels) to the theorized psychological construct of PA. The researchers, adhering to the cautionary advice of Thompson and Borello (1989) designed more than one model to avoid assumptions that only a single model fits the data.
The theoretical basis for LAPS item categories and item characteristics addressing the PS continuum incrementally from research point to content validity in general from the genesis of the LAPS discussed earlier. To further explore validity, exploratory factor analysis was conducted to determine simple patterns and pattern of relationships among LAPS levels. Can LAPS levels (variables) be explained largely in terms of a much smaller number of factors (variables)? Thus, this factor analysis seeks to heuristically unearth patterns and relationships from many dependent variables (LAPS levels) to discern something about the nature of the independent latent (factors) that effect them. Unlike multidimensional scaling, in factor analysis, two variables cannot be organized along one cluster, thus enabling insight into each variables tendency to load with a factor.
Results of the CFA factor loadings revealed a one-factor solution indicating that PS consists of a unidimensional construct domain, although rhyme received the lowest correlation (see descriptive statistics in Table 2 for means, standard deviations across LAPS types).
LAPS Descriptive Statistics for Phonological and Phonemic Items
Note: LAPS = Leveled Assessment of Phonological Sensitivity.
SEM was utilized to visually portray CFA results along pathways as correlational coefficients. The goal is to visually portray and evaluate the latent variable in light of the factors. Thus, the bubble represents PS latent variable in Model 1, the two bubbles represent Phonological and Phonemic latent variables in Model 2 and the three bubbles represent Phonological, Onset-Rime, and Phonemic latent variables in Model 3. The square shapes represent the dependent variables, which are the 11 item types consisting of phonological and phonemic categories (PgA through PmF; see Tables 3 and 4).
LAPS Reliability Analyses
Note: LAPS = Leveled Assessment of Phonological Sensitivity.
The items are PgA, PgB, PgC, PgD, PgE, PmA.
The items are PmB, PmC, PmD, PmE, PmF.
Covariance–Variance Matrix of LAPS Item Types
Note: LAPS = Leveled Assessment of Phonological Sensitivity.
SEM was utilized to visually portray CFA results along pathways as correlational coefficients. Three models of PS were tested: (a) PS as a unidimensional construct, (b) PG and PM as two separate constructs, and (3) PG, Onset-Rime, and PM as a construct. The structural equation model results (see Table 5) reveal qualities of “fit” or the extent to which the proposed model fits the distribution according to the articulated construct. In the case of the current study, the authors proposed three models as suggested in Yaun (2005) and followed Yaun’s protocol for qualifying goodness of fit. Goodness of fit can be viewed as how trustworthy a given model is in terms of how well it fits the scores or results in juxtaposition to the corresponding theoretical rationale for those scores. Yaun, in his review of structural equation model results interpretation, found that generally, comparative fit index (CFI) and TFI scores above .90 are acceptable and root mean square error of approximation (RMSEA) scores close to .05 are also acceptable. Based on this criteria, LAPS scores for Model 1 (PS is a unidimensional construct) do contain an acceptable fit (again see Table 5) as TFI; CFI scores are quite close to .90, and RMSEA is .08. The structural equation model tests used to assess the goodness of fit thus revealed that LAPS scores, consisting of phonological, onset-rime, and phonemic items can be interpreted as an overall measure of students’ ability to discriminate large and small sound units and that their ability to do so is a singular, or unidimensional, ability of PS (Figure 3).
Comparisons of Fit Indices for Models Examining the Distinguishableness of LAPS Levels With PS Domains
Note: LAPS = Leveled Assessment of Phonological Sensitivity; CFI = comparative fit index; RMSEA = root mean square error of approximation; PS = phonological sensitivity.
Structural equation Model 1
Discussion of LAPS Value and Utility
Value (Helpfulness of Measure)
The value or helpfulness of a measure is determined by whether the measure or assessment tool builds teacher knowledge, pinpoints student knowledge, and affects teacher practice (Novak et al., 1996). Will the tool affect classroom practice? LAPS provides a formative, performance-based assessment that links teacher–student language to educational objectives. Just as Ganske’s (1999) spelling assessment provides a tool for determining students’ developmental spelling level that guides instructional delivery and provides common language for teacher and student, the LAPS serves as a tool for determining students’ PS level for determining initial instruction, as well as continuing instruction. In addition, LAPS can be used as a progress-monitoring tool to determine students’ progress along the developmental continuum and to affect instructional decisions. Salinger (2006) stated that classroom-based assessments should follow the developmental progression of literacy tasks, the LAPS provides that fluid movement of PS development.
Utility (Technical Qualities)
The utility of an assessment tool has been defined as a product of its reliability, validity, cost-effectiveness, acceptability, and educational impact (Gearhart et al., 1995). According to Salinger (2006), assessments should be quick and easy to administer, score, and interpret, and have immediate utility to teachers. LAPS provides evidence of the characteristics defined as important for assessment utility. As described previously in the article, LAPS reliability and validity measures are robust. In addition, teachers accepted LAPS during the pilot program with positive comments about the tool. Finally, LAPS can provide educational impact when teachers utilize the data to inform instructional decisions.
Conclusion
The goal of this study is to bring the research base into the classroom where teachers are able to use LAPS as an “in-practice” tool as opposed to sole reliance on tests, screens, and inventories designed to identify deficits. LAPS was created as a tool to assist teachers in making instructional decisions, as well as in communicating with students (in the form of instructional prompts) and colleagues along the PS continuum in ways that mirror a phonics curriculum without the presence of letter symbols. Given students’ developmental nature and students’ variability with experience to spoken language, teachers must differentiate instruction along a specific learning domain, in this case PS, to scaffold individual students’ ability to reach higher levels of sensitivity. For scaffolding to occur, assessment must support teachers’ ability to make data-based decisions that focus instruction on the needs of the learner. The LAPS can be used in ways that support teachers in their quest to provide PS instruction that is based on individual student data.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.