In this study, the authors examined the validity of a holistically scored retell within a confirmatory factor analysis framework by comparing the fit of a three-factor model of reading with the data from a diverse sample of seventh and eighth graders. The final model demonstrated adequate fit, χ2(32) = 97.316; comparative fit index = .96; Tucker–Lewis index = .94; and root mean square error of approximation = .08. Retell’s chi-square difference, Δχ2(1) = 16.652, p < .001, and factor loading (.250, p < .001) were higher for the comprehension construct. Similarly, retell’s correlation to comprehension measures (r = .155–.257, p < .01) was stronger than its relationship to measures of fluency (r = .158–.183, p < .01) or word identification (r = .132, p < .05). However, retell had a large residual variance (.938) and low interrater reliability (κ = .37), suggesting that improvements to the instrument are needed. Despite overall latent differences, retell did not demonstrate differential item functioning.
BestR. M.FloydR. G.McNamaraD. S. (2008). Differential competencies contributing to children’s comprehension of narrative and expository texts. Reading Psychology, 29, 137–164.
4.
BovairdJ. A. (2007). Multilevel structural equation models for contextual factors. In LittleT. A.BovairdJ. A. (Eds.), Modeling contextual effects in longitudinal studies (pp.149-182). Mahwah, NJ: Lawrence Erlbaum.
5.
BrownT. A. (2006). Confirmatory factor analysis for applied research: Methodology in the social sciences. New York, NY: Guilford.
6.
BulyM. R.ValenciaS. (2003). Meeting the needs of ailing readers: Cautions and considerations for state policy. Seattle: University of Washington, Center for the Study of Teaching and Policy. Retrieved from: http://depts.washington.edu/ctpmail/PDFs/Reading-MRBSV-04-2003.pdf
7.
BurkeM. D.Hagan-BurkeS. (2007). Concurrent criterion-related validity of early literacy indicators for middle of first grade. Assessment for effective Intervention, 32, 66–77.
8.
BurkeM. D.Hagan-BurkeS.KwokO.ParkerR. (2009). Predictive validity of early literacy indicators from the middle of kindergarten to second grade. Journal of Special Education, 42, 209–226.
9.
CarlisleJ. F. (1999). Free recall as a test of reading comprehension for students with learning disabilities. Learning Disability Quarterly, 22, 11–22.
10.
CattsH. W.GillispieM.LeonardL. B.KailR. V.MillerC. A. (2002). The role of speed processing, rapid naming, and phonological awareness in reading achievement. Journal of Learning Disabilities, 35, 510–525.
11.
CohenJ. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.
12.
CoyneM. D.Kame’enuiE. J.SimmonsD. C.HarnB. (2004). Beginning reading intervention as inoculation or insulin: First-grade reading performance of strong responders to kindergarten intervention. Journal of Learning Disabilities, 37, 90–105.
13.
CromleyJ. G.AzevedoR. (2007). Testing and refining the direct and inferential mediation model of reading comprehension. Journal of Educational Psychology, 99, 311–325.
14.
CurranC. E.KintschE.HedbergN. (1996). Learning-disabled adolescents’ comprehension of naturalistic narratives. Journal of Educational Psychology, 88, 494–507.
15.
CuttingL. E.ScarboroughH. S. (2006). Prediction of reading comprehension: Relative contribution of word recognition, language proficiency, and other cognitive skills can depend on how comprehension is measured. Scientific Studies of Reading, 10, 277–299.
16.
DuffelmeyerF. A.DuffelmeyerB. B. (1987). Main idea questions on informal reading inventories. Reading Teacher, 41, 162–166.
17.
FletcherJ. M.LyonG. R.FuchsL. S.BarnesM. A. (2007). Learning disabilities: From identification to intervention. New York, NY: Guilford.
18.
FuchsL. S.FuchsD.HospM. K.JenkinsJ. R. (2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading, 5, 239–256.
19.
FuchsL. S.FuchsD.MaxwellL. (1988). The validity of informal reading comprehension measures. Remedial and Special Education, 9, 20–29.
20.
GambrellL. B.JawitzP. B. (1993). Mental imagery, text illustrations, and children’s story comprehension and recall. Reading Research Quarterly, 28, 265-276.
21.
GambrellL. B.KoskinenP. S.KapinusB. A. (1991). Retelling and the reading comprehension of proficient and less-proficient readers. Journal of Educational Research, 84, 356–362.
22.
GoodR. H.KaminskiR. A. (2002). Vital indicators of progress (VIP). Dallas, TX: Voyager Expanded Learning.
23.
HockM. F.BrasseurI. F.DeshlerD. D.CattsH. W.MarquisJ. G.MarkC. A.StriblingJ. W. (2009). What is the reading component skill profile of adolescent struggling readers in urban schools?Learning Disability Quarterly, 32, 21–38.
24.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structural analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
25.
JacksonN. E. (2005). Are university students’ component reading skills related to their text comprehension and academic achievement?Learning and Individual Differences, 15, 113–139.
26.
JohnsJ. L. (2008). Basic reading inventory: Pre-primer through grade twelve and early literacy assessments (10th ed.). Dubuque, IA: Kendall/Hunt.
27.
JoreskogK. G.GoldbergerA. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70, 631–639.
28.
KazdinA. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification, 1, 427–451.
29.
KeenanJ. M.BetjemannR. S.OlsonR. K. (2008). Reading comprehension tests vary in the skills they assess: Differential dependence on decoding and oral comprehension. Scientific Studies of Reading, 12, 281–300.
30.
KintschW.van DijkT. A. (1978). Towards a model of text comprehension and production. Psychological Review, 85, 363–349.
31.
KlesiusJ. P.HomanS. P. (1985). A validity and reliability update on the informal reading inventory with suggestions for improvement. Journal of Learning Disabilities, 18, 71–76.
32.
KucerS. B. (2009). Examining the relationship between text processing and text comprehension in fourth-grade readers. Reading Psychology, 30, 340–358.
33.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
LyonG. R.FletcherJ. M.ShaywitzS. E.ShaywitzB. A.TorgesenJ. K.WoodF. B.. . . OlsonR. (2001). Rethinking learning disabilities. In FinnC. E.Jr.RotherhamR. A. J.HokansonC. R.Jr. (Eds.), Rethinking special education for a new century (pp. 259–287). Washington, DC: Thomas B. Fordham Foundation.
37.
MacCallumR. C.BrowneM. W.SugawaraH. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Method, 1, 130–149.
38.
MacIntoshR.HashimS. (2003). Variance estimation for converting MIMIC model parameters to IRT parameters in DIF analysis. Applied Psychological Measurement, 27, 372–379.
39.
MadelaineA.WheldallK. (2005). Identifying low-progress readers: Comparing teacher judgment with a curriculum-based measurement procedure. International Journal of Disability, Development, and Education, 52, 33–42.
40.
MarcotteA. M.HintzeJ. M. (2009). Incremental and predictive utility of formative assessment methods of reading comprehension. Journal of School Psychology, 47, 315–335.
41.
MehtaP.NealeM. (2005). People are variables too: Multilevel structural equation models. Psychological Methods, 10, 259–284.
42.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (pp. 13–104). New York, NY: Macmillan.
43.
MuthenB. O.KaoC.BursteinL. (1991). Instructional sensitivity in mathematics achievement test items: Applications of a new IRT-based detection technique. Journal of Educational Measurement, 28, 1–22.
44.
MuthenL.MuthenB. (2009). Mplus(version 5.21) [Computer Software]. Los Angeles, CA: Author.
45.
NilssonN. L. (2008). A critical analysis of eight informal reading inventories. Reading Teacher, 61, 526–536.
46.
ReedD. K.VaughnS. (in press). Retell as an indicator of reading comprehension. Scientific Studies of Reading
47.
RichgelsD. J.McGeeL. M.LomaxR. G.SheardC. (1987). Awareness of text structures: Effects on recall of expository text. Reading Research Quarterly, 22, 177–196.
48.
RobertsG.GoodR.CorcoranS. (2005). Story retell: A fluency-based indicator of reading comprehension. School Psychology Quarterly, 20, 304–317.
SavageR. (2006). Reading comprehension is not always the product of nonsense word decoding and linguistic comprehension: Evidence from teenagers who are extremely poor readers. Scientific Studies of Reading, 10, 143–164.
51.
ScardamaliaM.BereiterC. (1987). Knowledge telling and knowledge transforming in written composition. In RosenbergS. (Ed.), Advances in applied psycholinguistics: Disorders of first-language development (Vol. 1, pp. 142–175). New York, NY: Cambridge University Press.
52.
SchaferJ. L.GrahamJ. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147–177.
53.
SchatschneiderC.BuckJ.WagnerR.HasslerL.HechtS.Powell-SmithK. (2004). A multivariate study of individual differences in performance on the reading portion of the Florida Comprehensive Assessment Test: A brief report (Technical Report No. 50). Tallahassee: Florida Center for Reading Research.
54.
ShinnM. R.GoodR.KnutsonN.TillyW. D.CollinsV. L. (1992). Curriculum-based measurement of oral reading fluency: A confirmatory analysis of its relation to reading. School Psychology Review, 21, 459–479.
SnowC. (2002). Reading for understanding: Toward an R&D program in reading comprehension. Santa Monica, CA: RAND Education.
57.
SnyderL.CaccamiseD.WiseB. (2005). The assessment of reading comprehension: Considerations and cautions. Topics in Language Disorders, 25, 33–50.
58.
Spear-SwerlingL. (2006). Children’s reading comprehension and oral reading fluency in easy text. Reading and Writing: An Interdisciplinary Journal, 19, 199–220.
StageS. A.JacobsenM. D. (2001). Predicting student success on a state-mandated performance-based assessment using oral reading fluency. School Psychology Review, 30, 407–419.
61.
SteckerP. M.FuchsL. S. (2000). Effecting superior achievement using curriculum-based measurement: The importance of individual progress monitoring. Learning Disability Research & Practice, 15, 128–134.
62.
SteigerJ. H.LindJ. M. (1980, May). Statistically based tests for the number of common factors. Paper presented at the meeting of the Psychometric Society, Iowa City, IA.
63.
TalbottE.LloydJ. WTankersleyM. (1994). Effects of reading comprehension interventions for students with learning disabilities. Learning Disability Quarterly, 17, 223–232.
64.
Texas Education Agency, University of Houston, & The University of Texas System. (2008). Texas middle school fluency assessment, version 1.0. Austin, TX: Author.
65.
TorgesenJ. K.AlexanderA. W.WagnerR. K.RashotteC. A.VoellerK. K. S.ConwayT. (2001). Intensive remedial instruction for children with severe reading disabilities: Immediate and long-term outcomes from two instructional approaches. Journal of Learning Disabilities, 34, 33–58.
66.
TuckerL. R.LewisC. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.
67.
ValenciaS. W.BulyM. R. (2004). Behind test scores: What struggling readers really need. Reading Teacher, 57, 520–531.
68.
van den BroekP.TzengY.RisdenK.TrabassoT.BascheP. (2001). Inferential questioning: Effects on comprehension of narrative texts as a function of grade and timing. Journal of Educational Psychology, 93, 521–592.
69.
VelluntinoF.ScanlonD.SipayE.SmallS.PrattA.ChenR.DencklaM. Y. Y. (1996). Cognitive profiles of difficult-to-remediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of specific reading disability. Journal of Educational Psychology, 88, 601–638.
70.
VukovicR. K.WilsonA. M.NashK. K. (2004). Naming speed deficits in adults with reading disabilities: A test of the double-deficit hypothesis. Journal of Learning Disabilities, 37, 440–450.
71.
WagnerR. K.MuseA. E.TannenbaumK. R. (2007). Promising avenues for better understanding implications of vocabulary development for reading comprehension. In WagnerR. K.MuseA. E.TannenbaumK. R. (Eds.), Vocabulary acquisition: Implications for reading comprehension (pp. 276–291). New York, NY: Guilford.
72.
WalczykJ. J.RaskaL. J. (1992). The relation between low- and high-level reading skills in children. Contemporary Educational Psychology, 17, 38–46.
73.
WileyH. I.DenoS. L. (2005). Oral reading and maze measures as predictors of success for English learners on a state standards assessment. Remedial and Special Education, 26, 207–214.
74.
WilliamsJ. P. (1991). Comprehension by learning-disabled and nondisabled adolescents of personal/social problems presented in text. American Journal of Psychology, 104, 563–586.
75.
WolfM. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.
76.
WoodsM. L.MoeA. J. (2007). Analytical reading inventory: Comprehensive standards-based assessment for all students, including gifted and remedial (8th ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.
77.
ZinarS. (1990). Fifth-graders’ recall of propositional content and causal relationships from expository prose. Journal of Reading Behavior, 22, 181–199.
