The Woodcock-Johnson Tests of Achievement, Fourth Edition (WJ IV ACH) is purported to align with Cattell–Horn–Carroll (CHC) theory and offers upward of 20 scores within its interpretive and scoring system. The Technical Manual does not furnish validity evidence for the scores reported by the scoring system, suggesting that evidentiary support may be incomplete. Exploratory bifactor analysis (EBFA; maximum likelihood extraction with a bigeomin [orthogonal] rotation) was applied to the two school-aged correlation matrices at ages 9 to 19. Results indicated nonalignment with CHC theory and do not support the interpretation of most of the scores suggested by the scoring system. Instead, the results of this study suggest that the loading patterns diverge significantly from the interpretive system produced by the WJ IV ACH. Only the academic fluency and academic knowledge clusters emerged following the use of EBFA. Implications for clinical interpretation of the WJ IV ACH are offered.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing (4th ed.). Washington, DC: Authors.
2.
BandalosD. L.GerstnerJ. J. (2016). Using factor analysis in test construction. In SchweizerK.DiStefanoC. (Eds.), Principles and methods of test construction: Standards and recent advancements (pp. 26-51). Gottingen, Germany: Hogrefe.
3.
BeaujeanA. A. (2015). John Carroll’s views on intelligence: Bi-factor vs. higher-order models. Journal of Intelligence, 3, 121-136. doi:10.3390/jintelligence3040121
4.
BeaujeanA. A.BensonN. F. (2018) Theoretically-consistent cognitive ability test development and score interpretation. Contemporary School Psychology. doi:10.1007/s40688-018-0182-1
5.
BollenK. A.BauldryS. (2011). Three Cs in measurement models: Causal indicators, composite indicators, and covariates. Psychological Methods, 16, 265-284.
6.
BorsboomD.CramerA. O. J.KievitR. A.ScholtenA. Z.FranićS. (2009). The end of construct validity. In LissitzR. W. (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 135-170). Charlotte, NC: Information Age Publishing.
7.
BorsboomD.MellenberghG. J.van HeerdenJ. (2004). The concept of validity. Psychological Review, 111, 1061-1071. doi:10.1037/0033-295X.111.4.1061
8.
BrunnerM.NagyG.WilhelmO. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796-846. doi:10.1111/j.1467-6494.2011.00749.x
9.
CanivezG. L. (2017). Review of the Woodcock-Johnson IV. In CarlsonJ. F.GeisingerK. F.JonsonJ. L. (Eds.), The twentieth mental measurements yearbook (pp. 875-882). Lincoln, NE: Buros Center for Testing.
10.
CattellR. B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54, 1-22. doi:10.1037/h0046743
11.
CattellR. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276. doi:10.1207/s15327906mbr0102_10
12.
CohenJ. (1959). The factorial structure of the WISC at ages 7-6, 10-6, and 13-6. Journal of Consulting Psychology, 23, 285-299.
13.
CormierD. C.BulutO.McGrewK. S.FrisonJ. (2016). The role of Cattell-Horn-Carol (CHC) cognitive abilities in predicting writing achievement during the school-age years. Psychology in the Schools, 53, 787-803. doi:10.1002/pits.21945
14.
CormierD. C.BulutO.McGrewK. S.SinghD. (2017). Exploring the relations between Cattell-Horn-Carroll (CHC) cognitive abilities and mathematics achievement. Applied Cognitive Psychology, 31, 530-538. doi:10.1002/acp.3350
15.
CormierD. C.McGrewK. S.BulutO.FunamotoA. (2017). Revisiting the relations between the WJ-IV measures of Cattell-Horn-Carroll (CHC) cognitive abilities and reading achievement during the school-age years. Journal of Psychoeducational Assessment, 35, 731-754. doi:10.1177/0734282916659208
16.
DombrowskiS. C. (2015). Exploratory bifactor analysis of the WJ III Achievement at school age via the Schmid-Leiman procedure. Canadian Journal of School Psychology, 30, 34-50. doi:10.1177/0829573514560529
17.
DombrowskiS. C.DiStefanoC.NoonanK. (2004). Review of the Stanford-Binet, fifth edition. Communiqué, 33, 12-15.
18.
DombrowskiS. C.McGillR. J.CanivezG. L. (2017). Exploratory and hierarchical factor analysis of the WJ IV Cognitive at school age. Psychological Assessment, 29, 394-407. doi:10.1037/pas0000350
19.
DombrowskiS. C.McGillR. J.CanivezG. L. (2018a). An alternative conceptualization of the theoretical structure of the Woodcock-Johnson IV Tests of Cognitive Abilities at school age: A confirmatory factor analytic investigation. Archives of Scientific Psychology, 6, 1-13. doi:10.1037/arc0000039
20.
DombrowskiS. C.McGillR. J.CanivezG. L. (2018b). Hierarchical exploratory factor analyses of the Woodcock-Johnson IV full test battery: Implications for CHC application in school psychology. School Psychology Quarterly, 33, 235-250. doi:10.1037/spq0000221
21.
DombrowskiS. C.NoonanK. (2004). Review of the WISC-IV. Communiqué, 33, 35-38.
22.
FayersP. M.HandD. J. (2002). Causal variables, indicator variables and measurement scales: An example from quality of life. Journal of the Royal Statistical Society. Series A (Statistics in Society), 165, 233-261.
23.
Groth-MarnatG.WrightA. J. (2016). Handbook of psychological assessment (6th ed.). Hoboken, NJ: Wiley.
24.
HancockG. R.MuellerR. O. (2001). Rethinking construct reliability within latent variable systems. In CudeckR.du ToitS.SorbomD. (Eds.), Structural equation modeling: Present and future: A festschrift in honor of Karl Joreskog (pp. 195-216). Lincolnwood, IL: Scientific Software International.
25.
HornJ. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185. doi:10.1007/BF02289447
26.
JacksonJ. S. H.MaraunM. (1996). The conceptual validity of empirical scale construction: The case of the sensation seeking scale. Personality and Individual Differences, 21, 103-110. doi:10.1016/0191-8869(95)00217-0
JennrichR. I.BentlerP. M. (2012). Exploratory bifactor analysis: The oblique case. Psychometrika, 77, 442-454. doi:10.1007/s11336-012-9269-1
29.
JensenA. R. (1993). Psychometric g and achievement. In GiffordB. R. (Ed.), Policy perspectives on educational testing. Boston: Kluwer.
30.
JensenA. R.WengL. (1994). What is a good g?Intelligence, 18, 231-258.
31.
KaiserH. F. (1974). An index of factorial simplicity. Psychometrika, 39, 31-36.
32.
KaufmanA. S. (1975). Factor analysis of the WISC-R at 11 age levels between 61/2 and 161/2 years. Journal of Consulting and Clinical Psychology, 43, 135-147. doi:10.1037/h0076502
33.
KaufmanA. S.LichtenbergerE. O. (2006). Assessing adolescent and adult intelligence (3rd ed.). Hoboken, NJ: Wiley.
34.
KaufmanS. B.ReynoldsM. R.LiuX.KaufmanA. S.McGrewK. S. (2012). Are cognitive g and academic achievement g one and the same g? An exploration on the Woodcock–Johnson and Kaufman tests. Intelligence, 40, 123-138. doi:10.1016/j.intell.2012.01.009
35.
KrauseM. S. (2005). How the psychotherapy research community must work toward measurement validity and why. Journal of Clinical Psychology, 61, 269-283. doi:10.1002/jclp.20020
36.
KrauseM. S. (2012). Measurement validity is fundamentally a matter of definition, not correlation. Review of General Psychology, 16, 391-400. doi:10.1037/a0027701
37.
MacmannG. M.BarnettD. W. (1997). Myth of the master detective: Reliability of interpretations for Kaufman’s “intelligent testing” approach to the WISC-III. School Psychology Quarterly, 12, 197-234. doi:10.1037/h0088959
38.
MaraunM. D. (1998). Measurement as a normative practice: Implications of Wittgenstein’s philosophy for measurement in psychology. Theory & Psychology, 8, 435-461. doi:10.1177/0959354398084001
39.
MatarazzoJ. D.PrifiteraA. (1989). Subtest scatter and premorbid intelligence: Lessons from the WAIS-R standardization sample. Psychological Assessment, 1, 186-191. doi:10.1037/1040-3590.1.3.186
40.
MatherN.WendlingB. J. (2014). Examiner’s manual. Woodcock-Johnson IV Tests of Achievement. Rolling Meadows, IL: Riverside.
41.
McDermottP. A.FantuzzoJ. W.GluttingJ. J. (1990). Just say no to subtest analysis: A critique on Wechsler theory and practice. Journal of Psychoeducational Assessment, 8, 290-302. doi:10.1177/073428299000800307
42.
McGrewK. S.LaForteE. M.SchrankF. A. (2014). Technical Manual: Woodcock-Johnson IV. Rolling Meadows, IL: Riverside.
43.
MuthénL. KMuthénB. O. (1998-2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Author.
44.
Pearson. (2009). Wechsler Individual Achievement Test (3rd ed.). San Antonio, TX: Author.
45.
PetriD.MariL.CarboneP. (2015). A Structured Methodology for Measurement Development. IEEE Transactions on Instrumentation and Measurement, 64, 2367-2379.
46.
ReiseS. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. doi:10.1080/00273171.2012.715555
47.
ReiseS. P.BonifayW. E.HavilandM. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. doi:10.1080/00223891.2012.725437
48.
ReiseS. P.MooreT. M.HavilandM. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. doi:10.1080/00223891.2010.496477
49.
R Development Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
50.
RevelleW. (2017). psych: Procedures for psychological, psychometric, and personality research (version 1.7.8) [computer software]. Evanston, IL: Northwestern University.
51.
ReynoldsC. R. (1979). Factor structure of the Peabody Individual Achievement Test at five grade levels between grades one and 12. Journal of School Psychology, 17, 270-274.
52.
RodriguezA.ReiseS. P.HavilandM. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137-150. doi:10.1037/met0000045
53.
SattlerJ. M. (2008). Assessment of Children: Cognitive Applications (5th ed.). Jerome M. Sattler Publisher: La Mesa, CA.
54.
SchmidJ.LeimanJ. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53-61. doi:10.1007/BF02289209
55.
SchneiderW. J.McGrewK. S. (2012). The Cattell-Horn-Carroll model of intelligence. In FlanaganD. P.HarrisonP. L. (Eds.), Contemporary intellectual assessment Theories tests, and issues (3rd ed., pp. 99-144). New York, NY: Guilford.
56.
SchrankF. A.MatherN.McGrewK. S. (2014). Woodcock-Johnson IV Tests of Achievement. Rolling Meadows, IL: Riverside.
57.
ShapiroE. S. (2011). Academic skills problems: Direct assessment and intervention. New York: Guilford Press.
58.
SireciS.Faulkner-BondM. (2014). Validity evidence based on test content. Psicothema, 26, 100-107.
59.
VelicerW. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321-327. doi:10.1007/BF02293557
60.
WatkinsM. W. (2000). Cognitive profile analysis: A shared professional myth. School Psychology Quarterly, 15, 465-479. doi:10.1037/h0088802
61.
WatkinsM. W. (2003). IQ subtest analysis: Clinical acumen or clinical illusion?The Scientific Review of Mental Health Practice, 2, 118-141.
62.
WatkinsM. W.KushJ. C. (1994). Wechsler subtest analysis: The right way, the wrong way, or no way?School Psychology Review, 23, 640-651. Available from http://www.nasponline.org
63.
WilliamsT. O.EavesR. C. (2001). Exploratory and confirmatory factor analyses of the Woodcock reading mastery tests–revised with special education students. Psychology in the Schools, 38, 561-567.
64.
WilliamsT. O.FallA.EavesR. C.DarchC.Woods-GrovesS. (2007). Factor analysis of the KeyMath—Revised normative update form A. Assessment for Effective Intervention, 32, 113-120. doi:10.1177/15345084070320020201
65.
WoodcockR. W.McGrewK. S.MatherN. (2001). Woodcock Johnson III Tests of Achievement. Rolling Meadows, IL: Riverside.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
