Evidence supporting the internal structure of an assessment can provide support for its scoring mechanisms. A total of 1,691 young children in the United States were assessed by their parents using the 16-month questionnaire of the Ages and Stages Questionnaires, Third Edition (ASQ-3), and results were used to study the internal structure and dimensionality of the ASQ-3 using item response theory (IRT)/Rasch models. Data were analyzed using IRT including unidimensional and multidimensional Rasch Partial Credit Models to examine four theory-based internal structures. Results supported the hypothesis that the ASQ-3 measures a unidimensional construct—the overall development of children—while preference was given to a more precise multidimensional construct of five interrelated domains. However, concerns emerged regarding the extremely high correlations between items in the fine-motor and problem-solving domains. Findings provide implications for the test developers in terms of item refinement to improve measurement of underlying constructs.
AckermanT. A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255-278.
2.
AdamsR. J.WilsonM. R.WangW. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
3.
AdamsR. J.WuM. L.WilsonM. R. (2015). ConQuest (Version 4.4) [Computer software]. Camberwell: Australian Council for Educational Research.
4.
AlkherainejK. (2012). Examination of the factor structure and agreement of three questionnaires for identifying young children at risk for autism spectrum disorders (Unpublished doctoral dissertation), University of Oregon, Eugene.
5.
American Educational Research Association, American Psychological Association, & National Council for Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
6.
BayleyN. (2006). Bayley Scales of Infant and Toddler Development (3rd ed.). San Antonio, TX: Harcourt Assessment.
7.
BerkL. E. (2012). Infants, children and adolescents (7th ed.). Boston, MA: Pearson.
8.
BeverlyF. (2010). Supporting the education and care of young children: Putting into practice what we know. In HargreaversA.LievermanA.FullanM.HopkinsD. (Eds.), Second international handbook of educational change (pp. 933-951). Philadelphia, PA: Springer.
9.
BondT.FoxC. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences. New York, NY: Routledge.
10.
BrickerD.MacyM.SquiresJ.MarksK. (2013). Developmental screening in your community. Baltimore, MD: Brookes.
11.
BukatkoD.DaehlerM. W. (2011). Child development: A thematic approach (6th ed.). Belmont, CA: Wadsworth.
12.
ChengY. Y.WangW. C.HoY. H. (2008). Multidimensional Rasch analysis of a psychological test with multiple subtests: A statistical solution for the Bandwidth-Fidelity Dilemma. Educational and Psychological Measurement, 69, 369-388.
13.
EmbretsonS. E.ReiseS. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
14.
FeldmanH. M.DaleP. S.CampbellT. F.ColbornD. K.Kurs-LaskyM.RocketteH. E.ParadiseJ. L. (2005). Concurrent and predictive validity of parent reports of child language at ages 2 and 3 years. Child Development, 76, 856-868.
15.
FensonL.MarchmanV. A.ThalD. J.DaleP. S.ReznickJ. S.BatesE. (2000). MacArthur-Bates Communicative Development Inventories: User’s guide and technical manual. Baltimore, MD: Brookes.
16.
FilgueirasA.PiresP.MaissonetteS.Landeira-FernandezJ. (2013). Psychometric properties of the Brazilian-adapted version of the Ages and Stages Questionnaire in public child daycare centers. Early Human Development, 89, 561-576.
17.
FinchH. (2010). Multidimensional item response theory parameter estimation with nonsimple structure items. Applied Psychological Measurement, 35, 67-82.
18.
GuralnickM. J. (2011). Why early intervention works: A system perspective. Infants and Young Children, 24, 6-28.
19.
HambletonR. K.MurrayL. (1983). Some goodness of fit investigations for item response models. In HambletonR. K. (Ed.), Applications of item response theory (pp. 71-94). Vancouver, Canada: Educational Research Institute of British Columbia.
20.
HornmanJ.KerstjensJ. M.de WinterA. F.BosA. F.ReijneveldS. A. (2013). Validity and internal consistency of the Ages and Stages Questionnaire 60-month version and the effect of three scoring methods. Early Human Development, 89, 1011-1015.
21.
JiangS.WangC.WeissD. J. (2016). Sample size requirements for estimation of item parameters in the multidimensional graded response model. Frontiers in Psychology, 7, 109.
22.
King-DowlingS.RodriguezM. C.MissiunaC.CairneyJ. (2016). Validity of the Ages and Stages Questionnaire to detect risk of developmental coordination disorder in preschoolers. Child Care, Health and Development, 42, 188-194.
23.
KoseI. A.DemirtasliN. C. (2012). Comparison of unidimensional and multidimensional models based on item response theory in terms of both variables of test length and sample size. Procedia: Social and Behavioral Sciences, 46, 135-140.
24.
KwunY.ParkH. W.KimM.-j.LeeB. S.KimA.-R. (2015). Validity of the Ages and Stages Questionnaires in Korean compared to Bayley Scales of Infant Development-II for screening preterm infants at corrected age of 18-24 months for neurodevelopmental delay. Journal of Korean Medical Science, 30, 450-455.
RadeckiL.Sand-LoudN.O’ConnorK. G.SharpS.OlsonL. M. (2011). Trends in the use of standardized tools for developmental screening in early childhood: 2002-2009. Pediatrics, 128, 14-19.
27.
ReckaseM. (2009). Multidimensional item response theory. New York, NY: Springer.
28.
RosenbergS. A.ZhangD.RobinsonC. C. (2008). Prevalence of developmental delays and participation in early intervention services for young children. Pediatrics, 121, 1503-1509.
29.
SalviaJ.YsseldykeJ. E.BoltS. (2013). Assessment in special and inclusive education (12th ed.). Belmont, CA: Wadsworth.
30.
ScaliseK. (2012). Crowdsourcing and education with relation to the knowledge economy. In KaracapilidisN. (Ed.), Web-based and blended educational tools and innovations (pp. 136-150). Hershey, PA: IGI Global.
31.
SchonhautL.ArmijoI.SchönstedtM.AlvarezJ.CorderoM. (2013). Validity of the ages and stages questionnaires in term and preterm infants. Pediatrics, 131, e1468-e1474.
32.
ShonkoffJ. P.MeiselsS. J. (Eds.). (2000). Handbook of early childhood intervention. New York, NY: Cambridge University Press.
33.
SigafoosJ. (2000). Communication development and aberrant behavior in children with developmental disabilities. Education and Training in Mental Retardation and Developmental Disabilities, 35, 168-176.
34.
SmithC.WallenM.WalkerK.BundyA.RolinsonR.BadawiN. (2012). Validity of the fine motor area of the 12-month Ages and Stages Questionnaire in infants following major surgery. Physical & Occupational Therapy in Pediatrics, 32, 260-271.
U.S. Department of Health and Human Services. (2014). Birth to Five: Watch Me Thrive! A compendium of screening measures for young children. Washington, DC: Author.
39.
WrightB. D.LinacreJ. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions. Retrieved from http://www.rasch.org/rmt/rmt83b.htm.
40.
WuM. L.AdamsR. J.WilsonM. R. (1998). ACER ConQuest: Generalised item response modelling software [Computer software and manual]. Hawthorn: Australian Council for Educational Research.
41.
WuM. L.AdamsR. J.WilsonM. R.HaldaneS. (2007). ACER ConQuest version 2.0: Generalised item response modelling software [Computer software and manual]. Camberwell: Australian Council for Educational Research.
