Although many fluid reasoning (Gf) tests have been developed, there is a lack of figural tests measuring its lower-order process factors simultaneously. The present article introduces the development of the Multidimensional Induction-Deduction Computerized Adaptive Test (MID-CAT) to measure two process factors of Gf. The MID-CAT is designed to provide an instrument that is flexible, efficient, and entirely free for non-commercial use. We created 530 items and administered them to a sample of N = 2,247. Items were fitted and calibrated using the Rasch model. The results indicate that the final item pool has a wide range of difficulties that could precisely measure a wide range of test-takers’ abilities. A simulation study also indicates that MID-CAT provides greater measurement efficiency than separate-unidimensional CAT or fixed-item test. In the discussion, we provide perspectives on how the MID-CAT can be used for future research.
AdamsR. J.WilsonM.WangW. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23. https://doi.org/10.1177/0146621697211001
2.
AdamsR. J.WuM. L. (2007). The mixed-coefficients multinomial logit model: A generalized form of the Rasch model. In von DavierM.CarstensenC. H. (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 57–75). Springer. https://doi.org/10.1007/978-0-387-49839-3_4
BardosA. N. (2001). General Ability Measure for Adults (GAMA). In DorfmanW. I.HersenM. (Eds.), Understanding psychological assessment (pp. 43–58). Springer. https://doi.org/10.1007/978-1-4615-1185-4_3
5.
BergstromB. A.LunzM. E.GershonR. C. (1992). Altering the level of difficulty in computer adaptive testing. Applied Measurement in Education, 5(2), 137–149. https://doi.org/10.1207/S15324818AME0502_4
BrownL.SherbenouR. J.JohnsenS. K. (2010). Test of nonverbal intelligence (4th ed.). PRO-ED.
8.
CanivezG. L.YoungstromE. A. (2019). Challenges to the Cattell-Horn-Carroll theory: Empirical, clinical, and policy implications. Applied Measurement in Education, 32(3), 232–248. https://doi.org/10.1080/08957347.2019.1619562
CattellR. B. (1943). The measurement of adult intelligence. Psychological Bulletin, 40(3), 153.
11.
CattellR. B.KrugS. E.BartonK. (1973). Technical supplement for the Culture Fair Intelligence Tests, Scales 2 and 3. Institute for Personality and Ability Testing.
12.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06
13.
ChalmersR. P. (2016). Generating adaptive and non-adaptive test interfaces for multidimensional item response theory applications. Journal of Statistical Software, 71(5), 1–38. https://doi.org/10.18637/jss.v071.i05
14.
ChangH.-H.YingZ. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213–229. https://doi.org/10.1177/014662169602000303
15.
ChierchiaG.FuhrmannD.KnollL. J.Pi-SunyerB. P.SakhardandeA. L.BlakemoreS.-J. (2019). The matrix reasoning item bank (MaRs-IB): Novel, open-access abstract reasoning items for adolescents and adults. Royal Society Open Science, 6(10), 190232. https://doi.org/10.1098/rsos.190232
16.
ColeJ. C.LopezB. R.DaleoD. V. (2004). Latent relationships of fluid, visual, and simultaneous cognitive tasks. Psychological Reports, 94(2), 547–561. https://doi.org/10.2466/pr0.94.2.547-561
17.
CondonD. M.RevelleW. (2014). The international cognitive ability resource: Development and initial validation of a public-domain measure. Intelligence, 43(1), 52–64. https://doi.org/10.1016/j.intell.2014.01.004
18.
DasJ.NaglieriJ.KirbyJ. (1994). Assessment of cognitive processes: The PASS theory of intelligence.
19.
FlanaganD. P.OrtizS.AlfonsoV. (2013). Essentials of cross-battery assessment (3rd ed.). Wiley.
HammillD. D.PearsonN. A.WeiderholtJ. L. (2009). Comprehensive test of nonverbal intelligence (2nd ed.). PRO-ED.
23.
HebbD. O. (1941). Clinical evidence concerning the nature of normal adult test performance. Psychological Bulletin, 38, 593.
24.
HeydaschT.HaubrichJ.RennerK.-H. (2013). The short version of the Hagen Matrices Test (HMT-S): 6-item induction intelligence test. Methods, Data, Analyses, 7(2), Article 2. https://doi.org/10.12758/mda.2013.011
25.
HornJ. L. (1968). Organization of abilities and the development of intelligence. Psychological Review, 79, 242–259.
26.
HornJ. L.NollJ. (1997). Human cognitive capabilities: Gf-Gc theory. In FlanaganD.GenshaftJ.HarrisonP. (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 53–91). The Guilford Press.
KleinB.RavenJ.FodorS. (2018). Scrambled Adaptive Matrices (SAM)—A new test of eductive ability. Psychological Test and Assessment Modeling, 60(4), 451–492.
31.
KochM.SpinathF. M.GreiffS.BeckerN. (2022). Development and validation of the open matrices item bank. Journal of Intelligence, 10(3), 41. https://doi.org/10.3390/jintelligence10030041
32.
KovacsK.ConwayA. R. A. (2016). Process overlap theory: A unified account of the general factor of intelligence. Psychological Inquiry, 27(3), 151–177. https://doi.org/10.1080/1047840X.2016.1153946
33.
KovacsK.ConwayA. R. A. (2019). A unified cognitive/differential approach to human intelligence: Implications for IQ testing. Journal of Applied Research in Memory and Cognition, 8(3), 255–272. https://doi.org/10.1016/j.jarmac.2019.05.003
34.
KroehneU.GoldhammerF.PartchevI. (2014). Constrained Multidimensional Adaptive Testing without intermixing items from different dimensions. Psychological Test and Assessment Modeling, 56(4), 348–367.
35.
KyllonenP.HartmanR.SprengerA.WeeksJ.BertlingM.McGrewK.KrizS.BertlingJ.FifeJ.StankovL. (2019). General fluid/inductive reasoning battery for a high-ability population. Behavior Research Methods, 51(2), 507–522. https://doi.org/10.3758/s13428-018-1098-4
36.
LeeN. Y. L.GoodwinG. P.Johnson-LairdP. N. (2008). The psychological puzzle of Sudoku. Thinking and Reasoning, 14(4), 342–364. https://doi.org/10.1080/13546780802236308
MakranskyG.GlasC. A. W. (2013). The applicability of multidimensional computerized adaptive testing for cognitive ability measurement in organizational assessment. International Journal of Testing, 13(2), 123–139. https://doi.org/10.1080/15305058.2012.672352
39.
MatzkeD.DolanC. V.MolenaarD. (2010). The issue of power in the identification of “g” with lower-order factors. Intelligence, 38(3), 336–344. https://doi.org/10.1016/J.INTEIL.2010.02.001
40.
McGrewK. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1–10. https://doi.org/10.1016/j.intell.2008.08.004
41.
MulderJ.van der LindenW. J. (2008). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74(2), 273–296. https://doi.org/10.1007/s11336-008-9097-5
42.
NaglieriJ. A. (2016). Naglieri Nonverbal Ability Test–Third Edition (NNAT3). Pearson.
43.
OteroT. M. (2017). Brief review of fluid reasoning: Conceptualization, neurobasis, and applications. Applied Neuropsychology: Child, 6(3), 204–211. https://doi.org/10.1080/21622965.2017.1317484
44.
PaapM. C. S.BornS.BraekenJ. (2019). Measurement efficiency for fixed-precision multidimensional computerized adaptive tests: Comparing health measurement and educational testing using example banks. Applied Psychological Measurement, 43(1), 68–83. https://doi.org/10.1177/0146621618765719
45.
RaschG. (1960). Probabilistic models for some intelligence and attainment tests. University of Chicago Press.
46.
RavenJ. C.CourtJ. H.RavenJ. (1988). Manual for Raven’s Progressive Matrices and Vocabulary Scales. H. K. Lewis.
47.
R Core Team. (2012). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org
48.
ReckaseM. D. (2009). Multidimensional item response theory. Springer.
49.
RiosJ. A.GuoH.MaoL.LiuO. L. (2017). Evaluating the impact of careless responding on aggregated-scores: To filter unmotivated examinees or not?International Journal of Testing, 17(1), 74–104. https://doi.org/10.1080/15305058.2016.1231193
50.
RiosJ. A.SolandJ. (2021). Parameter estimation accuracy of the effort-moderated item response theory model under multiple assumption violations. Educational and Psychological Measurement, 81(3), 569–594. https://doi.org/10.1177/0013164420949896
51.
RobitzschA. (2023). sirt: Supplementary Item Response Theory Models (R package version 3.13-228) [Computer software]. https://CRAN.R-project.org/package=sirt
RothB.BeckerN.RomeykeS.SchäferS.DomnickF.SpinathF. M. (2015). Intelligence and school grades: A meta-analysis. Intelligence, 53, 118–137. https://doi.org/10.1016/j.intell.2015.09.002
54.
RuizP. E. (2009). Measuring fluid intelligence on a ratio scale: Evidence from nonverbal classification problems and information entropy. Behavior Research Methods, 41(2), 439–445. https://doi.org/10.3758/BRM.41.2.439
55.
SahinA.AnilD. (2017). The effects of test length and sample size on item parameters in item response theory. Educational Sciences: Theory & Practice, 17(1), 321–335. https://doi.org/10.12738/estp.2017.1.0270
56.
SantarnecchiE.EmmendorferA.Pascual-LeoneA. (2017). Dissecting the parieto-frontal correlates of fluid intelligence: A comprehensive ALE meta-analysis study. Intelligence, 63, 9–28. https://doi.org/10.1016/J.INTELL.2017.04.008
57.
SchmidtF. L. (2002). The role of general cognitive ability and job performance: Why there cannot be a debate. Human Performance, 15(1–2), 187–210. https://doi.org/10.1207/s15327043hup1501&02_12
58.
SchneiderW. J.McGrewK. S. (2012). The Cattell-HornCarroll Model of Intelligence. In FlanaganD. P.HarrisonP. L. (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 99–144). The Guilford Press.
59.
SchneiderW. J.McGrewK. S. (2018). The Cattell–Horn–Carroll theory of cognitive abilities. In FlanaganD. P.McDonoughE. M. (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (4th ed., pp. 73–163). The Guilford Press.
60.
SchwarzG. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.
ShengY.WikleC. K. (2014). Comparing multi-unidimensional and unidimensional IRT models. Educational and Psychological Measurement, 67(6), 899–919.
63.
SpearmanC. (1904). “General Intelligence,” Objectively determined and measured. The American Journal of Psychology, 15(2), 201–292. https://doi.org/10.2307/1412107
64.
StoetG. (2010). PsyToolkit: A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 1096–1104. https://doi.org/10.3758/BRM.42.4.1096
65.
StoneE.DaveyT. (2011). Computer-adaptive testing for students with disabilities: A review of the literature (ETS Research Report No. RR-11-32). ETS.
66.
StoetG. (2017). PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24–31. https://doi.org/10.1177/0098628316677643
67.
SunderlandM.BatterhamP.CarragherN.CalearA.SladeT. (2019). Developing and validating a computerized adaptive test to measure broad and specific factors of internalizing in a community sample. Assessment, 26(6), 1030–1045. https://doi.org/10.1177/1073191117707817
68.
Ul HassanM.MillerF. (2022). Discrimination with unidimensional and multidimensional item response theory models for educational data. Communications in Statistics—Simulation and Computation, 51(6), 2992–3012. https://doi.org/10.1080/03610918.2019.1705344
69.
VeldkampB. P.van der LindenW. J. (2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67(4), 575–588. https://doi.org/10.1007/BF02295132
70.
VispoelW. P. (2000). Reviewing and changing answers on computerized fixed-item vocabulary tests. Educational and Psychological Measurement, 60(3), 371–384. https://doi.org/10.1177/00131640021970600
71.
WainerH. (1993). Some practical considerations when converting a linearly administered test to an adaptive format. Educational Measurement: Issues and Practice, 12(1), 15–20. https://doi.org/10.1111/j.1745-3992.1993.tb00519.x
72.
WainerH. (2000). Computerized adaptive testing: A primer (2nd ed.). Lawrence Erlbaum.
73.
WangC.ChangH.-H.BoughtonK. A. (2013). Deriving stopping rules for multidimensional computerized adaptive testing. Applied Psychological Measurement, 37(2), 99–122. https://doi.org/10.1177/0146621612463422
74.
WangW.-C.ChenP.-H. (2004). Implementation and measurement efficiency of multidimensional computerized adaptive testing. Applied Psychological Measurement, 28(5), 295–316. https://doi.org/10.1177/0146621604265938
75.
WangW.-C.ChenP.-H.ChengY.-Y. (2004). Improving measurement precision of test batteries using multidimensional Item response models. Psychological Methods, 9(1), 116–136. https://doi.org/10.1037/1082-989X.9.1.116
76.
WeissL. G.KeithT. Z.ZhuJ.ChenH. (2013). WAIS-IV and clinical validation of the four- and five-factor interpretative approaches. Journal of Psychoeducational Assessment, 31(2), 94–113. https://doi.org/10.1177/0734282913478030
77.
WilhelmO. (2005). Measuring reasoning ability. In WilhelmO.EngleR. W. (Eds.), Handbook of understanding and measuring intelligence (pp. 373–392). Sage. https://doi.org/10.4135/9781452233529.n21
78.
WiseS. L.DeMarsC. E. (2006). An application of item response time: The effort-moderated IRT model. Journal of Educational Measurement, 43(1), 19–38. https://doi.org/10.1111/j.1745-3984.2006.00002.x
79.
WiseS. L.KongX. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. https://doi.org/10.1207/s15324818ame1802_2
80.
WiseS. L.MaL. (2012, January).
Setting response time thresholds for a CAT item pool: The normative threshold method
[Paper presentation]. The 2012 annual meeting of the National Council on Measurement in Education, Vancouver, Canada.
81.
WrightB. D.LinacreJ. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370–371.
82.
YaoL. (2014). Multidimensional CAT item selection methods for domain scores and composite scores with item exposure control and content constraints: MCAT for domain scores and composite scores. Journal of Educational Measurement, 51(1), 18–38. https://doi.org/10.1111/jedm.12032
83.
YaoL.PommerichM.SegallD. O. (2014). Using multidimensional CAT to administer a short, yet precise, screening test. Applied Psychological Measurement, 38(8), 614–631.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
