We develop and validate a self-report measure of intrinsic and extrinsic cognitive load suitable for measuring the constructs in a variety of learning contexts. Data were collected from three independent samples of college students in the U.S. (Ntotal= 513; Mage= 21.13 years). Kane’s (2013) framework was used to validate the measure. Three types of validity evidence were presented: scoring, generalization, extrapolation. After establishing evidence of validity, especially measurement invariance, we then compared group mean differences based on students’ demographic characteristics. These findings support the psychometric integrity of this measure of cognitive load, which may be used to investigate cognitive load in various learning contexts, particularly examining factors that may perpetuate or mitigate differences in cognitive load between students. Such a measure could be useful in educational and clinical settings as a mechanism for early identification of potential learning challenges.
AndersenM. S.MakranskyG. (2021a). The validation and further development of a multidimensional cognitive load scale for virtual environments. Journal of Computer Assisted Learning, 37(1), 183–196. https://doi.org/10.1111/jcal.12478
2.
AndersenM. S.MakranskyG. (2021b). The validation and further development of the multidimensional cognitive load scale for physical and online lectures (MCLS-POL). Frontiers in Psychology, 12, 642084. https://doi.org/10.3389/fpsyg.2021.642084
3.
AnmarkrudØ.AndresenA.BråtenI. (2019). Cognitive load and working memory in multimedia learning: Conceptual and measurement issues. Educational Psychologist, 54(2), 61–83. https://doi.org/10.1080/00461520.2018.1554484
4.
AyresP. (2006). Impact of reducing intrinsic cognitive load on learning in a mathematical domain. Applied Cognitive Psychology, 20(3), 287–298. https://doi.org/10.1002/acp.1245
5.
BettencourtG. M.ManlyC. A.KimballE.WellsR. S. (2020). STEM degree completion and first-generation college students: A cumulative disadvantage approach to the outcomes gap. The Review of Higher Education, 43(3), 753–779. https://doi.org/10.1353/rhe.2020.0006
6.
BrünkenR.PlassJ. L.LeutnerD. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38(1), 53–61. https://doi.org/10.1207/S15326985EP3801_7
7.
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
8.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
9.
CierniakG.GerjetsP.ScheiterK. (2009a). Expertise reversal in multimedia learning: Subjective load ratings and viewing behavior as cognitive process indicators. Proceedings of the Annual Meeting of the Cognitive Science Society, 31(31). https://escholarship.org/uc/item/99r5z5fb
10.
CierniakG.ScheiterK.GerjetsP. (2009b). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load?Computers in Human Behavior, 25(2), 315–324. https://doi.org/10.1016/j.chb.2008.12.020
11.
CookD. A.BrydgesR.GinsburgS.HatalaR. (2015). A contemporary approach to validity arguments: A practical guide to Kane’s framework. Medical Education, 49(6), 560–575. https://doi.org/10.1111/medu.12678
12.
CooperG.Tindall-FordS.ChandlerP.SwellerJ. (2001). Learning by imagining. Journal of Experimental Psychology Applied, 7(1), 68–82. https://doi.org/10.1037/1076-898X.7.1.68
13.
DebueN.van de LeemputC. (2014). What does germane load mean? An empirical contribution to the cognitive load theory. Frontiers in Psychology, 5, 1099. https://doi.org/10.3389/fpsyg.2014.01099
14.
DikaS. L.D’AmicoM. M. (2016). Early experiences and integration in the persistence of first-generation college students in STEM and non-STEM majors. Journal of Research in Science Teaching, 53(3), 368–383. https://doi.org/10.1002/tea.21301
15.
De JongT. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science, 38(2), 105–134. https://doi.org/10.1007/s11251-009-9110-0
GuenoleN.BrownA. (2014). The consequences of ignoring measurement invariance for path coefficients in structural equation models. Frontiers in Psychology, 5, 980. https://doi.org/10.3389/fpsyg.2014.00980
18.
HeoM.ToomeyN. (2020). Learning with multimedia: The effects of gender, type of multimedia learning resources, and spatial ability. Computers & Education, 146, 103747. https://doi.org/10.1016/j.compedu.2019.103747
19.
HessB. J.KvernB. (2021). Using Kane’s framework to build a validity argument supporting (or not) virtual OSCEs. Medical Teacher, 43(9), 999–1004. https://doi.org/10.1080/0142159X.2021.1910641
20.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
21.
JinY. (2020). A note on the cutoff values of alternative fit indices to evaluate measurement invariance for ESEM models. International Journal of Behavioral Development, 44(2), 166–174. https://doi.org/10.1177/0165025419866911
22.
KalyugaS. (2011). Cognitive load theory: How many types of load does it really need?Educational Psychology Review, 23(1), 1–19. https://doi.org/10.1007/s10648-010-9150-7
23.
KalyugaS.ChandlerP.SwellerJ. (2001). Learner experience and efficiency of instructional guidance. Educational Psychology, 21(1), 5–23. https://doi.org/10.1080/01443410124681
24.
KaneM. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
25.
KirschnerP. A.AyresP.ChandlerP. (2011). Contemporary cognitive load theory research: The good, the bad and the ugly. Computers in Human Behavior, 27(1), 99–105. https://doi.org/10.1016/j.chb.2010.06.025
26.
KlepschM.SchmitzF.SeufertT. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in Psychology, 8, 1997. https://doi.org/10.3389/fpsyg.2017.01997
27.
KlepschM.SeufertT. (2020). Understanding instructional design effects by differentiated measurement of intrinsic, extraneous, and germane cognitive load. Instructional Science, 48(1), 45–77. https://doi.org/10.1007/s11251-020-09502-9
28.
KorbachA.BrünkenR.ParkB. (2018). Differentiating different types of cognitive load: A comparison of different measures. Educational Psychology Review, 30(2), 503–529. https://doi.org/10.1007/s10648-017-9404-8
29.
KrellM. (2015). Evaluating an instrument to measure mental load and mental effort using item response theory. Science Education Review Letters, 1(6). https://edoc.hu-berlin.de/handle/18452/8864
30.
KroehneU.GoldhammerF. (2018). How to conceptualize, represent, and analyze log data from technology-based assessments? A generic framework and an application to questionnaire items. Behaviormetrika, 45(2), 527–563. https://doi.org/10.1007/s41237-018-0063-y
31.
LawsonG. M.HookC. J.FarahM. J. (2018). A meta-analysis of the relationship between socioeconomic status and executive function performance among children. Developmental Science, 21(2), Article e12529. https://doi.org/10.1111/desc.12529
32.
LeonardJ. A.MackeyA. P.FinnA. S.GabrieliJ. D. E. (2015). Differential effects of socioeconomic status on working and procedural memory systems. Frontiers in Human Neuroscience, 9, 554. https://doi.org/10.3389/fnhum.2015.00554
33.
LeppinkJ.PaasF.Van der VleutenC. P. M.Van GogT.Van MerriënboerJ. J. G. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058–1072. https://doi.org/10.3758/s13428-013-0334-1
34.
LeppinkJ.PaasF.Van GogT.van Der VleutenC. P.Van MerrienboerJ. J. (2014). Effects of pairs of problems and examples on task performance and different types of cognitive load. Learning and Instruction, 30, 32–42. https://doi.org/10.1016/j.learninstruc.2013.12.001
35.
LeppinkJ.Pérez-FusterP. (2019). Mental effort, workload, time on task, and certainty: Beyond linear models. Educational Psychology Review, 31(2), 421–438. https://doi.org/10.1007/s10648-018-09460-2
36.
LeungS. O.XuM. L. (2013). Single-item measures for subjective academic performance, self-esteem, and socioeconomic status. Journal of Social Service Research, 39(4), 511–520. https://doi/org/10.1080/01488376.2013.794757
37.
MaguireM. J.SchneiderJ. M. (2019). Socioeconomic status related differences in resting state EEG activity correspond to differences in vocabulary and working memory in grade school. Brain and Cognition, 137, 103619. https://doi.org/10.1016/j.bandc.2019.103619
38.
MayerR. E.MorenoR. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. https://doi.org/10.1207/S15326985EP3801_6
39.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
40.
MillerD. J.NobleP.MedlenS.JonesK.MunnsS. L. (2021). Brief research report: Psychometric properties of a cognitive load measure when assessing the load associated with a course. The Journal of Experimental Education, 91(2), 401–410. https://doi.org/10.1080/00220973.2021.1947763
41.
MooneyK. E.PradyS. L.BarkerM. M.PickettK. E.WatermanA. H. (2021). The association between socioeconomic disadvantage and children’s working memory abilities: A systematic review and meta-analysis. PLoS One, 16(12), Article e0260788. https://doi.org/10.1371/journal.pone.0260788
42.
MorenoR. (2010). Cognitive load theory: Historical development and relation to other theories. In PlassJ. L.MorenoR.BrünkenR. (Eds), Cognitive load theory. Cambridge University Press. (pp. 9–28).
43.
MurphyG.GroegerJ. A.GreeneC. M. (2016). Twenty years of load theory—where are we now, and where should we go next?Psychonomic Bulletin & Review, 23(5), 1316–1340. https://doi.org/10.3758/s13423-015-0982-5
44.
National Center for Science and Engineering Statistics (NCSES). (2023). Diversity and STEM: Women, minorities, and persons with disabilities 2023. Available at
45.
OberT. M.ChengY. (2022). December 16 cognitive load and mental effort inventory - scale validation. Open Science Framework Repository. Retrieved from
46.
OberT. M.LiuC.ChengY. (2023). Development, validation, and evidence of measurement invariance of a shortened measure of trait test anxiety. European Journal of Psychological Assessment. https://doi.org/10.1027/1015-5759/a000761
47.
PaasF.AyresP. (2014). Cognitive load theory: A broader view on the role of memory in learning and education. Educational Psychology Review, 26(2), 191–195. https://doi.org/10.1007/s10648-014-9263-5
48.
PaasF.RenklA.SwellerJ. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/S15326985EP3801_1
49.
PaasF.van GogT.SwellerJ. (2010). Cognitive load theory: New conceptualizations, specifications, and integrated research perspectives. Educational Psychology Review, 22(2), 115–121. https://doi.org/10.1007/s10648-010-9133-8
50.
PaasF.van MerriënboerJ. J. G. (2020). Cognitive-load theory: Methods to manage working memory load in the learning of complex tasks. Current Directions in Psychological Science, 29(4), 394–398. https://doi.org/10.1177/0963721420922183
51.
PaasF. G. W. C. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429–434. https://doi.org/10.1037/0022-0663.84.4.429
52.
ParkB.BrünkenR. (2015). The rhythm method: A new method for measuring cognitive load—an experimental dual-task study. Applied Cognitive Psychology, 29(2), 232–243. https://doi.org/10.1002/acp.3100
53.
PutnickD. LBornsteinM. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
54.
RediferJ. L.BaeC. L.ZhaoQ. (2021). Self-efficacy and performance feedback: Impacts on cognitive load during creative thinking. Learning and Instruction, 71, 101395. https://doi.org/10.1016/j.learninstruc.2020.101395
SkulmowskiA.XuK. M. (2021). Understanding cognitive load in digital and online learning: A new perspective on extraneous cognitive load. Educational Psychology Review, 34, 171–196. https://doi.org/10.1007/s10648-021-09624-7
58.
St JohnA. M.KibbeM.TarulloA. R. (2019). A systematic assessment of socioeconomic status and executive functioning in early childhood. Journal of Experimental Child Psychology, 178, 352–368. https://doi.org/10.1016/j.jecp.2018.09.003
SwellerJ. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138. https://doi.org/10.1007/s10648-010-9128-5
SwellerJ. (2016). Cognitive load theory, evolutionary educational psychology, and instructional design. Evolutionary Perspectives on Child Development and Education, pp. 291–306. https://doi.org/10.1007/978-3-319-29986-0_12
WhitneyB. M.ChengY.BrodersenA. S.HongM. R. (2019). The scale of student engagement in statistics: Development and initial validation. Journal of Psychoeducational Assessment, 37(5), 553–565. https://doi.org/10.1177/0734282918769983
68.
WiseS. L.KongX. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. https://doi.org/10.1207/s15324818ame1802_2
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
