Two studies were conducted to examine and compare the construct validity of scores on the Junior Metacognition Awareness Inventory (JMAI) and problem-solving interview protocols. Participants consisted of 183 middle and high school students attending a university summer program for academically talented youth. Study 1 results indicated that JMAI scores were internally consistent and yielded an interpretable two-factor structure after the elimination of several items; however, the scores were not significantly or meaningfully related to GPA or current and future mathematics achievement. In Study 2 (n = 30), JMAI scores did not predict students’ metacognitive behaviors during mathematics problem-solving tasks. In contrast, students’ metacognitive behaviors observed during problem solving were meaningfully related to mathematics achievement with medium to high effect sizes. Findings support the predictive validity of metacognition with regard to academic achievement when operationalized with problem-solving interviews, but call into question the criterion-related validity of JMAI scores.
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
ArumR.RoksaJ. (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.
3.
AydinU.UbuzB. (2010). Structural model of metacognition and knowledge of geometry. Learning and Individual Differences, 20, 436-445. doi:10.1016/j.lindif.2010.06.002
4.
BarfurthM. A.RitchieK. C.IrvingJ. A.ShoreB. M. (2009). A metacognitive portrait of gifted learners. In ShavininaL. V. (Ed.), International handbook on giftedness (pp. 397-427). New York, NY: Springer.
5.
BensonJ. (1998). Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practice, 17, 10-22.
6.
BransfordJ. D.BrownA. L.CockingR. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school (Exp. ed.). Washington, DC: National Academies Press.
7.
BrownA. L. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In WeinertF. E.KluweR. H. (Eds.), Metacognition, motivation and understanding (pp. 65-116). Hillsdale, NJ: Lawrence Erlbaum.
8.
ByrneB. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.
9.
Cardelle-ElawarM. (1995). Effects of metacognitive instruction on low achievers in mathematics problems. Teaching and Teacher Education, 11, 81-95. doi:10.1016/0742-051X(94)00019-3
10.
CarrM.AlexanderJ.SchwanenflugelP. (1996). Where gifted children do and do not excel on metacognitive tasks. Roeper Review, 18, 212-217.
11.
ChengP.-W. (1993). Metacognition and giftedness: The state of the relationship. Gifted Child Quarterly, 37, 105-112. doi:10.1177/001698629303700302
12.
ColemanE. B.ShoreB. (1991). Problem-solving processes of high and average performers in Physics. Journal for the Education of the Gifted, 14, 366-379. doi:10.1177/016235329101400403
13.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. doi:10.1007/BF02310555
14.
De CorteE.GreerB.VerschaffelL. (1996). Mathematics teaching and learning. In BerlinerD. C.CalfeeR. C. (Eds.), Handbook of educational psychology (pp. 491-549). New York, NY: Macmillan Library Reference USA.
15.
DesoeteA. (2008). Multimethod assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition Learning, 3, 189-206. doi:10.1007/s11409-008-9026-0
16.
DesoeteA.RoeyersH.De ClercqA. (2003). Can offline metacognition enhance mathematical problem solving?Journal of Educational Psychology, 95, 188-200. doi:10.1037/0022-0663.95.1.188
17.
DignathC.ButtnerG. (2008). Components of fostering self-regulated learning among students: A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3, 231-264. doi:10.1007/s11409-008-9029-x
18.
Edwards-LeisC. (2006, December). Variations to stimulated recall protocols to enhance student reflection: I did, I saw, I remembered. Paper presented at the Australian Association for Research in Education Annual Conference, Adelaide, Australia.
19.
EricssonK. A.SimonH. A. (1993). Protocol analysis: Verbal reports as data (rev. ed.). Cambridge, MA: MIT Press.
20.
FlavellJ. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-development inquiry. American Psychologist, 34, 906-911. doi:10.1037/0003-066X.34.10.906
21.
FloydF. J.WidamanK. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. doi:10.1037/1040-3590.7.3.286
22.
FoxM. C.EricssonK. A.BestR. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137, 316-344. doi:10.1037/a0021663
23.
GarofaloJ.LesterF. K. (1985). Metacognition, cognitive monitoring, and mathematical performance. Journal for Research in Mathematics Education, 16, 163-176. doi:10.2307/748391
24.
HammouriH. A. M. (2003). An investigation of undergraduates’ transformational problem solving strategies: Cognitive/metacognitive processes as predictors of holistic/analytic strategies. Assessment and Evaluation in Higher Education, 28, 571-586. doi:10.1080/0260293032000130225
25.
HannahC. L.ShoreB. M. (2008). Twice-exceptional students use of metacognitive skills on a comprehension monitoring task. Gifted Child Quarterly, 52, 3-18. doi:10.1177/0016986207311156
26.
HattieJ. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.
27.
HaytonJ. C.AllenD. G.ScarpelloV. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7, 191-205. doi:10.1177/1094428104263675
28.
JacobsJ. E.ParisS. G. (1987). Children’s metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22, 255-278. doi:10.1207/s15326985ep2203&4_4
29.
JacobseA. E.HarskampE. G. (2012). Towards efficient measurement of metacognition in mathematical problem solving. Metacognition and Learning, 7, 133-149. doi:10.1007/s11409-012-9088-x
30.
KarakelleS.SaraçS. (2007). Validity and factor structure of Turkish versions of the Metacognitive Awareness Inventory for Children (Jr. MAI)—A and B forms. Türk Psikoloji Yazilari [Turkish Archive of Psychology], 10, 87-103.
31.
KeF. (2008a). A case study of computer gaming for math: Engaged learning from gameplay?Computers and Education, 51, 1609-1620. doi:10.1016/j.compedu.2008.03.003
32.
KeF. (2008b). Computer games application within alternative classroom goal structures: Cognitive, metacognitive, and affective evaluation. Educational Technology Research and Development, 56, 539-556. doi:10.1007/s11423-008-9086-5
33.
KimH. J.PedersenS. (2010). Young adolescents’ metacognition and domain knowledge as predictors of hypothesis-development performance in a computer-supported context. Educational Psychology, 30, 565-582. doi:10.1080/01443410.2010.491937
34.
KleinA. (1991). Changing directions: A program for academically talented students re-examines its most basic assumptions. Educator, 5, 4-7.
35.
KrutetskiiV. A. (1976). The psychology of mathematical abilities in school children. Chicago, IL: University of Chicago Press.
36.
LawsonM. J.RiceD. N. (1987). Thinking aloud: Analysing students’ mathematical performance. School Psychology International, 8, 233-243.
Mathematics Diagnostic Testing Project. (2006). The California State University/University of California Mathematics Diagnostic Testing Project. Retrieved from https://mdtp.ucsd.edu/about/index.html
39.
McDonaldR. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
40.
MevarechZ. R. (1999). Effects of metacognitive training embedded in cooperative settings on mathematical problem solving. Journal of Educational Research, 92, 195-205. doi:10.3102/00028312040001281
41.
MevarechZ. R.KramarskiB. (2003). The effects of metacognitive training versus worked-out examples on students’ mathematical reasoning. British Journal of Educational Psychology, 73, 449-471.
42.
MontagueM. (1991). Gifted and learning-disabled gifted students’ knowledge and use of mathematical problem-solving strategies. Journal for the Education of the Gifted, 14, 393-411. doi:10.1177/016235329101400405
43.
MontagueM.ApplegateB. (1993). Middle school students’ mathematical problem solving: An analysis of think-aloud protocols. Learning Disability Quarterly, 16, 19-32. doi:10.2307/1511157
44.
O’BrienJ. (1993). Action research through stimulated-recall. Research in Science Education, 23, 214-221. doi:10.1007/BF02357063
45.
PintrichP. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory Into Practice, 41, 219-225. doi:10.1207/s15430421tip4104_3
46.
PintrichP. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16, 385-407. doi:10.1007/s10648-004-0006-x
47.
PolyaG. (1945). How to solve it: A new aspect of mathematical method. Princeton, NJ: Princeton University Press.
48.
PuustinenM.PulkkinenL. (2001). Models of self-regulated learning: A review. Scandinavian Journal of Educational Research, 45, 269-286. doi:10.1080/00313830120074206
49.
Santos-TrigoM. (1996). An exploration of strategies used by students to solve problems with multiple ways of solution. Journal of Mathematical Behavior, 15, 263-284. doi:10.1016/S0732-3123(96)90006-1
50.
SchoenfeldA. H. (1981, April). Episodes and executive decisions in mathematical problem solving. Paper presented at the Annual Meeting of the American Educational Research Association, Los Angeles, CA.
51.
SchoenfeldA. H. (1985). Control. In SchoenfeldA. H. (Ed.), Mathematical problem solving (pp. 97-144). New York, NY: Academic Press.
52.
SchoenfeldA. H. (1987). What’s all the fuss about metacognition? In SchoenfeldA. H. (Ed.), Cognitive science and mathematics education (pp. 189-215). Hillsdale, NJ: Lawrence Erlbaum.
53.
SchrawG. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125.
SchwartzN. H.AndersenC.HongN.HowardB.McGeeS. (2004). Influence of metacognitive skills on learners’ memory of information in a hypermedia environment. Journal of Educational Computing Research, 31, 77-93. doi:10.2190/JE7W-VL6W-RNYF-RD4M
56.
ShoreB. M.DoverA. C. (1987). Metacognition, intelligence and giftedness. Gifted Child Quarterly, 31, 37-39. doi:10.1177/001698628703100108
57.
SnyderK. E.NietfeldJ. L.Linnenbrink-GarciaL. L. (2011). Giftedness and metacognition: A short-term longitudinal investigation of metacognitive monitoring in the classroom. Gifted Child Quarterly, 55, 181-193. doi:10.1177/0016986211412769
58.
SperlingR. A.HowardB. C.MillerL. A.MurphyC. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51-79. doi:10.1006/ceps.2001.1091
59.
SperlingR. A.RichmondA. S.RamsayC. M.KlappM. (2012). The measurement and predictive ability of metacognition in middle school learners. Journal of Educational Research, 105, 1-7. doi:10.1080/00220671.2010.514690
60.
SteinerH. H.CarrM. (2003). Cognitive development in gifted children: Toward a more precise understanding of emerging differences in intelligence. Educational Psychology Review, 15, 215-246. doi:10.1023/A:1024636317011
61.
SternbergR. J. (1980). Sketch of a componential subtheory of human intelligence. Behavioral and Brain Sciences, 3, 573-614.
62.
SubotnikR. F.Olszewski-KubiliusP.WorrellF. C. (2011). Rethinking giftedness and gifted education: A proposed direction forward based on psychological science. Psychological Science in the Public Interest, 12, 3-54. doi:10.1177/1529100611418056
63.
SwansonH. L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82, 306-314. doi:10.1037/0022-0663.82.2.306
64.
SwansonH. L. (1992). The relationship between metacognition and problem solving in gifted children. Roeper Review, 15, 43-48. doi:10.1080/02783199209553457
VandeveldeS.Van KeerH.De WeverB. (2011). Exploring the impact of student tutoring on at-risk fifth and sixth graders’ self-regulated learning. Learning and Individual Differences, 21, 419-425. doi:10.1016/j.lindif.2011.01.006
67.
VandiverB. J.WorrellF. C. (2002). The reliability and validity of scores on the Almost Perfect Scale-Revised with academically talented middle school students. Journal of Secondary Gifted Education, 13, 108-119.
68.
VerschaffelL.De CorteE.LasureS.Van VaerenberghG.BogaertsH.RatinckxE. (1999). Learning to solve mathematical application problems: A design experiment with fifth graders. Mathematical Thinking and Learning, 1, 195-229. doi:10.1207/s15327833mtl0103_2
69.
ViseuF.OliveiraI. B. (2012). Open-ended tasks in the promotion of classroom communication in mathematics. International Electronic Journal of Elementary Education, 4, 287-300.
70.
WaiJ.LubinskiD.BenbowC. P.SteigerJ. H. (2010). Accomplishment in science, technology, engineering, and mathematics (STEM) and its relation to STEM educational dose: A 25-year longitudinal study. Journal of Educational Psychology, 102, 860-871.
71.
WatkinsM. W. (2000). Monte Carlo PCA for parallel analysis. State College, PA: Ed & Psych Associates.
72.
WorrellF. C. (2007). Ethnic identity, academic achievement, and global self-concept in four groups of academically talented adolescents. Gifted Child Quarterly, 51, 23-38. doi:10.1177/0016986206296655
73.
Yilmaz-TüzünÖ.TopçuM. S. (2010). Investigating the relationships among elementary school students’ epistemological beliefs, metacognition, and constructivist science learning environment. Journal of Science Teacher Education, 21, 255-273. doi:10.1007/s10972-009-9163-6
74.
YoungA. E. (2011a, April). Examining the context-specific nature of metacognition in mathematics problem solving. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
75.
YoungA. E. (2011b). Explorations of metacognition among academically talented middle and high school mathematics students (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3413529)
76.
YoungA. E.WorrellF. C.GabelkoN. H. (2011). Predictors of success in accelerated and enrichment summer mathematics courses for academically talented students. Journal of Advanced Academics, 22, 558-577. doi:10.1177/1932202X11413886
77.
ZimmermanB. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41, 64-70. doi:10.1207/s15430421tip4102_2
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
