Comparing Metacognition Assessments of Mathematics in Academically Talented Students

Abstract

Two studies were conducted to examine and compare the construct validity of scores on the Junior Metacognition Awareness Inventory (JMAI) and problem-solving interview protocols. Participants consisted of 183 middle and high school students attending a university summer program for academically talented youth. Study 1 results indicated that JMAI scores were internally consistent and yielded an interpretable two-factor structure after the elimination of several items; however, the scores were not significantly or meaningfully related to GPA or current and future mathematics achievement. In Study 2 (n = 30), JMAI scores did not predict students’ metacognitive behaviors during mathematics problem-solving tasks. In contrast, students’ metacognitive behaviors observed during problem solving were meaningfully related to mathematics achievement with medium to high effect sizes. Findings support the predictive validity of metacognition with regard to academic achievement when operationalized with problem-solving interviews, but call into question the criterion-related validity of JMAI scores.

Keywords

Junior Metacognition Awareness Inventory mathematics achievement metacognition problem solving scale development

Get full access to this article

View all access options for this article.

References

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Arum

Roksa

(2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.

Aydin

Ubuz

(2010). Structural model of metacognition and knowledge of geometry. Learning and Individual Differences, 20, 436-445. doi:10.1016/j.lindif.2010.06.002

Barfurth

M. A.

Ritchie

K. C.

Irving

J. A.

Shore

B. M.

(2009). A metacognitive portrait of gifted learners. In Shavinina

L. V.

(Ed.), International handbook on giftedness (pp. 397-427). New York, NY: Springer.

Benson

(1998). Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practice, 17, 10-22.

Bransford

J. D.

Brown

A. L.

Cocking

R. R.

(Eds.). (2000). How people learn: Brain, mind, experience, and school (Exp. ed.). Washington, DC: National Academies Press.

Brown

A. L.

(1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In Weinert

F. E.

Kluwe

R. H.

(Eds.), Metacognition, motivation and understanding (pp. 65-116). Hillsdale, NJ: Lawrence Erlbaum.

Byrne

B. M.

(2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.

Cardelle-Elawar

(1995). Effects of metacognitive instruction on low achievers in mathematics problems. Teaching and Teacher Education, 11, 81-95. doi:10.1016/0742-051X(94)00019-3

10.

Carr

Alexander

Schwanenflugel

(1996). Where gifted children do and do not excel on metacognitive tasks. Roeper Review, 18, 212-217.

11.

Cheng

P.-W.

(1993). Metacognition and giftedness: The state of the relationship. Gifted Child Quarterly, 37, 105-112. doi:10.1177/001698629303700302

12.

Coleman

E. B.

Shore

(1991). Problem-solving processes of high and average performers in Physics. Journal for the Education of the Gifted, 14, 366-379. doi:10.1177/016235329101400403

13.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. doi:10.1007/BF02310555

14.

De Corte

Greer

Verschaffel

(1996). Mathematics teaching and learning. In Berliner

D. C.

Calfee

R. C.

(Eds.), Handbook of educational psychology (pp. 491-549). New York, NY: Macmillan Library Reference USA.

15.

Desoete

(2008). Multimethod assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition Learning, 3, 189-206. doi:10.1007/s11409-008-9026-0

16.

Desoete

Roeyers

De Clercq

(2003). Can offline metacognition enhance mathematical problem solving? Journal of Educational Psychology, 95, 188-200. doi:10.1037/0022-0663.95.1.188

17.

Dignath

Buttner

(2008). Components of fostering self-regulated learning among students: A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3, 231-264. doi:10.1007/s11409-008-9029-x

18.

Edwards-Leis

(2006, December). Variations to stimulated recall protocols to enhance student reflection: I did, I saw, I remembered. Paper presented at the Australian Association for Research in Education Annual Conference, Adelaide, Australia.

19.

Ericsson

K. A.

Simon

H. A.

(1993). Protocol analysis: Verbal reports as data (rev. ed.). Cambridge, MA: MIT Press.

20.

Flavell

J. H.

(1979). Metacognition and cognitive monitoring: A new area of cognitive-development inquiry. American Psychologist, 34, 906-911. doi:10.1037/0003-066X.34.10.906

21.

Floyd

F. J.

Widaman

K. F.

(1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. doi:10.1037/1040-3590.7.3.286

22.

Fox

M. C.

Ericsson

K. A.

Best

(2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137, 316-344. doi:10.1037/a0021663

23.

Garofalo

Lester

F. K.

(1985). Metacognition, cognitive monitoring, and mathematical performance. Journal for Research in Mathematics Education, 16, 163-176. doi:10.2307/748391

24.

Hammouri

H. A. M.

(2003). An investigation of undergraduates’ transformational problem solving strategies: Cognitive/metacognitive processes as predictors of holistic/analytic strategies. Assessment and Evaluation in Higher Education, 28, 571-586. doi:10.1080/0260293032000130225

25.

Hannah

C. L.

Shore

B. M.

(2008). Twice-exceptional students use of metacognitive skills on a comprehension monitoring task. Gifted Child Quarterly, 52, 3-18. doi:10.1177/0016986207311156

26.

Hattie

(2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.

27.

Hayton

J. C.

Allen

D. G.

Scarpello

(2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7, 191-205. doi:10.1177/1094428104263675

28.

Jacobs

J. E.

Paris

S. G.

(1987). Children’s metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22, 255-278. doi:10.1207/s15326985ep2203&4_4

29.

Jacobse

A. E.

Harskamp

E. G.

(2012). Towards efficient measurement of metacognition in mathematical problem solving. Metacognition and Learning, 7, 133-149. doi:10.1007/s11409-012-9088-x

30.

Karakelle

Saraç

(2007). Validity and factor structure of Turkish versions of the Metacognitive Awareness Inventory for Children (Jr. MAI)—A and B forms. Türk Psikoloji Yazilari [Turkish Archive of Psychology], 10, 87-103.

31.

(2008a). A case study of computer gaming for math: Engaged learning from gameplay? Computers and Education, 51, 1609-1620. doi:10.1016/j.compedu.2008.03.003

32.

(2008b). Computer games application within alternative classroom goal structures: Cognitive, metacognitive, and affective evaluation. Educational Technology Research and Development, 56, 539-556. doi:10.1007/s11423-008-9086-5

33.

Kim

H. J.

Pedersen

(2010). Young adolescents’ metacognition and domain knowledge as predictors of hypothesis-development performance in a computer-supported context. Educational Psychology, 30, 565-582. doi:10.1080/01443410.2010.491937

34.

Klein

(1991). Changing directions: A program for academically talented students re-examines its most basic assumptions. Educator, 5, 4-7.

35.

Krutetskii

V. A.

(1976). The psychology of mathematical abilities in school children. Chicago, IL: University of Chicago Press.

36.

Lawson

M. J.

Rice

D. N.

(1987). Thinking aloud: Analysing students’ mathematical performance. School Psychology International, 8, 233-243.

37.

MacCallum

R. C.

Widaman

K. F.

Zhang

Hong

(1999). Sample size in factor analysis. Psychological Methods, 4, 84-99. doi:10.1037/1082-989X.4.1.84

38.

Mathematics Diagnostic Testing Project. (2006). The California State University/University of California Mathematics Diagnostic Testing Project. Retrieved from https://mdtp.ucsd.edu/about/index.html

39.

McDonald

R. P.

(1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.

40.

Mevarech

Z. R.

(1999). Effects of metacognitive training embedded in cooperative settings on mathematical problem solving. Journal of Educational Research, 92, 195-205. doi:10.3102/00028312040001281

41.

Mevarech

Z. R.

Kramarski

(2003). The effects of metacognitive training versus worked-out examples on students’ mathematical reasoning. British Journal of Educational Psychology, 73, 449-471.

42.

Montague

(1991). Gifted and learning-disabled gifted students’ knowledge and use of mathematical problem-solving strategies. Journal for the Education of the Gifted, 14, 393-411. doi:10.1177/016235329101400405

43.

Montague

Applegate

(1993). Middle school students’ mathematical problem solving: An analysis of think-aloud protocols. Learning Disability Quarterly, 16, 19-32. doi:10.2307/1511157

44.

O’Brien

(1993). Action research through stimulated-recall. Research in Science Education, 23, 214-221. doi:10.1007/BF02357063

45.

Pintrich

P. R.

(2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory Into Practice, 41, 219-225. doi:10.1207/s15430421tip4104_3

46.

Pintrich

P. R.

(2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16, 385-407. doi:10.1007/s10648-004-0006-x

47.

Polya

(1945). How to solve it: A new aspect of mathematical method. Princeton, NJ: Princeton University Press.

48.

Puustinen

Pulkkinen

(2001). Models of self-regulated learning: A review. Scandinavian Journal of Educational Research, 45, 269-286. doi:10.1080/00313830120074206

49.

Santos-Trigo

(1996). An exploration of strategies used by students to solve problems with multiple ways of solution. Journal of Mathematical Behavior, 15, 263-284. doi:10.1016/S0732-3123(96)90006-1

50.

Schoenfeld

A. H.

(1981, April). Episodes and executive decisions in mathematical problem solving. Paper presented at the Annual Meeting of the American Educational Research Association, Los Angeles, CA.

51.

Schoenfeld

A. H.

(1985). Control. In Schoenfeld

A. H.

(Ed.), Mathematical problem solving (pp. 97-144). New York, NY: Academic Press.

52.

Schoenfeld

A. H.

(1987). What’s all the fuss about metacognition? In Schoenfeld

A. H.

(Ed.), Cognitive science and mathematics education (pp. 189-215). Hillsdale, NJ: Lawrence Erlbaum.

53.

Schraw

(1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125.

54.

Schraw

Dennison

R. S.

(1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475. doi:10.1006/ceps.1994.1033

55.

Schwartz

N. H.

Andersen

Hong

Howard

McGee

(2004). Influence of metacognitive skills on learners’ memory of information in a hypermedia environment. Journal of Educational Computing Research, 31, 77-93. doi:10.2190/JE7W-VL6W-RNYF-RD4M

56.

Shore

B. M.

Dover

A. C.

(1987). Metacognition, intelligence and giftedness. Gifted Child Quarterly, 31, 37-39. doi:10.1177/001698628703100108

57.

Snyder

K. E.

Nietfeld

J. L.

Linnenbrink-Garcia

L. L.

(2011). Giftedness and metacognition: A short-term longitudinal investigation of metacognitive monitoring in the classroom. Gifted Child Quarterly, 55, 181-193. doi:10.1177/0016986211412769

58.

Sperling

R. A.

Howard

B. C.

Miller

L. A.

Murphy

(2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51-79. doi:10.1006/ceps.2001.1091

59.

Sperling

R. A.

Richmond

A. S.

Ramsay

C. M.

Klapp

(2012). The measurement and predictive ability of metacognition in middle school learners. Journal of Educational Research, 105, 1-7. doi:10.1080/00220671.2010.514690

60.

Steiner

H. H.

Carr

(2003). Cognitive development in gifted children: Toward a more precise understanding of emerging differences in intelligence. Educational Psychology Review, 15, 215-246. doi:10.1023/A:1024636317011

61.

Sternberg

R. J.

(1980). Sketch of a componential subtheory of human intelligence. Behavioral and Brain Sciences, 3, 573-614.

62.

Subotnik

R. F.

Olszewski-Kubilius

Worrell

F. C.

(2011). Rethinking giftedness and gifted education: A proposed direction forward based on psychological science. Psychological Science in the Public Interest, 12, 3-54. doi:10.1177/1529100611418056

63.

Swanson

H. L.

(1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82, 306-314. doi:10.1037/0022-0663.82.2.306

64.

Swanson

H. L.

(1992). The relationship between metacognition and problem solving in gifted children. Roeper Review, 15, 43-48. doi:10.1080/02783199209553457

65.

Topçu

M. S.

Yilmaz-Tüzün

. (2009). Elementary students’ metacognition and epistemological beliefs considering science achievement, gender and socioeconomic status. Elementary Education Online, 8, 676-693.

66.

Vandevelde

Van Keer

De Wever

(2011). Exploring the impact of student tutoring on at-risk fifth and sixth graders’ self-regulated learning. Learning and Individual Differences, 21, 419-425. doi:10.1016/j.lindif.2011.01.006

67.

Vandiver

B. J.

Worrell

F. C.

(2002). The reliability and validity of scores on the Almost Perfect Scale-Revised with academically talented middle school students. Journal of Secondary Gifted Education, 13, 108-119.

68.

Verschaffel

De Corte

Lasure

Van Vaerenbergh

Bogaerts

Ratinckx

(1999). Learning to solve mathematical application problems: A design experiment with fifth graders. Mathematical Thinking and Learning, 1, 195-229. doi:10.1207/s15327833mtl0103_2

69.

Viseu

Oliveira

I. B.

(2012). Open-ended tasks in the promotion of classroom communication in mathematics. International Electronic Journal of Elementary Education, 4, 287-300.

70.

Wai

Lubinski

Benbow

C. P.

Steiger

J. H.

(2010). Accomplishment in science, technology, engineering, and mathematics (STEM) and its relation to STEM educational dose: A 25-year longitudinal study. Journal of Educational Psychology, 102, 860-871.

71.

Watkins

M. W.

(2000). Monte Carlo PCA for parallel analysis. State College, PA: Ed & Psych Associates.

72.

Worrell

F. C.

(2007). Ethnic identity, academic achievement, and global self-concept in four groups of academically talented adolescents. Gifted Child Quarterly, 51, 23-38. doi:10.1177/0016986206296655

73.

Yilmaz-Tüzün

Ö.

Topçu

M. S.

(2010). Investigating the relationships among elementary school students’ epistemological beliefs, metacognition, and constructivist science learning environment. Journal of Science Teacher Education, 21, 255-273. doi:10.1007/s10972-009-9163-6

74.

Young

A. E.

(2011a, April). Examining the context-specific nature of metacognition in mathematics problem solving. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

75.

Young

A. E.

(2011b). Explorations of metacognition among academically talented middle and high school mathematics students (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3413529)

76.

Young

A. E.

Worrell

F. C.

Gabelko

N. H.

(2011). Predictors of success in accelerated and enrichment summer mathematics courses for academically talented students. Journal of Advanced Academics, 22, 558-577. doi:10.1177/1932202X11413886

77.

Zimmerman

B. J.

(2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41, 64-70. doi:10.1207/s15430421tip4102_2

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB