An Integration of Multiple Student Engagement Dimensions into a Single Measure and Validity-Based Studies

Abstract

Student engagement is typically regarded as being a multidimensional construct, but there remains no clear consensus about its precise conceptualization. Several current measures of student engagement are limited by their omission of relevant dimensions and/or poor correspondence between dimension labels and item content. To address these limitations, we integrated dimensions from two validated student engagement instruments, each with different approaches to the dimensionality of student engagement, into a single measure and assessed its psychometric properties. We concluded from factor analyses that this scale captures seven distinct-yet-related engagement dimensions, including students’ perceptions of contextual influences, which load on a global higher-order student engagement construct. Structural equation modeling (SEM) indicated that poor academic performance was linked to lower student engagement and that lower student engagement was associated with worse emotional well-being, confirming the concurrent validity of the scale. The results validate the integrated measure of engagement as a comprehensive assessment of student engagement in Portuguese adolescents.

Keywords

student engagement dimensionality integration psychometrics bifactor model multilevel

Get full access to this article

View all access options for this article.

References

Appleton

J. J.

Christenson

S. L.

Kim

Reschly

A. L.

(2006). Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument. Journal of School Psychology, 44, 427-445. doi:10.1016/j.jsp.2006.04.002

Appleton

J. J.

Christenson

S. L.

Furlong

M. J.

(2008). Student Engagement with School: Critical Conceptual and Methodological Issues of the Construct. Psychology in the Schools, 45, 369-386. doi:10.1002/pits

Betts

J. E.

Appleton

J. J.

Reschly

A. L.

Christenson

S. L.

Huebner

S. E.

(2010). A study of the factorial invariance of the Student Engagement Instrument (SEI): Results from middle and high school students. School Psychology Quarterly, 25, 84-93. doi:10.1037/a0020259

Birch

S. H.

Ladd

G. W.

(1997). The teacher-child relationship and children’s early school adjustment. Journal of School Psychology, 35, 61-79.

Borsboom

Mellenbergh

G. J.

Van Heerden

(2004). The concept of validity. Psychological Review, 111, 1061-1071. doi:10.1037/0033-295X.111.4.1061

Browne

M. W.

Cudeck

(1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230-258. doi:10.1177/0049124192021002005

Carbonaro

(2005). Tracking, students’ effort, and academic achievement. Sociology of Education, 78, 27-49.

Chase

P. A.

Hilliard

L. J.

Geldhof

G. J.

Warren

D. J. A.

Lerner

R. M.

(2014). Academic achievement in the high school years: The changing role of school engagement. Journal of Youth and Adolescence, 43, 884-896. doi:10.1007/s10964-013-0085-4

Chen

F. F.

West

S. G.

Sousa

K. H.

(2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225. doi:10.1207/s15327906mbr4102

10.

Connell

J. P.

Wellborn

J. G.

(1991). Competence, autonomy and relatedness: A motivational analysis of self-system processes. In Gunnar

M. R.

Sroufe

L. A.

(Eds.), Minnesota Symposium on Child Psychology (pp. 43-77). Hillsdale, NJ: Lawrence Erlbaum.

11.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302. doi:10.1037/h0040957

12.

Deci

E. L.

Ryan

R. M.

(1985). Intrinsic motivation and self-determination in human behaviour. New York, NY: Plenum.

13.

Dunn

T. J.

Baguley

Brunsden

(2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399-412. doi:10.1111/bjop.12046

14.

Fall

A. M.

Roberts

(2012). High school dropouts: Interactions between social context, self-perceptions, school engagement, and student dropout. Journal of Adolescence, 35, 787-798. doi:10.1016/j.adolescence.2011.11.004

15.

Ferguson

C. J.

(2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40, 532-538. doi:10.1037/a0015808

16.

Finn

J. D.

(1989). Withdrawing from school. Review of Educational Research, 59, 117. doi:10.2307/1170412

17.

Finn

J. D.

Rock

D. A.

(1997). Academic success among students at risk for school failure. Journal of Applied Psychology, 82, 221-234. doi:10.1037/0021-9010.82.2.221

18.

Fredricks

J. A.

Blumenfeld

P. C.

Friedel

Paris

A. H.

(2005). School engagement. In Moore

K. A.

Lippman

(Eds.), Conceptualizing and measuring indicators of positive development: What do children need to flourish? (pp. 305-321). New York, NY: Kluwer Academic/Plenum Press.

19.

Fredricks

J. A.

Blumenfeld

P. C.

Paris

A. H.

(2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59-109. doi:10.3102/00346543074001059

20.

Fredrickson

B. L.

(2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56, 218-226. doi:10.1037/0003-066X.56.3.218

21.

Galinha

I. C.

Pais-Ribeiro

(2005). Contribuição para o estudo da versão portuguesa da positive and negative affect schedule (PANAS): II—Estudo psicométrico [Contribution to the study of the positive and negative affect schedule (PANAS) Portuguese version: II—Psychometric Study]. Análise Psicológica, 23, 219-227.

22.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185. doi:10.1007/BF02289447

23.

Hox

J. J.

(2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum.

24.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118

25.

Jimerson

S. R.

Campos

Greif

J. L.

(2003). Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist, 8, 7-27. doi:10.1007/BF03340896

26.

Kruyen

P. M.

Emons

W. H. M.

Sijtsma

(2012). Test length and decision quality in personnel selection: When is short too short? International Journal of Testing, 12, 321-344. doi:10.1080/15305058.2011.643517

27.

Lam

Jimerson

Wong

B. P. H.

Kikas

Shin

Veiga

F. H.

. . . Zollneritsch

(2014). Understanding and measuring student engagement in school: The results of an international study from 12 countries. School Psychology Quarterly, 29, 213-232. doi:10.1037/spq0000057

28.

Lee

J. S.

(2014). The relationship between student engagement and academic performance: Is it a myth or reality? The Journal of Educational Research, 107, 177-185. doi:10.1080/00220671.2013.807491

29.

Lerner

R. M.

(2011). Trajectories of school engagement during adolescence: Implications for grades, depression, delinquency, and substance use. Developmental Psychology, 47, 233-247.

30.

(2000). Socioeconomic gaps in academic achievement within schools: Are they consistent across subject areas? Educational Research and Evaluation, 71, 543-562.

31.

Moreira

P. A. S.

Cunha

Castro

Inman

R. A.

(2018). The Student School Engagement Survey (SSES): A measure of emotional, cognitive, and behavioral engagement? Manuscript Submitted for Publication.

32.

Moreira

P. A. S.

Dias

Matias

Castro

Gaspar

Oliveira

(2018). School effects on students’ engagement with school: Academic performance moderates the effect of school support for learning on students’ engagement. Learning and Individual Differences, 67, 67-77. doi:10.1016/j.lindif.2018.07.007

33.

Moreira

P. A. S.

Dias

M. A.

(2018). Tests of factorial structure and measurement invariance for the Student Engagement Instrument: Evidence from middle and high school students. International Journal of School & Educational Psychology. Advance online publication. doi:10.1080/21683603.2017.1414004

34.

Moreira

P. A. S.

Vaz

F. M.

Dias

P. C.

Petracchi

(2009). Psychometric properties of the Portuguese version of the Student Engagement Instrument. Canadian Journal of School Psychology, 24, 303-317. doi:10.1177/0829573509346680

35.

National Center for School Engagement. (2006). Quantifying School Engagement: Research Report. Denver, CO. Retrieved from http://schoolengagement.org/school-engagement-services/ncse-publications/reports/

36.

Nguyen

T. D.

Cannata

Miller

(2016). Understanding student behavioral engagement: Importance of student interaction with peers and teachers. Journal of Educational Research, 111, 163-174. doi:10.1080/00220671.2016.1220359

37.

Nunnally

J. C.

(1967). Psychometric theory. New York, NY: McGraw-Hill.

38.

Reise

S. P.

Moore

T. M.

Haviland

M. G.

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. doi:10.1080/00223891.2010.496477.Bifactor

39.

Reise

S. P.

Scheines

Widaman

K. F.

Haviland

M. G.

(2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73, 5-26. doi:10.1177/0013164412449831

40.

Reschly

A. L.

Christenson

S. L.

(2012). Jingle, Jangle, and Conceptual Haziness: Evolution and Future Directions of the Engagement Construct. In Christenson

S. L.

Reschly

A. L.

Wylie

(Eds.), Handbook of Research on Student Engagement (pp. 3-19). Boston, MA: Springer.

41.

Reschly

A. L.

Huebner

S. E.

Appleton

J. J.

Antaramian

(2008). Engagement as flourishing: The contribution of positive emotions and coping to adolescents’ engagement at school and with learning. Psychology in the Schools, 45, 419-431. doi:10.1002/pits.20306

42.

Sinclair

M. F.

Christenson

S. L.

Lehr

C. A.

Reschly

A. L.

(2003). Facilitating student engagement: Lessons learned from Check & Connect longitudinal studies. The California School Psychologist, 8, 29-41. doi:10.1007/BF03340894

43.

Skinner

E. A.

Furrer

Marchand

Kindermann

(2008). Engagement and disaffection in the classroom: Part of a larger motivational dynamic? Journal of Educational Psychology, 100, 765-781. doi:10.1037/a0012840

44.

Skinner

E. A.

Pitzer

J. R.

(2012). Developmental dynamics of student engagement, coping, and everyday resilience. In Skinner

Pitzer

Brule

(Eds.), Handbook of research on student engagement (pp. 21-44). Boston, MA: Springer.

45.

Virtanen

T. E.

Moreira

Ulvseth

Andersson

Tetler

Kuorelahti

(2018). Analyzing measurement invariance of the students’ engagement instrument brief version: The cases of Denmark, Finland, and Portugal. Canadian Journal of School Psychology, 33, 297-313. doi:10.1177/0829573517699333

46.

Wang

M.-T.

Chow

Hofkens

Salmela-Aro

(2015). The trajectories of student emotional engagement and school burnout with academic and psychological development: Findings from Finnish adolescents. Learning and Instruction, 36, 57-65. doi:10.1016/j.learninstruc.2014.11.004

47.

Wang

M.-T.

Eccles

J. S.

(2012). Social support matters: Longitudinal effects of social support on three dimensions of school engagement from middle to high school. Child Development, 83(3), 877-895. doi:10.1111/j.1467-8624.2012.01745.x

48.

Wang

M.-T.

Fredricks

J. A.

(2014). The reciprocal links between school engagement, youth problem behaviors, and school dropout during adolescence. Child Development, 85, 722-737. doi:10.1111/cdev.12138

49.

Wang

M.-T.

Fredricks

J. A.

Hofkens

Linn

J. S.

(2017). Conceptualization and assessment of adolescents’ engagement and disengagement in school: A Multidimensional School Engagement Scale. European Journal of Psychological Assessment. Advance online publication. doi:10.1027/1015-5759/a000431

50.

Wang

M.-T.

Holcombe

(2010). Adolescents’ perceptions of school environment, engagement, and academic achievement in middle school. American Educational Research Journal, 47, 633-662. doi:10.3102/ 0002831209361209

51.

Wang

M.-T.

Peck

S. C.

(2013). Adolescent educational success and mental health vary across school engagement profiles. Developmental Psychology, 49, 1266-1276. doi:10.1037/a0030028

52.

Watson

Clark

L. A.

Tellegen

(1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063-1070. doi:10.1037/0022-3514.54.6.1063

53.

Yazzie-Mintz

(2007). Voices of Students on Engagement: A Report on the 2006 High School Survey of Student Engagement. Center for Evaluation and Education Policy, Indiana University. Retrieved from https://files.eric.ed.gov/fulltext/ED495758.pdf%0Ahttp://ezproxy.lib.utexas.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED495758&site=ehost-live

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.33 MB