Restricted accessResearch articleFirst published online 2021-6
Measuring Statistics Attitudes at the Student and Instructor Levels: A Multilevel Construct Validity Study of the Survey of Attitudes Toward Statistics
Numerous studies have been conducted using the Survey of Attitudes Toward Statistics-36 (SATS-36). Recently, large-scale assessment studies have begun to examine the extent to which students vary in their statistics attitudes across instructors. Yet, empirical evidence linking student responses to the SATS items to instructor-level constructs is still lacking. Using multilevel confirmatory factor analysis, we investigated the factor structure underlying the measure of students’ statistics attitudes at both the student and instructor levels. Results from 13,507 college students taught by 160 introductory statistics instructors support a correlated six-factor model at each level. Additionally, there is evidence for the structural validity of a shared teacher–student attitude impacts construct that may capture meaningful patterns of teaching characteristics and competencies tied to student development of statistics attitudes. These findings provide empirical support for the use of the SATS-36 in studying contextual variables in relation to statistics instructors. Implications for educational practice are discussed.
AndridgeR. R.LittleR. J. A. (2010). A review of hot deck imputation for survey non-response. International Statistical Review, 78(1), 40-64. doi:10.1111/j.1751-5823.2010.00103.x
2.
BennettD. A. (2001). How can I deal with missing data in my study?Australian and New Zealand Journal of Public Health, 25(5), 464-469. doi:10.1111/J.1467-842X.2001.TB00294.X
3.
BlazarD.KraftM. A. (2017). Teacher and teaching effects on students’ attitudes and behaviors. Educational Evaluation and Policy Analysis, 39(1), 146-170. doi:10.3102/0162373716670260
4.
ByrneB. M.van de VijverF. J. R. (2010). Testing for measurement and structural equivalence in large-scale cross-cultural studies: Addressing the issue of nonequivalence. International Journal of Testing, 10(2), 107-132. doi:10.1080/15305051003637306
5.
CarlsonK. A.WinquistJ. R. (2011). Evaluating an active learning approach to teaching introductory statistics: A classroom workbook approach. Journal of Statistics Education, 19(1), 23. doi:10.1080/10691898.2011.11889596
6.
CashinS. E.ElmoreP. B. (2005). The survey of attitudes toward statistics scale: A construct validity study. Educational and Psychological Measurement, 65(3), 509-524. doi:10.1177/0013164404272488
7.
ChanceB.WongJ.TintleN. (2016). Student performance in curricula centered on simulation-based inference: A preliminary report. Journal of Statistics Education, 24(3), 114-126. doi:10.1080/10691898.2016.1223529
8.
ChiesiF.PrimiC. (2009). Assessing statistics attitudes among college students: Psychometric properties of the Italian version of the survey of attitudes toward statistics (SATS). Learning and Individual Differences, 19(2), 309-313. doi:10.1016/J.LINDIF.2008.10.008
9.
DavidovE.DülmerH.CieciuchJ.KuntzA.SeddigD.SchmidtP. (2018). Explaining measurement nonequivalence using multilevel structural equation modeling: The case of attitudes toward citizenship rights. Sociological Methods & Research, 47(4), 729-760. doi:10.1177/0049124116672678
10.
DedrickR. F.GreenbaumP. E. (2011). Multilevel confirmatory factor analysis of a scale measuring interagency collaboration of children’s mental health agencies. Journal of Emotional and Behavioral Disorders, 19(1), 27-40. doi:10.1177/1063426610365879
11.
DongY.PengC.-Y. J. (2013). Principled missing data methods for researchers. Springerplus, 2(1), 222. doi:10.1186/2193-1801-2-222
12.
EcclesJ. S.RoeserR. W. (1999). School and community influences on human development. In BornsteinM. H.LambM. E. (Eds.), Developmental psychology: An advanced textbook (pp. 503-554). Mahwah, NJ: Lawrence Erlbaum.
13.
EcclesJ. S.WigfieldA. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109-132. doi:10.1146/annurev.psych.53.100901.135153
14.
EmmioğluE.Capa-AydinY. (2012). Attitudes and achievement in statistics: A meta analysis study. Statistics Education Research Journal, 11(2), 95-102.
15.
GeldhofG. J.PreacherK. J.ZyphurM. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72-91. doi:10.1037/a0032138
16.
GrundS.LüdtkeO.RobitzschA. (2018). Multiple imputation of missing data for multilevel models: Simulations and recommendations. Organizational Research Methods, 21(1), 111-149. doi:10.1177/1094428117703686
17.
HuangF. L.CornellD. G. (2016). Using multilevel factor analysis with clustered data: Investigating the factor structure of the positive values scale. Journal of Psychoeducational Assessment, 34(1), 3-14. doi:10.1177/0734282915570278
18.
HuL. t.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. doi:10.1080/10705519909540118
19.
JakS.JorgensenT. D. (2017). Relating measurement invariance, cross-level invariance, and multilevel reliability. Frontiers in Psychology, 8, 1640. doi:10.3389/FPSYG.2017.01640
20.
KimE. S.DedrickR. F.CaoC.FerronJ. M. (2016). Multilevel factor analysis: Reporting guidelines and a review of reporting practices. Multivariate Behavioral Research, 51(6), 881-898. doi:10.1080/00273171.2016.1228042
21.
KowarikA.TemplM. (2016). Imputation with the R package VIM. Journal of Statistical Software, 74(7), 1-16. doi:10.18637/jss.v074.i07
22.
KozlowskiS. W. J.KleinK. J. (2000). A multi-level approach to theory and research in organizations: Contextual, temporal, and emergent processes. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods inorganizations: Foundations extensions, and new directions (pp. 3-90). San Francisco, CA: Jossey-Bass.
23.
KraftM. A. (2019). Teacher effects on complex cognitive skills and social-emotional competencies. Journal of Human Resources, 54(1), 1-36. doi:10.3368/jhr.54.1.0916.8265R3
24.
LaiB. S.LivingsM. S.D’AmicoM. P.HayatM. J.WilliamsJ. (2018). A growth mindset pilot intervention for a graduate-level biostatistics sourse. Statistics Education Research Journal, 17(2), 104-119.
25.
LanceC. E.ButtsM. M.MichelsL. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say?Organizational Research Methods, 9(2), 202-220. doi:10.1177/1094428105284919
26.
LauC.ChiesiF.HofmannJ.RuchW.SaklofskeD. H. (2020). The Italian version of the state-trait cheerfulness inventory trait form: Psychometric validation and evaluation of measurement invariance. Journal of Psychoeducational Assessment, 38(5), 613-626. doi:10.1177/0734282919875639
27.
LavidasK.BarkatsasT.ManesisD.GialamasV. (2020). A structural equation model investigating the impact of tertiary students’ attitudes toward statistics, perceived competence at mathematics, and engagement on statistics performance. Statistics Education Research Journal, 19(2), 27-41.
28.
LesserL. M.PearlD. K.WeberJ. J. (2016). Assessing fun items’ effectiveness in increasing learning of college introductory statistics students: Results of a randomized experiment. Journal of Statistics Education, 24(2), 54-62. doi:10.1080/10691898.2016.1190190
29.
LittleJ. (2013). Multilevel confirmatory ordinal factor analysis of the life skills profile-16. Psychological Assessment, 25(3), 810-825. doi:10.1037/A0032574
30.
LittleT. D.RhemtullaM.GibsonK.SchoemannA. M. (2013). Why the items versus parcels controversy needn’t be one. Psychological Methods, 18(3), 285-300. doi:10.1037/a0033266
31.
MarshH. W.HauK.-T.BallaJ. R.GraysonD. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33(2), 181-220. doi:10.1207/S15327906MBR3302_1
32.
MarshH. W.LüdtkeO.NagengastB.TrautweinU.MorinA. J. S.AbduljabbarA. S.KöllerO. (2012). Classroom climate and contextual effects : Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47(2), 106-124. doi:10.1080/00461520.2012.670488
33.
MartinA. J.MalmbergL.-E.LiemG. A. D. (2010). Multilevel motivation and engagement: Assessing construct validity across students and schools. Educational and Psychological Measurement, 70(6), 973-989. doi:10.1177/0013164410378089
34.
MuenksK.WigfieldA.EcclesJ. S. (2018). I can do this! The development and calibration of children’s expectations for success and competence beliefs. Developmental Review, 48, 24-39. doi:10.1016/j.dr.2018.04.001
35.
NolanM. M.BeranT.HeckerK. G. (2012). Surveys sssessing students’ attitudes toward statistics: A systematic review of validity and reliability. Statistics Education Research Journal, 11(2), 103-123.
36.
PaulW. L.CunningtonC. R. (2017). An exploration of student attitudes and satisfaction in a GAISE-influenced introductory statistics course. Statistics Education Research Journal, 16(2), 487-510.
37.
PerssonI.KrausK.HanssonL.WallentinF. (2019). Confirming the structure of the survey of attitudes toward statistics (SATS-36) by Swedish students. Statistics Education Research Journal, 18(1), 83-93.
38.
PiantaR. C.HamreB. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38(2), 109-119. doi:10.3102/0013189X09332374
39.
RamirezC.SchauC.EmmioğluE. (2012). The importance of attitudes in statistics education. Statistics Education Research Journal, 11(2), 57-71.
40.
RosseelY. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. doi:10.18637/jss.v048.i02
41.
SarikayaE. E.OkA.AydinY. C.SchauC. (2018). Turkish version of the survey of attitudes toward statistics: Factorial structure invariance by gender. The International Journal of Higher Education, 7(2), 121-127. doi:10.5430/ijhe.v7n2p121
42.
SchauC. (2003). Students’ attitudes: The “other” important outcome in statistics education. Paper presented atThe Joint Statistical Meetings, San Francisco, CA, 2-5 August 2003.
43.
SchauC.EmmioğluE. (2012). Do introductory statistics courses in the United States improve students’ attitudes?Statistics Education Research Journal, 11(2), 86-94.
44.
SchauC.StevensJ.DauphineeT. L.VecchioA. D. (1995). The development and validation of the survey of attitudes toward statistics. Educational and Psychological Measurement, 55(5), 868-875. doi:10.1177/0013164495055005022
45.
SchweigJ. (2014). Cross-level measurement invariance in school and classroom environment surveys: Implications for policy and practice. Educational Evaluation and Policy Analysis, 36(3), 259-280. doi:10.3102/0162373713509880
46.
StapletonL. M.YangJ. S.HancockG. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41(5), 481-520. doi:10.3102/1076998616646200
47.
TempelaarD. T.van der LoeffS. S.GijselaersW. H. (2007). A structural equation model analyzing the relationship of students’ attitudes toward statistics, prior reasoning abilities and course performance. Statistics Education Research Journal, 6(2), 78-102.
48.
TippinG. K.LafreniereK. D.PageS. (2012). Student perception of academic grading: Personality, academic orientation, and effort. Active Learning in Higher Education, 13(1), 51-61. doi:10.1177/1469787411429187
49.
VanHoofS.KuppensS.SotosA. E. C.VerschaffelL.OnghenaP. (2011). Measuring statistics attitudes: Structure of the survey of attitudes toward statistics (SATS-36). Statistics Education Research Journal, 10(1), 35-52.
50.
WigfieldA.CambriaJ. (2010). Students’ achievement values, goal orientations, and interest: Definitions, development, and relations to achievement outcomes. Developmental Review, 30(1), 1-35. doi:10.1016/J.DR.2009.12.001
51.
XuC.PetersM.BrownS. (2020). Instructor and instructional effects on students’ statistics attitudes. Statistics Education Research Journal, 19(2), 7-26.
52.
XuC.SchauC. (2019). Exploring method effects in the six-factor structure of the survey of attitudes toward statistics (SATS-36). Statistics Education Research Journal, 18(2), 39-53.
53.
ZyphurM. J.KaplanS. A.ChristianM. S. (2008). Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions. Group Dynamics: Theory, Research, and Practice, 12(2), 127-140. doi:10.1037/1089-2699.12.2.127
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
