Few studies have examined the psychometric properties of the test-related items from the Achievement Emotions Questionnaire (AEQ). Using a sample of 955 university students, we examined the factor structure of 12 emotion items measuring test-related anger, boredom, enjoyment, and pride. Results indicated the four emotions were distinct, allowing for further examination of measurement invariance of the items across testing platform (computer-based vs. pencil-paper test) and gender. Configural, metric, and scalar invariance was supported for testing platform and gender. Given interest in modeling change in emotions, we also examined whether the items were invariant across time by measuring emotions at 3 times during a test. Using a sample of 651 university students, configural, metric, and scalar invariance was supported across time. The findings provide important structural validity evidence for the item responses and support the comparison of emotions across testing platform, gender, and time.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
2.
BarryC. L.FinneyS. J. (2009). Does it matter how data are collected? A comparison of testing conditions and the implications for validity. Research & Practice in Assessment, 3, 1–15.
3.
BashkovB.FinneyS. J. (2013). Applying Longitudinal Mean and Covariance Structures (LMACS) analysis to assess construct stability over two time points: An example using psychological entitlement. Measurement and Evaluation in Counseling and Development, 46, 289–314.
4.
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464–504.
5.
CheungG. W.RensvoldR. B. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1–27.
6.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.
7.
FinneyS. J.DiStefanoC.KoppJ. P. (2016). Overview of estimation methods and preconditions for their application with structural equation modeling. In SchweizerK.DiStefanoC. (Eds.), Principles and methods of test construction: Standards and recent advancements (pp. 135–165). Hogrefe.
8.
FinneyS. J.MathersC. E.MyersA. J. (2016). Investigating the dimensionality of examinee motivation across instruction conditions in low-stakes testing contexts. Research & Practice in Assessment, 11, 5–17.
9.
FinneyS. J.SatkusP.PerkinsB. A. (in press). The effect of perceived test importance and examinee emotions on expended effort during a low-stakes test: A longitudinal panel model. Educational Assessment.
10.
GoetzT.PreckelF.PekrunR.HallN. C. (2007). Emotional experiences during test taking: Does cognitive ability make a difference?Learning and Individual Differences, 17, 3–16.
11.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
JurichD.BradshawL. (2014). Diagnosing psychosocial research attributes: An illustration of diagnostic classification modeling. International Journal of Testing, 14, 49–72.
14.
MarshH. W.HauK.-T.WenZ. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers to overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320–341.
15.
MyersA. J.FinneyS. J. (in press). Change in self-reported motivation before to after test completion: Relation with performance. Journal of Experimental Education.
16.
PekrunR. (2017). Achievement emotions. In ElliotA. J.DweckC. S.YeagerD. S. (Eds.), Handbook of competence and motivation: Theory and application (pp. 251–271). Guilford Press.
17.
PekrunR.GoetzT.FrenzelA. C.BarchfeldP.PerryR. P. (2011). Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36, 36–48.
18.
PekrunR.GoetzT.PerryR. P.KramerK.HochstadtM.MolfenterS. (2004). Beyond test anxiety: Development and validation of the Test Emotions Questionnaire (TEQ). Anxiety, Stress, and Coping, 17, 287–316.
19.
PekrunR.GoetzT.TitzW.PerryR. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37, 91–105.
20.
PekrunR.LichtenfeldS.MarshH. W.MurayamaK.GoetzT. (2017). Achievement emotions and academic performance: Longitudinal models of reciprocal effects. Child Development, 88, 1653–1670.
21.
PerkinsB. A.FinneyS. J.SatkusP. (2019, July). Predicting examinee effort and test performance from change in emotions during a test: A latent growth model [Paper presentation]. The European Conference on Psychological Assessment, Brussels, Belgium.
22.
ReeveC. L.BonaccioS.WinfordE. C. (2014). Cognitive ability, exam-related emotions and exam performance: A field study in a college setting. Contemporary Educational Psychology, 39, 124–133.
23.
RhemtullaM.Brosseau-LiardP.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373.
24.
SteenkampJ.BaumgartnerH. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25, 78–90.
U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. The Library Quarterly, 77, 89–92.
27.
WiseS. L.DeMarsC. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10, 1–17.
28.
ZeidnerM. (2014). Anxiety in education. In PekrunR.Linnenbrink-GarciaL. (Eds.), International handbook of emotions in education (pp. 265–288). Taylor & Francis.
