Engagement, as a multidimensional construct, fluctuates due to various factors. Previous research that examined engagement as a subjective experience was mainly concerned with the qualitative structure of engagement in different contexts (e.g., task difficulty, aesthetics). Few studies have examined the user’s subjective engagement as a given task progresses over time. This paper reports findings from an online study that aimed to assess the usefulness of subjective measures in capturing momentary feelings related to task engagement, under varying task duration and difficulty conditions. The Short Stress State Questionnaire (SSSQ) and the User Engagement Scale (UES) were used to capture self-reported engagement during a Tetris game. The sensitivity of scales to task conditions and relationships among subscales were examined. Results showed that changes in SSSQ were sensitive to difficulty levels, and Engagement obtained from the SSSQ was highly correlated with UES subscales. SSSQ may be a particularly useful tool to capture participants’ momentary feelings during a task via its Engagement, Distress, and Worry subscales.
Get full access to this article
View all access options for this article.
References
1.
BerkaC.LevendowskiD. J.LumicaoM. N.YauA.DavisG.ZivkovicV. T.OlmsteadR. E.TremouletP. D.CravenP. L. (2007). EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks. 78(5), 14.
2.
ChapmanP. M. (1997). Models of engagement: Intrinsically motivated interaction with multimedia learning software (Doctoral dissertation, University of Waterloo). (n.d.). Models of engagement: Intrinsically motivated interaction with multimedia learning software (Doctoral dissertation, University of Waterloo).
3.
ComfreyA.LeeH. (1992). Interpretation and application of factor analytic results. Comrey AL, Lee HB. A First Course in Factor Analysis, 2, 1992.
4.
CsikszentmihalyiM. (1990). Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990.
5.
DohertyK.DohertyG. (2019). Engagement in HCI: Conception, Theory, and Measurement. ACM Computing Surveys, 51(5), 1–39.
6.
EwingK. C.FaircloughS. H.GilleadeK. (2016). Evaluation of an Adaptive Game that Uses EEG Measures Validated during the Design Process as Inputs to a Biocybernetic Loop. Frontiers in Human Neuroscience, 10, 223.
7.
HeltonW. S. (2004). Validation of a Short Stress State Questionnaire. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 48(11), 1238–1242.
8.
HigginsE. T. (2006). Value from hedonic experience and engagement. Psychological Review, 113(3), 439–460.
9.
HornJ. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185.
10.
JordanC. (1992). Experimental study of the effect of an instantaneous self assessment workload recorder on task performance. Report No. DRA/TM (CAD5)/92011. Farnborough: Defence Evaluation & Research Agency.
11.
KellerJ.RingelhanS.BlomannF. (2011). Does skills–demands compatibility result in intrinsic motivation? Experimental test of a basic notion proposed in the theory of flow-experiences. The Journal of Positive Psychology, 6(5), 408–417.
12.
KimS.BaekT. H. (2018). Examining the antecedents and consequences of mobile app engagement. Telematics and Informatics, 35(1), 148–158.
13.
KölesM.SzegletesL.ForstnerB. (2015). Towards a physiology-based difficulty control system for serious games. 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), 323–328.
14.
LaurelB. (1991). Computers as theatre reading. Mas: Addison-Wesley Publishing Company.
15.
Lora ArizaD. S.Sánchez-RuizA. A.González-CaleroP. A. (2017). Time Series and Case-Based Reasoning for an Intelligent Tetris Game. In AhaD. W.LieberJ. (Eds.), Case-Based Reasoning Research and Development (Vol. 10339, pp. 185–199). Springer International Publishing.
16.
MatthewsG.CampbellS. E.FalconerS.JoynerL. A.HugginsJ.GillilandK.GrierR.WarmJ. S. (2002). Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry. Emotion, 2(4), 315–340.
17.
MatthewsG.EmoA.FunkeG.ZeidnerM.RobertsR.CostaP.SchulzeR. (2006). Emotional Intelligence, Personality, and Task-Induced Stress. Journal of Experimental Psychology. Applied, 12, 96–107.
18.
MatthewsG.SzalmaJ.PanganibanA. R.NeubauerC.WarmJ. S. (2013). Profiling task stress with the dundee stress state questionnaire. Psychology of stress: New research, 1, 49-90.
19.
MatthewsG.WarmJ. S.Reinerman-JonesL. E.LangheimL. K.WashburnD. A.TrippL. (2010). Task engagement, cerebral blood flow velocity, and diagnostic monitoring for sustained attention. Journal of Experimental Psychology: Applied, 16(2), 187–203.
20.
MontgomeryH.SharafiP.HedmanL. R. (2004). Engaging in Activities Involving Information Technology: Dimensions, Modes, and Flow. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46(2), 334–348.
21.
O’BrienH. L.TomsE. G. (2010). The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology, 61(1), 50–69.
22.
O’BrienH. L.TomsE. G. (2013). Examining the generalizability of the User Engagement Scale (UES) in exploratory search. Information Processing & Management, 49(5), 1092–1107.
23.
SeddonK.SkinnerN. C.PostlethwaiteK. C. (2008). Creating a model to examine motivation for sustained engagement in online communities. Education and Information Technologies, 13(1), 17–34.
24.
SharekD.WiebeE. (2011). Using Flow Theory to Design Video Games as Experimental Stimuli. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 55(1), 1520–1524.
25.
SharekD.WiebeE. (2015). Investigating Real-time Predictors of Engagement: Implications for Adaptive Videogames and Online Training. International Journal of Gaming and Computer-Mediated Simulations, 7(1), 20–37.
26.
SinhaA.GavasR.ChatterjeeD.DasR.SinharayA. (2015). Dynamic assessment of learners’ mental state for an improved learning experience. 2015 IEEE Frontiers in Education Conference (FIE), 1–9.
27.
TabachnickB. G.FidellL. S.UllmanJ. B. (2019). Using multivariate statistics (Seventh edition). Pearson.
28.
WiebeE. N.LambA.HardyM.SharekD. (2014). Measuring engagement in video game-based environments: Investigation of the User Engagement Scale. Computers in Human Behavior, 32, 123–132.
