Abstract
This paper describes the development and validation of an instrument to assess how students deal with emotionally challenging classroom situations (the DECCS Questionnaire). The questionnaire is based on a vignette with one learning and one performance situation in a classroom, and is intended for students in grades 4 to 7. On a sample of N = 639 students (Mage = 10.6 years; SD = 1.25, 52.4% girls) in northwest Germany, factor analytic procedures for the subdimensions of students dealing with emotionally challenging classroom situations show that two factors can be distinguished each for students’ appraisals, for students’ adaptive behavior, for students’ maladaptive behavior, and for students’ desired teacher behavior. The questionnaire and subscales demonstrate good reliability and validity values. Thus, the DECCS questionnaire shows good psychometric properties and is well suited for determining how students deal with emotionally challenging classroom situations.
Keywords
Introduction
Every day, both students and teachers are confronted with emotionally challenging situations in school. These are particularly important in considering the impact on learning and teaching. According to Lindqvist (2019), emotionally challenging situations are ones that cause difficult feelings to arise and that they frequently affect both teachers and students in everyday school life. Some studies have investigated how teachers and educators deal with challenging situations caused mainly by students’ diverse needs and behavioral problems (Markkanen et al., 2020; Obrusnikova & Dillon, 2011). However, so far, studies that focus on learners’ perspectives on challenging situations are rare.
There are studies that refer to higher education students, e.g., socially challenging situations such as collaborative learning among student teachers, and other collaborative learning situations (Järvenoja et al., 2013; Järvenoja & Järvelä, 2010; Koivuniemi et al., 2018), or boredom in over- and under-challenging situations for college students (Acee et al., 2010). However, the results from these studies are only slightly transferable to elementary and lower secondary school age students. Furthermore, studies of emotionally challenging situations in the classroom – besides focusing on the teacher’s perspective – follow a rather qualitative research approach (e.g., Kurki et al., 2015; Lindqvist, 2019; van Driel et al., 2021), so that valid quantitative measures for the elementary and lower secondary school remain a desideratum.
In a preliminary qualitative study, it could be shown that situations that are challenging for students can be clustered according to learning and performance situations (Schlesier, 2020). This approach is aligned with goal orientation theory and the dichotomy of learning/mastery and performance goal approaches (Dweck, 1986; Elliot, 1999; Vandewalle et al., 2019). Accordingly, it is unsurprising that teachers have attributed different triggers and emotions to students in both learning and performance situations (Schlesier, 2020). Teachers report that students experience predominantly negative emotions (e.g., anxiety, anger, frustration, disappointment, shame) in performance-related situations (Schlesier, 2020). In learning situations, the range of students’ emotions is broader (e.g., pride, defiance, joy of learning, hope, boredom, anger, frustration, anxiety, etc.) (Schlesier, 2020). This confirms the findings of Raccanello et al. (2013) who conducted a broad cross-sectional study with students in the 4th, 7th, and 11th grades. They also found that the setting (and explicitly not the subject) determines which emotions are experienced by students. Thus, evaluative settings (such as performance situations) are associated with significantly higher levels of hope, anxiety, shame, and hopelessness; in contrast, emotions as joy, relief, and boredom are more likely to be experienced in non-evaluative settings (Raccanello et al., 2013).
Moreover, the preliminary study (Schlesier, 2020) hints that the triggers of emotionally challenging classroom situations differ in learning and in performance situations. Qualitative data in that study shows a wide spectrum of learning situations (e.g., working on a worksheet, group work or class talk), while a considerably smaller range of performance situations was detected, (e.g., tests, oral examination or controlled homework). Thus, since both the trigger and the students’ emotions seemingly vary in emotionally challenging learning and performance-related classroom situations, we may deduce that learning situations are associated with different student appraisals, student adaptive and maladaptive behavior, and teacher behavior (that the students hope for). These aspects of emotionally challenging situations – student appraisals, student adaptive and maladaptive behavior as well as teacher behavior that the students hope for – are of particular interest because they offer promising and comprehensive approaches to possible following interventions. The following sections present an overview of each of these topics: students’ appraisals, students’ adaptive and maladaptive behavior and students’ desired teacher behavior.
Student Appraisals of Emotionally Challenging Classroom Situations
How students deal with emotionally challenging situations in the classroom depends mainly on how they interpret the situations. These so-called ‘appraisals’ have already been shown to be crucial in handling emotionally challenging classroom situations (see the previous qualitative pilot study; Schlesier, 2020). This is also evident from numerous studies over the last two decades on Pekrun’s Control-Value Theory (CVT; Pekrun, 2000, 2006; Pekrun & Stephens, 2010) that show that it is the interpretation of a situation that determines what emotions are experienced. According to CVT, two aspects are particularly relevant for the appraisal of a situation: subjective control and the subjective value of the situation (Pekrun & Stephens, 2010). The subjective value of a situation is the extent to which a person assesses a situation as being important for him or herself. Whether a person subjectively perceives a situation as controllable can be traced back to causal expectations, (performance) attributions, competence appraisals (e.g., academic self-concept), and attributions, the appraisal of whether the situation is a success or a failure, and whether it is an internal, self-determined controllable situation or an external, externally controlled situation (Frenzel & Goetz, 2018; Pekrun, 2000; 2006; Pekrun et al., 2007; Pekrun & Stephens, 2010).
Control-Value Theory in turn, is based on appraisal theory, which is traceable to Arnold (1960) and Lazarus (1966). Appraisal theory – which has become known primarily as Lazarus’ Transactional Stress Theory (TST; Lazarus & Folkman, 1987; Moors et al., 2013) – assumes that it is cognitive appraisals of situations that determine which emotions are felt. According to TST, a person evaluates a situation according to available information, and the implication of this information for his or her own well-being (appraisal) (Lazarus & Folkman, 1987). The term primary appraisal refers to evaluation of the situation as positive, dangerous or irrelevant, depending on motivational and cognitive factors. The term secondary appraisal then refers to assessment of coping resources that may be available. If no resources are available, then stress occurs for the individual (Lazarus & Folkman, 1987). The situation is then coped with, in either a problem- or emotion-oriented way. All thoughts and actions can be traced back to the secondary appraisal; and the individual’s situational and generalized beliefs are decisive in determining their response (Lazarus & Folkman, 1987). Applying TST to testing situations, test anxiety occurs when a student evaluates the situation and feels that they are unable to successfully master the test in accordance with internal/external expectations and resources (Raufelder & Hoferichter, 2018). Accordingly, the student interprets the situation as dangerous or challenging, resulting in (over-)stress and test anxiety (Raufelder & Hoferichter, 2018).
According to both the TST and the CVT, one’s general beliefs in terms of academic self-concept and attributed competence are crucial regarding how a situation is interpreted and experienced. Consequently, the topics of academic self-concept and attributed competence have been the subject of much research over the past decade (Arens et al., 2013; Lohbeck et al., 2017). There are also connections with achievement emotions as, for example, (test) anxiety is more likely to occur in elementary and secondary school children who have a low academic self-concept (Lazarides & Raufelder, 2021; Lohbeck et al., 2016). Enjoyment, in turn, results from a high academic self-concept (Lazarides & Raufelder, 2021). Moreover, it is known that, in secondary school students, academic self-concept partially modulates student feelings of more enjoyment and less anxiety and boredom, especially if teachers have better diagnostic skills (Westphal et al., 2018). All these theoretical approaches and research findings indicate that students’ appraisals are particularly relevant in handling emotionally challenging classroom situations. However, no research has been done specifically on this subject, such as how students use appraisals differently in emotionally challenging situations.
Students’ Adaptive and Maladaptive Behavior in Emotionally Challenging Classroom Situations
Dealing with emotionally challenging situations involves student behavior as well as the aforementioned appraisals (Schlesier, 2020). Since challenging classroom situations are accompanied by students’ experience of emotions, student behavior is related primarily to an emotion regulation process. Accordingly, in this section, we consider emotion regulation strategies, as well as their function in relation to student behavior, in order to define adaptive and maladaptive coping responses.
There are various emotion regulation strategies with corresponding classifications (Schlesier et al., 2019). One systematization of emotion regulation strategies that is frequently used refers to functionality in terms of the coping efficiency of the emotion – this is the distinction between adaptive and maladaptive behavior (Aldao et al., 2015; Grob & Smolenski, 2009; Vierhaus et al., 2016). Adaptive emotion regulation strategies (described as problem solving, seeking social support, or ‘grit’ behavior) (Duckworth & Yeager, 2015; Schlesier et al., 2019) are assumed more likely to have positive consequences (e.g., social, cognitive outcomes), because they lead to coping with the emotion in the short and long term (Aldao et al., 2015; Graziano et al., 2007). Thus, adaptive emotion regulation strategies are more likely to achieve an individual’s goal (Aldao et al., 2015). Maladaptive emotion regulation strategies, on the other hand, are more likely to result in the individual not coping with the emotion and hence not achieving their goal. According to the functionality of emotion regulation strategies, flexibility plays a crucial role—students must learn to modify various strategies according to the situational context in order for them to work adaptively (Aldao et al., 2015).
There are already several inventories to assess emotion regulation in childhood and early adolescence (Dorn et al., 2013; Kullik & Petermann, 2012). However, in Schlesier (2020), it is pointed out that existing inventories – such as the FEEL-KJ (Grob & Smolenski, 2009) – do not focus on the (elementary) school context. In addition, existing questionnaires are mostly related only to a specific emotion (e.g., KÄRST; Salisch & Pfeiffer, 1998), do not include functionality (e.g., ERQ-CA; Gullone & Taffe, 2012), or are exclusively related to the dysfunctionality of emotion regulation behaviors (EDS-C; Morrongiello et al., 2012). All in all, existing emotion regulation inventories map traits rather than situation-specific behavior. As was shown in the previous qualitative study (Schlesier, 2020), these points are highly relevant for the analysis of how students deal with emotionally challenging classroom situations in the elementary school. Therefore, there is a clear need to develop a new quantitative measurement tool in order to gain insight into students’ emotion regulation in emotionally challenging classroom situations and, at the same time, to include both functionality and dysfunctionality.
Teacher Behavior in Emotionally Challenging Classroom Situations
A highly relevant factor that influences whether students are able to deal as adaptively as possible with classroom situations that challenge them emotionally, is how teachers react in these situations (Schlesier, 2020). There are several indications in the body of research of the importance of teachers’ behavior that affects students’ emotions, appraisals and stress levels, as well as their own behavior. For example, Kember and Leung (2006) show that a positive teacher-student relationship (teacher encouraging students, feedback, assistance from teaching staff) is significantly positively related to students’ perceived workload and thus their stress levels. It is also known that teacher support leads to significantly less anxiety and boredom in secondary school students, and is associated with significantly more enjoyment in mathematics in grades nine and ten (Lazarides & Buchholz, 2019). A strongly positive teacher–student relationship leads to fewer behavioral disorders and less depression in students, benefiting children from at-risk groups the most (Liu et al., 2015). A teacher’s praise influences the learning environment in a positive way, thus contributing substantially to the classroom climate longitudinally (Ingemarson et al., 2020). In addition, studies have shown that a teacher’s praise can enhance students’ learning and prosocial behavior, as well as contribute to a positive learning environment and reduce stress and school burnout (Cimpian et al., 2007; Gable et al., 2009; Hoferichter & Raufelder, 2022; Zentall & Morris, 2012). Overall, research studies until now have shown that the relationship aspect is more prominent than the interaction aspect (Schlesier, 2020). However, exactly what students desire from their teachers’ behavior in emotionally challenging situations has not been studied so far.
Some instruments exist that assess teacher behavior traits from different perspectives (referring mainly to the teacher–student relationship). These include the ICEQ (Fraser, 1982), the QTI (Wubbels & Levy, 1993), and the NRI (Furman & Buhrmester, 2009). The dimensions used in the CLASS instrument (Pianta et al., 2008) have become widely accepted. Accordingly, dimensions that are often differentiated include teachers’ emotional support (e.g., teacher sensitivity, regard for student perspectives), classroom organization (e.g., productivity, behavior management), and instructional support (e.g., quality of feedback) (Jennings et al., 2017; Pianta et al., 2008). In this regard, a comprehensive intervention study by Jennings et al. (2017) shows that it is mainly teachers’ emotional support and sensitivity to students’ needs, rather than their organizational skills, that improve classroom climate.
Surveys of instructional quality – especially in the field of research on the professionalization of teachers – should also be mentioned here. Baier and colleagues (2019) for example, surveyed learning support, classroom management and cognitive activation as levels of instructional quality. In particular, the dimension of learning support seems to be partly consistent with the statements made by teachers in the previous qualitative study with regard to students’ emotionally challenging situations (Schlesier, 2020). Learning support (Baier et al., 2019) includes the extent to which teachers provide their students with adaptive explanations of tasks (drawn from Baumert et al., 2008), emotional responsiveness to students’ problems (drawn from Saldern et al., 1986), and constructive responses to errors (drawn from Baumert et al., 2008). However, those items are based on a general assessment of teachers by students and are not directly transferable to the wishes of students in situations in which they feel emotionally challenged. Nonetheless, all the existing instruments target the assessment of teacher behavior; but they do not relate to emotionally challenging situations, nor do they reflect teacher behavior that is desired by students.
The Transition from Elementary to Secondary School: A Particularly Emotionally Challenging Time for Students
The current study focuses on students from 4th to 7th grade due to the fact that their transition from elementary to secondary school is a critical life event and a particularly challenging time emotionally (Filipp, 1995; van Ophuysen, 2009). On the one hand, transition effects such as experiencing intensively positive emotions have been described at the very beginning of secondary school (Meyer & Schlesier, 2021; Pekrun et al., 2007). On the other hand, there seem to be many unfavorable developments regarding emotions and emotion regulation during the first years after school transition (Ahmed et al., 2013; Bieg et al., 2019; Meyer & Schlesier, 2021; Obermeier et al., 2021; 2022; Raccanello et al., 2013; Schumacher & Denner, 2017; Vierhaus et al., 2016).
During the elementary school years, student emotions seem to remain relatively stable, but with the transition to secondary school, they seem to take a rather unfavourable course. Positive emotions decrease in the course of 5th grade, while negative emotions increase (Meyer & Schlesier, 2021; Vierhaus et al., 2016). The increase in negative emotions such as anxiety over the course of the first year in secondary school then leads to children feeling less well in school (Obermeier et al., 2022). Furthermore, while adaptive emotion regulation strategies increase during the elementary school years, they decrease significantly between grades 4 and 7 (Vierhaus et al., 2016). At the same time, maladaptive strategies such as avoidant coping increase significantly from the transition phase onwards (Vierhaus et al., 2016). In addition to the intrapersonal emotional and biophysiological changes that children experience during transition, they also face other changes in their school environment. For example, there are more teachers, which makes both the students and teachers perceive their relationship as less positive compared to their primary school years. Students in secondary school tend to perceive their teachers as being cold and distant, and they feel insufficiently supported (Eccles et al., 1993; Hargreaves, 2000; Harter, 1996).
In studies investigating the transition from elementary school, either elementary or secondary school students are usually examined (e.g., Lichtenfeld et al., 2012; Obermeier et al., 2022), as it is logistically difficult to implement data collection at both types of schools at the same time. However, for a comprehensive overview, it is particularly important to examine both the end of elementary school (in Germany: mostly 4th grade, in other countries often 5th or 6th grade), and the beginning of secondary school (in Germany: mostly 5th grade, in other countries: 6th or 7th grade). For this reason, the questionnaire developed in this study should be applied in grades 4 to 7.
Aim and Hypotheses
Due to the reasons mentioned above, it is particularly important to examine the extent to which students deal with emotionally challenging situations around the transition from elementary to secondary school. It is therefore the aim of this study to develop and validate an appropriate measurement instrument, namely the DECCS (Dealing with Emotionally Challenging Classroom Situations) questionnaire for students. In line with the previous qualitative study (Schlesier, 2020), we therefore examine student appraisals, student behavior, and student-desired teacher behavior in relation to two emotionally challenging classroom situations for their factor structure.
In accordance with the goal orientation approach and the distinction between mastery and performance goals (Dweck, 1986; Elliot, 1999; Vandewalle et al., 2019), we assume that student appraisals, student adaptive and maladaptive behavior, and student desired teacher behavior can each be divided into two factors: learning-specific and performance-specific appraisals, learning-specific and performance-specific student behavior, and learning-specific and performance-specific student desired teacher behavior (Hypotheses 1a-c).
In addition, on the basis of prior studies on emotion regulation strategies (e.g., Aldao et al., 2015; Grob & Smolenski, 2009; Vierhaus et al., 2016), we assume that different strategies of adaptive and maladaptive student behavior can be identified (Hypothesis 2). Regarding desired teacher behavior, we anticipate – on the basis of the results of our preliminary qualitative study (Schlesier, 2020) – that we can identify controlling, praising and supporting behavior (Hypothesis 3). When examining the construct validity of the DECCS, we assume that school well-being is positively related to adaptive student behavior and high values of teacher support; conversely, well-being is presumably negatively related to maladaptive student behavior (Grigoryeva & Shamionov, 2014; Obermeier et al., 2021; 2022; Tian et al., 2015) (Hypothesis 4). Furthermore, we assume that lower grades (whether earned in a student’s favorite or least favorite subject) are associated with more unfavorable appraisals (overstress, low academic self-concept, low perceived competence) and more maladaptive student behavior (e.g., Weber & Petermann, 2016) (Hypothesis 5).
Material and Methods
Sample and Procedures
The participants (N = 639) were 8- to 15-year-old 4th to 7th grade students (M age = 10.6 years; SD = 1.25, 52.4% girls) in 27 elementary and secondary schools in the federal states of Lower Saxony and Bremen, Germany. This age group was examined due to the fact that it is reasonable to assume that students handle emotionally challenging classroom situations differently during the transition from elementary to secondary school and in the early years thereafter. All types of schools that provide education in at least one of the grades 4–7 were included in the study. Thus, students from public schools as well as private schools were part of the study, which comprises elementary schools, middle/high schools (different German middle school types: Gymnasium, Gesamtschule, Oberschule, Realschule), and schools for children with special educational needs (support for emotional-social development and physical development). All students who wanted to participate in the study and who had a signed parental consent form were included in the study.
Initially, the ethics committee at the University of Oldenburg approved the study and research design. Subsequently, the participating schools and the state education authority gave their written consent. Before data collection began in the schools, written consent was obtained from parents or guardians. Classrooms were used for questionnaire-based data collection on normal school days, in the fall of 2020 1 by trained student teachers or teachers. Students were informed that they could withdraw their participation at any time, without any disadvantage, and they could also leave some questions unanswered if they felt uncomfortable about them.
Questionnaire Construction
How elementary and early secondary school students deal with emotionally challenging situations and how they interact with their teachers in such situations was investigated in a preliminary qualitative study. In the prior study (Schlesier, 2020), descriptions from 31 interviews with teachers about students’ emotionally challenging classroom situations were examined. It was found that the interaction of teachers and students in emotionally challenging situations seems to follow a similar pattern (see Figure 1). Therefore, a model of teacher – student interaction in achievement-emotions situations (Learning and performance Emotions, Emotion Regulation - Teacher Student Interaction (LEmoR-TSI); Schlesier, 2020) was developed in a three-stage qualitative analysis process (qualitative content analyses and heuristic techniques), and tested (for the first time) using quantitative content analyses (Schlesier, 2020). According to the LEmoR-TSI model, emotionally challenging situations seem to begin with a trigger; this is interpreted by the student (appraisal) and thus, a discrete learning or performance emotion is experienced (Schlesier, 2020). The student then displays this experience in the form of either adaptive or maladaptive (emotion regulation) behavior, which in turn is interpreted by the teacher (Schlesier, 2020). The teacher then also displays more or less supporting, praising and controlling emotion regulation behavior (in accordance with teacher-specific interaction factors) (Schlesier, 2020). This process then leads either to a consequence, or to a further interaction loop, depending on how the teacher’s behavior is again (re-)interpreted by the student (Schlesier, 2020). Concepts within the original LEmoR-TSI model and the DECCS questionnaire. Note. The conceptualization of the DECCS questionnaire is based on the model of teacher–student interaction in achievement-emotions situations (Learning and performance Emotions, Emotion Regulation – Teacher Student Interaction, LEmoR-TSI; Schlesier, 2020). Further student and teacher intrapersonal factors (e.g., general beliefs, prior experiences, teachers’ emotions etc.) are not included in the figures.
The DECCS questionnaire is based on the foundation of the LEmoR-TSI model and its qualitative subcategories. The situations mentioned by the teachers in the preliminary qualitative study that play a particularly important role, were used as the basis for developing two vignettes: one on an emotionally challenging classroom learning situation, and one on a performance situation. These case vignettes offer many advantages, as listed in Steiner and colleagues (2016), including high internal validity, construct validity, and reliability. Moreover, one important advantage of these case vignettes is that the concrete interaction process in the concrete classroom situation is described. That means that the students are better able to empathize with the items since the corresponding questions are embedded in a concrete and realistic context (Steiner et al., 2016). It can thus be assumed that the elementary and early secondary school children are better able to answer the questions, as they are more realistic and less abstract than traditional trait surveys (Steiner et al., 2016).
Questionnaire Design and Rotated Component Matrix of the DECCS items for Each Phase, based on a Random Sample of 323 Elementary and Lower Secondary Secondary School Students.
Note. Factor loadings >.40 are in boldface. CV1 = Case Vignette 1, Performance Situation, CV2 = Case Vignette 2, Learning Situation; F1 = Factor 1, F2 = Factor 2. The case vignettes can be adapted to be domain-specific or -non-specific; in the present study, we referred to the least favorite subject and mathematics.
Subsequently, the appraisal of the situation which led to the identified emotion, is assessed. The appraisal includes generalized beliefs about self-concept, overstress, as well as attributions of competence, in accordance with the stress theory of Lazarus (Lazarus & Folkman, 1987). On the basis of the teachers’ statements in the preliminary study, we developed a total of eight items for appraisals of both types of emotionally challenging situations, i.e., four items for each case vignette referring to one of the two classroom situations (e.g., “You feel this way because you think you can’t do it anyway.”, or “…it overwhelms you.”), which were answered on a 5-point Likert-type response scale (1 = “strongly disagree”, 2 = “disagree”, 3 = “half and half”, 4 = “agree”, 5 = “strongly agree”). Higher scores on this scale indicate a more unfavorable appraisal, i.e. the student appraised the situation as overwhelming, overstressing, being incompetent in dealing with the situation.
In accordance with Grob and Smolenski (2009), we distinguish between adaptive and maladaptive behavior in the DECCS. We developed six items for adaptive behavior (three items per case vignette; e.g., “Then you skip the task and solve the others first.”, or “Then you definitely want to do well on this task sheet anyway.”) that were assessed on a 5-point Likert-type scale (1 = “strongly disagree”, 2 = “disagree”, 3 = “half and half”, 4 = “agree”, 5 = “strongly agree”). Higher scores on this scale indicate more adaptive student behavior. For maladaptive behavior, we developed eight items (four items per case vignette; e.g., “Then you tear up the paper on which the test is written.“, or “Then you don’t work on the whole task sheet again.”). Students answered these items on a 5-point Likert-type scale (1 = “strongly disagree”, 2 = “disagree”, 3 = “half and half”, 4 = “agree”, 5 = “strongly agree”). Higher scores on this scale indicate more unfavorable maladaptive behavior.
The preliminary qualitative study revealed that teacher behavior seems to play a crucial role in the process of students dealing with emotionally challenging classroom situations. Therefore, we posed eight items (four items per case vignette) to students about the teacher behavior that they would wish for in the referring emotionally challenging situations (e.g., “That she comes to me and explains the test tasks.”, or “That she tells me I’m doing great.”). Items were assessed on a 5-point Likert-type scale (1 = “strongly disagree”, 2 = “disagree”, 3 = “half and half”, 4 = “agree”, 5 = “strongly agree”). Higher scores on this scale indicate more attentive, positive teacher behavior.
Measures (Construct Validity)
School well-being. Students’ well-being in school was assessed by three items (e.g., “I am really looking forward to school”, or “I have a lot of fun at school”) on a 5-point Likert scale from 1 (“true”) to 5 (“not true at all”) (α = .73).
Grades. We asked students for their grades in their favorite and their least favorite subjects from their last certificate. German school grades range from 1 (“very good”) to 6 (“fail”).
Statistical Analyses
The underlying factor structure of the 26 items of the DECCS questionnaire for students for (a) the appraisal of the case vignette; the students’ (b) adaptive or (c) maladaptive behavior; and (d) the students’ desired teacher behavior was examined in a three-step process using structural equation modeling (SEM) in Mplus version 8.5 (Muthén & Muthén, 2017), and maximum likelihood estimation.
Initially, the sample was randomly divided in two, so that mutually independent samples were obtained for the exploratory factors analysis (EFA) and the confirmatory factor analysis (CFA), respectively. The random subsample for the EFA consists of 323 students (M age = 10.7 years; SD = 1.26, 52.9% girls); the random subsample for the CFA consists of 316 students (M age = 10.6 years; SD = 1.23, 51.9% girls).
In the first step, in order to check the interdependence between the factors underlying the items for each aspect (a – d), exploratory factors analysis (EFA) with oblique promax rotation was conducted. The following criteria were used to determine the number of factors to retain: eigenvalues of the unrotated factors ≥1, scree plot, variance accounted for by unrotated factors to reduce the risk of extracting too many minor factors, internally reliable factors, and factors that yield meaningful psychological constructs.
In the second step, confirmatory factor analyses (CFA) were run to identify the final factor structure. The internal consistency reliability (Cronbach’s alpha) of the scores was examined. In the third step, in order to demonstrate construct validity, associations between the two factors for each condition and independent measures of related constructs (i.e., school well-being and grades) were examined. The discriminant validity was tested by estimating the confidence intervals (CIs) of the paired correlations among the latent variables. If the confidence interval of the paired correlation does not include the value of 1, it is evidence for discriminant validity (Torkzadeh et al., 2003).
Due to the Covid-19 pandemic, a relevant part of this sample of students was not assigned to a specific class throughout the school year, i.e. the data are not nested. Accordingly, a multilevel approach could not be conducted.
The following fit indices were used to determine model fit (Hu & Bentler, 1999): Chi-Square Test of Model Fit (Χ2), Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), and Standardized Root Mean Square Residuals (SRMR). CFI values greater than .90, and RMSEA and SRMR values lower than .08 are usually interpreted to reflect an acceptable fit to the data. CFI values greater than .95, as well as RMSEA and SRMR values lower than .05 or .06, are generally interpreted to reflect a good fit to the data.
Little’s Missing Completely at Random (MCAR) test was performed and suggested that data were missing completely at random: (a) for the appraisal of the case vignette: χ2 (76) = 88.43, p > .05; (b) for the students’ adaptive behavior: χ2 (39) = 52.63, p > .05; (c) for students’ maladaptive behavior: χ2 (107) = 124.73, p > .05; and (d) for students’ desired teacher behavior: χ2 (57) = 70.49, p > .05. Because the MCAR test confirmed the use of full information maximum likelihood (FIML) to account for missing data, all analyses were run with maximum likelihood estimation with robust standard errors (MLR) with FIML.
Results
Exploratory Factor Analyses
For the first aspect of the DECCS questionnaire for students – the (a) appraisal of the situation presented in the two case vignettes – a two-factor structure best met the criteria for an adequate factor analytic solution and 7 items remained with factor loading >.40. One item was eliminated due to weak factor loading (<.40) and/or cross-loadings on several factors. Three items with factor loadings ranging from 0.47 to 0.89 were loaded on Factor 1 (44.74% explained variance). This factor was labelled “performance appraisal” as all items concerned the first case vignette, which focuses on a test situation. Four items with factor loadings ranging from 0.55 to 0.75 were loaded on Factor 2 (16.28% explained variance). This factor was labelled “learning appraisal” as all items concerned the second case vignette, in which a learning situation is described.
For student’s (b) adaptive behavior, a two-factor structure best met the criteria for an adequate factor analytic solution and 5 items with factor loading >.40. One item was eliminated due to weak factor loading (<.40) and/or cross-loadings on several factors. Three items with factor loadings ranging from 0.46 to 0.85 were loaded on Factor 1 (49.04% explained variance). This factor was labelled “grit” as all items focus on the goal of solving the test/task, despite obstacles. Two items with factor loadings ranging from 0.50 to 0.84 were loaded on Factor 2 (21.47% explained variance). This factor was labelled “problem solving” as all items focus on solving the problems described in the case vignettes.
For student’s (c) maladaptive behavior, a two-factor structure best met the criteria for an adequate factor analytic solution and eight items with factor loading >.40. Four items with factor loadings ranging from 0.50 to 0.84 were loaded on Factor 1 (41.56% explained variance). The factor was labelled “rebelliousness” as all items focus on rebellious behavior in reaction to the case vignettes. Four items with factor loadings ranging from 0.46 to 0.85 were loaded on Factor 2 (19.41% explained variance). This factor was labelled “avoidance” as all items focus on behavior to avoid the challenges described in the case vignettes.
For (d) students’ desired teacher behavior, a two-factor structure best met the criteria for an adequate factor analytic solution and 6 items with factor loading >.40. Two items were eliminated due to weak factor loading (<.40) and/or cross-loadings on several factors. Four items with factor loadings ranging from 0.61 to 0.86 were loaded on Factor 1 (47.84% explained variance). This factor was labelled “teacher’s support” as all items focus on the desired behavior of instructional support to deal with the challenges described in the case vignettes. Two items with factor loadings ranging from 0.67 to 1.00 were loaded on Factor 2 (20.62% explained variance). This factor was labelled “teacher’s praise” as all items focus on praise from the teacher in reaction to the described case vignettes.
The DECCS questionnaire design and all remained items, as well as the results of the Rotated Component Matrix are shown in Table 1.
Confirmatory Factor Analyses
In the second step of the data analysis, confirmatory factor analyses were conducted. Four separate structural equation models were run for each aspect of the DECCS: (a) appraisal of the case vignettes; students’ (b) adaptive or (c) maladaptive behavior; and (d) students’ desired teacher behavior.
The latent factor model for the appraisal of the case vignettes reached a good fit (χ2 (12) = 11.26, p > .05, CFI = 1.00, RMSEA = 0.00 (0.00–0.06), SRMR = .02) (see Figure 2). Students’ appraisal of the emotionally challanging classroom situation.
The latent factor model for students’ adaptive behavior reached a good fit (χ2 (4) = 1.98, p > .05, CFI = 1.00, RMSEA = 0.00 (0.00–0.06), SRMR = .02) (see Figure 3). Students’ adaptive behavior.
The latent factor model for the students’ maladaptive behavior reached a good fit (χ2 (17) = 39.02, p < .05, CFI = .96, RMSEA = 0.06 (0.04–0.09), SRMR = .05) (see Figure 4). Students’ maladaptive behavior.
The latent factor model for the students’ desired teacher behavior reached a good fit (χ2 (7) = 10.01, p > .05, CFI = .99, RMSEA = 0.04 (0.00–0.08), SRMR = .02) (see Figure 5). Students’ desired teacher behavior.
In order to test discriminant validity, the 95% confidence interval for the correlations between the latent variables was computed: (a) performance appraisal and learning appraisal (CI .45, .71); (b) grit and problem solving (CI .54, .82); (c) avoidance and rebelliousness (CI .46, .69); and (d) teacher’s support and teacher’s praise (CI .31, .57). The correlation values are rather low, which provides further support for discriminant validity (Torkzadeh et al., 2003).
Means, Standard Deviations, Intercorrelations and Reliabilities of DECCS Subscales and Related Constructs.
Notes. All measures are standardized; Appr Perform = Appraisal Performance Situation, Appr Learning = Appraisal Learning Situation, Teacher’s Support = Student’s Desired Teacher Support, Teacher’s Praise = Student’s Desired Teacher Praise, Well-being = School Well-being, Grade 1 = Grade in favorite subject, Grade 2 = Grade in least favorite subject; α = Reliability Chronbach’s Alpha.
*p < .05, **p < .01, ***p < .001, N = 639.
Construct Validity
In order to demonstrate construct validity, students’ scores on the DECCS scales were correlated with their grades and a measure of well-being in school. Table 2 reports all bivariate correlations and mean values. Scores on the DECCS subscale are significantly associated with both grades (favourite subject and least favourite subject) and well-being in school.
Discussion
The purpose of this study was to develop and validate an instrument that can be employed to assess how students deal with emotionally challenging classroom situations (DECCS) in elementary and early secondary school. The instrument offers two case vignettes: one related to a learning situation the other referring to a performance situation. Based on these case vignettes, students can first state their perceived emotions (descriptively) and then (a) interpret the situation (appraisals); assess their own (b) adaptive as well as (c) maladaptive behavior; and (d) describe information about the teacher’s desired behavior in such situations.
According to Hypotheses 1a-c, exploratory analyses – conducted using two random samples according to the split-half method – have shown that the distinction between emotionally challenging learning and performance situations is only relevant in terms of the students’ appraisal. With regard to student behavior and desired teacher behavior, such a differentiation has not been proven – contrary to the assumption of the mastery and goal approach (Dweck, 1986; Vandewalle et al., 2019). Rather, with regard to student behavior, the systematic approach according to functionality (adaptive and maladaptive strategies) has been shown to be useful; discrete strategies could be identified, as assumed (Hypothesis 2), which is consistent with Grob and Smolenski (2009).
With regard to student’s desired teacher behavior, only task-related teacher support could be distinguished from emotional support (teacher’s praise), as also described by Pianta et al. (2008). Controlling teacher behavior could not be identified, contrary to our assumptions (Hypothesis 3). Consequently, it can be deduced from these results that student behavior — as well as the students’ desired teacher behavior — is relatively independent of the respective emotionally challenging situation, i.e., independent of the classroom context being a learning or a performance situation. At the same time, a student’s appraisal of an emotionally challenging classroom situation does depend on the context, i.e., whether the student is in an evaluative or non-evaluative situation.
Concerning the examination of the construct validity, the correlations reveal expected results on school well-being (Hypothesis 4). High values of school well-being are significantly associated with higher values of grit, problem solving, and teacher’s praise (Grigoryeva & Shamionov, 2014; Obermeier et al., 2021; 2022; Tian et al., 2015). In contrast, unfavorable appraisals in a learning or performance situation and rebelliousness are significantly negatively related to the students’ school well-being.
Not least, it should be noted that, as described in previous studies for grades obtained in German and math (e.g., Weber & Petermann, 2016), avoidance is significantly related to lower grades (in both the favorite and least favorite subject). It is also noticeable that there are more significant unfavorable correlations with grades obtained in the least favorite subject than with those obtained in the favorite subject (see Table 2). For example, an unfavorable appraisal of a situation (in both learning and performance situations) is significantly correlated with worse grades in the least favorite subject (Hypothesis 5).
Strengths, Limitations and Future Directions
As with any study, some limitations must be considered when interpreting the results of the present study. Since the study is based on a cross-sectional design, no causal relationships among the variables can be revealed. Also, due to the first-time and one-time use of the questionnaire, no re-test reliability could be calculated. Future longitudinal studies with two or more measurement points would be desirable to overcome these limitations. Validation studies with different age groups and from other nations would be useful additions to test whether the instrument proves to be a reliable and valid measurement tool for other languages, cultural groups, and age groups.
In addition, social desirability could have led to some bias in the data. As can be seen in Table 2, higher mean values resulted for the adaptive strategies compared to the maladaptive strategies. However, since the data were collected using a 5-point Likert-type scale and a scattering is recognizable despite the left shift, we assume that the values are nevertheless meaningful. Also, some items unexpectedly dropped out in the exploratory analysis, resulting partly in an unbalanced item amount of the subscales. Moreover, due to COVID-19 restrictions, the data were collected in a rather unique time period that exhibited very different classroom structures (e.g., remote, hybrid, and half-split classes). These circumstances may have influenced the data, particularly responses to the students’ desired teacher behavior, as students may have wished for closer interactions with their teachers due to social distancing in classrooms. Further data collection at times without restrictions should investigate any different results in the students’ desired teacher behavior.
Furthermore, it was indeed the case that missing values ranged between 0 and 2% for all variables, but there was one exception for appraisals of the learning situation (which was 8%). Also, the conception of teacher behavior was rather limited compared to common constructs (see Baier et al., 2019; Baumert et al., 2008), as only task-related and emotional support could be identified. This is due to the facts that, firstly, the vignettes required explicit teacher behavior (and not traits); and, secondly, the situations were very specific and thus required specific answers in terms of teacher behavior.
Some voices might criticize the fact that the instrument is a self-reporting one and that other measurement approaches would be desirable in order to draw as comprehensive a picture as possible. However, the focus of the DECCS questionnaire is explicitly on the perceptions of the students. In order not to ignore the teachers’ perspective, the development of a DECCS teachers’ questionnaire could be a useful addition. For a DECCS for teachers, the use of video vignettes of classroom situations might be useful.
Besides these limitations, the study has several strengths. After all, it is the first questionnaire of its kind that addresses emotionally challenging classroom situations as a starting point, thereby considering not only different aspects of the emotional appraisal of the situation but also the adaptive and maladaptive behavior of the students, as well as their expectations or wishes with regard to the behavior of the teacher. The inclusion of two case vignettes enables the differentiation of performance and learning situations, as well as allowing age-appropriate access for elementary school students and students in the lower grades of secondary schools (Steiner et al., 2016).
Footnotes
Acknowledgments
We thank the student teachers from the University of Oldenburg for collecting the data and Dr. Jill Fresen for proofreading the paper.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Note
Appendix
Questionnaire design of the DECCS items in German (original items). Anmerkung. CV1 = Fallvignette 1, Leistungssituation, CV2 = Fallvignette 2, Lernsituation. Die Fallvignetten können domänenspezifisch adaptiert oder domänenunspezifisch eingesetzt werden. 1) Diese Items referieren auf Frage 2.
Du hast Unterricht in
[Fach einfügen]
. Die Lehrerin teilt einen Test aus und du siehst sofort, dass du die erste Aufgabe nicht kannst.
Stell dir vor, du sitzt gerade in einer
Unterrichts
stunde in [Fach einfügen] und ihr sollt ein Arbeitsblatt bearbeiten. Die Lehrerin kommt zu dir und weist dich darauf hin, dass du einen Fehler gemacht hast.
1. Frage: Du bist/hast…
traurig
enttäuscht
trotzig
Angst
schämst dich
verärgert
hilflos
hoffnungslos
frustriert
(Selbst-)Mitleid
neugierig
Sonstiges: …
Items des DECCS
CV1
CV2
2. Frage: Du fühlst dich so, weil…
du denkst, dass du das eh nicht kannst
X
dich das überfordert
X
du das bestimmt nicht hinkriegen wirst
X
du denkst, dass du das eh nicht kannst
X
dich das überfordert
X
du das bestimmt nicht hinkriegen wirst
X
du denkst, dass die Lehrerin denkt, dass du das nicht gut kannst
X
3. Frage: Dann…
du diesen Test unbedingt trotzdem gut machen willst.
1)
X
versuchst du trotzdem, die Aufgaben zu lösen
X
du dieses Aufgabenblatt unbedingt trotzdem gut bearbeiten willst.
1)
X
überspringst du die Aufgaben und löst erstmal die anderen
X
überspringst du die Aufgaben und löst erstmal die anderen
X
zerreißt du das Papier, auf dem die Aufgaben stehen
X
schreibst du den ganzen Test nicht mit
X
zerreißt du das Papier, auf dem die Aufgaben stehen
X
bearbeitest du eben das ganze Aufgabenblatt nicht
X
du diesen Test eh nicht schreiben willst.
1)
X
drehst du dich erstmal weg und wartest ab
X
du dieses Aufgabenblatt eh nicht bearbeiten willst.
1)
X
drehst du dich erstmal weg und wartest ab
X
4. Frage: Was würdest du dir von der Lehrerin in diesem Moment wünschen?
Dass sie zu mir kommt und mir die Aufgaben erklärt
X
Dass sie mir bei den Aufgaben hilft und sie gemeinsam mit mir bearbeitet.
X
Dass sie zu mir kommt und mir die Aufgaben erklärt
X
Dass sie mir bei den Aufgaben hilft und sie gemeinsam mit mir bearbeitet.
X
Dass sie mir sagt, dass ich das super mache
X
Dass sie mir sagt, dass ich das super mache
X
