Learning Scientific Explanations by Means of Worked Examples – Promoting Psychology Students’ Explanation Competence

Abstract

The aim of this study is to investigate whether worked examples are effective in fostering psychology students’ explanation competence. Explanation competence is a context-specific cognitive disposition that enables a person to construct a causal model of an observable psychological phenomenon by drawing on psychological theories. We set up a training intervention using worked examples to demonstrate how the observed psychological phenomenon (e.g., cognitive dissonance) is represented in an explanation. Instructional support was implemented using a fading procedure. We investigated the effects of worked examples on explanation competence using a sample of psychology students (n = 46) from a German university. In an experimental between-group pre- and post-tests design, the participants in the experimental condition received the training intervention, whereas the participants in a control condition did not receive the intervention. The experimental and control groups did not differ in their explanation competence before the training intervention. Participants in the experimental condition had a significant higher explanation competence after the training intervention than the participants in the control condition. Thus, our results indicate that worked examples effectively foster psychology students’ explanation competence. Considerations on how the results could be implemented in actual teaching settings are provided.

Keywords

Scientific explanation explanation competence worked example fading

Introduction

Using psychological theories to explain psychological phenomena or, simply put, explaining a psychological phenomenon, is not only a part of the definition of psychology as a science (e.g., Gerrig & Zimbardo, 2010), but also a component of scientific competence in psychology (Dietrich et al., 2015). In psychology teaching, learning the content of scientific theories is emphasized, especially for novice students (see Birke, Rosman & Mayer, 2016). Thus, psychology students mainly acquire knowledge about the content of psychological theories. According to Ohlsson (1992), the problem herein is that content knowledge of psychological theories is not sufficient to use these theories to explain real-world situations. Theories contain claims about the relation of different entities, but they do not contain rules on how theories are applied. Thus, it is necessary to teach psychology students how to use psychological theories to explain real-world situations such as the behavior of a person. This paper presents the evaluation of a training intervention featuring a worked examples approach to fostering psychology students’ explanation competence.

The nature of scientific explanations and their structure

Scientific explanations answer the question why a certain phenomenon happened (Ohlsson, 2010). Ohlsson’s (1992) concept of theory articulation conceptualizes an explanation¹ as the application of a scientific theory to a situation by mapping the theory onto that situation. Mapping relates the theoretical causes and consequences to the observed situation. An example would be the prospective university student Eva, who has to choose between two universities, each of which has its advantages and disadvantages. After the decision, we observe that Eva talks favorably about the advantages of her choice. This behavior can be explained by applying the theory of cognitive dissonance (Festinger, 1957).

Explanations follow a given structure. A widely known structural model for explanations is the deductive–nomological model² (DN model; Hempel & Oppenheim, 1948). The DN model consists of two components: the explanans and the explanandum. The explanandum contains the phenomenon to be explained, the explanans consists of a set of sentences that are stated to account for the phenomenon. The first sentence in the explanans is a theory mentioning the causes of the phenomenon at hand. The second sentence (antecedence condition) states that the cause(s) mentioned in the theory is/are present in the observed situation. The explanandum is deduced from these two sentences. Table 1 shows the components of the DN model.

Table 1.

The DN model of scientific explanations.

Explanans	Theory	If cause A, then effect B
Explanans	Antecedence condition	Cause A is present
Explanandum	Phenomenon	Therefore effect B is present

At this point, we have to distinguish between situation and phenomenon. In this paper, we always refer to “phenomenon” as part of the explanandum, whereas “situation” refers to the phenomenon and its antecedence condition as part of the explanans. In the example, the phenomenon is that Eva talks more favorably about her choice, whereas the situation means that Eva has to choose between two universities and that afterwards she talks more favorably about her choice.

The application of the DN model to the example of Eva given above is straightforward. The theory of cognitive dissonance states that if two or more relevant cognitions are dissonant, then there is a motivation for dissonance reduction. This is especially important after a forced choice. Dissonance reduction is attained by means of adding consonant information. In our example both universities are appealing to Eva, which results in two dissonant cognitions. Thus, she is forced to make a decision causing cognitive dissonance. In order to reduce cognitive dissonance, she adds favorable information (consonant cognitions) about one of the universities, thus devaluing the other and easing the decision process. Taken together this leads to Eva appraising her choice more favorably after the decision.

The key in constructing an explanation is the elaboration of how the abstract terms of the theory manifest themselves empirically in the situation. Table 2 illustrates how the abstract theoretical terms are mapped onto the given situation in our example. Such an elaboration of the relation between a theory and a given situation is the basis for a scientific understanding of the phenomenon (see McCain, 2015).

Table 2.

DN model applied to the theory of cognitive dissonance (Festinger, 1957).

Question: “Why does Eva talk favorably about the advantages of her choice?”
Theory of cognitive dissonance (Festinger, 1957)	Abstract term	Empirical manifestation

Antecedence condition	Relevant cognitions are dissonant	Both universities are attractive to Eva
	Especially after forced choice decisions	She is forced to make a decision for one university
	Adding consonant information	Eva adds consonant information about the chosen university
Phenomenon	Change of appraisal of the decision	Eva appraises her choice more favorably

Norms for scientifically valid explanations

Constructing an explanation is a form of deductive reasoning in which theories and the observed situation are coordinated (see Tytler & Peterson, 2003). To consider this deductive inference as valid, an explanation has to adhere to several norms (Rosenberg, 2012; Westermann, 2000). The first norm stipulates that the theory used in the explanation must be proven empirically and thus prevents the use of non-scientific “everyday” or subjective theories (see Stark, 2005). Based on this norm, we set up a second norm that a sound explanation has to mention the used theory explicitly.

A third norm requires a detailed elaboration of the relationship between the theory and the situation, hereafter referred to as theory-evidence-coordination. This norm draws on the notion of the distinction between theories and evidence (Kuhn, 2005) and, in the context of explanations, refers to the ability to distinguish the abstract terms of the theory from their manifestation in the observed situation and to decide which available theory best fits the observed situation.

Logical correctness is a fourth norm to which a good explanation must adhere. As the explanandum constitutes a deduction from the two parts of the explanans, an explanation must not contain circular reasoning (Woodward, 2014). The logical correctness of the explanation is inherent in an adequate theory-evidence-coordination. Thus, a logically correct explanation requires a proper mapping between the antecedence condition and the phenomenon and its respective empirical manifestation. Circular reasoning, for example, would be evident if the same empirical observations were mentioned both in the antecedence condition and in the phenomenon.

The fifth and final norm refers to the consideration of alternative theories that: (a) also explain the situation; (b) can only explain a part of the situation; or (c) cannot explain the situation at all. We call this norm “multiperspectivity,” and it draws on argumentation theory (Walton, 1989). Because explanations are used in scientific argumentative discourse it is important to protect or to support one’s own argument. This can be achieved by mentioning other potentially relevant theories or by ruling out theories that do not fit the observed situation.

To exemplify these norms we revisit Eva’s example. When compiling an actual, that is, a written or oral, explanation, the first two norms require that the explanation contains an explicit reference to Festinger’s (1957) theory of cognitive dissonance. This requirement may be fulfilled by simply adding a citation referring to Festinger’s work. A more elaborate form of this requirement could include an additional summary of Festinger’s theory. Furthermore, to fulfil the norm that a theory must be empirically proven, the explanation has to contain references that support Festinger’s theory. This norm is easily complied by adding citations of suitable studies.

The next norm referring to the theory-evidence-coordination requires a mapping of the elements of Festinger’s theory and the observed situation as shown in Table 2. To provide an adequate theory-evidence-coordination in a written or oral explanation, it must be evident that theoretical statements are coordinated with the appropriate elements of the situation. For instance, the theoretical statement that relevant cognitions must be dissonant has to be mapped with the related statement that both universities are attractive to Eva. Such a mapping has to be completed for all parts of the DN model, that is, the theory and the antecedence condition in the explanans as well as the phenomenon in the explanandum. Thus, in a valid explanation there must be statements containing this mapping. If these mappings in the theory-evidence-coordination are correct, the explanation automatically complies with the norm of logical correctness. Thus, these two norms are related.

To fulfil the final norm, the explanation should consider other relevant theories. In the case of cognitive dissonance, Heider’s (1958) balance theory is an example of a potentially relevant theory. A sound explanation should therefore figure out why the balance theory is not as suitable as the cognitive dissonance theory to explain the situation.

Explanation competence and learning from worked examples

According to Klieme and Leutner (2006), competences are context-specific cognitive dispositions necessary to cope with situations and demands in specific domains. In their model of scientific competences in the social sciences, Dietrich et al. (2015) conceptualize explanation as a cognitive process in which a causal relation is constructed. Following these two lines of reasoning, we define explanation competence as a context-specific cognitive disposition that enables a person to construct a causal model (see Kim, 1994) of an observable psychological phenomenon (like cognitive dissonance) by drawing on psychological theories. Explanation competence, as defined here, can be further divided into distinct competences necessary to apply each of the norms for sound explanations outlined in the previous section. Thus, explanation competence is an overarching construct that has several facets. These facets form a set of cognitive dispositions enabling the application of the norms of good explanations.

A promising approach to foster explanation competence is learning with worked examples (hereafter WE). This approach is mainly used for initial knowledge application in well-structured domains (Renkl, 2014). The effectiveness of WE in well-structured domains has been frequently documented. For example, learning with WE supports knowledge acquisition in mathematics (Durkin & Rittle-Johnsson, 2012). But WE have also been shown to be effective in teaching skills in the complex and ill-structured domain of scientific argumentation (Hefter, Berthold, Renkl, Riess, Schmid & Fries, 2014; Schworm & Renkl, 2007). As explanations in psychology also constitute a part of a complex and ill-structured domain, WE should be an appropriate tool to foster students’ explanation competence.

WE present to the learner the problem itself, the steps involved in the problem-solving process as well as the solution. The learner’s task is to elaborate the WE. As the learner does not have to find the solutions themselves, they are able to focus their attention towards understanding the solution and, most importantly, the problem-solving steps. The lower cognitive demands of example elaboration allow the learner to allocate more cognitive resources to the acquisition of problem-solving schemata (Sweller, 2005). Furthermore, it is standard practice to present several isomorphic WE to the learner (Renkl, 2014). The repeated presentation of isomorphic WE should support the learner’s construction of a problem-solving schema and provide practice for the automation of the problem-solving process (Renkl, 2014). Didactically, WE constitute so-called double content examples. According to Hefter et al. (2014), double content examples consist of two domains, that is, the learning domain and the exemplifying domain. In our case, the learning domain represents knowledge about explanations, whereas the exemplifying domain provides the content area in which aspects of the learning domain are applied. In the cognitive dissonance example above, knowledge about the DN model is the learning domain, whereas knowledge about the theory of cognitive dissonance is the exemplifying domain.

WE do not, however, ensure learning success per se. Using WE alone often results in a so-called illusion of understanding (Renkl, 2014). Therefore, learning with WE is usually combined with different forms of cognitive activation and additional instructional support. One such form of activation is the presentation of incomplete WE (Stark, 1999; Van Merrienboer & Paas, 1990). Incomplete WE require the learner to complete some of the solution steps by themselves, that is, one or more solution steps have been omitted. This form of cognitive activation should limit illusions of understanding, especially when they are followed by some form of feedback (Renkl, 2014). Incomplete WE are especially promising when they are realized in a fading procedure. Such a procedure consists of a complete initial/introductory WE followed by several isomorphic incomplete WE (Renkl, 2014). The learner has to fill in the faded step(s), and receives feedback about their solution with regard to the missing steps afterwards.

Drawing on WE principles, Hefter et al. (2014) developed a short-term training intervention to enhance argumentation competences. Their intervention featured examples of an argumentative discourse. In order to cope with low prior knowledge about argumentative discourse the authors presented an introductory instructional text explaining the elements of such a discourse (see Schworm & Renkl, 2007). Inspired by Hefter et al.’s (2014) short-term training intervention, we set up a similar training intervention to foster psychology students’ explanation competence. As we did not expect our participants to have declarative knowledge of the basics of explanations and the respective terminology, we also presented an introductory instructional section explaining the DN model. The introduction presented information about the basic concepts and the various parts of an explanation. In the following section, WE examples were used to foster explanation competence. In these WE we used the DN model to illustrate the solution steps necessary to compile a written explanation. The necessary solution steps consist of first relating the elements of the DN model to the elements of the situation and the theory. The second and final solution step requires the compilation of a written narrative form of the DN model. Furthermore, the WE were double content examples: a given situation and a suitable psychological theory were used in the exemplifying domain, whereas the DN model was used in the learning domain. In order to implement a fading procedure we used two isomorphic structured WE, in which the first WE was completely worked out, and in the second WE several solution steps were omitted.

Hypotheses

The goal of this study is to evaluate the effectiveness of a training intervention to foster psychology students’ declarative knowledge about the DN model and the norms as well as explanation competence. Following the argumentation above, WE seem to be an effective tool for improving psychology students’ explanation competence and, furthermore, enhance declarative knowledge about explanations. Because explanation competence as an overarching competence can be divided into a set of cognitive dispositions to apply the norms for scientifically sound explanations, we furthermore expect WE to have positive effects on these distinct competence facets. Thus, we assume that our training intervention will:

foster psychology students’ declarative explanation knowledge,

foster psychology students’ explanation competence in general, and

foster psychology students’ competence to apply the norms for sound explanations as facets of explanation competence.

Methods

Sample size planning and sample

Sample size was planned using GPower Version 3.1.3 software (Faul, Erdfelder, Buchner & Lang, 2009). For a mixed ANOVA with two measurements in two groups and an interaction between the within and between factors we assumed a nominal significance level of α = .05, a power of β = .95 and a moderate effect of f = .25. We further assumed a correlation of .5 between the pre- and the post-test measures. This setting yielded a total sample size of 54 participants. In order to account for possible dropout we rounded up to a total sample size of 60 participants.

Only 46 psychology students (12 male, 34 female), however, from a German university participated in the study. The mean age was 21.5 years (SD = 2.63) and the median semester was 3 (range 1–7). The participants received a two-hour credit for participation. Since not enough students volunteered for the study, we weighted the assignment scheme so that approximately two-thirds of the participants were randomly allocated to the experimental condition, resulting in an experimental group of 30 (8 male, 22 female) and a control condition of 16 participants (4 male, 12 female).

The mean age in the experimental condition was 21.43 (SD = 2.32) and the median semester was 3 (range 1–6) whereas the mean age in the control condition was 21.63 (SD = 2.47) and the median semester was 4 (range 1–8).

Design and procedure

The study had a pre–post control group design, with the between-factor training intervention (experimental condition vs. control condition) and the pre-test and post-test as the repeated-measurements factor. The pre- and post-tests consisted of identical tests to measure declarative explanation knowledge and explanation competence. In addition, the pre-test included a demographic questionnaire. In the experimental condition, participants worked with the training intervention, whereas the participants in the control condition received a fake training intervention in which they had to evaluate arguments. Its content was not related to the contents of the training intervention. All materials were in a paper-and-pencil format, and the participants worked self-paced through the entire session.

The sessions took place in a laboratory. We offered participants several possible time frames to participate from which they could choose. A student research assistant guided each session.

A session consisted of the pre-test, the training intervention and the post-test. After the participant finished either the pre-test or the training intervention, they signaled to the examiner, who then handed the next portion of the material to the participant. Standardized instructions were contained in the material, but participants were free to ask comprehension questions at any time. There was no time limit for each individual part of a session, but we instructed the participants that the general time limit was 120 minutes. No participant reached this time limit. The pre-test consisted of a self-constructed multiple-choice test to measure declarative knowledge about explanations and an open-response task to measure the participant’s competence to compile an explanation. After the pre-test they received the training intervention (experimental condition) or the fake training intervention (control condition). Allocation to the training and the fake intervention was randomized according to the allocation scheme for each participant. Finally, participants received the post-test, which consisted of the same self-constructed instruments as the pre-test.

The procedure was the same for participants in the experimental and control conditions, expect for the fake training intervention in the control group. The fake training intervention consisted of texts regarding the history of rhetoric. In between these texts, argument evaluation tasks (see Stanovich & West, 1997) were presented. The fake training intervention had the same number of text pages as the training intervention, and we verified in a pre-test that the reading time for the training and the fake training intervention corresponded.

Training intervention

The intervention was carried out via a booklet consisting of three parts. The first part contained an introductory text on explanations. The text provided information about the DN model, the structure of explanations, the function of scientific theories in explanations and the norms for valid explanations. The DN model was explained using a simple example analogous to the one shown in Table 1. The introduction also featured a heuristic presenting how an explanation can be constructed if the description of an observed situation is given. This heuristic was exemplified using the same example as the one used to explain the DN model. At the end of the first part, students had to answer some multiple choice (MC) items asking about the contents of the instructional part. Feedback about the solutions was provided.

The second part contained two WE. Each of the two WE were structured as follows. First, we provided a situation to be explained as well as the theory that was used in the WE. After that, the generation of the explanation was modeled by means of a so-called explanation tool (McNeill & Krajcik, 2007). The explanation tool is a didactic aid that helps to improve the abstraction of the problem-solving steps. The explanation tool is structured according to the DN model and assists the participants in contrasting the abstract theoretical terms with their situational manifestation. The structure of the explanation tool for the first WE is shown in Table 2. Thus, the explanation tool firstly helped the learner to state clearly which theory was used to explain the observed phenomenon and, secondly, to contrast the abstract elements of the theory with the elements of the observed situation.

The first WE consisted of the cognitive dissonance example from above and thus contained only one theory. Also, the first WE was completely worked out. At the beginning of the example, participants were prompted to figure out the question to be answered by the explanation. We provided feedback containing the correct response. After that, the WE presented a written form of the explanation. The first WE focused on theory-evidence-coordination and the compilation of a written explanation from the components of the explanation tool.

The second WE provided the example of a pupil choosing a task of moderate difficulty and some information about her achievement motivation and risk avoidance. Afterwards we provided Atkinson’s (1957) choice and risk theory to explain the situation. Furthermore, we also provided Weiner’s (1992) theory of causal attribution and the self-determination theory (Deci & Ryan, 2000). The WE focused again on theory-evidence-coordination. Table 3 shows the explanation tool for the second WE. In addition to the first WE, the concept of multiperspectivity was introduced. As in the first WE, participants were prompted to figure out the question to be answered by the explanation at the beginning of the example. We again provided feedback about the correct response. After that, the participants received the explanation tool. In order to implement the fading procedure, the participants first received an empty explanation tool. They were then prompted to fill out the explanation tool by themselves. We provided feedback in the form of a completely filled-out explanation tool. Afterwards we presented the written form of the explanation. After the two WE the training intervention finished.

Table 3.

Explanation tool for the second worked example.

Question: “Why does the pupil choose a moderately difficult task?”
Choice under risk theory (Atkinson, 1957)	Abstract term	Empirical manifestation
Antecedence condition	High achievement motivation	Test indicates a high achievement motivation
Antecedence condition	High risk avoidance	Test indicates high risk avoidance
Phenomenon	Preference for moderately difficult task	Pupil chooses a moderately difficult task
Multiperspectivity
According to the self-determination theory (Deci & Ryan, 2000): the pupil may have experience of competence but the observed situation does not contain any information about other consequences resulting from that theory
Causal attribution (Weiner, 1992): theory is not applicable because it focuses on the attribution of causes after a successful or unsuccessful task

Dependent variables

Declarative explanation knowledge

Declarative explanation knowledge was measured by a self-constructed multiple-choice test. The test included questions about the three different explanation parts, the five norms that characterize scientifically sound explanations, four questions regarding the role of scientific theories in explanation and one question regarding the heuristic of how an explanation can be constructed. Every question had three options: one attractor; two distractors; and the option to indicate that the participant did not know the answer. An example item is:

“The DN model consists of the following three components:

Scientific theory, the causes of the phenomenon and its consequences;

Core statement of a phenomenon, abstract premise and specific premise;

Scientific theory, observed phenomenon and the causes of the observed phenomenon”.

where option (c) is the attractor.

Participants were instructed not to guess. Answers were dichotomously coded right or wrong. A maximum of 12 points could be attained. The participants completed the same test as both the pre-test and the post-test. Guttman’s split-half reliability for the pre-test was .71 for the pre-test and .60 for the post-test.

Explanation competence

The participants’ ability to explain a real-world situation using psychological theories was measured with a self-constructed open-response task. The same task was used in both the pre- and post-tests. The test contained a description of a pupil showing physiological symptoms of test anxiety and a decline in academic performance in mathematics. The physiological symptoms were shivering and illness before a math exam. The decline in academic performance was a series of bad grades in successive math exams. The situation also contained a description of the pupil’s family background of a father having high expectations for his son’s math performance and exerting pressure to achieve good math grades. The pupil was characterized as having been a very shy and anxious toddler. The description was 534 words long. Furthermore, we provided a summary of theories of test anxiety (e.g., Zeidner, 1998, 232 words) as well as summaries of two theories that could not explain the situation (causal attribution (Weiner, 1992) and social learning theory (e.g., Bandura, 1971)) and a summary of a theory that could only explain a part of the situation (transactional model (e.g., Sameroff & Chandler, 1975); in total 137 words for the description of the three alternative theories). Students were instructed to write down their explanation of the situation using the given theories and compose whole sentences. In the post-test, participants were additionally instructed to apply the knowledge about the explanations that they had learned in the training intervention.

To assess explanation competence we rated the presence of the explanation norms in the participants’ writings. In order to write an adequate explanation, students should mention explicitly the theory of test anxiety, provide a logically correct deduction from the explanans to the explanandum and elaborate on the theory-evidence-coordination. Firstly, we rated whether the explanation mentioned the theory (Theory mentioned). We assigned points according to the level of detail that the participants provided. No points were assigned if explanations contained no reference to the theory at all. One point was assigned if the participants mentioned the theory of test anxiety and two points were assigned if the explanation contained an explicit reference to the theory, that is, the participants briefly laid out the theory of test anxiety.

Second, we rated if the explanation contrasted the abstract elements and their manifestation in the observed situation (Theory-evidence-coordination). As theory-evidence-coordination refers to both the antecedence condition in the explanans and the phenomenon in the explanandum, it is necessary to split theory-evidence-coordination into the elements referring to each part. The participants were expected to recognize that the family background described constitutes the antecedence condition, whereas the physiological symptoms and the decline in academic performance constitute the phenomenon. Accordingly, Theory-evidence-coordination splits into three parts: Theory-evidence-coordination: antecedence condition family background, Theory-evidence-coordination: phenomenon symptoms and Theory-evidence-coordination: phenomenon decline academic performance. Each of these parts was rated according to the same scheme: No points were assigned if the explanation did not contain anything about theory-evidence-coordination. One point was assigned if the explanation contained either the element of the theory or the element of the observed situation but the relation between these was not laid out. Two points were assigned if the explanation contained both the element of the theory and the element of the observed situation but the relation between these was not elaborated. Three points were assigned if the explanation contained both the element of the theory and the element of the observed situation as well as a comment about their relation. Thus, for each part of Theory-evidence-coordination, a maximum of three points could be achieved. In order to get an overall measure we calculated the sum score of the three parts. A maximum of nine points could be achieved for the overall score for Theory-evidence-coordination.

The norm of the logical correctness of an explanation depends on theory-evidence-coordination. A logically correct explanation depends on a correct theory-evidence-coordination. If the relation of the elements of the observed situation to the elements of the theory is correct, then the explanation is automatically correct. Thus, the participants had to recognize that the family background described constitutes the antecedence condition, whereas the physiological symptoms and the decline in academic performance follow from the antecedence condition when considered in conjunction with the theory of test anxiety. Therefore, there was no separate variable for logical correctness.

Third, we rated the multiperspectivity of the explanation. To fulfil the criterion of multiperspectivity, the explanation was required to exclude the theory of causal attribution as well as social learning theory. Furthermore, the explanation must acknowledge that the transactional model explains only a part of the situation. Thus, multiperspectivity splits into three parts: Multiperspectivity: social learning theory, Multiperspectivity: causal attribution and Multiperspectivity: transactional model. The coding schemes for Multiperspectivity: social learning theory and Multiperspectivity: causal attribution were the same. No points were assigned if the explanation did not mention the respective theories. One point was assigned if the exclusion was mentioned but justified falsely. Two points were assigned if the exclusion was mentioned but not justified. Three points were assigned if the exclusion was mentioned and justified superficially. Finally, four points were assigned if the exclusion was mentioned and justified profoundly. Thus, the participants could achieve a maximum of four points for each of the two parts of Multiperspectivity. The coding scheme for Multiperspectivity: transactional model was different, because the transactional model fitted to the antecedence condition, but not to the phenomenon. We assigned no points if the explanation did not mention the transactional model or if the transactional model was excluded.³ One point was assigned if the exclusion was only mentioned, two points were assigned if the exclusion was superficially justified and three points were assigned if the exclusion was profoundly justified. In order to get an overall measure for multiperspectivity we calculated the sum score of the three parts, yielding a maximum of 11 points.

Finally, in order to get an overall measure for explanation competence we calculated the sum score of all facets of explanation competence. Consequently, the dependent variable Explanation competence equals the sum of Theory mentioned, Theory-evidence-coordination, as well as Multiperspectivity. A maximum of 22 points could be reached.

The rating procedure was as follows. Two independent raters (the first author and a student assistant holding a bachelor’s degree in psychology) coded seven randomly drawn explanations from the pre- and post-tests in three rounds. We used Cohen’s κ as the measure for inter-rater agreement. In the first round the two raters coded the explanation and afterwards discussed the rating. Cases of disagreement between the raters were discussed, which led to an adaptation of the coding scheme after the first round. In a second round the same explanations were rated and the codings compared repeatedly. Again, cases of disagreement were discussed. Afterwards, we conducted a third coding round with the same explanation. As the inter-rater agreement was sufficiently high, the second rater coded the remaining explanations. For Theory mentioned there was an absolute agreement for both the second and third coding round. For Theory-evidence-coordination: antecedence condition family background Cohen’s κ was .70 after the second and .73 after the third round, for Theory-evidence-coordination: phenomenon symptoms Cohen’s κ was .74 after the second and .72 after the third round. For Theory-evidence-coordination: phenomenon decline academic performance there was an absolute agreement after the second and third rounds; Cohen’s κ was .72. Multiperspectivity: social learning theory yielded a Cohen’s κ of .58 in the second round and .61 in the third round. For Multiperspectivity: causal attribution we had an absolute agreement in the second round and a Cohen’s κ of .68 in the third round. For Multiperspectivity: transactional model Cohen’s κ was .70 after the second round and .73 after the third round.

For further information about the assessment of explanation competence we include in Appendix 2 a table (Table 8) containing an overview of the coding criteria as well as an explanation tool featuring the core elements of the required explanation.

Statistical analyses

We used SPSS Version 20 (IBM Corporation, 2011) for all statistical analyses and applied the following analysis strategy. For every dependent variable we conducted a mixed ANOVA with pre- and post-tests as within-factor and the intervention (experimental vs. control condition) as between-factor. Afterwards, we calculated comparisons between the pre- and post-tests in each condition by means of a pairwise t-test, or we calculated comparisons between the experimental and the control condition for the pre- and post-tests separately by means of independent samples t-tests. Results of these comparisons were only reported if they were significant. We use partial-η² (η_p²) and Cohen’s κ as effect size measures.

Results

Internal validity

Experimental and control condition did not differ significantly in the distribution of sex, χ²(1) = 0.02, n.s.; nor in the mean semester, t(44) = −1.31, n.s. There were also no differences in prior declarative knowledge about explanation and explanation competence between the experimental and control conditions. These findings suggest that the randomization procedure was effective.

Declarative explanation knowledge

Descriptive statistics are presented in Table 4. A mixed ANOVA yielded a significant effect of the within-factor, F(1,44) = 68.97, p < .001, η_p²= .61; a significant effect of the intervention factor, F(1,44) = 11.81, p = .002, η_p²= .20; as well as a significant interaction, F(1,44) = 51.17, p < .001, η_p²= .54. There was no significant difference between the experimental and control conditions at pre-test, t(44) = −0.92, n.s.; whereas there was a highly significant and substantial difference between the two groups at post-test, t(44) = 8.62, p < .001, d = 2.67. A paired samples t-test revealed a significant and substantial increase in declarative explanation knowledge in the experimental condition, t(29) = 11.63, p < .001, d = 2.12; whereas no change occurred in the control condition, t(15) = 1.03, n.s.

Table 4.

Means and standard deviations for declarative explanation knowledge and explanation competence total score in the pre- and post-tests.

	Pre-test M (SD)	Post-test M (SD)
Declarative explanation knowledge
Experimental condition	3.10 (2.39)	8.13 (1.61)
Control condition	3.75 (2.08)	4.12 (1.26)
Applicable explanation knowledge
Experimental condition	4.17 (2.02)	7.07 (3.56)
Control condition	3.88 (2.58)	3.69 (3.07)

Explanation competence

A similar picture emerged with respect to Explanation competence. Descriptive statistics are presented in Table 4. A mixed ANOVA yielded a significant effect of the within-factor, F(1,44) = 7.62, p = .008, η_p²= .15; a significant effect of the intervention factor F(1,44) = 6.33, p = .016, η_p²= .13; as well as a significant interaction, F(1,44) = 10.81, p = .002, η_p²= .20. There was no significant difference between experimental and control conditions for the pre-test, t(44) = 0.38, n.s.; but for the post-test, t(44) = 3.22, p = .002, d = 0.99. A paired samples t-test indicated a significant increase in explanation competence in the experimental condition, t(29) = 4.82, p < .001, d = 0.88; whereas no change occurred in the control condition, t(15) = − 0.38, n.s.

Facets of explanation competence

We examined the effect of the training intervention for each explanation competence facet separately. Means and standard deviations for each facet are provided in Table 5; the correlations between the facets are shown in Table 6. For Theory mentioned a mixed ANOVA neither indicated an effect of the within-factor, F(1,44) = 0.70, n.s.; nor an effect of the intervention factor, F(1,44) = 0.39, n.s.; nor an interaction, F(1,44) = 0.70, n.s.

Table 5.

Means and standard deviations for the facets of explanation competence in the pre- and post-tests.

	Pre-test M (SD)	Post-test M (SD)
Experimental condition
Theory mentioned	0.80 (0.41)	0.97 (0.41)
Theory-evidence-coordination	2.40 (1.50)	2.10 (1.67)
Multiperspectivity	0.93 (1.39)	3.93 (2.78)
Control condition
Theory mentioned	0.81 (0.54)	0.81 (0.66)
Theory-evidence-coordination	2.00 (1.83)	1.94 (1.39)
Multiperspectivity	1.06 (1.34)	0.88 (1.41)

Table 6.

Correlations between declarative explanation knowledge, explanation competence total score and explanation facets.

		1	2	3	4	5
1	Declarative explanation knowledge	–	.43**	.16	.23	.57***
2	Explanation competence	.34*	–	.54***	.37*	.87***
3	Theory mentioned	.04	.51***	–	.44**	.29
4	Theory-evidence-coordination	.15	.74***	.35*	–	.11
5	Multiperspectivity	.35*	.55***	.07	−.13	–

The correlations for the pre-test are below the main diagonal, and the correlations for the post-test are above (**p < .05, **p < .01, ***p < .001).

A similar picture emerged for Theory-evidence-coordination. A mixed ANOVA neither revealed an effect of the within-factor, F(1,44) = 0.55, n.s.; nor an effect of the intervention factor, F(1,44) = 0.43, n.s.; nor an interaction effect, F(1,44) = 0.24, n.s.

For Multiperspectivity a mixed ANOVA showed a significant effect of the within-factor, F(1,44) = 14.74, p < .001, η_p²= .25; a significant effect of the intervention factor, F(1,44) = 9.24, p = .004, η_p²= .17; as well as a significant interaction, F(1,44) = 18.93, p < .001, η_p²= .30. There was no significant difference between experimental and control condition at the pre-test, t(44) = −.30, n.s.; but at the post-test, t(44) = 4.12, p < .001, d = 1.01. A paired samples t-test indicated a significant increase in Multiperspectivity in the experimental condition, t(29) = 5.89, p < .001, d = 1.07; whereas no change occurred in the control condition, t(15) = −0.64, n.s.

Discussion

Overall, the results demonstrate that the training intervention fostered, in particular, the participants’ declarative explanation knowledge, but only parts of explanation competence. Regarding declarative explanation knowledge, the pre-test results support our assumption that psychology students do not have declarative knowledge about scientific explanations. The strong effects of the experimental condition shows that our training intervention effectively enhanced the participants’ declarative knowledge of scientific explanations. The same picture emerged for the overall score for explanation competence. There was a strong effect on the experimental condition, suggesting evidence for the effectiveness of our training intervention.

On the facet level of explanation effects, however, a positive effect only emerged for Multiperspectivity. There were no learning effects for Theory mentioned and Theory-evidence-coordination. A closer look at the descriptive statistics revealed almost no gain for the facet Theory mentioned (Table 5). The means indicate that the participants had only a slightly higher tendency to mention the used theory in the post-test as compared to the pre-test. The norm to mention and elaborate on the theory seems almost absent in both the pre- and post-tests. Thus, the absence of a significant effect on Theory mentioned can be attributed to a floor effect. We observed a diametrically opposed effect for Multiperspectivity. The relatively low values in the pre-test indicate that Multiperspectivity is almost absent in the participants’ pre-test explanations. This finding may reflect a rather naïve concept of explanation in which it is sufficient to name only the theory used but not to reflect about possible alternative theories. The exposition to the norm of Multiperspectivity in our training intervention thus may have altered such a naïve concept of explanation to a more advanced and more scientific concept.

Another possible explanation of this finding is the understanding of the task demands by the participants. Instead of changing their naïve concept of explanation to a scientific one, it is possible that the participants interpreted the demand of the explanation task according to the requirements of explanations presented in the WE. In this way, there is no conceptual change (see Lambrozo, 2012) from a naïve, everyday concept of explanation to a scientific one, and the participants only fulfilled the task requirements. This view is supported by the descriptive statistics in Table 5, indicating a shift from the application of Theory-evidence-coordination to Multiperspectivity in the post-test. As Multiperspectivity was the last norm covered in the intervention, it is possible that this norm was especially salient to the participants and that they interpreted the explanation task in the post-test as a task to apply the norm of Multiperspectivity.

One more possibility besides simply fulfilling task requirements is that the participants kept their naïve concept and only added the scientific concept of explanation. This view is in accordance with conceptual change research that argues that naïve concepts are not discarded, but newer and more sophisticated concepts are additively amended (see Chinn & Samarapungavan, 2009). Thus, the task in the post-test activated the acquired scientific concept about explanations, whereas the task in the pre-test activated a naïve, pre-scientific concept of explanation. This, however, required the participants at the very least to recognize, from the context, which concept of explanation applies to a given task. From this perspective, learning is successful if, after the training intervention, the participants are able to activate the scientific concept at will and on cue from the context (Caravita & Hallden, 1994).

This interpretation, again, assumes that students associated the scientific concepts with the norm of Multiperspectivity because it was salient to them. Both of the interpretations mentioned above, of the findings regarding multiperspectivity, can be seen in the light of a recency effect – especially as the post-test took place immediately after the training intervention – but the data is not sufficient to decide which interpretation of the findings is more plausible.

A striking result on the facet level is the absence of an effect on Theory-evidence-coordination. As correct theory-evidence-coordination is at the heart of an explanation and, furthermore, a prerequisite for the logical correctness of an explanation, this is a severe shortcoming of our training intervention. The absence of this effect supports our initial notion that content knowledge of psychological theories, as taught to psychology students and as provided in the form of theory summaries in both the pre- and post-tests and in our training intervention, is in itself not sufficient to recognize how abstract theory elements can manifest in real-life situations.

This situation is analogous to the development of expertise in the domain of medical diagnosis (Charlin, Boshuizen, Custers & Feltovich, 2007). In medical diagnosis, novices learn various symptoms of illness. When trying to make a diagnosis, a novice simply compares a patient’s symptoms with the list of symptoms typical for some illnesses. During the development of expertise the knowledge of typical illness symptoms is incorporated into networks, a process known as knowledge encapsulation (Rikers, Schmidt & Boshuizen, 2000). When working with patients, so-called “illness scripts” are generated, which contain relevant information regarding the illness symptoms and their possible manifestation in real cases. These illness scripts therefore facilitate the diagnostic process. An analogous development may apply to the acquisition of expertise in the application of psychological theories. In working with psychological theories, students acquire knowledge about what a theory “looks like in practice.” So, in the development of expertise, the students’ content knowledge of psychological theories is enriched with knowledge on how the abstract theoretical terms may manifest in specific situations. As indicated by the pre-test data, the participants are novices regarding the application of psychological theories (Birke et al., 2016), as most of them were in the first half of their undergraduate studies. This could explain why the participants in our training intervention failed to acquire adequate theory-evidence-coordination skills.

This perspective has two important consequences. Firstly, from an instructional point of view, if the goal is to foster students’ explanation competence, it seems necessary to not only provide worked examples demonstrating the compilation of an explanation, but also to provide examples on how psychological theories are applied in practice. Secondly, from a cognitive point of view, more fine-grained analyses of the various kinds of knowledge involved in explanation competence are necessary. For example, the types of knowledge involved in explanations could be analyzed with regard to Anderson et al.’s (2001) knowledge taxonomy. Such an analysis in the context of that knowledge taxonomy would allow for the inference of specific instructional measures to support the acquisition of various types of knowledge presumably involved in explanation competence.

Besides such an explanation drawing on the expertise in the use of psychological theories, there is also the possibility that our training intervention does not effectively foster the competence to express a generated explanation in a written form. In the training intervention, theory-evidence-coordination was always demonstrated by means of the explanation tool. The transition from the explanation to a textual explanation was not, however, instructionally supported. Thus, the participants may have been able to construct a causal model of the phenomenon drawing on a psychological theory but were not able to express this model in the form of a written explanation. As explanation competence was assessed solely by means of a written explanation, the assessment does not allow separating the competence to construct such a model and the ability to verbally express the model.

Lastly, another potential explanation of the missing effect of theory-evidence-coordination is that two WE might simply not be enough to automatize the steps required in theory-evidence-coordination, and the participants might not have received enough training to coordinate theory with evidence without the use of the explanation tool.

Since we do not have data on the learning process itself, however, further studies should attempt to amend the data on learning outcomes with additional process data. Furthermore, a more fine-grained assessment of explanation competence is needed.

Limitations

One of the major limitations of the present study is the use of the same test for declarative explanation knowledge and explanation competence in the pre- and post-tests. Although such a measure ensures—at least superficially—the equivalence and thus the comparability of pre- and post-tests, at the same time it hinders the generalizability of the results. This is less problematic for declarative explanation knowledge, but more so for explanation competence. The current results provide no evidence that the participants can transfer the acquired explanation competence to novel situations.

Another possible limitation is the possibility that the participants were simply trained to the task. Since the explanation task in the pre-and the post-tests was different from the task in the worked examples, however, such a training process seems unlikely. Besides, if teaching-to-the-task had happened, it seems plausible that all facets of explanation competence would have been affected and not only Multiperspectivity. In order to rule out such a possibility, a transfer task would be needed that had not been administered in a pre-test beforehand.

A third limitation refers to the authenticity of the test task measuring explanation competence. As Chinn and Malhotra (2001) argue, tasks differ in their authenticity. Making an explanation requires the selection of suitable theories, as well as the consideration of alternative theories and/or ruling out unsuitable theories from the vast number of theories available. In our explanation competence task, however, we provided the participants with a set of preselected theories from which they had to choose suitable ones. Thus, this task is not wholly authentic and therefore provides no evidence that our training intervention fosters the ability to find a suitable theory without effective instructional support. In order to construct a more authentic task, it would be necessary to draw on the principles of problem-based learning (e.g., Schmidt, 1983) and provide an open task in which the participants have to look for suitable theories by themselves. With regard to the lack of expertise in working with psychological theories mentioned above, as well as organizational difficulties associated with the access to scientific databases and the students’ experience in the autonomous search and review of scientific literature, however, the feasibility of such a task is questionable. Lastly, a limitation is the implementation of the whole intervention in one session. Because the post-test occurred immediately after the training, it is not possible to draw any conclusions about the sustainability of the learning effects.

Conclusions

The results indicate that it is possible to foster declarative knowledge about explanation and explanation competence with our training intervention. More research is needed concerning the number of necessary WE, and the role of content knowledge of psychological theories for explanation competence. Besides fading, further research could investigate if instructional explanations are also helpful in fostering explanation competence.

From a didactic point of view, our results can be seen as hints about how to implement the concept of explanation competence in psychology teaching. Our results suggest providing declarative knowledge about the scientific concept of explanation before the compilation of explanation is demonstrated. Instructors should firstly use structural aids like the explanation tool to demonstrate how explanations of psychological phenomena are constructed. Furthermore, to ensure correct theory-evidence-coordination, instructors should also emphasize content knowledge and its relation to the observed situation, and also use an adequate number of examples to ensure that students acquire the necessary solution steps. Fading procedures are also easily implemented in classroom settings. The fading of our training intervention can easily be implemented by means of classroom questions or exercises.

Lastly, we want to emphasize that explanation competence is not only an import construct with regard to the science of psychology and the scientific competences of psychology. Constructing explanations is also a vital part of applied psychology, for example, presenting expert opinions in court cases. Thus, the instructional possibilities for fostering the explanation competence of psychology students should be studied in more detail.

Footnotes

Acknowledgements

We want to thank Martin Klein and Stephanie Knoll for their support in the preparation of the manuscript and Neele Oetjen for her support in coding the explanation task.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author biographies

Eric Klopp, Dr. phil., is a research associate at the chair for Personality Development and Education at the Department of Education, Saarland University. His research focuses on epistemological beliefs, scientific competencies and argumentation, learning from worked examples and error-based learning, as well as statistical methods, especially latent variable models.

Robin Stark is professor for Personality Development and Education at the Department of Education, Saarland University. His research focuses on learning with worked examples, problem-based learning, conceptual change, scientific competencies, medical education, teacher education, music-related aesthetical argumentation and error-based learning.

Appendix 1

References

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., …Wittrock, M. C. (Eds.), (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Addison Wesley Longman.

Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64, 359–372.

Bandura

(1971) Psychological modeling: Conflicting theories, Chicago, IL: Aldine Atherton.

Birke, P., Rosman, T., & Mayer, A.-K. (2016). Entwicklung fachspezifischer epistemologischer Überzeugungen bei Studienanfängern der Psychologie und Informatik [Development of domain-specific epistemological beliefs in beginning psychology and information sciences students]. In A.-K. Mayer, & T. Rosman, T. (Eds.), Denken über Wissen und wissenschaft. Epistemologische überzeugungen [Thinking about knowledge and science – Epistemological beliefs] (pp. 101–119). Lengerich, Germany: Pabst Science Publishers.

Caravita

Halldén

(1994) Reframing the problem of conceptual change. Learning and Instruction 4: 89–111.

Charlin

Boshuizen

Custers

E. J.

Feltovich

P. J.

(2007) Scripts and clinical reasoning. Medical Education 41: 1178–1184.

Chinn

Malhotra

B. M.

(2001) Epistemological authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks. Science Education 86: 175–218.

Chinn

Samarapanguvan

(2009) Conceptual change – Multiple routes, multiple mechanism: A commentary on Ohlsson (2009). Educational Psychologist 44: 48–57.

Deci, E. L. & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227–268.

10.

Dietrich

Zhang

Klopp

Brünken

Krause

U.-M.

Spinath

F. M.

Stark

Spinath

(2015) Scientific competencies in the social sciences. Psychology Learning and Teaching 14: 115–130.

11.

Durkin

Rittle-Johnson

(2012) The effectiveness of using incorrect examples to support learning about decimal magnitude. Learning and Instruction 22: 206–214.

12.

Faul

Erdfelder

Buchner

Lang

A.-G.

(2009) Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods 41: 1149–1160.

13.

Festinger

(1957) A theory of cognitive dissonance, Stanford, CA: Stanford University Press.

14.

Gerrig

R. J.

Zimbardo

P. G.

(2010) Psychology and life, Boston, MA: Allyn and Bacon.

15.

Heider

(1958) The Psychology of Interpersonal Relations, New York, NY: John Wiley & Sons.

16.

Hefter

Berthold

Renkl

Riess

Schmid

Fries

(2014) Effects of a training intervention to foster argumentation skills while processing conflicting scientific positions. Instructional Science 42: 929–947.

17.

Hempel

Oppenheim

(1948) Studies in the logic of explanation. Philosophy of Science 15: 135–175.

18.

IBM Corporation (2011). IBM SPSS Statistics for Macintosh, Version 20.0. Armonk: IBM Corporation.

19.

Kim

(1994) Explanatory knowledge and metaphysical dependence. Philosophical Issues 5: 51–69.

20.

Klieme

Leutner

(2006) Kompetenzmodelle zur Erfassung individueller Lernergebnisse und zur Bilanzierung von Bildungsprozessen. Beschreibung eines neu eingerichteten Schwerpunktprogramms der DFG [Competence models for assessing individual learning outcomes and evaluating educational processes. Description of a new priority program of the German Research Foundation (DFG)]. Zeitschrift für Pädagogik 52: 876–903.

21.

Kuhn

(2005) Education for thinking, Cambridge, MA: Harvard University Press.

22.

Lambrozo

(2012) Explanation and abductive inference. In: Holyoak

K. J.

Morrison

R. G.

(eds) The Oxford handbook of thinking and reasoning, Oxford, UK: Oxford University Press, pp. 260–276.

23.

McCain

(2015) Explanation and the nature of scientific knowledge. Science & Education 24: 827–854.

24.

McNeill

K. L.

Krajcik

(2007) Instructional strategies to support students writing scientific explanations. In: Luft

Gess-Newsome

Bell

(eds) Science as inquiry in the secondary setting, Washington, DC: National Science Foundation.

25.

Ohlsson

(1992) The cognitive skill of theory articulation: A neglected aspect of science education. Science & Education 1: 181–192.

26.

Ohlsson

(2010) Questions, patterns, and explanations, not hypothesis testing, is the core of psychology as of any science. In: Toomela

Valsiner

(eds) Methodological thinking in psychology: 60 years gone astray?, Charlotte, NC: Information Age Publisher, pp. 27–43.

27.

Renkl

(2014) Toward an instructionally oriented theory of example-based learning. Cognitive Science 38: 1–37.

28.

Rikers

M. P. J.

Schmidt

H. G.

Boshuizen

H. P. A.

(2000) Knowledge encapsulation and the intermediate effect. Contemporary Educational Psychology 25: 150–166.

29.

Rosenberg

(2012) Philosophy of science, New York, NY: Routledge.

30.

Sameroff

A. J.

Chandler

M. J.

(1975) Reproductive risk and the continuum of caretaker casualty. In: Horowitz

F. D.

(ed.) Review of child development research Vol 4, Chicago, IL: University of Chicago Press, pp. 187–244.

31.

Schmidt

H. G.

(1983) Problem-based learning: Rationale and description. Medical Education 17: 11–16.

32.

Schworm

Renkl

(2007) Learning argumentation skills through the use of prompts for self-explaining examples. Journal of Educational Psychology 99: 285–296.

33.

Stanovich

K. E.

West

R. F.

(1997) Reasoning independently of prior belief and individual differences in actively open-minded thinking. Journal of Educational Psychology 89: 342–357.

34.

Stark, R. (1999). Lernen aus Lösungsbeispielen. Einfluss unvollständiger Lösungsbeispiele auf Beispielelaboration, Lernerfolg und Motivation. [Learning with worked-out examples. Effects of incomplete worked-out examples on example elaboration, learning success, and motivation]. Goüttingen, Germany: Hogrefe.

35.

Stark

(2005) Constructing arguments in educational discourses. In: Gruber

Harteis

Mulder

Rehrl

(eds) Bridging individual, organisational, and cultural aspects of professional learning, Regensburg, Germany: S. Roderer, pp. 64–71.

36.

Sweller

(2005) Implications of cognitive load theory for multimedia learning. In: Mayer

R. E.

(ed.) The Cambridge handbook of multimedia learning, New York, NY: Cambridge University Press, pp. 19–30.

37.

Tytler

Peterson

(2003) Tracing young children’s scientific reasoning. Research in Science Education 33: 433–465.

38.

Van Merrienboer

J. J. G.

Paas

(1990) Automation and schema acquisition in learning elementary computer programming: Implications for the design of practice. Computers in Human Behavior 6: 273–289.

39.

Walton

D. N.

(1989) Dialogue theory for critical thinking. Argumentation 3: 169–184.

40.

Weiner

(1992) Human motivation: Metaphors, theories and research, Newbury Park, CA: Sage Publications.

41.

Westermann

(2000) Wissenschaftstheorie und Experimentalmethodik. [Philosophy of science and experimental design], Göttingen, Germany: Hogrefe.

42.

Woodward, J. (2014). Scientific explanation. In E. N. Zalta (Ed.) The Stanford Encyclopedia of Philosophy Archive (Winter 2014 Edition). Retrieved from https://plato.stanford.edu/archives/win2014/entries/scientific-explanation.

43.

Zeidner

(1998) Test anxiety: The state of the art, New York, NY: Plenum.