Sage Journals: Discover world-class research

Abstract

This study investigated the impact of the knowledge revision components (KReC) framework on enhancing student conceptual understanding and eliminating misconceptions about matter and change in science lessons. The study used refutation texts and integrated them into the Online Advanced Organizer Concept Teaching Material (ONACOM). The research involved a three-week intervention with three groups: Treatment Group I (received refutation texts + ONACOM), Treatment Group II (received refutation texts only), and Control Group (did not receive refutation texts or online materials). One hundred eighty-eight fourth-grade students from four classes in two public primary schools in Antalya, Türkiye, participated in the study. The MANCOVA analysis was used to evaluate the results. Results indicated that when the pretest scores were controlled, conceptual understanding scores revealed a significant positive difference in treatment group I compared to the other groups. Moreover, there was a significant difference in the misconception scores between the treatment groups and the control group in favor of the treatment groups. It also was found that despite the interventions made in the treatment groups, some misconceptions of the students resisted revision. In conclusion, the intervention with integrating refutation texts into ONACOM had a significant positive impact on students’ conceptual understanding and the knowledge revision process. The study also discussed students’ misconceptions and conceptual understandings about matter and change within the KReC framework, providing suggestions for future research.

Keywords

refutation text online material misconceptions knowledge revision

Introduction

In science education, it is important to provide students with progressive information according to their learning processes, to eliminate the reasons for misconceptions such as pre-existing beliefs and over-generalizations based on experience, to guide their science learning process, and to develop their conceptual understanding (Korur, Toker & Eryılmaz, 2016; Vosniadou, 2019). Studies on conceptual understanding commonly employ two- or three-tier tests, which are usually scored based on the total correct item score (Korur, 2015; Korur, Enil & Göçer, 2016). Students’ implementation of the concept test and its conjunction with the related concept maps was a primitive technique to determine the conceptual understanding level of students (Aydın Ceran & Ateş, 2020). Unfortunately, misconceptions are a reality among students, prospective teachers, and even science teachers, which negatively affects the understanding of scientific concepts (Kanli, 2014; Korur, 2015). Addressing misconceptions in science education involves identifying students’ misconceptions and implementing effective ways to eliminate them (Kaltakci Gurel et al., 2015; Korur, 2015).

A comprehensive approach that describes the process of revision of misconceptions is the Knowledge Revision Components framework (KReC; Kendeou & O’Brien, 2014). Although different theories target the revision of misconceptions or conceptual change (Bensley & Lilienfeld, 2017; Chi, 2008; diSessa, 1993), the present study uses the KReC framework as a cognitive mechanism to support the learning process to be used for the revision of misconceptions. In this respect, the present study is not a direct test of this theoretical approach, but rather a framework that is utilized to eliminate students’ misconceptions and construct their conceptual understanding processes in the specified science topics.

Refutation texts are considered one of the remarkable solutions for overcoming misconceptions in recent years (Hunsu et al., 2023; Prinz et al., 2022; Schroeder & Kucera, 2022). Kendeou (2024) states that refutation text studies provide systematic support for the main processes of the KReC framework. Although some experimental studies on refutation texts generally provide strong evidence of their effectiveness in overcoming misconceptions (Flemming et al., 2020; Hunsu et al., 2023) and facilitate knowledge revision (Weingartner & Masnick, 2019), Zengilowski et al. (2021) emphasized that more research is needed on how refutation texts can most effectively contribute to conceptual change or knowledge acquisition of students. They believed that the refutation text phenomenon should be crucially analyzed in terms of theoretical and methodological issues, such as including interpretation of attentional measures, identifying the knowledge domains that lend themselves to refute, and clarifying unexplored assumptions about how conceptual change occurs. Tippett (2010) stated that refutation text alone was not enough to provide knowledge revision. He also argued that “Refutation text used in conjunction with other types of text, or with videos, demonstrations, hands-on experiments, and other activities, will increase the likelihood of conceptual change” (Tippett, 2010, p. 966). Similar to Tippett’s (2010) view, Butterfuss and Kendeou (2021) further expanded the KReC framework into the KReC multiple documents (KReC-MD) and utilized it across multiple documents, providing knowledge revision by revising student misconceptions. However, Kendeou (2024) underlines the need to understand how multimodal texts, which contain various modes (i.e., dynamic visuals, sound, text, gestures) and have become a dominant form of communication in social media, affect the revision mechanism.

Based on these needs emphasized in the refutation text literature, the present study uses the Online Advanced Organizer Concept Teaching Material (ONACOM), an online tool that offers multimodal use in concept teaching, unlike other studies. As an innovation in this experimental study, the effects of interventions designed by integrating the refutation texts into ONACOM on strengthening the knowledge revision of 4th-grade primary school students on matter and change and reducing the impact of misconceptions are examined. Thus, we aim to contribute to the comprehensive and detailed explanation of knowledge revision by combining refutation texts with multimodal in the online environment.

Background

Theory of Conceptual Change and Knowledge Revision

Undoubtedly, the process of correcting misconceptions should be grounded in the theory of conceptual change. This theory can be viewed from two perspectives: knowledge-as-theory and knowledge-as-elements (Özdemir & Clark, 2007). In the knowledge-as-theory view, based on Piaget’s learning theories, students are unlikely to change their understanding unless they encounter problems that cannot be resolved within their current conceptual framework. For a radical conceptual change to occur, the student must be dissatisfied with their initial understanding, and the new scientific knowledge to be assimilated must be understandable, reasonable, and effective (Posner et al., 1982). In addition, the knowledge-as-elements perspective, proposed by diSessa (1993) focuses on the nature of knowledge elements. It suggests that learners’ knowledge is an unstructured collection of simple elements formed through interaction with the physical world offering an organic view of their development and complexity (diSessa, 1993). Neither this nor the knowledge-as-theory perspective offers a singular explanation for the complexity of conceptual change. Instead, both provide different dimensions for understanding the richness and multidimensionality of this process (Özdemir & Clark, 2007).

On the other hand, Kendeou (2024) characterizes conceptual change as a fundamental transformation of a student’s understanding of a concept and specifically states that it is used in fundamental misconceptions. He offered the theory of knowledge revision, which mainly focused on the learner’s simple, factual inaccuracies or misunderstandings instead of false mental models or deep-rooted misconceptions. This theory explains that learners will modify their existing knowledge and schemas if they are dissatisfied with their current understanding when it fails to resolve a problem scientifically and accurately. However, the existing literature provides limited insights into readers’ cognitive processing during knowledge revision (Kendeou, 2024). Kendeou envisaged the need to investigate situations in which knowledge revision can occur when conceptual change does not occur and that this can be considered a paradigm shift in this field. Kendeou makes sense of the philosophy and theoretical structure in this approach by building on refutation texts (Butterfuss & Kendeou, 2021; Kendeou & O’Brien, 2014). The KReC and KReC-MD frameworks are used to form the knowledge revision theory, which replaces students’ background knowledge with accurate information and reorganizes conflicting knowledge, thereby creating conditions that facilitate knowledge revision during refutation text reading and supporting successful learning in the presence of misconceptions (see Kendeou, 2024 for the details of the theory). The theoretical pattern of the present study is based on the knowledge revision approach developed from Kendeou’s philosophical perspective.

Refutation Text and Knowledge Revision

Refutation texts state previously acquired but incorrect knowledge, then directly refute it, and finally provide an alternative and correct knowledge with a causal explanation (Kendeou et al., 2014, 2016; van den Broek & Kendeou, 2008). These powerful persuasion tools (Hynd, 2001) have three basic features (Weingartner & Masnick, 2019): (i) misconception, (ii) refutation, and (iii) evidence. It usually starts by stating a common misconception among students. Then, this misconception is refuted. Refutation is supported by describing evidence that supports currently accepted scientific views. Thus, students are consecutively confronted by their misconceptions, the correct expression of these misconceptions, and the related scientific evidence.

The effectiveness of these tools in persuading the reader can be explained by the fact that they create cognitive dissonance between existing erroneous ideas and new information and that they are understandable, reasonable, and useful (Hynd, 2001). Tippett (2010) states that readers’ metacognitive awareness about the inaccuracy of their current knowledge, consistent and reasonable presentation of correct information, and the nature of the information highlights the power of these texts.

Prior knowledge has a critical impact on the processes and products of learning experiences (Vosniadou, 2019). It has been known for a long time that if this prior knowledge contains misconceptions, serious problems arise in the process of learning new knowledge (Dellantonio & Pastore, 2021). In knowledge revision terminology, prior knowledge and new information on the relevant subject are taken into account simultaneously (Kendeou et al., 2017). In knowledge revision, the students’ misconception arising from previous knowledge is gradually reduced according to newly encoded correct information (Butterfuss & Kendeou, 2021).

The KReC explains understanding how knowledge revision can occur during reading (Kendeou et al., 2017). The KReC covers five key principles: encoding, passive activation, coactivation, integration, and competing activation (see Kendeou et al., 2014). According to this framework, information in long-term memory (which cannot be erased) can be activated while reading a text, by the use of encoding and passive activation principles. Such activation supports the integration of new information with the student’s prior knowledge. These two pieces of information (prior knowledge and newly acquired knowledge) compete during the activation process. Knowledge revision occurs when new knowledge wins the competition thanks to its activation over prior knowledge (Kendeou et al, 2014; Will et al., 2019). Kendeou et al. (2014) argue that “the KReC framework can contribute to a deeper understanding of many of the existing models and theories of conceptual change and/or knowledge revision” (p. 391).

The examination of the literature on the refutation texts reveals that the researchers generally try to explain the changes in the student’s scientific knowledge base (Kendeou et al., 2017) by using a single term, namely “conceptual change” or “knowledge revision.” It is noteworthy that the term “knowledge revision” has been extensively used in the science learning literature (Flemming et al., 2020; Harsch & Kendeou, 2023; Hunsu et al., 2023) since the introduction of the KReC framework (Kendeou et al., 2014). Kendeou et al. (2017) explain their use of this term to express intentional revision or modification of misconceptions in two aspects: (i) Knowledge revision directly addresses the difficulty of making certain assumptions about the structure and nature of prior knowledge. (ii) It is operationalized (rejects the delete-and-replace view of conceptual change). Therefore, it causes the activation of students’ existing misconceptions to be gradually reduced and/or eliminated, thus intervening in pre-existing misconceptions.

As many researchers similarly state (Kim & Kendeou, 2021; Weingartner & Masnick, 2019; Will et al., 2019), refutation texts contain short-term interventions, while misconceptions are deep-rooted and resistant structures to change (Asberger et al., 2021; Dellantonio & Pastore, 2021). Therefore, this study follows the methods of (i) explaining the change in students’ misconceptions with the term knowledge revision and (ii) conducting a research process based on the KReC framework.

Refutation Text Interventions in the Literature

Evidence from past refutation text intervention studies suggests that when students misconceptions and scientific concepts are simultaneously activated, integrated, and compared, the attention is focused on the conflict between these two beliefs, and refutation texts become more effective tools for resolving misconceptions than texts without refutation (Danielson et al., 2016; Schroeder & Kucera, 2022; Torsney et al., 2021). In addition, previous studies have used some textual, visual, rhetorical, or multimodal tools in conjunction with refutation texts to enhance their effect of them. For example, Danielson et al. (2016) showed that refutation texts combined with analogy or graphics and analogy were more successful in encouraging revision of scientific topics in college students. Butterfuss and Kendeou (2021) extended the understanding of knowledge revision from the process of readers reading a single refutation text to interacting with multiple texts or documents, thus extending KReC to knowledge revision components-multiple documents (KReC-MD). These two frameworks contain the same assumptions and conditions; KReC-MD also proposes that “source reliability and intertextual integration are two key factors that influence the success of knowledge revision in the context of multiple documents” (Kendeou, 2024, p.11). Butterfuss (2020) conducted a series of experiments testing the assumptions of the KReC-MD framework and these key factors. These experiments indicated that undergraduate students demonstrated greater intertextual integration when reading refutation texts from high-credibility sources compared to non-refutation texts from low-credibility sources, and that students who engaged in greater intertextual integration experienced more substantial knowledge revision. In addition to these, Johnson et al. (2022) found that a multimodal refutation intervention using an authentic video from an expert source was effective in reducing misconceptions about a socio-scientific topic such as the COVID-19 pandemic compared to a group that did not receive the intervention. In the present study, considering the different interventions and integrations of refutation text in past studies, we brought the refutation text to bear on a digital content organizer, which we can integrate into multiple and different types of educational content.

The Online Advance Organizer Concept Teaching Material (ONACOM)

With a versatile structure that can be integrated into teaching methods, ONACOM is a digital content organizer that offers the opportunity to upload different digital content (such as video, image, URL, and text) to concepts on a concept map. ONACOM differs from the examples in the related literature with its features such as having its database structure, users having their profile pages, and teachers and students having different user authorizations (see Korur, Toker & Eryılmaz, 2016). This material has been used in teaching science topics such as light, sound, force, and motion to students at different grade levels, and evidence has been provided that it has increased students’ achievement and attitudes (Korur, Toker & Eryılmaz, 2016; Yılmaz & Korur, 2021). In addition, Türksoy (2019), in his experimental study integrating augmented reality applications into ONACOM, provided strong evidence that this material helped fourth-grade primary school students to construct and retain new knowledge in science subjects. ONACOM has been developed to increase the conceptual understanding of students in various studies (Korur, Toker & Eryılmaz, 2016; Türksoy, 2019). There is no study in the literature on the integration of ONACOM with refutation texts to contribute to students’ concept acquisition and knowledge revision processes. The usage of ONACOM in this study is compatible with the KReC framework (Kendeou et al., 2014) since it can support the coactivation, integration, and competing activation processes.

The Present Study

This study was conducted experimentally on three groups. In the first group (Treatment-1), the teaching process of the “matter and change” subject was carried out using the refutation texts integrated into ONACOM in line with the KReC framework (ONACOM-RT). As for the second group (Treatment-2), the teaching process was carried out using the refutation texts in line with the KReC framework (RT). In the third group (Control), the teaching process was carried out by a method included in the related curriculum (TMC), in which no refutation texts or refutational explanations were used. In this context, to test the effects of the applied methods among the groups and within the groups, the null hypotheses of the study are as follows:

The ONACOM-RT, RT, and TMC methods do not lead to significant differences in students’ conceptual understanding or misconceptions about Matter and Change when accounting for prior knowledge.

The ONACOM-RT method does not significantly improve students’ conceptual understanding of Matter and Change.

The ONACOM-RT method does not significantly reduce students’ misconceptions about Matter and Change.

The RT method does not significantly improve students’ conceptual understanding of Matter and Change.

The RT method does not significantly reduce students’ misconceptions about Matter and Change.

The TMC method does not significantly improve students’ conceptual understanding of Matter and Change.

The TMC method does not significantly reduce students’ misconceptions about Matter and Change.

In addition to testing the seven null hypotheses, we aimed to identify robust misconceptions across all groups. The sample of the study, method, measurement tools, and ethical considerations were approved by the Provincial Directorate for National Education of Antalya, Türkiye (#22025226). All participants in the study were volunteers.

Method

Design and Participants

The study had a quasi-experimental pretest-posttest, treatment-control group research design. In this design, ONACOM-RT and RT were applied to the treatment groups, whereas TM-C was applied to the control group. The two treatment groups received different interventions than a single control group. While multiple control groups could enhance the robustness of the findings by providing additional comparisons, they also increase the complexity of the experimental design. Kirk (2013) notes that multiple control groups can be beneficial but are not always required. Given the context of this study—specifically the number of variables, available sample size, and the hypothesis being tested—a single control group was sufficient to address the specific research questions and the variables under investigation. Matter and Change Three Tier Test (MCT) was administered to all groups at the beginning of the study and 3 weeks after the treatment to determine the students’ conceptual understanding and misconception scores (see Table 1). The school applications were performed in line with the Matter and Change unit for fourth graders.

Table 1.

The Quasi-Experimental Design of the Study.

Groups	Pre-test/pre-scores	Methods	Treatment period	Post-test/post-scores
Treatment-1	MCT/	ONACOM-RT		MCT/
Treatment-2	Pre-conceptual understanding score (CUS)	RT	3 weeks	Post-conceptual understanding score (CUS)
Control	Pre-misconception score (MS)	TMC		Post-misconception score (MS)

The accessible population includes all fourth graders in Konyaaltı and Muratpaşa, the two biggest central districts of the province of Antalya, Türkiye. Since individual student selection and group assignment are not feasible, a two-stage stratified cluster random sampling was employed for a representative sample. Schools served as the strata, and classes as the clusters. First, the two most crowded state schools in these districts were chosen. Second, two classes were randomly selected from each school.

Data were collected from 188 students from the 2 schools in the mentioned districts of Antalya. There were 63 students (30 girls and 33 boys) in the control group, 62 students (32 girls and 30 boys) in the Treatment-1 group, and 63 students (28 girls and 35 boys) in the Treatment-2 group.

Variables and Score Types

Pretest scores were designated as covariates, posttest scores as dependent variables, and the teaching methods used (ONACOM-RT, RT, and TMC) as independent variables. Two types of scores were calculated based on both students and items. The scores based on students were calculated as pretest and posttest scores to perform inferential statistical analyses. The scores based on items were those that reflected percentages based on tiers and those that examined the validity of the data collection instrument.

Matter and Change Test Three-Tier Conceptual Understanding Student Score (Max. 14)

It is the number of items a student answers correctly in the first two tiers of a test and marks as “Confident” (1-1-1) in the third one. Higher scores indicate a better conceptual understanding. It was included in the analyses as a conceptual understanding pre-test score (CUS-PRE) and post-test score (CUS-POST).

Matter and Change Test Three-Tier Misconception Student Score (Max. 14)

It is the number of items a student answers incorrectly in the first two tiers of a test and marks as “Confident” (0-0-1) in the third one. Higher scores indicate a greater number of misconceptions. It was included in the analyses as “misconception pretest score” (MS-PRE) and “misconception posttest score” (MS-POST).

One-Tier Misconception Based on Item Percentage Score (MBI-1)

For each misconception, the number of items with zero in the first tier is divided by the total number of students and multiplied by 100.

Two-Tier Misconception Based on Item Percentage Score (MBI-2)

For each misconception, the number of items with zero in the first two tiers is divided by the total number of students and multiplied by 100.

Three-Tier Misconception Based on Item Percentage Score (MBI-3)

For each misconception, the number of items with zero in the first two tiers and with one in the third tier is divided by the total number of students and multiplied by 100.

False Negative Percentage Score (FN; 0-1-1)

For each item, the number of scores with zero in the first tier and one in the second and third tiers is divided by the total number of students and multiplied by 100.

False Positive Percentage Score (FP; 1-0-1)

For each item, the number of scores with one in the first and third tiers and zero in the second tier divided by the total number of students and multiplied by 100.

Data Collection Instrument

Matter and Change Three Tier Test (MCT)

The test was developed by the first two authors of this study. Initially, an item pool was created to design the MCT, aimed at identifying misconceptions and assessing conceptual understanding. Most items were crafted by the authors to align with the learning outcomes in the Turkish science curriculum (The Turkish Ministry of National Education [TMNE], 2018). The MCT consisted of three tiers: the first tier presented a question, the second tier asked for the reasoning behind the response, and the third tier gauged participants’ confidence in their answers (options: “I am sure,”“I just guess,”“I am not sure”). The face and content validity of the MCT was confirmed by experts (three faculty members, two doctoral students, and two master students). A pilot study was conducted with 156 fifth graders who had studied “Matter and Change” to assess the validity and reliability of the MCT. The pilot results showed a significant correlation between students’ scores on the first two tiers and their confidence in the third tier (r[156] = .718, p < .05).

A classical item analysis was conducted for the first two tiers, as the third tier had only yes–no responses. The difficulty and discrimination indices were calculated for the first two tiers of each item. For item difficulty, one item (item 6) was classified as very difficult (.00–.19), five items (items 1, 3, 4, 5, and 7) were difficult (.20–.39), two items (items 2 and 9) were moderate (.40–.59), six items (items 8, 10, 11, 12, 15, and 16) were easy (.60–.79), and two items (items 13 and 14) were very easy (.80–1.00). According to the threshold values of Crocker and Algina (2008), the test had moderate difficulty.

In misconception tests, it is recommended to detect assessment errors by identifying correct answers with incorrect reasoning (false positives) or incorrect answers with correct reasoning (false negatives) (Hestenes & Halloun, 1995). Minimizing false positive and false negative rates to below 10% is crucial for improving content validity in multiple-choice tests (Hestenes & Halloun, 1995). Based on these analyses, items 2 and 9, which had discrimination values below .19 and false positive/negative rates above 10%, were removed, as other items adequately covered their learning outcomes. The remaining items had discrimination indices between .200 and .417, indicating the discrimination was acceptable (Ebel, 1954). After removing two items, the KR-20 coefficient for test reliability was .77, reflecting acceptable consistency in student responses (Fraenkel et al., 2012). Following the pilot study, expert feedback was re-collected for the MCT, and three items were revised. An expert also reviewed the grammar and vocabulary. The final version of the MCT included 14 items assessing nine misconceptions. Alternative sets representing these misconceptions are shown in Table 2 (see for details Korumaz, 2018).

Table 2.

Alternative Sets Showing Misconceptions According to Three Tiers.

Misconception	Misconceptions alternative sets	Number of items
1. Heat and temperature are the same	1.1 A 1.2 A 1.3 A	1
2. Temperature varies depending on the size of the object	3.1 A 3.2 A 3.3 A	1
2. Temperature varies depending on the size of the object	3.1 C 3.2 A 3.3 A	1
3. Heat can flow in all directions continuously	2.1 B 2.2 A 2.3 A	3
	2.1 C 2.2 A 2.3 A
	4.1 A 4.2 A 4.3 A
	4.1 C 4.2 A 4.3 A
	6.1 A 6.2 A 6.3 A
	6.1 C 6.2 A 6.3 A
4. The material of the object affects the temperature	5.1 A 5.2 A 5.3 A	1
4. The material of the object affects the temperature	5.1 A 5.2 C 5.3 A	1
5. Even if we perceive properties of matter such as color and smell, if the matter is not visible, it does not exist in the environment	7.1 B 7.2 A 7.3 A	2
	13.1 A 13.2 A 13.3 A
	13.1 C 13.2 A 13.3 A
6. Gases are not matter because most gases are invisible, and they do not have mass	8.1 A 8.2 A 8.3 A	1
7. The change of state is a physical and chemical change	9.1 A 9.2 A 9.3 A	3
	12.1 B 12.2 A 12.3 A
	12.1 C 12.2 A 12.3 A
	14.1 A 14.2 A 14.3 A
	14.1 C 14.2 A 14.3 A
8. Materials may only exhibit the properties of one of the states of matter	10.1 B 10.2 A 10.3 A	1
	10.1 C 10.2 A 10.3 A	1
9. Melting/freezing/boiling/evaporation is mostly considered to be properties of only water	11.1 A 11.2 A 11.3 A	1

Teaching Materials

Refutation Texts

Refutation texts for misconceptions in that specific unit were included in certain slides in presentations. During the treatments, nine refutation texts were distributed to the Treatment-1 and Treatment-2 groups in printed form and shown in PowerPoint presentations. In presentations, the instructional presentation method was used to guide refutation text. Two refutation texts were used in the first week, four in the second week, and three in the third week. The refutation texts were designed similarly to the studies of Hynd et al. (1997) and Tippett (2010). The introduction of the refutation text included a question that expresses the misconception and emphasizes that the student who read the text might also have that misconception, which helped internalization and caught attention. The rest of the text consisted of scientific knowledge involving statements aiming to correct and refute the misconception. An example of an English translation of the refutation text about heat and temperature is presented in Figure 1.

Figure 1.

Sample refutation text and its components.

Online Advance Organizer Concept Teaching Material (ONACOM)

Treatments in the Treatment-1 group were performed using the worksheet integrated into ONACOM, the “Matter and Change” PowerPoint presentation, and refutation texts. ONACOM, which was integrated into the teaching methods, is a digital content organizer enhanced by both network and computer facilities. This online teaching material is different from the other examples in the literature because each user has their profile page, both teachers and students have different authorizations, and it has a semantic network structure and its database (Korur, Toker & Eryılmaz, 2016). ONACOM is also an online concept-mapping tool as it enables teachers to organize multimedia content freely and construct the concept map of the subject to be covered in the concept mapping editor (Korumaz, 2018; Korur, Toker & Eryılmaz 2016; Yilmaz & Korur, 2021).

Treatment and Control Group Instructions

The KReC-based ONACOM-RT method was applied to the Treatment-1 group. Before the treatment started, the teachers in the Treatment-1 group were informed by the researcher about using the ONACOM. One of the researchers, who participated in lessons during the process alongside the classroom teacher, acted as merely a guide and kept details related to the process of application of the method by filling in a method application form. The ONACOM-RT method was applied to one class in each of the two selected schools for 3 weeks by dividing one class hour into four stages. In the first stage of the method, the related page of the ONACOM map was displayed (see Figure 2). The concepts in the related concept map were elaborated. In the second stage, the teacher made explanations related to the content and introduced the main concepts of the subject from the map. The refutation texts prepared to eliminate the misconceptions found in the topics of that week were used in proper steps. At this stage, the teacher presented some of the digital content that was assigned to the concepts and students analyzed them. This stage essentially reflected the coactivation principle of the KReC framework since it was the time that new information came into contact with the misconception.

Figure 2.

Sample ONACOM map prepared for matter and change unit.

In the third stage, using the relevant digital content from ONACOM, the subject matter was explained through examples, and the students were asked to provide further examples. During lecturing, the teacher offered more explanations related to the subjects by using digital content (such as pictures, and videos) on ONACOM. This process corresponds to the integration principle in the KReC framework. In the fourth stage, students were given a little extra time to review the content they wanted. At this stage, the teacher also asked the students to re-read the refutation texts, aiming to ensure that correct scientific knowledge had been established and misconceptions were eliminated. Since the activation of the newly encoded true scientific knowledge has increased considerably and since the objective was to reduce or eliminate the activation of false information, this stage mainly reflected the KReC framework’s competing activation principle. At the end of the stage, students solved the ONACOM assessment questions. Students were told that they could also use ONACOM from their homes by logging on to their profile pages.

In the Treatment-2 group (RT) the whole process was applied in the same manner except for ONACOM. Nine refutation texts, related teaching materials, and digital contents were classified according to lessons in 3 weeks; and in every lesson, the KReC framework principles were taken into account carefully. At the end of the lessons, the students solved questions from a worksheet, the assessment questions were reviewed one by one, and the teacher clarified any.

In the control group (TMC) teachers covered the subjects by using methods and techniques they preferred concerning the learning outcomes of the Matter and Change unit given in the science curriculum. In the TMC group, the application also lasted 3 weeks. The teachers also used educational videos or digital content such as animations from an online learning environment (e.g., www.morpakampus.com) in several lessons. Moreover, the teachers also prepared worksheets for the students about the subject every week and made the students take various notes using the science course book and notebook during the lecture. On the other hand, neither refutation texts nor refutation explanations were used during lecturing in the TMC group. One of the researchers also participated in control group lessons, filling in a method application form to keep details about the process of application of the method, and usage of various materials.

Data Analysis

Data cleaning and missing data analysis were initially conducted. Outliers affecting the normality of the distribution were identified using Mahalanobis distance values. Two participants from the control group (participants 170 and 188) had Mahalanobis values with p < .05, classifying them as extreme outliers, and were removed from further analyses. MANCOVA assumptions, which included those of the paired-sample t-test, were assessed. Skewness and kurtosis values for all variables fell within the acceptable range (−1.5 to +1.5), indicating normal data distribution (Field, 2009). The maximum observed skewness was 1.052, and the maximum kurtosis was −.998, further confirming normality.

Regression homogeneity, variance/covariance homogeneity, linearity, sphericity, and independence of observations in MANCOVA were all tested, with no significant violations observed. The interaction term Method * MS-PRE * CUS-PRE yielded p = .156 for MS-POST and p = .760 for CUS-POST, confirming the homogeneity of the regression assumption. Additionally, the similarity in group sizes (Treatments-1, 2/control < 1.5) supported regression homogeneity. The equality of variance and covariance matrices was verified using Box’s M-test (M = 24.097, F[6, 851702.7] = 3.953, p = .001). Box’s M-test, which assesses the homogeneity of covariance matrices across groups, is highly sensitive, particularly when group sizes exceed 30 participants. As such, a significance threshold of .001 was deemed acceptable (Hair et al., 2006). Levene’s test was used to evaluate the homogeneity of variance, revealing a non-homogeneous distribution of variances for both dependent variables (MS-POST, p = .003; CUS-POST, p = .019). In instances where Levene’s test indicates variance differences, it is appropriate to examine the variance ratio for the dependent variables. For sample groups ranging from 30 to 60 participants, variances are considered homogeneous if the ratio of the largest to smallest variance falls between 2 and 3 (Field, 2009). The variance ratio for MS-POST was 2.74 (2.917/1.063) and for CUS-POST, it was 1.15 (7.323/6.378), both under the critical value of 3.

All analyses were conducted using SPSS (IBM SPSS, 2013) and MS Excel (Microsoft Corporation, 2013). This study, or its analysis plan, was not preregistered. Due to data-sharing restrictions imposed by the Ministry of National Education’s General Directorate of Legal Services, supporting data is unavailable but can be provided upon reasonable request.

Results

The between-group analysis with MANCOVA and its post hoc analysis indicated the possible effects of the methods on students’ MS and CUS scores when the pretest results were controlled. Within-group analysis showed notable improvements in CUS scores and reductions in misconception scores for the three groups. The findings also included several robust misconceptions in the Matter and Change unit.

Results of the Between-Group Analysis

The MANCOVA analysis was performed to test the first hypothesis of the study. The MANCOVA indicated that when pretest scores were controlled, there was a significant difference between the effects of the treatments in the groups on collective dependent variables (F[4, 364] = 20.404, p < .005, Wilks’Λ = .667, partial η² = .183). In summary, the methods had significant treatment effects on MS-POST (F[2, 183] = 11.854, p < .001, partial η² = .115) and with large effect sizes on the CUS_POST (F[2, 183] = 41.701, p < .001, partial η² = .313) when the MS-PRE and CUS-PRE were controlled. The methods explained 18.3% of the variance of the dependent variables. To evaluate the post hoc tests in terms of adjusted alpha, the alpha constituent needs to be recalculated using the alpha adjustment approach. Since there were two dependent variables in this study, the alpha should be reduced from .050 to .025 to maintain the Type I error rate for the joint null hypothesis at the pre-specified α joint of .050 (Rubin, 2021). There were still statistically significant effects of the methods on the two dependent variables for the adjusted alpha level. One of the reasons for this significant difference might be that students not only realized their misconceptions but also learned the concepts through a certain systematic knowledge revision framework with the help of refutation texts and refutation texts integrated online teaching material (Butterfuss & Kendeou, 2021; Hunsu et al., 2023; Kendeou et al., 2017).

According to the results of MS-POST in Table 3, there were no significant differences between the mean misconception scores of the ONACOM-RT and RT. However, the mean MS-POST of the students in the control group (TMC) was significantly higher than that of the students in both the ONACOM-RT and RT groups separately. It can be said that the TMC method applied to the control group was not as effective as the methods applied to the treatment groups in eliminating misconceptions of students. The ONACOM-RT group students had significantly higher CUS-POST mean scores than the students in the other two groups. The mean CUS-POST of the students in the RT group was significantly higher than that of the students in the TMC group. The advanced graphical analysis in Figure 3a shows that the sharp decrease in the mean misconception scores might indicate that in the ONACOM-RT and RT groups’ misconceptions were mostly eliminated during the process. However, the moderate decrease in the scores of the TMC group might indicate that the elimination of the misconceptions in this group was quite limited. According to the advanced graphical analysis of conceptual understanding scores in Figure 3b, the highest increase in mean scores was in the ONACOM-RT group. Therefore, the ONACOM-RT was the most effective method among the three groups in reducing students’ misconceptions while increasing their conceptual understanding.

Table 3.

Pairwise Comparisons.

Dependent variable	(I)	(J)	Mean variance (I-J)	Std. error	p	95% confidence interval for difference
Dependent variable	(I)	(J)	Mean variance (I-J)	Std. error	p	Lower	Upper
MS-POST	TMC	ONACOM-RT	1.051*	.228	.000	.501	1.601
	TMC	RT	.831*	.228	.000	.283	1.380
	ONACOM-RT	RT	−0.220*	.229	.712	−.772	.333
CUS-POST	TMC	ONACOM-RT	−3.735*	.409	.000	−4.720	−2.749
	TMC	RT	−1.833*	.408	.000	−2.816	−.851
	ONACOM-RT	RT	1.901*	.411	.000	−.911	2.891

Note. N = 188.

p < .025.

Figure 3.

(a) MS advanced graphical analysis and (b) CUS advanced graphical analysis.

Results of the Within-Group Analysis and the Robust Misconceptions

The conceptual understanding scores and misconception scores were analyzed within the groups to analyze the second to seventh hypotheses of the study. At the beginning, the students’ mean CUS scores (CUS-PRE; $\bar{X}$ _{ONACOM_RT} = 2.77, $\bar{X}$ _RT = 2.57, $\bar{X}$ _TMC = 2.63) and MS scores (PRE-MS; $\bar{X}$ _{ONACOM_RT} = 2.18, $\bar{X}$ _RT = 1.78, $\bar{X}$ _TMC = 2.03) were close to each other for all three groups. The paired sample t-test related to the data obtained from the students in the Treatment-1 group was performed to test the second null hypothesis. A statistically significant difference was found between CUS-PRE ( $\bar{X}$ = 2.77; SD = 1.68) and CUS-POST ( $\bar{X}$ = 7.24; SD = 2.18) with a large effect size (t[61] = −14.295, p < .001; Cohen’s d = 2.30), then the null hypothesis was rejected. At the end of the lecturing performed with the ONACOM-RT method, the conceptual understanding scores of the students in the Matter and Change unit increased by 32% (4.47 out of 14).

When the misconception scores in the same group were investigated with the paired sample t-test to test the third null hypothesis, a statistically significant difference was detected between MS-PRE ( $\bar{X}$ = 2.18; SD = 1.71) and MS-POST ( $\bar{X}$ = .95; SD = 1.03) with a large effect size (t[61] = 6.058, p < .05; Cohen’s d = .871). Also, there was a noticeable decrease (approximately 9%; 1.23 decrease out of 14) in the students’ misconception scores. It was determined that the ONACOM-RT method considerably increased students’ conceptual understanding and eliminated their misconceptions about the Matter and Change unit. The misconceptions of the ONACOM-RT were analyzed in terms of item scores for three tiers. There were three different scores before (PRE) and after (POST) lecturing respectively: MBI-1, MBI-2, and MBI-3.

In the ONACOM-RT group, five out of six misconceptions were eliminated, the misconception percentage was reduced, but not eliminated for one misconception, and no misconceptions were detected in three of them (see Figure 4). An example of a misconception that was eliminated is “Heat can flow in all directions continuously” (see Table 1). After the ONACOM-RT treatment, 26.9% of the students answered the items corresponding to this misconception incorrectly in the first tier. This rate dropped to 8.1% after the ONACOM-RT treatment. Since the POST-MBI-3 value is below 10%, it can be said more properly and consistently that the detected misconception has been eliminated. On the other hand, the misconception that “Temperature varies depending on the size of the object” was not eliminated and it could be named “robust.”

Figure 4.

Misconceptions of the ONACOM-RT group for the tiers before and after the treatment.

The data of the students in the RT group was used to test the fourth hypothesis. The paired sample t-test indicated that statistically significant difference with a large effect size between CUS-PRE ( $\bar{X}$ = 2.57; SD = 1.65) and CUS-POST ( $\bar{X}$ = 5.22; SD = 2.66) scores (t[62] = −7.739, p < .05; Cohen’s d = 1.20). After the treatment through the RT, the conceptual understanding scores of the students increased by 19% (2.65 out of 14). Concerning the fifth hypothesis, the paired sample t-test performed for the misconception scores in this group indicated a statistically significant difference with a medium effect size between MS-PRE ( $\bar{X}$ = 1.78; SD = 1.42) and MS-POST ( $\bar{X}$ = 1.04; SD = 1.22) scores (t[62] = 3.176, p = .002; Cohen’s d = .56). After the treatment through the RT, the students’ misconception scores decreased by 5% (.74 out of 14). The misconceptions of the RT group were also analyzed in terms of item scores for three tiers to identify robust misconceptions.

It was observed that two of the six misconceptions were eliminated, and the percentage of misconceptions was reduced in four of them (see Figure 5). After the treatment applied in the RT group, the robust misconceptions that could not be eliminated were “Temperature varies depending on the size of the object,”“Heat can flow in all directions continuously,”“The material of the object affects the temperature,” and “Gases are not matter, because most gases are invisible, and they do not have mass.”

Figure 5.

Misconceptions of the RT group according to the tiers before and after the treatment.

In the control group (TMC), a statistically significant difference with a medium effect size was found between CUS-PRE ( $\bar{X}$ = 2.63; SD = 1.94) and CUS-POST ( $\bar{X}$ = 3.42; SD = 2.52) scores (t[62] = −3.160, p = .002; Cohen’s d = .351), indicating the sixth hypothesis was rejected. At the end of the lecturing through the TMC method, the conceptual understanding scores of the students increased by 6% (.79 out of 14). When the misconception scores were examined, no statistically significant difference was detected between the MS-PRE ( $\bar{X}$ = 2.03; SD = 1.69) and MS-POST ( $\bar{X}$ = 1.95; SD = 1.71) scores (t[62] = .361, p = .719); so, the seventh hypothesis was accepted. The mean post-misconception score of the students only decreased by 0.6% (.08 out of 14). At the end of the lectures using the TMC method, the misconception percentages of the students did not considerably decrease (see Figure 6).

Figure 6.

Misconceptions of the TMC group according to the tiers before and after the treatment.

One of the five misconceptions was eliminated, the misconception percentage of four misconceptions decreased, and the percentage of one misconception increased, there was a possible misconception, namely “Even if we perceive properties of matter such as color and smell, if the matter is not visible, it does not exist in the environment” in TMC. In this direction, it was revealed that it would be a challenging process to eliminate misconceptions with a teaching method that was not planned with regards to a systematic approach to eliminating misconceptions, and it was also possible to create misconceptions. The robust misconceptions in this group were, “Heat and temperature are the same,”“Temperature varies depending on the size of the object,”“Heat can flow in all directions continuously,” and “The material of the object affects the temperature.”

Discussion

This study explored the effectiveness of the ONACOM-RT, RT, and TMC groups in addressing misconceptions of fourth graders and improving their conceptual understanding, particularly regarding the Matter and Change unit. In this study, the ONACOM-RT group showed significant improvement in conceptual understanding, moreover, the RT group played a key role in addressing misconceptions. Additionally, this study identified the robust misconceptions across the groups and tracked their improvement in conceptual understanding.

The methods applied in treatment groups were not superior to each other in eliminating misconceptions. One of the reasons might be related to the structure of the digital contents assigned to the ONACOM were not directly focused on eliminating misconceptions, except for the refutation texts. The digital contents in ONACOM did not support the further elimination of misconceptions since they were individually more focused on teaching or supporting learning than on addressing misconceptions. It seems that the principles of the KReC framework such as coactivation, integration, and competing activation (see Kendeou et al., 2014) were not fully realized for the digital contents in the ONACOM. They most probably even did not dominate a conflict between prior knowledge and new information (Akerson & Abd-El-Khalick, 2004; Chi, 2008; Özdemir, 2013; Shen & Confrey, 2007). Therefore, it seems that the digital contents in the ONACOM did not force students to revise their existing schemata and they might be happy with what they know. We expected better performance of ONACOM-RT in eliminating misconceptions compared to the group given only RT. Butterfuss (2020) tested the main hypotheses derived from KReC-MD in two experiments, demonstrating that text structure, source credibility, and intertextual integration on knowledge revision, with readers who integrated intertextual information achieving superior outcomes of knowledge revision. The fact that refutation texts alone might be effective tools in eliminating misconceptions, but should not be seen as the only remedy for misconceptions, supports the idea that they can be used in different ways depending on the context (Schroeder & Kucera, 2022; Zengilowski et al., 2021). In this regard, Chi (2008) argued that the concepts that refutation texts aim to change should be presented multiple times in different formats, Tippett (2010) stated that refutation texts should be supported by activities such as videos or demonstrations, and Danielson et al. (2016) suggested that integration of a chart or some visuals to the refutation texts. Schroeder and Kucera (2022) stated that refutation texts including images did not add value as expected compared to the refutation texts without an image but they offered more research is needed to understand the effects on learning. Similarly, we may also offer to analyze the effects of refutation texts integrated into multimedia, which contains digital content focused on misconceptions, for future researchers.

On the other hand, the methods applied to the treatment groups were highly effective in eliminating students’ misconceptions compared to the method (TMC) that was emphasized in the curriculum (TMNE, 2018). This may be because the curriculum or science course books do not emphasize misconceptions about all concepts and do not make any suggestions. Another reason might be that the refutation texts in treatment groups were prepared in line with the KReC framework, which offered several principles to organize refutation texts. Eliminating misconceptions is not a process to be carried out by chance. As Pintrich et al. (1993) emphasized, knowledge revision should not begin with simply telling students, “Your knowledge is wrong; I will tell you the truth,” as this approach disregards the complexities of learners’ existing beliefs involved in conceptual change. Regarding the KReC framework, the knowledge encoded and stored in long-term memory cannot simply be changed (encoding), and revising the knowledge is a process that is independent of the learner (passive activation; Butterfuss & Kendeou, 2021; Kendeou et al., 2014). Like the application of Weingartner & Masnick (2019), in our refutation texts, the misconception and refutation cue with causal relations were given at the beginning of the text, then the currently accepted scientific knowledge with explanations from daily life were provided, and at the end of the text, further explanations were given to make correct information dominate. Another reason may be that teachers may not prefer the manipulation of misconceptions, because of some limitations such as time. The results of related studies of science learning were similar to the results of this study in a way that they emphasized the positive effects of refutation texts on knowledge revision and on eliminating misconceptions that students possess (Broughton et al., 2010; Gürkan & Korur, 2021; Korur, Enil, & Göçer, 2016; Lombardi et al., 2016).

In the middle school science curriculum (TMNE, 2018), the matter and change unit is one of the units with the highest number of misconceptions (Sözbilir, 2003). Therefore, MCT in this study was developed only for the level of primary students to identify their misconceptions. The MCT is an effective tool used to measure the misconceptions and conceptual understanding of fourth-grade students. According to the findings, fourth-grade students had strong misconceptions and a lack of knowledge in the unit of matter and change. As Zengilowski et al. (2021) indicated, it was very difficult to be sure that incorrect beliefs resulted from prior knowledge or guessing. To overcome this difficulty to some extent, we represented a level of certainty question in the last tier of MCT. However, since the students were young, we still cannot be sure how realistically they marked these certainty statements.

The KReC framework (Kendeou & O’Brien, 2014) underpinned the interventions used in the ONACOM-RT and RT groups to improve conceptual understanding. Conceptual understanding is a complex process that evolves over time and is reinforced through systematic repetition and revisiting of key concepts (Wild et al., 2013). The systematic treatments, particularly in the ONACOM-RT group, significantly enhanced students’ understanding of the Matter and Change topic compared to the RT and TMC groups. This can be attributed to the structures of both the ONACOM and the RT. In terms of ONACOM, developed according to Mayer’s (2017) Multimedia Learning Principles, it featured (i) the use of text combined with digital content (images, animations, videos), which supported dual coding and extended the processing time in short-term memory, (ii) content designed to align with students’ cognitive load, promoting meaningful learning, and (iii) an interactive learning environment that allowed students to efficiently integrate knowledge. These factors contributed to students’ learning, as supported by previous studies (Korur, Toker & Eryılmaz, 2016; Moreno & Mayer, 2007; Yılmaz & Korur, 2021). Another contributing factor was the structure of the refutation texts, which first presented the misconception, followed by a refutation cue, and then the correct scientific knowledge. This structure helped reduce confusion between lack of knowledge and misunderstanding (Korur, Enil & Göçer, 2016), a finding supported by McCrudden and Kendeou (2014), who highlighted similar refutation text structures in improving understanding of abstract scientific concepts and Guzzetti et al. (1993) who emphasized that refutation texts are effective in promoting conceptual change by directly addressing and correcting misconceptions.

In this study, conceptual understanding scores increased in all three groups. The ONACOM-RT group showed the highest increase (32%), followed by the RT group (19%), while the TMC group had only a 6% increase. These results suggest that the interventions, especially ONACOM-RT, can significantly support the conceptual understanding of 4th graders. Extending the intervention period could lead to more holistic development in conceptual understanding levels. Over 3 weeks, the three to five-fold increase in conceptual understanding in the RT and ONACOM-RT groups compared to the control group can be attributed to several reasons. One reason may be that increased correct knowledge suppressed misconceptions raised student awareness, and facilitated the acquisition of scientific knowledge and conceptual understanding. This aligns with Kendeou and O’Brien’s (2014) finding that conceptual understanding improves when misconceptions are co-activated with scientific truths. Additionally, Diakidoy et al. (2003) and Mason et al. (2008) also highlight the positive impact of refutation texts on primary students’ conceptual understanding. Another reason may be that ONACOM likely supports students in developing opinions and enhances their understanding and use of new scientific knowledge. Through KReC-MD, ONACOM effectively promotes knowledge integration with its diverse digital multimedia content. It encourages self-regulated learning, helping students strengthen their conceptual understanding without external support (Mefferd & Bernacki, 2023). In related studies, such short-term interventions using online materials like ONACOM also improve students’ academic achievement (Korur, Toker & Eryılmaz, 2016; Yılmaz & Korur, 2021).

In the present study, the misconceptions were mostly eliminated in the ONACOM-RT and RT groups, but they could not be eliminated in the control group. In all three groups, the common robust misconception (before and after treatments) was “Temperature varies depending on the size of the object.” In terms of KReC principles (coactivation, integration, and competing activation), there was no difference between the refutation text prepared for this misconception and those prepared for other misconceptions. However, Chi (2005) indicated that students think of temperature as a measurement of the heated amount of some substance or concrete object. Therefore, to support competing activation, further explanations and examples might be given by considering students misconceiving the definition of temperature. The misconceptions about the emerging processes are robust, and changing them requires students to overcome their, perhaps, innate predispositions by gaining knowledge about the emergent kind (Chi, 2005). Heat and temperature misconceptions were among the robustly misconceived concepts mentioned here. Another robust misconception for RT and control group students was “Heat can flow in all directions continuously.” Heat flow is defined as an emergent process that might be a robust misconception. The students’ misconceptions can be described as a misrepresentation of the process as a substance (Chi, 2005). Misconceptions result from the activation of an inappropriate schema. While the use of an existing schema can lead to robust misconceptions, the use of an emergent schema helps the student to learn and understand the emergent processes that include possible robust misconceptions (Chi et al., 2012). Therefore, it will be important to accurately determine that they have such a schema to apply the necessary treatment. The results of some studies support the findings of this study by showing the existence of robust misconceptions even after applications of well-planned interventions related to heat and temperature (Leinonen, et al., 2013; Prince et al., 2013).

Limitations and Future Research

The present study has several limitations. First, while results indicate improved conceptual understanding and reduced misconceptions in the ONACOM-RT and RT groups, these findings occurred immediately post-intervention. Future research should assess long-term effects to address concerns about the sustainability of refutation texts (Prinz et al., 2022; Will et al., 2019; Zengilowski et al., 2021). Secondly, all interventions were conducted by student’s own teachers in different classes, which could introduce variability. However, one of the authors of this study observed the process to ensure fidelity to the interventions. Third, the ONACOM online tool allowed students to create profiles and access refutation texts outside class; however their usage was not tracked. While we viewed this as an opportunity for enhanced learning (Yılmaz & Korur, 2021), future studies should measure how often students engage with such tools. Fourth, a total of nine refutation texts were used at different times, which may have influenced recall and targeted knowledge revision due to the varying time intervals between the texts and the posttest. Fifth, students could not indicate confidence in their answers at each tier, potentially leading to inflated scores (Kaltakci Gurel et al., 2015). This limitation existed in all three-tier tests. Lastly, the study’s 3-week duration may not have been sufficient for younger students to achieve deeper conceptual understanding. Longer exposure to these tools could lead to a more comprehensive evaluation of their effects, such as a greater reduction in misconceptions.

Future studies should explore the impact of extended intervention periods to determine students’ understanding and retention of concepts. This would offer valuable insights into the time needed to achieve more robust learning outcomes. Future studies could also investigate how an innovative online tool (such as ONACOM) can be designed to address and eliminate misconceptions in educational settings, particularly for primary school students. Finally, educators should be mindful of outlined principles (such as co-activation, integration, and competing activation) of the KReC when developing teaching materials to better support students in overcoming robust misconceptions.

Conclusion

This study’s findings suggest that using the KReC framework for developing refutation texts effectively facilitates knowledge revision, highlighting the importance of adopting evidence-based frameworks for addressing misconceptions. As a theoretical implication, the results of the study reinforce existing literature on the effectiveness of refutation texts alone or the texts integrated into an online teaching material, such as ONACOM, in revising knowledge and eliminating misconceptions, advocating for broader implementation of such texts in science education. There were also a couple of practical implications: (i) Teachers using refutation texts with online tools by leading KReC-MD principles, can reference this study as a guide to addressing key factors in correcting 4th graders’ misconceptions and improving their understanding of the Matter and Change unit. The statistically significant differences observed between the groups in this study likely indicate practical significance when considered alongside the effect size and power values of this study; (ii) Despite the effectiveness of interventions, the study highlights the persistence of certain robust misconceptions. This implies that while some misconceptions can be addressed, others may require more intensive and tailored instructional approaches; (iii) The Misconception Assessment Tool (MCT) proves effective for identifying misconceptions and gauging conceptual understanding among fourth graders, emphasizing the importance of appropriate assessment tools for younger students; and (iv) The high prevalence of misconceptions in the matter and change unit underscores the need for targeted interventions in primary education to address these misconceptions early.

Footnotes

Author Note

This study utilized a portion of the data from the master’s thesis written by the second author under the supervision of the first author at the Institute of Educational Sciences, Burdur Mehmet Akif Ersoy University.

ORCID iDs

Fikret Korur

Kevser Korumaz

Dilek Erduran Avci

Ethical Considerations

This study was performed in line with the principles and permission of the Provincial Directorate for National Education of Antalya, Türkiye (#22025226). All participants in the study were volunteers.

Consent to Participate

Informed consent was obtained from the participants and their legal guardians, as required for studies involving minors.

Author Contributions

Fikret Korur: The project leader of the project for which ONACOM was developed, supervision, conceptualization, methodology, analysis, and manuscript writing and editing, and funding acquisition. Kevser Korumaz: Literature review, methodology, data collection, analysis, interpretation of results, manuscript writing. Dilek Erduran Avci: conceptualization, literature review, critical revision of the manuscript, manuscript writing, manuscript editing, contact with the journal, and funding acquisition.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We would like to thank TÜBİTAK for supporting the project carried out by the first author, during which the Online Advance Organizer Concept Teaching Material was developed under Grant No. (113K319).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The participants of this study did not give written consent for their data to be shared publicly, so due to the sensitive nature of the research, supporting data is not available.

References

Akerson

V. L.

Abd-El-Khalick

(2004). Learning as conceptual change: Factors mediating the development of preservice elementary teachers’ views of nature of science. Science Teacher Education, 88, 785–809. https://doi.org/10.1002/sce.10143

Asberger

Thomm

Bauer

(2021). On predictors of misconceptions about educational topics: A case of topic specificity. PLoS One, 16(12), Article e0259878. https://doi.org/10.1371/journal.pone.0259878

Aydın Ceran

Ateş

(2020). Conceptual understanding levels of students with different cognitive styles: An evaluation in terms of different measurement techniques. Eurasian Journal of Educational Research, 20(88), 149–178.

Bensley

D. A.

Lilienfeld

S. O.

(2017). Psychological misconceptions: Recent scientific advances and unresolved issues. Current Directions in Psychological Science, 26(4), 377–382. https://doi.org/10.1177/0963721417699026

Broughton

S. H.

Sinatra

G. M.

Reynolds

R. E.

(2010). The nature of the refutation text effect: An investigation of attention allocation. The Journal of Educational Research, 103(6), 407–423. https://doi.org/10.1080/00220670903383101

Butterfuss

(2020). Revising misconceptions using multiple documents. The University of Minnesota Digital Conservancy.

Butterfuss

Kendeou

(2021). KReC-MD: Knowledge revision with multiple documents. Educational Psychology Review, 33, 1475–1497. https://doi.org/10.1007/s10648-021-09603-y

Chi

M. T. H.

(2005). Commonsense conceptions of emergent processes: Why some misconceptions are robust. Journal of the Learning Sciences, 14(2), 161–199. https://doi.org/10.1207/s15327809jls1402_1

Chi

M. T. H.

(2008). Three types of conceptual change: Belief revision, mental model transformation, and categorical shift. In Vosniadou

(Ed.), Handbook of research on conceptual change (pp. 61–82). Erlbaum.

10.

Chi

M. T. H.

Roscoe

R. D.

Slotta

J. D.

Roy

Chase

C. C.

(2012). Misconceived causal explanations for emergent processes. Cognitive Science, 36(1), 1–61. https://doi.org/10.1111/j.1551-6709.2011.01207.x

11.

Crocker

Algina

(2008). Introduction to classical and modern test theory. Cengage Learning.

12.

Danielson

R. W.

Sinatra

G. M.

Kendeou

(2016). Augmenting the refutation text effect with analogies and graphics. Discourse Processes, 53(5–6), 392–414. https://doi.org/10.1080/0163853X.2016.1166334

13.

Dellantonio

Pastore

(2021). Ignorance, misconceptions, and critical thinking. Synthese, 198(8), 7473–7501. https://doi.org/10.1007/s11229-019-02529-7

14.

Diakidoy

I.-A. N.

Kendeou

Ioannides

(2003). Reading about energy: The effects of text structure in science learning and conceptual change. Contemporary Educational Psychology, 28(3), 335–356. https://doi.org/10.1016/S0361-476X(02)00039-5

15.

diSessa

A. A.

(1993). Towards an epistemology of physics. Cognitive and Instruction, 10(2–3), 105–225. https://doi.org/10.1080/07370008.1985.9649008

16.

Ebel

R. L.

(1954). Procedures for the analysis of classroom tests. Educational and Psychological Measurement, 14, 352–364.

17.

Field

(2009). Discovering statistics using SPSS (3rd ed.). Sage Publications Ltd.

18.

Flemming

Kimmerle

Cress

Sinatra

G. M.

(2020). Research is tentative, but that’s okay: Overcoming misconceptions about scientific tentativeness through refutation texts. Discourse Processes, 57(1), 17–35. https://doi.org/10.1080/0163853X.2019.1629805

19.

Fraenkel

J. R.

Wallen

N. E.

Hyun

H. H.

(2012). How to design and evaluate research in education (8th ed.). McGraw Hill.

20.

Guzzetti

B. J.

Snyder

T. E.

Glass

G. V.

Gamas

W. S.

(1993). Promoting conceptual change in science: A comparative meta-analysis of instructional interventions from reading education and science education. Reading Research Quarterly, 28(2), 116–159.

21.

Gürkan

Ş.

Korur

(2019, May). Çürütme metni ve dijital hikaye destekli kavramsal değişim yaklaşımlarının kavram yanılgılarının giderilmesine etkisi: Yer kabuğu ve dünya’mızın hareketleri ünitesi [The effect of refutation text and digital story-supported global change relations on eliminating misconceptions: Earth’s crust and Movements of our Earth unit] [Paper presentation]. 14. Ulusal Fen Bilimleri ve Matematik Eğitimi Kongresi, Burdur Mehmet Akif Ersoy Üniversitesi, Burdur, Türkiye.

22.

Hair

Black

Babin

Anderson

Tatham

(2006). Multivariate data analysis (6th ed.). Pearson Prentice Hall.

23.

Harsch

R. M.

Kendeou

(2023). Learning from refutation texts about scientific topics with analogical and causal explanations. Contemporary Educational Psychology, 73, Article 102172. https://doi.org/10.1016/j.cedpsych.2023.102172

24.

Hestenes

Halloun

(1995). Interpreting the force concept inventory. The Physics Teacher, 33, 502–506.

25.

Hunsu

N. J.

Adesope

McCrudden

M. T.

(2023). The effects of text structure on students’ use of comprehension strategies and cognitive outcomes during science text processing. Frontiers Education, 8, Article 1112804. https://doi.org/10.3389/feduc.2023.1112804

26.

Hynd

(2001). Refutational texts and the change process. International Journal of Educational Research, 35(7–8), 699–714. https://doi.org/10.1016/S0883-0355(02)00010-1

27.

Hynd

Alvermann

Qian

(1997). Preservice elementary school teachers’ conceptual change about projectile motion: Refutation text, demonstration, affective factors and relevance. Science Education, 81, 1–27. https://doi.org/10.1002/(SICI)1098-237X(199701)81:1<1::AID-SCE1>3.0.CO;2-M

28.

IBM SPSS. (2013). IBM SPSS Statistics for Windows [Version 22.0]. IBM Corp.

29.

Johnson

Butterfuss

Kim

Orcutt

Harsch

Kendeou

(2022). The ‘Fauci Effect’: Reducing COVID-19 misconceptions and vaccine hesitancy using an authentic multimodal intervention. Contemporary Educational Psychology, 70, Article 102084.

30.

Kaltakci Gurel

Eryilmaz

McDermott

L. C.

(2015). A review and comparison of diagnostic instruments to identify students’ misconceptions in science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008. https://doi.org/10.12973/eurasia.2015.1369a

31.

Kanli

(2014). A study on identifying the misconceptions of pre-service and in-service teachers about basic astronomy concepts. Eurasia Journal of Mathematics, Science & Technology Education, 10(5), 471–479. https://doi.org/10.12973/eurasia.2014.1120a

32.

Kendeou

(2024). A theory of knowledge revision: The development of the KReC framework. Educational Psychology Review, 36(2), Article 44. https://doi.org/10.1007/s10648-024-09885-y

33.

Kendeou

Braasch

J. L.

Bråten

(2016). Optimizing conditions for learning: Situating refutations in epistemic cognition. The Journal of Experimental Education, 84(2), 245–263. https://doi.org/10.1080/00220973.2015.1027806

34.

Kendeou

Butterfuss

Van Boekel

O’Brien

E. J.

(2017). Integrating relational reasoning and knowledge revision during reading. Educational Psychology Review, 29, 27–39. https://doi.org/10.1007/s10648-016-9381-3

35.

Kendeou

O’Brien

E. J.

(2014). Knowledge revision: Processes and mechanisms. In Rapp

D. N.

Braasch

J. L. G.

(Eds.), Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences (pp. 353–378). MIT Press.

36.

Kendeou

Walsh

E. K.

Smith

E. R.

O’Brien

E. J.

(2014). Knowledge revision processes in refutation texts. Discourse Processes, 51(5–6), 374–397. https://doi.org/10.1080/0163853X.2014.913961

37.

Kim

Kendeou

(2021). Knowledge transfer in the context of refutation texts. Contemporary Educational Psychology, 67, Article 102002. https://doi.org/10.1016/j.cedpsych.2021.102002

38.

Kirk

R. E.

(2013). Experimental design: Procedures for the behavioral sciences (4th ed.). Sage Publications.

39.

Korumaz

(2018). The effects of different combined methods on the conceptual understanding and the elimination of misconceptions of fourth-grade students [Master’s thesis, Burdur Mehmet Akif Ersoy University]. Council of Higher Education Thesis Center Repository. https://tez.yok.gov.tr/UlusalTezMerkezi/tezSorguSonucYeni.jsp

40.

Korur

(2015). Exploring seventh-grade students’ and pre-Service science teachers’ misconceptions in astronomical concepts. Eurasia Journal of Mathematics, Science & Technology Education, 11(5), 1041–1060. https://doi.org/10.12973/eurasia.2015.1373a

41.

Korur

Enil

Göçer

(2016). Effects of two combined methods on the teaching of basic astronomy concepts. The Journal of Educational Research, 109(2), 205–217.

42.

Korur

Toker

Eryılmaz

(2016). Effects of the integrated online advance organizer teaching materials on students’ science achievement and attitude. Journal of Science Education and Technology, 25(4), 628–640, https://doi.org/10.1007/s10956-016-9618-4

43.

Leinonen

Asikainen

M. A.

Hirvonen

P. E.

(2013). Overcoming students’ misconceptions concerning thermal physics with the aid of hints and peer interaction during a lecture course. Physical Review Special Topics - Physics Education Research, 9(2), 1–22. https://doi.org/10.1103/PhysRevSTPER.9.020112

44.

Lombardi

Nussbaum

E. M.

Sinatra

G. M.

(2016). Plausibility judgments in conceptual change and epistemic cognition. Educational Psychologist, 51(1), 35–56. https://doi.org/10.1080/00461520.2015.1113134

45.

Mason

Gava

Boldrin

(2008). On warm conceptual change: The interplay of text, epistemological beliefs, and topic interest. Journal of Educational Psychology, 100, 291–309.

46.

Mayer

R. E.

(2017). Using multimedia for e-learning. Journal of Computer Assisted Learning, 33(5), 403–423. https://doi.org/10.1111/jcal.12197

47.

McCrudden

M. T.

Kendeou

(2014). Exploring the link between cognitive processes and learning from refutational text. Journal of Research in Reading, 37, S116–S140. https://doi.org/10.1111/j.1467-9817.2011.01527.x

48.

Mefferd

K. C.

Bernacki

M. L.

(2023). Tracing undergraduate science learners’ digital cognitive strategy use and relation to performance. Journal of Science Education and Technology, 32, 837–857. https://doi.org/10.1007/s10956-022-10018-9

49.

Microsoft Corporation. (2013). Microsoft Excel. https://office.microsoft.com/excel

50.

Moreno

Mayer

R. E.

(2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326.

51.

Özdemir

O. F.

(2013). Transfer and conceptual change: The change process from the theoretical perspectives of coordination classes and phenomenological primitives. Instructional Science, 41, 81–103. https://doi.org/10.1007/s11251-012-9219-4

52.

Özdemir

Clark

D. B.

(2007). An overview of conceptual change theories. Eurasia Journal of Mathematics, Science & Technology Education, 3(4), 351–361. https://doi.org/10.12973/ejmste/75414

53.

Pintrich

P. R.

Marx

R. W.

Boyle

R. A.

(1993). Beyond cold conceptual change: The role of motivational beliefs and classroom contextual factors in the process of conceptual change. Review of Educational Research, 63(2), 167–199. https://doi.org/10.3102/0034654306300216

54.

Posner

Strike

Hewson

Gertzog

(1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66, 211–227. https://doi.org/10.1002/sce.3730660207

55.

Prince

M. J.

Vigeant

M. A.

Nottis

K. E. K.

(2013, June). Assessment and repair of critical misconceptions in engineering heat transfer and thermodynamics [Paper presentation]. ASEE Annual Conference & Exposition, Atlanta, GA, USA.

56.

Prinz

Kollmer

Flick

Renkl

Eitel

(2022). Refuting student teachers’ misconceptions about multimedia learning. Instructional Science, 50(1), 89–110. https://doi.org/10.1007/s11251-021-09568-z

57.

Rubin

(2021). When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing. Synthese, 199, 10969–11000. https://doi.org/10.1007/s11229-021-03276-4

58.

Sarı Ay

Ö.

Aydoğdu

(2015). The effect of conceptual change texts in the misconceptions identified removal in the unit of states of matter and heat. Hacettepe University Journal of Education, 30(2), 99–111.

59.

Schroeder

N. L.

Kucera

A. C.

(2022). Refutation text facilitates learning: A meta-analysis of between-subjects experiments. Educational Psychology Review, 34(2), 957–987. https://doi.org/10.1007/s10648-021-09656-z

60.

Shen

Confrey

(2007). From conceptual change to transformative modeling: A case study of an elementary teacher in learning astronomy. Science Education, 91, 948–966. https://doi.org/10.1002/sce.20224

61.

Sözbilir

(2003). A review of selected literature on students’ misconceptions of heat and temperature. Boğaziçi University Journal of Education, 20(1), 25–41.

62.

The Turkish Ministry of National Education (TMNE). (2018). Fen bilimleri dersi öğretim programı [Science course curriculum]. T.C. Milli Eğitim Bakanlığı. https://mufredat.meb.gov.tr/ProgramDetay.aspx?PID=325

63.

Tippett

C. D.

(2010). Refutation text in science education: A review of two decades of research. International Journal of Science and Mathematics Education, 8, 951–970. https://doi.org/10.1007/s10763-010-9203-x

64.

Torsney

B. M.

Korstange

Symonds

J. E.

(2021). Changing misconceptions of abilities: The impact of a brief mindset refutation text intervention. Psychology in the Schools, 58(12), 2293–2312. https://doi.org/10.1002/pits.22592

65.

Türksoy

(2019). The effect of teaching methods integrated with augmented reality and online materials on achievement and retention in science lesson: Mixed design [Doctoral dissertation, Burdur Mehmet Akif Ersoy University]. Council of Higher Education Thesis Center Repository. https://tez.yok.gov.tr/UlusalTezMerkezi/tezDetay.jsp?id=TetojXi2BHO4RCX-UbyQwg&no=gYi-4AsWH109nZD4y50a2A

66.

van den Broek

Kendeou

(2008). Cognitive processes in comprehension of science texts: The role of co-activation in confronting misconceptions. Applied Cognitive Psychology, 22, 335–351. https://doi.org/10.1002/acp.1418

67.

Vosniadou

(2019). The development of students’ understanding of science. Frontiers Education, 4, Article 32. https://doi.org/10.3389/feduc.2019.00032

68.

Weingartner

K. M.

Masnick

A. M.

(2019). Refutation texts: Implying the refutation of a scientific misconception can facilitate knowledge revision. Contemporary Educational Psychology, 58, 138–148. https://doi.org/10.1016/j.cedpsych.2019.03.004

69.

Wild

T. A.

Hilson

M. P.

Hobson

S. M.

(2013). The conceptual understanding of sound by students with visual impairments. Journal of Visual Impairment & Blindness, 107(2), 107–116. https://doi.org/10.1177/0145482X1310700204

70.

Will

K. K.

Masad

Vlach

H. A.

Kendeou

(2019). The effects of refutation texts on generating explanations. Learning and Individual Differences, 69, 108–115. https://doi.org/10.1016/j.lindif.2018.12.002

71.

Yilmaz

Korur

(2021). The effects of an online teaching material integrated methods on students’ science achievement, attitude and retention. International Journal of Technology in Education, 4(1), 22–45. https://doi.org/10.46328/ijte.79

72.

Zengilowski

Schuetze

B. A.

Nash

B. L.

Schallert

D. L.

(2021). A critical review of the refutation text literature: Methodological confounds, theoretical problems, and possible solutions. Educational Psychologist, 56(3), 175–195. https://doi.org/10.1080/00461520.2020.1861948

Enhancing Students Conceptual Understanding: An Interventional Study on the Knowledge Revision Components Framework Using Online Materials

Abstract

Keywords

Introduction

Background

Theory of Conceptual Change and Knowledge Revision

Refutation Text and Knowledge Revision

Refutation Text Interventions in the Literature

The Online Advance Organizer Concept Teaching Material (ONACOM)

The Present Study

Method

Design and Participants

Variables and Score Types

Matter and Change Test Three-Tier Conceptual Understanding Student Score (Max. 14)

Matter and Change Test Three-Tier Misconception Student Score (Max. 14)

One-Tier Misconception Based on Item Percentage Score (MBI-1)

Two-Tier Misconception Based on Item Percentage Score (MBI-2)

Three-Tier Misconception Based on Item Percentage Score (MBI-3)

False Negative Percentage Score (FN; 0-1-1)

False Positive Percentage Score (FP; 1-0-1)

Data Collection Instrument

Matter and Change Three Tier Test (MCT)

Teaching Materials

Refutation Texts

Online Advance Organizer Concept Teaching Material (ONACOM)

Treatment and Control Group Instructions

Data Analysis

Results

Results of the Between-Group Analysis

Results of the Within-Group Analysis and the Robust Misconceptions

Discussion

Limitations and Future Research

Conclusion

Footnotes

Author Note

ORCID iDs

Ethical Considerations

Consent to Participate

Author Contributions

Funding

Declaration of Conflicting Interests

Data Availability Statement

References