Sage Journals: Discover world-class research

Abstract

As a dynamic and multifaceted construct, computational thinking (CT) has proven to be challenging to conceptualize and assess, which impedes the development of a workable ontology framework. To address this issue, the current article describes a novel approach towards understanding the ontological aspects of CT by using text mining and graph-theoretic techniques to elucidate teachers’ perspectives collected in an online survey (N = 105). In particular, a hierarchical cluster analysis, a knowledge representation method, was applied to identify sub-groups in CT conceptualization and assessment amongst teachers. Five clusters in conceptualization and two clusters in assessment were identified; several relevant and distinct themes were also extracted. The results suggested that teachers attributed CT as a competence domain, relevant in the problem- solving context, as well as applicable and transferrable to various disciplines. The results also shed light on the importance of using multiple approaches to assess the diversity of CT. Overall, the findings collectively contributed to a comprehensive and multi-perspective representation of CT that refine both theory and practice. The methodology employed in this article has suggested a minor but significant step towards addressing the quintessential questions of “what is CT?” and “how is it evidenced?”.

Keywords

computational thinking computational thinking assessment ontology text mining teachers’ perspectives

As the enthusiasm in promoting computational thinking (CT) grows, there is a surge of research seeking to investigate its role in computing activities and impact on students’ outcomes. Despite this, however, the fundamental understanding regarding its conceptualization and appropriate assessment approaches remain equivocal in the current literature (Rich & Hodges, 2017). This is indeed a challenging issue as CT is a complex and multifaceted construct that encompasses diverse skills and abilities, useful in multiple domains and contexts. Hence, a consensus for an agreeable definition is yet to be fulfilled. Nonetheless, numerous perspectives have been proposed, with each prioritizing different aspects and contexts; all of which can be subscribed to two larger domain views. A domain-specific view considers CT to be relevant to contexts related to programming and computing whereas a domain-general view regards it as a set of cross-disciplinary thinking and problem-solving skills (Israel-Fishelson et al., 2021; Lai, 2019). While some equate CT with programming and computing-centric skills (i.e. a domain-specific perspective), others consider CT as a general cognitive and problem-solving skill for diverse contexts (i.e. a domain-general perspective).

The ambiguity in conceptualization often translates into obscurity in assessment methods as the latter is the manifestation of the former. Indeed, there are multiple methods to measure different aspects of CT, depending on the specific learning goals, outcomes, and contexts. Some consider key to measure CT components such as abstraction, problem-decomposition, and debugging alongside coding proficiency (e.g., Moreno-León et al., 2016; Román-González et al., 2018; Weintrop & Wilensky, 2017), others take attitudes, engagement, sense of belonging, and creativity into account (e.g., Mesiti et al., 2019; Pinkard et al., 2020; Tissenbaum et al., 2018). Thus, given the state of the current literature, there is yet a comprehensive account in the ontology of CT that guides appropriate assessment design. Emerging from this, therefore, are two foundational and fundamental questions: what is CT and how is it evidenced? Do teachers perceive CT as programming skills, technology-dependent skills, or competency beyond these domains? What are the mechanisms that underpin CT and what are the core components that should be measured in a CT assessment? These perspectives from teachers can inform and further elucidate the nature of CT. In this regard, the aim of this article is to identify the domain view and associated themes that represent key ontological aspects of CT from teachers’ diverse viewpoints.

Teachers’ Ontological Views of CT

An ontological approach, as a knowledge representation method, formulates and represents a construct by identifying components it entails, relations between the components, as well as properties and contexts that co- exist and interrelate (Stancin et al., 2020). In other words, it is a formal and clear specification of a shared conceptualization for a construct, helping to directly address “what is CT?” (Gruber, 1993). According to Guarino (1995), the aim of an ontology is to build a model by highlighting the essential components/concepts of a construct in both a human-understandable manner and a machine-readable format. In the context of CT, an ontology approach can be used to define and refine the common understanding regarding the nature of CT so as to reach a clear conceptualization and to better inform assessment design. That is, to bridge theory and practice.

The views of teachers are crucial in the development of an ontology of CT as ontologies are socially constructed artefacts that represent a collective understanding and knowledge amongst domain experts (Fensel, 2001). This is particularly so considering the enactment perspective is slowly replacing the fidelity perspective in curriculum implementation (Van den Akker, 2010). There is increasing emphasis on the role of teachers in making curricula change to better support their students’ learning and classroom culture. This contrasts to the latter perspective in which teachers follow the “prescriptions'' from other stakeholders, sources, or parties (Van den Akker, 2010). Therefore, their shared understanding and assumptions of CT influence pedagogy, curriculum design, and assessment methods—all of which underpin, and therefore, help inform the ontology of CT. To approach such a framework, the semantics of how teachers consider CT, as well as the contexts and situations in which they perceive CT to emerge from are critical. Although such a framework is yet to be developed in a comprehensive manner, previous works have taken initial steps in investigating how teachers conceptualize CT, which could inform, identify, and validate the ontology that underlies it (Fürst et al., 2003).

Research, albeit scarce, has been conducted to gain insights into how teachers operationalize CT as a construct. For example, linking their research to teachers’ operationalization of CT, Yadav et al. (2018) used open-ended questions to explore American pre-service teachers’ attitudes towards CT. They found that teachers considered CT as a cognitive tool to solve problems with or without the use of technology. In general, the authors extracted four major themes regarding CT: problem-solving/logic, algorithms, use of technology, and critical thinking.

These results are similar to Bower and Falkner (2015) and Garvin et al. (2019). Bower and Falkner (2015) reveal that Australian pre-service teachers considered “problem-solving using technology”, “using technology”, “logical thinking” to be important to the conceptualization of CT; Garvin et al. (2019) demonstrate “problem- solving” was the most frequent description of CT amongst American teachers. Similarly, Fessakis and Prantsoudi (2019) used an open-ended question to investigate how Greek teachers perceived CT. From the responses, the authors discovered four main categories: problem-solving method (73.52%), equating CT with a CT dimension (7.37%), an epistemological method (8.09%), and unclear answers (9.56%). Although a small percentage of teachers were ambiguous about the concept of CT, the majority agreed that CT is a problem-solving method that includes algorithmic solution, a way of thinking about problems with or without a computer, solving problems like a computer scientist, and logical problem-solving. Moreover, the majority of the teachers perceived CT as a concept beyond computer science because it helps solve problems in other disciplines and does not necessitate the use of a computer.

The common understanding regarding CT appears to be generalizable and does not limit to only computer science teachers yet they could evolve. For example, when studying experienced American STEM (science, mathematics, engineering, and technology) and non-STEM in-service teachers, Sands et al. (2018) did not find contrasting responses between them. On the contrary, they reveal that regardless of the subject area, all teachers confidently considered CT to involve logical thinking (100%) and problem-solving (99%).

Despite sharing common views, teachers’ perspectives are not static but evolve as they gain experience and knowledge. To explicate, Good et al. (2017) compared teachers’ responses before and after their exposure to a CT framework. Through in-depth interviews, the authors found “programming” to be the most frequent term at the pre- framework stage, however, it was replaced by “problem-solving” at the post-framework stage. This suggests a switch in teachers’ perspectives as they gain knowledge and enhance their conceptual understanding of CT. Taken these changes along with all the aforementioned findings together, teachers’ views can be broadly categorized under the domain-specific or domain-general view. Indeed, the results in Good et al. (2017) suggest that teachers who are more knowledgeable about CT tend to subscribe to the latter view and perceive CT to be applicable in diverse problem types, beyond technology or programming. This is particularly the case with teachers who partook in interventions—from initially equating CT with computing they eventually began to see the potential of CT being utilized in various ways (Yadav et al., 2018).

Previous studies have highlighted a general understanding of CT that appears unified between teachers (regardless of subject, training, and country) albeit there is space to build upon this body of work. For example, the context in which the mechanisms or properties of CT are understood or manifested in was not directly addressed yet context is essential to the interpretation of teachers’ views. This raises the necessity to first establish the conceptual scope; for example, by exploring teachers’ domain view. Furthermore, the methods employed previously did not allow the interpretation of teachers’ perspectives at a semantic and relational level although semantic proximity can enhance knowledge modeling (Fürst et al., 2003). Lastly, there has been little focus on translating findings in conceptual understanding to assessment design though assessment has been highlighted as a pressing issue in the literature (Mueller et al., 2017; Rich & Hodges, 2017). The current article aims to build upon this body of work and address these potential gaps with a text mining approach.

The Current Study

Given that teachers seem to have a general common understanding of CT, between-group difference in how teachers conceptualize CT should be minor. That is, teachers should arguably subscribe to a single shared domain view. However, beyond the between-group difference is the within-group difference that shed light on the distinct themes in teachers’ perceptions. Do teachers who subscribe to a common domain view also have a shared understanding of the mechanisms that underpin CT—the set of components, skills, and processes that CT encompasses? Relatedly, are these mechanisms also considered essential in CT assessment and can they inform appropriate assessment approaches? These questions, which emphasize on components as well as their relations, are the basis to understanding the ontology that underlies CT.

In addressing these timely issues, this study aims to contribute in both theoretical, methodological, and practical ways. Theoretically, it attempts to take an initial step towards examining the ontological aspects of CT. In so doing, it identifies different sub-groups amongst teachers; key and meaningful themes are extracted from these groups to inform a comprehensive and multi-perspective understanding of CT. Methodologically, previous studies have mainly relied on interviews to investigate teachers’ perspectives. This study employs a novel approach by using text mining techniques to explicate teachers’ responses while aligning to Guarino (1995) suggestions regarding ontologies—in a “human-understandable” and “machine-readable” manner. Lastly, the findings in this study seek to facilitate the development of a useful ontology framework that guides curriculum and assessment design in the future. By investigating these issues, the following research questions are explored in this article:
What are the mechanisms that underpin CT?

Do teachers share the same domain view regarding CT?

Are there different sub-groups and themes in the conceptualization of CT?

Are there different sub-groups and themes in the assessment of CT?

Methods

Sample

The teacher survey was promoted and circulated in multiple platforms, including the Computing At School teachers’ forum, Research Bytes of the National Centre for Computing Education, and other computer science teacher social media groups. A total of 105 active computer science teachers from the United Kingdom (N = 95; 90.48%) and Ireland (N = 10; 9.52%) participated in the survey. The teachers participated in this study work in diverse geographic parts: London (N = 30; 28.57%); Cambridgeshire (N = 14; 13%; 13.33%), Birmingham (N = 7; 6.67%); Leeds (N = 6; 5.71%); Norwich (N = 5; 4.76%); Sheffield (N = 3; 2.86%); Liverpool (N = 2; 1.90%); with some non-responses (N = 28; 26.67%). Of all the participants, 97 participants completed the full questionnaire, resulted in a 92% response rate. The majority of the teachers have a strong computer science background with 90.29% having had at least some programming experience. Of this group of teachers, most of them have had more than 10 years of programming experience (N = 56; 60.87%), some have at least 5 years (N = 34; 20.64%), others have at least 2 years (N = 15; 16.3%). Only 2 participants reported having none to only 1 year of programming experience.

The Teacher Survey

Computer science teachers participated in the teacher survey online in Qualtrics. The survey consists of a consent form, basic demographic information, two multiple-answer questions, and two open-ended free text questions—all regarding CT operationalization and assessment. In particular, the open-ended questions were designed to allow teachers to express their views as freely as they wish without the constraints of options or space. Moreover, it provided an opportunity for them to elaborate on the multiple-choice questions that are relevant to the ontology of CT. Research has suggested that this type of question has the potential to increase response rate (O’Cathain & Thomas, 2004). As this is a mixed item survey, the data includes both structured and semi-structured text. Table 1 explains the questions in detail along with corresponding research questions and analytical techniques.

Table 1.
A Summary of the Survey, Research Questions and Analytical Techniques.

Format Question and responses Research question Analytical techniques

Multiple-Answer Which of the following do you personally think underlie computational thinking? (1) Descriptive statistics

You can choose as many answers as appropriate to you.

o Algorithm o Planning o Criticism o Abstraction

o Collaboration o Design o Logic o Debugging

o Generalization o Organization o Problem representation

o Problem decomposition o Other [ ]

Multiple-Answer Do you think the above skills associated with computational thinking are context-dependent such that they only occur in a coding/ programming environment? (2) Descriptive statistics

• Yes, it is context-dependent. They only occur in a coding/ programming environment.

• No, it is context-neural. They do not just occur in a coding/programming environment but in different contexts where problems need to be solved.

• I am not sure.

Open- Ended In your view, what does computational thinking, as a thinking skill or a set of cognitive processes, mean and entail? (3) Text mining

Open-Ended In your view, what are some important cognitive skills, processes or strategies to be measured in a computational thinking assessment? (4) Text mining

Analytical Techniques

To address research question (1) and (2), descriptive statistics are used as they are informative and handle structured data well. This step helps establish the general view of CT and key CT components. To address research question (3) and (4), a text mining approach is adopted to alleviate two major issues with textual data: the challenge and time in analyzing texts as well as subjectivity and potential bias in interpretation. Moreover, through text mining techniques, useful information in teachers’ responses can be unearthed and hidden insights can be discovered. There are several merits in this approach. For example, the challenge of finding patterns in a great amount of unstructured information can be solved by text mining (Gaikwad et al., 2014). Moreover, text mining is an exploratory data-driven approach that does not assume any pre-established taxonomies or presupposed hypotheses (Yu et al., 2011). This allows openness in investigating teachers’ insights without potential influence from researchers’ preconceptions. Lastly, data/text-mining is an appropriate method for addressing ontological questions (Chang et al., 2020).

Figure 1 displays a flowchart that explains the text mining procedures of this study. Prior to model building, textual data is processed and summarized. Cluster estimation, using the elbow and silhouette methods, is then applied to explore the optimal number of clusters in conceptualization and assessment. This is particularly useful in unearthing the patterns of teachers’ views so as to identify themes that are theoretically and practically meaningful in the next stage. Following this, an unsupervised classification using the agglomerative hierarchical clustering algorithm is employed on the pre-established number of clusters. The algorithm constructs a tree of clusters, or a nested sequence of clusters, known as a dendrogram (Cobo et al., 2012; Stanimirova & Daszykowski, 2018). The dendrogram visually displays relationships at multiple levels of granularity, hence its application is useful for taxonomy building (Ma & Dhavala, 2018). The agglomerative algorithm, a bottom-up approach, starts with every response or data point as a cluster on its own and recursively merges more clusters based on their similarity until an optimal number is reached. That is, until all teachers’ responses eventually get categorized into a group. The similarity of each cluster is measured using the sum of squared Euclidean distance, one of the most common distance matrices (Ward, 1963). Response groups that share a high degree of similarity in their conceptual understanding and assessment are then identified; distinct themes that emerge from these groups are further explored. If the teachers share a common understanding, then one single cluster could explain the variance amongst their views. Otherwise, multiple clusters should be identified to suggest some variations amongst the teachers who belong to different sub-groups.

Figure 1.
A Flow Chart of the Text Mining Procedures.

Data Processing

Using the text mining R package (tm; Meyer et al., 2008), the text content of the dataset was parsed. Data processing included the removal of stop words and word stemming. First, stop words were removed to increase the efficiency and accuracy of the analysis. Stop words are frequent words that present no information such as punctuation, conjunctions, and disjunctions, amongst others. Next, word stemming was applied using Porter’s suffix-stripping algorithm to reduce inflected or derived words (Porter, 1980). This was used with the assumption that different morphological variations of words with the same stem are thematically similar, and themes were the concern of this study (Huang, 2008). Lastly, a document-term matrix was generated, containing only keywords that have high-level information.

Based on the summary of the matrix, there was a high level of sparsity for both semi-structured variables. Hence, a weighting function was applied to term frequency (TF) and inverse document frequency (IDF) to compute tf- idf (Salton & Buckley, 1988). The use of tf-idf entails two premises: 1) the higher the term frequency, the more important a term is and; 2) those terms that are in shorter responses are more significant. Therefore, using the tf- idf helped avoid issues related to the biases in term frequency for terms that are in shorter or longer responses (Hiemstra, 2000). Using tf-idf, only words that were distinctively frequent, and therefore relevant, were considered. The weighting function is defined in the following:

TF-IDF (term, document) = TF (term, document)–× IDF(term)

Results

What Underpins Computational Thinking?

To understand the mechanisms that underpin CT, the following question was asked: “which of the following do you personally think underlie computational thinking? You can choose as many answers as appropriate to you.” As shown in Figure 2, the components identified by more than half of the teachers (>50%) were: problem decomposition (N = 89; 90%), algorithms (N = 81; 82%), patterns/generalization (N = 79; 80%), logic (N = 77; 78%), abstraction (N = 76; 77%), problem representation (N = 60; 61%), debugging/evaluation (N = 58; 59%), and planning (N = 53; 54%). Some teachers (N = 10; 10%) selected the “other” option and elaborated on diverse aspects which could be summarized as: psychological aspect (i.e. resilience, creativity); application (i.e. linking knowledge to application, real-world problems, using heuristics); data (i.e. data structure and data analysis) and; computer science and engineering elements (i.e. system, artefact representation and evaluation, code building).

Figure 2.
A Bar Graph of Key CT Components.

Teachers’ Domain View

Amongst all the teachers, 91.51% (N = 97) agreed that the important CT processes/skills identified above are not context-dependent. This group of teachers viewed those skills as: “they do not just occur in a coding/programming environment but in different contexts where problems need to be solved”. In contrast, a small number of teachers (N = 6; 5.66%) perceived those skills as context-dependent skills in which they “only occur in a coding/programming environment.” Several teachers (N = 3; 2.83%) were uncertain regarding their position in this question. The results suggested that there was a general common view shared by most teachers: the domain-general/context-neutral view. The teachers who subscribed to this account of CT viewed CT skills applicable to diverse contexts and not solely limited to programming/coding.

Computational Thinking: Conceptualization

Text Summarization

Term frequency analysis, word association and word clouds were conducted to summarize the text. Excluding the term “computational”, the most frequent terms were “problem-solve” (N = 196), “think” (N = 69), “use” (N = 32), “solution” (N = 29), “computer” (N = 24), “break” (N = 22), “process” (N = 22), and “ability” (N = 22). Pairwise association between the word “computational” and other tf-idf terms were also explored, set for a correlation limit of .5. It was found that “computational” was associated with “mathematician” (.55). Indeed, several teachers believed CT is analogous to mathematical thinking and constructive mathematics. Word clouds were then generated using both term frequency and tf-idf, displayed in Figure 3. In the tf-idf cloud, when considering only terms that were distinctively important, not only was “problem-solve” key but so were “step”, “ability”, and “process”. The bar graph in Figure 3 elaborates on the tf-idf frequency of these terms; the higher the tf-idf frequency values the more relevant the terms are.

Figure 3.
Term Frequency World Cloud, TF-IDF Word Cloud and Bar Graph of CT Conceptualization.

A semantic network, a method of knowledge representation, was constructed based on cosine similarity to provide additional contextual and relational information of and between the terms. Rather than visually representing individual key terms in a word cloud, the network illustrates the semantic relations between them, helping to establish the structure and taxonomies of CT that are important to its ontology. Figure 4 displays the network graph of CT conceptualization. The core of the network can be identified as “problem” and “solve”; hence, CT can be fundamentally attributed to problem-solving. Surrounded by them are interconnected key terms—terms that help understand the scope of problem-solving in the context of CT. For example, “pattern”, “abstraction”, “decomposition” are all close to “use”, “create”, and “develop”, which as a semantic field, are close to “solution” and “process” right underneath “computational”. Together, the relations suggested that CT is the application and development of these core skills and processes for solving problems. On the right-hand side of “problem” are “real”, “world”, “experience”, which are close to “design” and “understand”. This suggested the understanding of CT concepts is underpinned by designing artefacts related to real world problems, scenarios, and experiences. Linking these relations together then, CT is essentially a property of skills and processes for practical purposes in real-world contexts: to develop, use and create.

Figure 4.
A Semantic Network of CT Conceptualization.

Sub-Groups and Themes in Conceptualization

The elbow method was used to validate the optimal number of clusters (k) that represent the sub-groups of teachers in CT conceptualization. The intra-cluster distance in the elbow method estimates within-group homogeneity and variability; when plotting the total intra-cluster variation against the clusters, the “elbow” on the graph would indicate the most optimal cut-off point (Bholowalia & Kumar, 2014). As displayed in Figure 5, there was an obvious inflection point at the 5^th cluster, as such it can be determined that k = 5. That is, 5 sub- groups best represented the diverse conceptualizations amongst teachers.

Figure 5.
Optimal Number of Clusters for CT Conceptualization.

A hierarchical clustering was applied with the agglomerative algorithm using k = 5, as established previously. The results are visually displayed in the dendrogram in Figure 6. The resultant clusters are seen to be different in size and density; some of them are larger and denser while others are smaller and sparser. As shown in the dendrogram, 2 larger sub-groups and 3 smaller sub-groups were identified. The height of the links indicate the distance (i.e. dissimilarity) between pairs of clusters under each link (Cobo et al., 2012). Based on the height, the two bigger sub-groups had views that were closest in similarity while the smaller sub-groups had distinct views and unique themes that deviated most from the majority of the teachers. The three smaller sub-groups could be considered outliers, however as they carry important and unique information for the purpose of this study, they were treated as individual clusters.

Figure 6.
A Dendrogram Produced Using Ward’s Minimum Variance, Representing Clusters of CT Conceptualization.

Next, between-group and with-in group differences were explored. Inspecting individual responses based on the clusters, between-group difference was minor and only applied to one case. One teacher who was unsure about the domain view in research question (2) formed an individual cluster (5^th) by conceptualizing CT from a computer science curriculum perspective. When using domain-view as the x-axis of the dendrogram, Figure 7 suggests that there were not any differences between teachers who held different domain views. For example, out of the five teachers (highlighted in the dendrogram) who subscribed to the domain-specific/context-specific view, the way they conceptualized CT aligned with other teachers from the two bigger sub-groups: two belonged to the 1^st cluster and three belonged to the 2^nd cluster. Hence, regardless of the domain view, the perspectives of the majority of the teachers can still be categorized and grouped into the five sub-groups explained in Table 2.

Figure 7.
A Dendrogram Representing Clusters of CT Conceptualization with Domain View.

Table 2.
Five Clusters of CT Conceptualization.

Clusters and themes N Domain-view Raw responses

1. Problem- solving 52 (53.60%) Mix “Breaking down a problem and figuring it out in steps.” (40) “It's a process or path to solve a problem or set if problems.” (95) “The process of identifying a problem and the logical steps and data sets required to address the problem / provide a solution.” (83) “The ability to identify via abstraction those parts of a problem which can be automated and applying logical constructs to create an unambiguous set of instructions which can be used to solve the problem.” (31)

2. A transferrable cross-disciplinary skills fundamental to computing principles 41 (42.26%) Mix “Computational thinking, has, in my experience, a need to be able to recognise and understand real world problems and be able to decompose them. Once a problem has been understood then a series of steps can are both applicable to the real world but also consider the additional steps necessary to enable the problem to be solved by computational means. Whilst some of these skills transfer to other areas of problem solving, programming has its own unique skill-set. This 'analysis phase', is complete separate to the design (both in terms of graphic design and algorithm design) and is acknowledged as such in most SDLCs.” (77) “I think it's both thinking processes as well as practices - and mostly, a way to help non-computer science folks feel welcome to a field they often find mysterious and foreign. Looking for an exact list or a definition has never been important to me - I see computational thinking more as a scaffold to help introduce ideas of computer science + the benefit of being able to use the same processes and practices in other areas of life.” (58) “Computational thinking is a title for the context of computer science, but it contains the skills and abilities for all problem solving, in a diverse range of applications.” (61) “A transferrable skill developed through the teaching of computing that can equip children to thrive in an ever-changing technological environment.” (48)

3. Solution processes 1 (.01%) Domain-general; Context-neutral “Broadly, paying greater attention to the process by which solutions may be generated (as opposed to simply focusing on the solutions themselves) and gaining insight from the generalisations made in these processes that allow problems to be seen as being analogous to one another at some level of abstraction.” (16)

4. Programming-centric skills that require the use of computer 1 (.01%) Uncertain “In my personal view, the concept of computational thinking is good but a secondary concept. Really the students and young people should learn how to program/code and the concepts arise out of that. The emphasis on 'computational thinking' is somewhat misplaced, in my humble opinion. It is somewhat like the emphasis on the theory of music or the theory of playing tennis. The tennis player has to learn how to play tennis well if he/she wants to win a tournament or be able to do something constructive in tennis. Andy Murray or any other sports champion did not sit down and think 'let me think of sports…thinking', they got on with the practical aspects of the sports, practised day and night to achieve their goals. Similarly, the concept of computational thinking revolves around the computer - the computer gives rise to the concept and it is fundamental and crucial that people are acquainted with the computer in a practical manner in order to appreciate the concept of 'computational thinking' - which is essentially a secondary concept and we do not society favours by focusing so much on this.” (20)

5. Computer Science curriculum relevant 2 (.02%) Domain-general; Context-neutral “I see ct as an educational construct devised to support teachers and learners articulate different aspects of a computing curriculum and a curriculum in general. It is a subset of "thinking skills" and stands longside considerations such as thinking hats, problem solving strategies, cognitive strategies (metaphors, etc). ct helps articulate the value of computer programming in that we are presenting real-world, complex, scenarios and enabling, through physical and mental processes, making them computabe (or at least closer to being computable) and in so doing. making the complexity more understandable.” (60) “Algorithmic thinking is significant part of computational thinking. When I meet with students who are seeking to take my AP CS course without the required prerequisites I demonstrate the binomial theorem and ask them to generalize it. While this is a higher level maths concept, recognizing patterns and being able to generalize them is ONE key to Comp Thinking. Another general problem solving, outside of a strictly technical context. Given a scenario being able to 1) identify the problem 2) identify known information about the problem 3) articulate a strategy to solve the problem 4) check results of strategy. Debate and logic are also examples of computational thinking, structured identification of and response to facts. Review of situations, the ability to decompose.” (103)

Computational Thinking: Assessment

Text Summarization

The most frequent terms considered important to CT assessment were (excluding the term “computational”): “problem-solve” (N = 118), “ability” (N = 42), “solution” (N = 39), “think” (N = 35), “use” (N = 27), “pattern” (N = 22), “understand” (N = 22), and “algorithm” (N = 22). Pairwise association between the word “computational” and other terms were also explored, set for a correlation limit of .5. Using tf-idf, “computational” was found to be strongly associated with both “compute” (.61) and “progression” (.61). Term frequency and tf-idf word clouds were also generated, displayed in Figure 8. Again, results in the term frequency word cloud are expected to align with the term frequency reported above. When inspecting the tf-idf cloud, however, more relevant terms emerged, such as “decomposition”, “logic”, and “abstraction”.

Figure 8.
Term Frequency World Cloud, TF-IDF Word Cloud and Bar Graph of CT Assessment.

Figure 9 presents the semantic network in which “problem”, “solution”, and “ability” were in the center, forming three semantic fields. Compared to the key terms in the conceptualization network, the emphasis in assessment are the end goal of a “solution” and demonstration of “ability”— both surrounding “problem.” In the semantic field of “ability” are a set of common components of CT: “algorithms”, “pattern”, recognition”, “decomposition,” and “abstraction.”

Figure 9.
A Semantic Network of CT Assessment.

Whereas some of these components were considered alongside “use” and “develop” in the conceptualization network, they can be deemed as ability to demonstrate problem-solving in the assessment network.

Sub-Groups and Themes in Assessment

Figure 10 displays the results of the elbow method. The inflected point was not obvious in the graph although k = 2 could be reasonable. Due to the ambiguity, the silhouette method (Kaufman and Rousseeuw, 1990) was used in addition to further validate the number of k. As shown, the silhouette coefficient also peaked at k = 2 and so it can be determined that two clusters performed best at explaining the variations in teachers’ responses.

Figure 10.
The Elbow Method and the Silhouette Method.

Next, hierarchical clustering was applied with the agglomerative algorithm using k = 2. The results are demonstrated in the dendrogram in Figure 11. Overall, teachers’ views regarding assessment were organized into two large sub-groups (see Table 3). The height of the 2^nd cluster was relatively higher than the 1^st, indicating larger distances or variations amongst responses in the cluster. A further inspection into the 2^nd cluster suggested that three teachers’ responses (11, 77, and 87) located further away from the rest of the cluster, based on their height. These teachers had all given specific suggestions concerning diverse ways in assessing CT, surrounding mathematics, data, design, coding, and other cognitive skills.

Figure 11.
A Dendrogram Produced Using Ward’s Minimum Variance, Representing Clusters of CT Assessment.

Table 3.
Two Clusters of CT Assessment.

Final clusters (themes) N Domain-view Raw responses

1. Logical problem solving strategies 64 (65.97%) Mix “Understanding the logical steps to solve a problem” (69) “Abstraction. Decomposition. Logical thinking. Algorithmic thinking.” (30) “Look at the problem and work out a proposed solution using logic trying an solution and if it does not work not to give up. Try many solutions to achieve the best outcome.” (97) “High level of reasoning, being able to think beyond a concrete operational level.” (59) “Mental flexibility; Logical reasoning; Abstract reasoning; Problem-solving; Pattern recognition” (7)

2. Multiple approaches to assess the diversity of CT (both cognitive and non-cognitive) 33 (34.02%) Mix “The ability to look at a complex problem and make it much simpler by breaking it into manageable chunks. This can be done with many real-world scenarios that students can understand, before they get anywhere near a computer. By extension, computational thinking can be applied to everyday life. This can obviously be assessed in many ways, but simple, handwritten flowcharts and similar techniques are as good as anything.” (66) “Skills related to CT such as abstraction, algorithmic design, decomposition or pattern recognition should be based on a complementary set of measurement tools. These assessment tools should be able to measure different aspects (cognitive and noncognitive aspects of learning CT) of the skills to be evaluated. Innovation here could be the combination of several complementary measurement tools rather than the use of a single measuring instrument.” (6) “The ability to see layers of abstraction. The ability to play with mental models. One could also assess the breadth and variety of mental models the student possesses.” (100) “The cognitive skills requires students to think about their thinking. Often students will jump to the solution, desperate to be right, and fail to realized the steps they have taken to get to that solution. This is difficult to teach using problems they already know how to solve, because they often fail to recognise how they get to a solution. When it is a new problem they can't solve they fail to solve it, because it is expect that the teacher will show them how to solve the problem first.” (61) Metacognition - is the learner aware of what they are doing and how?” (90)

As illustrated by Figure 12, there were a mix of domain views in both clusters. Amongst teachers who subscribed to the domain-specific/context-specific view (highlighted in the dendrogram), one belonged to the 1st cluster and four belonged to the 2^nd. Given the number of clusters extracted, there appears to be stronger alignment in CT assessment (k = 2) than CT conceptualization (k = 5).

Figure 12.
A Dendrogram Representing Clusters of CT Assessment with Domain View.

Discussion

The necessity for a comprehensive and multi-perspective account of CT is evident and has been highlighted by numerous researchers (Grover & Pea, 2017; Kong, 2016; Rich & Hodges, 2017; Shute et al., 2017). Underlined by this need are the fundamental questions of “what is CT” and “how is it evidenced”. Diving deep into these questions means examining the ontological aspects of CT, which requires a collective process in specifying its concepts, components, as well as their relations, so as to inform the building blocks that give rise to the construct.

This study has invited the voices of teachers in joining and contributing to this collective process. Adopting a text mining approach, this study elucidates the common understanding of CT as well as diverse themes that emerged from teachers’ perspectives. Together, these themes provide practical guidance in taking the initial step towards approaching the ontological representation of CT that is useful for both researchers and educators. In this regard, the current work is part of an overarching effort in investigating the nature of CT as well as in bridging theory and practice.

The Nature of CT: Key Components and Domain View

Computer scientists, computer science education researchers, and teachers all have their unique understanding and perspectives concerning the nature of CT that could be helpful in developing a workable ontology framework. Partly, to learn from each other’s perspectives but mostly, to establish common grounds.

Interestingly, the findings in this study have shown a strong alignment between teachers and views in the literature concerning the set of key components that CT encompasses. Other than the five common components proposed by and agreed amongst computer science education researchers (Angeli et al., 2016; Selby & Woollard, 2013; Voogt et al., 2015), at least half of the teachers also believed that logic, problem representation, and planning to be essential. These ideas also echo how some researchers consider CT, particularly in the context of solving complex real-world problems. For example, Wing (2010) contends that CT overlaps with logical thinking and Voskoglou & Buckley (2012) define logical thinking as an important process in dealing with complexities. Likewise, the National Council for Research (2010) highlights that CT involves logical reasoning to better understand artefacts, procedures, and systems. Overall, the selection of logic is also compatible with how teachers perceive CT in previous studies (see Bower & Falkner, 2015; Sands et al., 2018; Yadav et al., 2018). Compared to the literature, what is unique in this study is teachers’ selection of planning albeit it is not surprising. Planning, a high-order cognitive skill, is associated with and resembles metacognitive skills. In the context of problem-solving, planning is essential: it determines the steps of actions in order to achieve or devise a solution for a given problem. As such, it is a crucial problem-solving technique. When situating CT within problem-solving (which the majority of teachers do), planning is considered a pivotal element.

The overlapped and distinct ideas between teachers in this study and researchers may be underpinned by context. It is feasible that within the programming and computer science curriculum domain, the five common CT components (i.e. problem decomposition, algorithm, patterns/generalization, abstraction, and debugging/evaluation) are considered important. However, when teachers viewed CT in a domain-general manner in which the emphasis was on problem-solving context then logic, problem representation, and planning also become relevant. This sheds light to the usefulness of designing CT related activities and programs based on a specific context. In a non-computer science context, teachers may wish to include logic and higher-order cognitive skills such as planning, metacognition, and executive functions, to facilitate real-world problem-solving.

The domain view of CT has only recently been introduced to the literature albeit the underpinning ideas have been circulated around for some time (Lai, 2019). Using the domain view to categorize the general understanding of CT has proven to be useful in this study, as a direct and efficient way to classify teachers’ perspectives contextually. This is important because context, as aforementioned, determines the skills recognized and expressed as CT: what we consider as “CT” is contextual. The question is whether CT, as a set of skills, is only useful for coding/programming context or beyond. Answers to this question define the scope of CT. The findings suggest that the majority of the teachers in this study subscribed to the domain-general/context-neutral view, recognizing that the key components of CT are beyond the programming context. This has helped set the conceptual scope of their responses in this study. Overall, the view that CT is beyond programming is similar to the teachers’ opinions discussed in Fessakis and Prantsoudi (2019). However, as the classification of domain view relied solely on one question, future studies can design and elaborate on more appropriate ways to evaluate participants’ position in the domain view.

CT Conceptualization: The Five Sub-Groups and Themes

The study set out to investigate whether teachers who subscribed to different domain views would conceptualize CT differently. The results suggested that between-group difference was minimal, but some within-group variations were identified. Regarding the between-group difference, only one teacher who was unsure of the domain view position formed a unique cluster/theme. The theme of the cluster focused on programming-centric skills and the use of a computer. According to this teacher, CT is a “secondary concept” to learning how to code/program. In this regard, the teacher believed that CT is closely associated with the use of the computer, in which the concept of CT emerges. Hence, the teacher thought the emphasis need not to be on CT but rather on the interactions with the computer through coding/programming. Although unique in this sample, this view reflects important ideas of some researchers in the computer science education community, such as Brennan and Resnick’s (2012) framework. Despite being uncertain about the position in domain view, the opinions of this teacher seem to emphasize ideas that are important to the domain-specific view.

Regardless of domain view, the perspectives of the majority of the teachers were found to be explained by five clusters, with 2 larger sub-groups and 3 smaller sub-groups. The themes extracted from the two larger sub- groups were shared by 96% of the teachers: 53% perceived CT as a problem-solving process and 42% considered CT as a transferrable cross-disciplinary skill fundamental to computing principles. While the first theme surrounds ideas that are relatively common in the literature (i.e., problem-solving), the second theme provides some interesting insights. It was revealed that teachers considered CT as a set of cross-disciplinary skills that are transferrable. This idea also corresponds to the word association with mathematician. As observed in the dendrogram, the height of the 2^nd cluster is relatively taller, indicating more variations compared to the 1^st cluster. Hence, there were more diverse insights as to how CT skills could be transferred across disciplines and problem types. While some teachers emphasized real-world problems, others perceived it applicable in a wide range of areas that students need to face in the future. Nonetheless, all have agreed that CT is fundamentally underpinned by computing principles. Together, the ideas within the 2^nd cluster accentuate the dynamic nature of competence rather than skill per se: it is generic, cross-disciplinary, transferrable, and incorporates computing knowledge for real-world problems. These perspectives highlight CT, as a general competence, is underpinned by domain-specific knowledge. That is, a combination of domain-general skills and domain-specific knowledge. Interpreting this view alongside the semantic network reflects CT as a property of skills and processes for practical purposes in real-world contexts, facilitating students to use and create in problem-solving. CT, as a competence domain, is indeed an emerging view; in particular, the findings in this study further support the arguments made in Yadav et al. (2017) and Grover and Pea (2017).

CT Assessment: The Two Sub-Groups and Themes

Compared to conceptualization, there were more commonalities in assessment as only two clusters were identified. The majority of the perspectives were classified into two sub-groups: logic in problem-solving (65%) and multiple approaches to assess the diversity of CT (34%). Investigating whether there is alignment between conceptualization and assessment was a goal in this study and the findings suggest this alignment to be strong: the two clusters corresponded to the two larger sub-groups in conceptualization. The 1^st assessment cluster on logical problem-solving strategies focused on logical thinking and reasoning as strategies in solving problems. This group also emphasized on several core components of CT, such as abstraction and decomposition. The 2^nd cluster on multiple approaches to measure the diversity of CT emphasized the importance of using complex and real-world problems to assess the breadth of students’ understanding and skills. The teachers suggested these problems to be something that students can relate to and applied in their day-to-day lives, echoing Kalelioglu et al. (2016) suggestion of the use of real-life scenarios to assess CT.

Not only is the theme in the 2^nd cluster compatible with the idea that CT is a competence domain, but it also reflects the growing interest to move away from the use of a single measurement tool (see Tang et al., 2020). Indeed, the dynamic nature of competence necessitates the need to employ multiple assessment instruments to evaluate the diversity of CT. In the last decade, there are emerging assessment tools and frameworks proposing for this purpose. For example, Kong (2019) advises the use of a combination of multiple-choice questions, programming projects, and survey instruments to assess the full spectrum of CT. To date, there are several developments and CT assessment approaches serving the same goal: systems of assessment (Grover, 2015), a combination of assessment tools (Román-González et al., 2019), and the commutative assessment (Weintrop & Wilensky, 2017). Echoing these researchers, teachers in this group also believed this approach is essential in measuring the breadth and depth of CT, including both cognitive and non-cognitive aspects. Cognitively, metacognition, thinking about one’s own thinking, was highlighted to be relevant. Recalling that at least half of the teachers in this sample selected planning as a key CT component— the emphasis of metacognition recapitulates the role of higher-order thinking skills. Non-cognitively, components such as abstraction, algorithms, and problem-decomposition were suggested but teachers believed these skills have to be measured using multiple measurement tools. In a recent systematic review conducted by Tang et al. (2020), a detailed summary of current assessments that fall under cognitive and non-cognitive domain is given. It appears that there is a paucity of assessments that holistically assess CT; they either measure the cognitive or non-cognitive aspect but not both in a single assessment. Despite this, there is a small number of studies that have utilized multiple approaches by matching traditional knowledge tests with other tools. To continue the works established by these researchers, future assessments should adopt a competence-based approach by integrating diverse cognitive and non-cognitive aspects of CT in a single assessment.

Conclusion

In closing, this study was conducted to contribute to the overarching goal of examining the ontological aspects of CT. In this regard, a novel text mining approach was employed to elucidate teachers’ perspectives of CT structurally (breadth) as well as relationally at a semantic level (depth). The findings help shed light on the challenging yet quintessential questions of “what is CT?” and “how it is evidenced?”. Although teachers shared a general domain view in this study, the sub-groups and themes identified in both conceptualization and assessment demonstrate the dynamic and complex nature of CT. Overall, the findings are theoretically and practically relevant. Theoretically, the findings highlight the building blocks that give rise to the concept of CT as a competence domain as well as reinforce its attribution to problem-solving. Practically, the findings help inform the design of reliable and valid CT assessment—assessments that consider both cognitive and non-cognitive aspects using diverse measurement approaches. It is hoped that this study has taken the first step towards facilitating the development of an ontology framework of CT.

Format	Question and responses	Research question	Analytical techniques
Multiple-Answer	Which of the following do you personally think underlie computational thinking?	(1)	Descriptive statistics
	You can choose as many answers as appropriate to you.
	o Algorithm	o Planning	o Criticism	o Abstraction
	o Collaboration	o Design	o Logic	o Debugging
	o Generalization	o Organization	o Problem representation
	o Problem decomposition		o Other [ ]
Multiple-Answer	Do you think the above skills associated with computational thinking are context-dependent such that they only occur in a coding/ programming environment?	(2)	Descriptive statistics
	• Yes, it is context-dependent. They only occur in a coding/ programming environment.
	• No, it is context-neural. They do not just occur in a coding/programming environment but in different contexts where problems need to be solved.
	• I am not sure.
Open- Ended	In your view, what does computational thinking, as a thinking skill or a set of cognitive processes, mean and entail?	(3)	Text mining
Open-Ended	In your view, what are some important cognitive skills, processes or strategies to be measured in a computational thinking assessment?	(4)	Text mining

Clusters and themes	N	Domain-view	Raw responses
1. Problem- solving	52 (53.60%)	Mix	“Breaking down a problem and figuring it out in steps.” (40) “It's a process or path to solve a problem or set if problems.” (95) “The process of identifying a problem and the logical steps and data sets required to address the problem / provide a solution.” (83) “The ability to identify via abstraction those parts of a problem which can be automated and applying logical constructs to create an unambiguous set of instructions which can be used to solve the problem.” (31)
2. A transferrable cross-disciplinary skills fundamental to computing principles	41 (42.26%)	Mix	“Computational thinking, has, in my experience, a need to be able to recognise and understand real world problems and be able to decompose them. Once a problem has been understood then a series of steps can are both applicable to the real world but also consider the additional steps necessary to enable the problem to be solved by computational means. Whilst some of these skills transfer to other areas of problem solving, programming has its own unique skill-set. This 'analysis phase', is complete separate to the design (both in terms of graphic design and algorithm design) and is acknowledged as such in most SDLCs.” (77) “I think it's both thinking processes as well as practices - and mostly, a way to help non-computer science folks feel welcome to a field they often find mysterious and foreign. Looking for an exact list or a definition has never been important to me - I see computational thinking more as a scaffold to help introduce ideas of computer science + the benefit of being able to use the same processes and practices in other areas of life.” (58) “Computational thinking is a title for the context of computer science, but it contains the skills and abilities for all problem solving, in a diverse range of applications.” (61) “A transferrable skill developed through the teaching of computing that can equip children to thrive in an ever-changing technological environment.” (48)
3. Solution processes	1 (.01%)	Domain-general; Context-neutral	“Broadly, paying greater attention to the process by which solutions may be generated (as opposed to simply focusing on the solutions themselves) and gaining insight from the generalisations made in these processes that allow problems to be seen as being analogous to one another at some level of abstraction.” (16)
4. Programming-centric skills that require the use of computer	1 (.01%)	Uncertain	“In my personal view, the concept of computational thinking is good but a secondary concept. Really the students and young people should learn how to program/code and the concepts arise out of that. The emphasis on 'computational thinking' is somewhat misplaced, in my humble opinion. It is somewhat like the emphasis on the theory of music or the theory of playing tennis. The tennis player has to learn how to play tennis well if he/she wants to win a tournament or be able to do something constructive in tennis. Andy Murray or any other sports champion did not sit down and think 'let me think of sports…thinking', they got on with the practical aspects of the sports, practised day and night to achieve their goals. Similarly, the concept of computational thinking revolves around the computer - the computer gives rise to the concept and it is fundamental and crucial that people are acquainted with the computer in a practical manner in order to appreciate the concept of 'computational thinking' - which is essentially a secondary concept and we do not society favours by focusing so much on this.” (20)
5. Computer Science curriculum relevant	2 (.02%)	Domain-general; Context-neutral	“I see ct as an educational construct devised to support teachers and learners articulate different aspects of a computing curriculum and a curriculum in general. It is a subset of "thinking skills" and stands longside considerations such as thinking hats, problem solving strategies, cognitive strategies (metaphors, etc). ct helps articulate the value of computer programming in that we are presenting real-world, complex, scenarios and enabling, through physical and mental processes, making them computabe (or at least closer to being computable) and in so doing. making the complexity more understandable.” (60) “Algorithmic thinking is significant part of computational thinking. When I meet with students who are seeking to take my AP CS course without the required prerequisites I demonstrate the binomial theorem and ask them to generalize it. While this is a higher level maths concept, recognizing patterns and being able to generalize them is ONE key to Comp Thinking. Another general problem solving, outside of a strictly technical context. Given a scenario being able to 1) identify the problem 2) identify known information about the problem 3) articulate a strategy to solve the problem 4) check results of strategy. Debate and logic are also examples of computational thinking, structured identification of and response to facts. Review of situations, the ability to decompose.” (103)

Final clusters (themes)	N	Domain-view	Raw responses
1. Logical problem solving strategies	64 (65.97%)	Mix	“Understanding the logical steps to solve a problem” (69) “Abstraction. Decomposition. Logical thinking. Algorithmic thinking.” (30) “Look at the problem and work out a proposed solution using logic trying an solution and if it does not work not to give up. Try many solutions to achieve the best outcome.” (97) “High level of reasoning, being able to think beyond a concrete operational level.” (59) “Mental flexibility; Logical reasoning; Abstract reasoning; Problem-solving; Pattern recognition” (7)
2. Multiple approaches to assess the diversity of CT (both cognitive and non-cognitive)	33 (34.02%)	Mix	“The ability to look at a complex problem and make it much simpler by breaking it into manageable chunks. This can be done with many real-world scenarios that students can understand, before they get anywhere near a computer. By extension, computational thinking can be applied to everyday life. This can obviously be assessed in many ways, but simple, handwritten flowcharts and similar techniques are as good as anything.” (66) “Skills related to CT such as abstraction, algorithmic design, decomposition or pattern recognition should be based on a complementary set of measurement tools. These assessment tools should be able to measure different aspects (cognitive and noncognitive aspects of learning CT) of the skills to be evaluated. Innovation here could be the combination of several complementary measurement tools rather than the use of a single measuring instrument.” (6) “The ability to see layers of abstraction. The ability to play with mental models. One could also assess the breadth and variety of mental models the student possesses.” (100) “The cognitive skills requires students to think about their thinking. Often students will jump to the solution, desperate to be right, and fail to realized the steps they have taken to get to that solution. This is difficult to teach using problems they already know how to solve, because they often fail to recognise how they get to a solution. When it is a new problem they can't solve they fail to solve it, because it is expect that the teacher will show them how to solve the problem first.” (61) Metacognition - is the learner aware of what they are doing and how?” (90)

Footnotes

Acknowledgments

First and foremost, I deeply appreciate all the teachers who participated in this study. My gratitude goes to the Computing At School and National Centre for Computing Education for circulating the teacher survey on their platforms. I am indebted to Ceredig Cattanach-Chell, Dr. Cornelia Connolly, James Robinson, and William Lau for promoting the survey to their circles. Thanks must also be given to the anonymous reviewers who took their time to give feedback and Kevin W.H. Tai for proofreading this article. I greatly appreciate Prof. Michelle Ellefson’s valuable advice in all my work. Lastly, this work would not be made possible without the support from Toyo Mall Limited.

Declaration of Conflicting Interests

The author/authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author/authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Rina P.Y. Lai

Author Biography

Rina P.Y. Lai is currently a PhD Candidate, Course Demonstrator, and Academic Supervisor in Psychology and Education at the University of Cambridge. She is also a member of the INSTRUCT Research Group and the Science & Technology Education Research Group. Her research interests include computational thinking, psychometrics, and learning sciences. Her work has been disseminated and presented at international conferences, expert meetings, and in podcasts.

References

Angeli

Voogt

Fluck

Webb

Cox

Malyn-Smith

Zagami

(2016). A K-6 computational thinking curriculum framework: Implications for teacher knowledge. Journal of Educational Technology & Society, 19(3), 47–57.

Bholowalia

Kumar

(2014). EBK-Means: A clustering technique based on elbow method and K- Means in WSN. International Journal of Computer Applications, 105(9), 17–24.

Bower

Falkner

(2015). Computational thinking, the notional machine. Pre-Service Teachers, and Research Opportunities, 160, 10.

Brennan

Resnick

(2012). New frameworks for studying and assessing the development of computational thinking. In Proceeding of the 2012 Annual American Educational Research Association meeting (Vol. 1, pp.1–25). Academic Press.

Chang

D’Aniello

Gaeta

Orciuoli

Sampson

Simonelli

(2020). Building ontology-driven tutoring models for intelligent tutoring systems using data mining. IEEE Access, 8, 48151–48162. https://doi.org/10.1109/ACCESS.2020.2979281

Cobo

García-Solórzano

Morán

J. A.

Santamaría

Monzo

Melenchón

(2012). Using agglomerative hierarchical clustering to model learner participation profiles in online discussion forums. In Proceedings of the 2nd International conference on learning analytics and knowledge (pp. 248–251). New York: ACM. https://doi.org/10.1145/2330601.2330660

Fensel

(2001). Ontologies: Dynamic networks of formally represented meaning. http://sw portal.deri.at/papers/publications/network.pdf

Fessakis

Prantsoudi

(2019). Computer science teachers’ perceptions, beliefs and attitudes on computational thinking in Greece. Informatics in Education, 18(2), 227–258. https://doi.org/10.15388/infedu.2019.11

Fürst

Leclère

Trichet

(2003). Ontological engineering and mathematical knowledge management: a formalization of projective geometry, Annals of Mathematics and Artificial Intelligence, 38, 65–89.

10.

Gaikwad

Chaugule

Patil

(2014). Text mining methods and techniques. International Journal of Computer Applications, 85(17), 42–45.

11.

Garvin

Killen

Plane

Weintrop

(2019). Primary school teachers’ conceptions of computational thinking. In Proceedings of the 50th ACM technical symposium on computer science education (pp. 899–905). New York: ACM.

12.

Good

Yadav

Mishra

(2017). Computational thinking in computer science classrooms: Viewpoints from CS educators (pp.51–59). Association for the Advancement of Computing in Education (AACE).

13.

Grover

(2015). “Systems of assessments” for deeper learning of computational thinking in K-12. In Proceedings of the 2015 Annual meeting of the American Educational Research Association (AERA ’15) (pp. 1–9). Academic Press.

14.

Grover

Pea

(2017). Computational thinking: A competency whose time has come. In Sentence

Barenden

Schulte

(Eds.), Computer science education. Perspective on teaching and learning in school (pp. 19–38). Bloomsbury Academic.

15.

Gruber

T. R.

(1993). A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2), 199–200. https://doi.org/10.1006/knac.1993.1008

16.

Guarino

(1995). Formal ontology, conceptual analysis and knowledge representation. International Journal of Human-Computer Studies, 43(5–6), 625–640.

17.

Hiemstra

(2000). A probabilistic justification for using tf-idf term weighting in information retrieval. International Journal on Digital Libraries, 3(2), 131–139. https://doi.org/10.1007/s007999900025

18.

Huang

(2008, April). Similarity measures for text document clustering [Paper presentation]. New Zealand Computer Science Research Student Conference (NZCSRSC), Christchurch.

19.

Israel-Fishelson

Hershkovitz

Eguíluz

Garaizar

Guenaga

(2021). A Log-Based analysis of the association between creativity and computational thinking. Journal of Educational Computing Research, 59(5), 926–959.

20.

Kalelioglu

Gulbahar

Kukul

(2016). A framework for computational thinking based on a systematic research review. Baltic Journal of Modern Computing, 4, 583–596.

21.

Kaufman, L., & Rousseeuw, P. J. (1990). Finding Groups in Data–An Introduction to Cluster Analysis. New York: John Wiley & Sons Inc. https://doi.org/10.1002/9780470316801

22.

Kong

S. C.

(2016). A framework of curriculum design for computational thinking development in K-12 education. Journal of Computers in Education, 3(4), 377–394. https://doi.org/10.1007/s40692-016-0076-z

23.

Kong

S.-C.

(2019). Components and methods of evaluating computational thinking for fostering creative Problem-Solvers in senior primary school education. In Kong

S.-C.

Abelson

(Eds.), Computational thinking education (pp. 119–141). Springer. https://doi.org/10.1007/978-981-13-6528-7_8

24.

Lai

R. P. Y.

(2019). What underlies computational thinking: Exploring its cognitive mechanism and educational implications. In S. C. Kong, D. Andone, G. Biswas, U. Hoppe, R. H. Huang, B. C. Kuo, K. Y. Li, C. K. Looi, M. Milrad, J. Sheldon, J. L. Shih, K. F. Sin, K. S. Song, & J. Vahrenhold (Eds.), Proceedings of international conference on computational thinking education 2019 (pp. 204–208). Education University of Hong Kong.

25.

Dhavala

(2018). Hierarchical clustering with prior knowledge. ArXiv:1806.03432 [Cs, Stat]. http://arxiv.org/abs/1806.03432

26.

Mesiti

L. A.

Parkes

Paneto

S. C.

Cahill

(2019). Building capacity for computational thinking in youth through informal education. Journal of Museum Education, 44(1), 108–121. https://doi.org/10.1080/10598650.2018.1558656

27.

Meyer

Hornik

Feinerer

(2008). Text mining infrastructure in R. Journal of Statistical Software, 25(5), 1–54.

28.

Moreno-León

Robles

Román-González

(2016). Comparing computational thinking development assessment scores with software complexity metrics. In 2016 IEEE global engineering education conference (EDUCON) (pp. 1040–1045). IEEE.

29.

Mueller

Beckett

Hennessey

Shodiev

(2017). Assessing computational thinking across the curriculum. In Rich

P. J.

Hodges

C. B.

(Eds.), Emerging research, practice, and policy on computational thinking (pp. 251–267). Springer International Publishing. https://doi.org/10.1007/978-3-319-52691-1_16

30.

O’Cathain

Thomas

K. J.

(2004). “Any other comments?” Open questions on questionnaires – A bane or a bonus to research? BMC Medical Research Methodology, 25(4), 1–7.

31.

Pinkard

Martin

C. K.

Erete

(2020). Equitable approaches: Opportunities for computational thinking with emphasis on creative production and connections to community. Interactive Learning Environments, 28(3), 347–361. https://doi.org/10.1080/10494820.2019.1636070

32.

Porter

M. E.

(1980). Competitive strategy: Techniques for analyzing industries and competitors. Free Press.

33.

Rich

Hodges

C. B.

(Eds.). (2017). Emerging research, practice, and policy on computational thinking. Springer International Publishing.

34.

Román-González

Moreno-León

Robles

(2019). Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In S.-C. Kong & H. Abelson (Eds), Computational Thinking Education. Singapore: Springer.

35.

Román-González

Pérez-González

J.-C.

Moreno-León

Robles

(2018). Can computational talent be detected? Predictive validity of the computational thinking test. International Journal of Child-Computer Interaction, 18, 47–58. https://doi.org/10.1016/j.ijcci.2018.06.004

36.

Salton

Buckley

(1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523. https://doi.org/10.1016/0306-4573(88)90021-0

37.

Sands

Yadav

Good

(2018). Computational thinking in K-12: In-service teacher perceptions of computational thinking. In Khine

M. S.

(Ed.), Computational thinking in the STEM disciplines: Foundations and research highlights (pp. 151–164). Springer International Publishing. https://doi.org/10.1007/978-3-319- 93566-9_8

38.

Selby

Woollard

(2013). Computational thinking: The developing definition [monograph]. https://eprints.soton.ac.uk/356481/

39.

Shute

Sun

Asbell-Clarke

(2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003

40.

Stancin

Poscic

Jaksic

(2020). Ontologies in education – State of the art. Education and Information Technologies, 25(6), 5301–5320. https://doi.org/10.1007/s10639-020-10226-z

41.

Stanimirova

Daszykowski

(2018). Chapter nine – Exploratory analysis of metabolomic data. In Jaumot

Bedia

Tauler

(Eds.), Comprehensive analytical chemistry (Vol. 82, pp. 227–264). Elsevier. https://doi.org/10.1016/bs.coac.2018.08.005

42.

Tang

Yin

Lin

Hadad

Zhai

(2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798. https://doi.org/10.1016/j.compedu.2019.103798

43.

Tissenbaum, M., Sheldon, J., & Sherman, M. (2018). The State of the field in computational thinking assessment. In the Proceedings of the 2018 International Conference of the Learning Sciences. London

44.

Van den Akker

(2010). Building bridges: How research may improve curriculum policies and classroom practices. In S. Stoney (Ed.), Beyond Lisbon 2010: Perspectives from research and development for education policy in Europe (CIDREE yearbook 2010) (pp. 177–195). National Foundation for Educational Research.

45.

Voogt

Fisser

Good

Mishra

Yadav

(2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728. https://doi.org/10.1007/s10639-015-9412-6

46.

Voskoglou

M. G.

Buckley

(2012). Problem solving and computational thinking in a learning environment. ArXiv:1212.0750 [Cs]. http://arxiv.org/abs/1212.0750

47.

Ward

J. H.

Jr. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58(301), 236–244. https://doi.org/10.1080/01621459.1963.10500845

48.

Weintrop

Wilensky

(2017). Comparing block-based and text-based programming in high school computer science classrooms. ACM Transactions on Computing Education, 18(1), 1–25. https://doi.org/10.1145/3089799

49.

Yadav, A., Good, J., Voogt, J., & Fisser, P. (2017). Computational thinking as an emerging competence domain. In Competence-based vocational and professional education (pp. 1051–1067). Springer International Publishing.

50.

Yadav

Krist

Good

Caeli

E. N.

(2018). Computational thinking in elementary classrooms: Measuring teacher understanding of computational ideas for teaching science. Computer Science Education, 28(4), 371–400. https://doi.org/10.1080/08993408.2018.1560550

51.

C. H.

Jannasch-Pennell

DiGangi

(2011). Compatibility between text mining and qualitative research in the perspectives of grounded theory, content analysis, and reliability. The Qualitative Report, 16(3), 730–744. https://doi.org/10.46743/2160-3715/2011.1085

Teachers’ Ontological Perspectives of Computational Thinking and Assessment: A Text Mining Approach

Abstract

Keywords

Teachers’ Ontological Views of CT

The Current Study

Methods

Sample

The Teacher Survey

Analytical Techniques

Data Processing

Results

What Underpins Computational Thinking?

Teachers’ Domain View

Computational Thinking: Conceptualization

Text Summarization

Sub-Groups and Themes in Conceptualization

Computational Thinking: Assessment

Text Summarization

Sub-Groups and Themes in Assessment

Discussion

The Nature of CT: Key Components and Domain View

CT Conceptualization: The Five Sub-Groups and Themes

CT Assessment: The Two Sub-Groups and Themes

Conclusion

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

ORCID iD

Author Biography

References