Abstract
The aim of this research is to introduce a method of constructing knowledge structures which represent the relationships among the knowledge items in a pedagogically useful way. A prototype, “MyTeLeMap,” was implemented to support both learners and instructors. Learners could visualize the knowledge structures of different knowledge domains and obtain links to corresponding study materials. The system also offered learning paths and recommendations, for example, to related structures. Instructors could create and share their knowledge structures. An experiment compared the learning outcomes of learners using MyTeLeMap and those using a free-browsing mode. The results showed that MyTeLeMap helped learners more than free browsing. Future work includes the incorporation of a search application for learners and of management features for instructors to upload and share learning materials linked to specific knowledge domains.
Keywords
Introduction
Knowledge representation (KR) is the study of representing knowledge formally and explicitly, thereby supporting unambiguous knowledge sharing. This issue becomes particularly important when machines facilitate knowledge management (Guarino, 1995). In the learning and teaching environment, KR has been used to: represent a content structure of learning documents (Yang and Sun, 2013), support learners in their own knowledge acquisition (Chu et al., 2011; Shaw, 2010), and enable suggestions to learners (Chiou et al., 2012). Knowledge relationships can be represented in many forms: maps, trees, networks, and graphs. They may be called concept maps, knowledge maps, and knowledge structures.
There are several methods of building knowledge structures (Li-Yu et al., 2012; Nitchot et al., 2011; Novak and Canas, 2006), and an interesting question is how such structures can pedagogically support learning and teaching activities. In the early stage of this research, literature related to methods of knowledge structure/concept map building and use in educational technology was reviewed. Thai school teachers were surveyed and interviewed (as discussed in ‘Conclusions and Future Work’ section) and asked about their teaching experiences and the uses of knowledge structures in teaching. The preliminary findings suggested that the teachers did not report using knowledge structures in teaching, but did use “mind maps” to support learners’ conceptualization and understanding of the course content. These were a rather rough note of content knowledge where the relationships among the content items were usually vague and imprecise (D'Antoni et al., 2010). The teachers felt that mind maps were useful and that they could benefit education, helping learners think about and understand the relationships in the knowledge shown.
“Resources” are an important factor to complement a knowledge structure and support its use. There is still a lack of good resources for learning (Thai Education Testing Organization, 2009). E-learning is one type of resource, comprising electronically supported online courses, online websites, lesson videos, instructional TV, and so on. Such resources allow learners to study and practice/solve lesson problems at times and in places which may suit them better. While learners can retrieve supplementary online resources and familiarize themselves with missing knowledge before their course starts, it is difficult for them to find and access materials which match their intended learning outcomes (ILOs) or their current level of ability. Learners may not be able to identify their competences on their own, identify the relationships among both prerequisite and desired knowledge, or obtain learning materials which properly relate to their current and desired abilities.
This research suggests a method of constructing and applying knowledge structures to support learning and teaching which is based on the ILOs in a targeted knowledge domain. The “MyTeLeMap” application is a tool for building and visualizing knowledge structures. Instructors or knowledge structure designers (which could include learners themselves) can use this tool to develop, extend, and share knowledge structures. Importantly, the tool supports identifying knowledge missing from a learner’s current understanding and suggesting both learning paths and learning resources in respect of particular desired learning outcomes.
KR and applications
Bench-Capon (2014) suggested KR is a set of syntactic and semantic conventions that makes it possible to describe things. Syntax refers to a set of rules for combining symbols to form valid expressions, and semantics specifies how such expressions are to be interpreted.
Liao (2003) reviewed the literature of KR applications and found it used within fields such as human resources, management, database management, agriculture, and e-learning.
Chiu and Pan (2014) designed a knowledge structure to represent the information related to research papers (topic, name, and cited frequency) and to explore the relationships among them, helping readers understand the relationships among selected topics, papers, and citation frequencies. Wickel et al. (2013) used knowledge structures to manage human resources within organizations, identifying project team roles and members’ relevant knowledge. Abel (2015) developed “E-MEMORAe” as a web platform for managing and sharing knowledge within an organization (Figure 1). This system was adapted to an educational environment, where learners could use the knowledge structures to access learning resources. In this research (Abel, 2015), knowledge was defined using ontologies which cover the knowledge structure and its functionality. Hao et al. (2014) proposed a method of constructing a knowledge structure by: defining the domain knowledge content, computing a keyword-by-knowledge item co-occurrence matrix, and calculating semantic similarity. A sample of a resulting knowledge structure is shown in Figure 2.
Knowledge map containing seven persons in a project and their identified knowledge (Abel, 2015).
An example of a knowledge map where k is the knowledge keyword (Hao et al., 2014).
Related work has adopted KR within e-learning. Melis et al. (2003) proposed “ACTIVEMATH,” an open Web-based learning environment for mathematics, where the KR was of the content structure of mathematical learning resource documents. It used “OMDoc” (Kohlhase, 2000), an extension of the OpenMath XML standard, and contained a grammar representation of mathematical objects and sets of standardized symbols (the content dictionaries). Marshall et al. (2003) proposed “GetSmart,” a tool to allow individuals to create and share knowledge, where users could construct concept maps and synthesize their ideas into personal KRs. XML was applied to enhance modularity for concept map sharing. Mendes et al. (2002) used a fuzzy clustering algorithm and “TopicMaps,” a tool for modelling and managing knowledge structures which are in the form of XML documents, to discover and represent knowledge. The relationships between learning materials were identified by fuzzy clustering and later used within adaptive link documents.
Personalized search within educational system
Gordon and Pathak (1999) discussed four different methods for locating information on the Web. Of direct interest is the use of search engines to find and then furnish information that hopefully relates to the search term. Griffiths and Brophy (2007) suggested that learners use search engines to find learning resources from the internet as a self-learning activity. Currently, there are a number of search engines, for example, Google, Bing, Yahoo, and Alta Vista. Google is often used for searches in a learning context, since it gives a high probability that the first result is relevant (Hawking et al., 2001). In addition, Google offers the largest index, useful services, and relatively good performance and usability (Mayr and Tosques, 2005; Pan et al., 2007).
There is an argument, however, against Google’s search engine based on the PageRank algorithm, which is that PageRank is not effective for identifying the best webpages in a university system because of its domination by internal links (Thelwall, 2003). Normally, Google search results contain various kinds of webpages such as blogs, forums, electronic books, and electronic files. While some of these results could contain pages with academic purposes, others may contain internal links with non-academic purposes. Nevertheless, a Google search remains an effective way to gather all resources from the Web which may be related to a learner’s search using competence terms.
Tang and Ng (2006) used Google as a diagnostic aid for both doctors and patients. Their results showed that web-based search engines such as Google are becoming the latest tools in clinical medicine, and doctors in training need to become proficient in their use. Some studies were conducted to assess the effectiveness of Google as a tool for personal learning. Griffiths and Brophy (2007) investigated user searching behavior and information-seeking strategies. Google was rated well for ease of use, success, and time taken to search, and was found to be the search engine of choice, although students found it difficult to locate information and resources and may trade quality of results for effort and time spent searching. Liaw et al. (2006) investigated individual attitudes toward search engines as a learning assistance tool. Their results suggested that experience with high-quality search engines positively influenced user perception of individual enjoyment and self-efficacy. Tsai and Tsai (2003) explored students’ strategies in searching for information via Web-based activities. They concluded that high internet self-efficacy students had better searching strategies and learned better than those with low internet self-efficacy. While Google helped learners find information and study resources, there were some limitations in the quality of found resources and in the students’ self-efficacy skills.
Constructing MyTeLeMap knowledge structures
Some methods and approaches of how to design and construct knowledge structures were reviewed in “Knowledge Representation and Applications” section. In this study, the “MyTeLeMap” knowledge structure of fundamental programming conforms to the requirements of a directed acyclic graph, in which case it is also known as a dependency graph, and a number of theorems of graph theory apply on reachability and path uniqueness. This knowledge structure is a particular instance of Sowa’s (2000) definition of KR as “a multidisciplinary subject that applies theories from three fields: logic, ontology, and computation.” Logic identifies the formal structure and rules of inference. Ontology refers to the kinds of things that exist in the application domain. Computation distinguishes the application of KR from pure philosophy.
Data on subject matter and learning outcomes was obtained from school teachers, followed by a task analysis to give a diagrammatic representation of the subject matter (Gilbert and Gale, 2008). Knowledge structures were built as follows, adapting the process described by Nitchot et al. (2011):
Step 1: Choose knowledge domain, identify ILOs, list subject matter
The construction of a pedagogically informed knowledge structure starts by identifying the ILOs that a student’s knowledge will support. For example, the ILO “define HTML” might be relevant for a knowledge domain of “Web Technologies.” The ILOs are analyzed to provide a list of subject matter content. The earlier example of an ILO suggests “HTML” as an item of subject matter to be added to the list.
Step 2: Undertake task analysis of subject matter
The subject matter content is categorized intofour types based on Merrill’s CDT (Gilbert and Gale, 2008): fact, concept, procedure, and principle. The task analysis provides the relationships and structures inherent in each type of subject matter, using a diagrammatic approach where the type of subject matter has a characteristic notation and representation.
A “fact” is represented by two elements which make a fact pair. Each element is notated as a circle. For example, the fact of “HTML” is represented as a pair of two facts, “standard markup language” and “tag elements,” as shown in Figure 3.
Task analysis of the fact ‘HTML’ as a standard markup language.
Comprising tag elements
A “concept” involves the concept name, its superordinate class, and a number of attribute-value pairs which appropriately characterize the concept. The relationship between class and superordinate class is “a kind of” or “type of.” The concept is notated as triangle, and its components are shown as facts. For example, the concept of “CSS” is illustrated in Figure 4, showing that the concept “CSS” is a kind of “style” characterized by its “compatibility” with “HTML” and its “composition” comprising “property” and “value.”
Task analysis of the concept ‘CSS’.
Task analysis of the procedure ‘Setting the Website Online’.
A “procedure” is represented as a set of steps (and optionally has associated facts and concepts such as the procedure name, the situation in which it may be appropriately applied, and the goal which it achieves). A step is notated as a square (or, more elaborately, the procedure may be notated as a UML activity diagram). For example, the procedure of “setting the website online” is shown in Figure 5.
A “principle” involves the specification of cause and effect. The principle itself is notated as a pentagon with something of a direction suggested by two sides. For example, the principle of CSS definition is shown in Figure 6. Causes are shown on the left side of pentagon and the right side shows the effect or result of the principle. Here, the set of causes and effects are represented as facts.
Task analysis of the principle ‘CSS Definition’.
Step 3: Completing the subject matter list
The task analysis of the subject matter is reviewedand the list of subject matter enhanced to ensure completeness that all required facts, concepts, procedures, and principles are present. For example, “HTML” comprises “tag elements,” and consideration may be given to enumerating the various tag elements in the subject matter list if these are not already incorporated. This might be appropriate if an ILO such as, “list the major HTML tags” is to be added to the knowledge domain.
Step 4: Structure the subject matter
Given the complete list of subject matter and the associated task analysis which identifies the relationships between the subject matter items, a knowledge structure is created by representing each subject matter item as a node, and by connecting nodes where the items are connected in the task analysis diagrams. The relationship between subject matter nodes is parent–child, and by convention is notated by an arrow pointing to the child node. Which is the parent and which the child is given by the relationship shown in the task analysis, and, pedagogically, identifies the prerequisite. This knowledge structure represents the domain subject matter. In order to develop a competence structure, each node of subject matter requires tagging with a corresponding capability (from the associated ILO) and a context (implied by the ILO). Figure 7 shows a sample of a knowledge structure of a mathematical subject (at high school level).
Knowledge structure of web technologies, where arrow indicates the prerequisite and circle indicates the subject matter.
Tools for suggesting learning resources’ links
A tool for suggesting links to learning resources based on knowledge structures has been implemented within the prototype “MyTeLeMap.” The current tool incorporates the designed knowledge structures and their associated learning resources (mainly html links). A graph visualization library (such as Graphviz (REF) and Microsoft Automatic Graph Layout (https://www.graphviz.org/)) display the graph nodes and edges from the knowledge database. The Google API is used to gather links from the web. The tool infrastructure is shown in Figure 8. Currently, a recommender system and a learning path service are under investigation (rounded rectangles with strong line).
MyTeLeMap tool infrastructure.
Figures 9 and 10 show screenshots of the prototype MyTeLeMap tool in use. In Figure 9, the chosen knowledge structure is shown, and links associated with the selected node are suggested. The links are obtained from a Google search using the Google API, where the search keywords are extracted from the knowledge keywords in the selected node. The search results can be filtered by website (e.g. YouTube, Wikipedia). In Figure 10, the nodes are suggested based on the current selected node and previously visited nodes. The teacher role can manage all designed knowledge structures, as shown in Figure 11, and can create new knowledge structures as shown in Figure 12.
Screenshot of the prototype MyTeLeMap suggesting study materials links.
Screenshot of the prototype MyTeLeMap giving some suggestions.
Screenshot of the Prototype MyTeLeMap listing All designed knowledge structures.
Screenshot of the Prototype MyTeLeMap for managing a designed knowledge structure.
Questionnaire and interview: Design and results
A preliminary survey on teachers’ experiences in using and designing knowledge structures was conducted, comprising a questionnaire and an interview. For the questionnaire study, participants were 173 school teachers from Songkhla province, Thailand. The results showed that the teachers have used some tools for building such structures, such as a mind-mapping tool and the Cmap tool. Most of them (136 school teachers) recognized and used mind maps during their teaching activities. None of them represented or designed pedagogical structures with linkage among the knowledge components. They were interested in using a tool for building pedagogical knowledge structure (interested = 106, highly interested = 61) and were willing to use the structures and their applications as educational aids (interested = 101, highly interested = 62).
The interview study was conducted with 10 school teachers from the schools mentioned. Even though knowledge structures have been recommended for many years, school teachers still did not report using them in their teaching. However, the teachers felt that knowledge structures were useful and could benefit education more. For example, such structures could help learners think about and understand the relationships among the knowledge shown. On the teachers’ side, they thought that designing knowledge structures could be another way of sharing their tacit knowledge with other teachers.
Experiment: design and results
An experiment was conducted to compare whether learning using the prototype MyTeLeMap was better than learning by freely browsing. The pictorial representations of these two learning modes are shown in Figures 13 and 14.
MyTeLeMap learning mode.
Freely browsing learning mode.
The experiment was concerned with the second, “learning,” level of Kirkpatrick’s four levels of evaluation (Kirkpatrick, 2007). The participants were assigned to one of two groups: one group experienced the MyTeLeMap learning mode and the other group experienced the freely browsing learning mode. All participants were required to take a pre-test and a post-test, before and after experiencing the respective learning modes. The pre-test and post-test were the same for all participants, being a multiple choice test consisting of 10 questions as shown in Table 1. The scores obtained from the pre-test and post-test were compared for each learning mode.
The required sample size of this experiment was 12 according to G*power, using an effect size f = 1, an alpha error probability = 0.05, power = 0.8, the test family as F-test, the number of groups = 2, and the statistical test as ANOVA repeated measures. The actual number of participants was 40 as shown in Table 2.
Examples of questions in pre-test and post-test.
Means and standard deviation of test scores.
Profile graph of mean ratings of test scores of pre-test and post-test for two learning modes (where Group 1 = MyTeLeMap and Group 2 = Freely Browsing).
The questions in the pre-test/post-test were based on selected knowledge of fundamental programming. The chosen knowledge was as follows:
Functions Operators String Variables
A two-way repeated measures ANOVA was used to analyze the obtained test scores, in order to determine the significance of any differences in learning mode outcomes. “Learning mode” comprised two levels, freely-browsing and MyTeLeMap. “Test type” comprised two levels, pre-test and post-test.
Tables 2 to 4 show the descriptive statistics, the tests of within-subjects effects, and the tests of between-subjects effects. Figure 15 displays the profile graphs.
Tests of within-subjects effects.
Tests of between-subjects effects.
The results from the statistical data obtained (as shown in Tables 2 to 4) were as follows.
There was no significant difference between the two groups of students at pre-test, suggesting that the students in both groups had similar knowledge of fundamental programming. Both learning modes helped students significantly increasing their knowledge. There was a significant interaction between learning mode and test types, such that MyTeLeMap helped students improve their learning significantly more than freely browsing.
From the experiment, it can be concluded that both learning modes helped learners in learning, however, MyTeLeMap helped significantly more than freely browsing.
Conclusions and future work
A knowledge-based system (MyTeLeMap) for suggesting study material links from the Web has been proposed in this research. The aim of the approach is to assist learners to achieve their desired knowledge. The system provides learners with suggestions based on their current and previously selected nodes in a given knowledge structure. A method of constructing the knowledge structure was also proposed. An experimental study showed that the MyTeLeMap system can support learning better than free browsing. The system depends on a search engine API which tends to change regularly, and this may affect the long-term usability of the application. Future work includes a personalized search engine, improvements to the system’s user interface and user experience, and some management features to allow teachers to attach their own study materials.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by Thailand Research Fund under contract number MRG5980096.