Sage Journals: Discover world-class research

Abstract

The aim of this research is to introduce a method of constructing knowledge structures which represent the relationships among the knowledge items in a pedagogically useful way. A prototype, “MyTeLeMap,” was implemented to support both learners and instructors. Learners could visualize the knowledge structures of different knowledge domains and obtain links to corresponding study materials. The system also offered learning paths and recommendations, for example, to related structures. Instructors could create and share their knowledge structures. An experiment compared the learning outcomes of learners using MyTeLeMap and those using a free-browsing mode. The results showed that MyTeLeMap helped learners more than free browsing. Future work includes the incorporation of a search application for learners and of management features for instructors to upload and share learning materials linked to specific knowledge domains.

Keywords

Personalized learning knowledge structure web-based learning internet supported learning pedagogy structure visualization

Introduction

Knowledge representation (KR) is the study of representing knowledge formally and explicitly, thereby supporting unambiguous knowledge sharing. This issue becomes particularly important when machines facilitate knowledge management (Guarino, 1995). In the learning and teaching environment, KR has been used to: represent a content structure of learning documents (Yang and Sun, 2013), support learners in their own knowledge acquisition (Chu et al., 2011; Shaw, 2010), and enable suggestions to learners (Chiou et al., 2012). Knowledge relationships can be represented in many forms: maps, trees, networks, and graphs. They may be called concept maps, knowledge maps, and knowledge structures.

There are several methods of building knowledge structures (Li-Yu et al., 2012; Nitchot et al., 2011; Novak and Canas, 2006), and an interesting question is how such structures can pedagogically support learning and teaching activities. In the early stage of this research, literature related to methods of knowledge structure/concept map building and use in educational technology was reviewed. Thai school teachers were surveyed and interviewed (as discussed in ‘Conclusions and Future Work’ section) and asked about their teaching experiences and the uses of knowledge structures in teaching. The preliminary findings suggested that the teachers did not report using knowledge structures in teaching, but did use “mind maps” to support learners’ conceptualization and understanding of the course content. These were a rather rough note of content knowledge where the relationships among the content items were usually vague and imprecise (D'Antoni et al., 2010). The teachers felt that mind maps were useful and that they could benefit education, helping learners think about and understand the relationships in the knowledge shown.

“Resources” are an important factor to complement a knowledge structure and support its use. There is still a lack of good resources for learning (Thai Education Testing Organization, 2009). E-learning is one type of resource, comprising electronically supported online courses, online websites, lesson videos, instructional TV, and so on. Such resources allow learners to study and practice/solve lesson problems at times and in places which may suit them better. While learners can retrieve supplementary online resources and familiarize themselves with missing knowledge before their course starts, it is difficult for them to find and access materials which match their intended learning outcomes (ILOs) or their current level of ability. Learners may not be able to identify their competences on their own, identify the relationships among both prerequisite and desired knowledge, or obtain learning materials which properly relate to their current and desired abilities.

This research suggests a method of constructing and applying knowledge structures to support learning and teaching which is based on the ILOs in a targeted knowledge domain. The “MyTeLeMap” application is a tool for building and visualizing knowledge structures. Instructors or knowledge structure designers (which could include learners themselves) can use this tool to develop, extend, and share knowledge structures. Importantly, the tool supports identifying knowledge missing from a learner’s current understanding and suggesting both learning paths and learning resources in respect of particular desired learning outcomes.

KR and applications

Bench-Capon (2014) suggested KR is a set of syntactic and semantic conventions that makes it possible to describe things. Syntax refers to a set of rules for combining symbols to form valid expressions, and semantics specifies how such expressions are to be interpreted.

Liao (2003) reviewed the literature of KR applications and found it used within fields such as human resources, management, database management, agriculture, and e-learning.

Chiu and Pan (2014) designed a knowledge structure to represent the information related to research papers (topic, name, and cited frequency) and to explore the relationships among them, helping readers understand the relationships among selected topics, papers, and citation frequencies. Wickel et al. (2013) used knowledge structures to manage human resources within organizations, identifying project team roles and members’ relevant knowledge. Abel (2015) developed “E-MEMORAe” as a web platform for managing and sharing knowledge within an organization (Figure 1). This system was adapted to an educational environment, where learners could use the knowledge structures to access learning resources. In this research (Abel, 2015), knowledge was defined using ontologies which cover the knowledge structure and its functionality. Hao et al. (2014) proposed a method of constructing a knowledge structure by: defining the domain knowledge content, computing a keyword-by-knowledge item co-occurrence matrix, and calculating semantic similarity. A sample of a resulting knowledge structure is shown in Figure 2.

Figure 1.

Knowledge map containing seven persons in a project and their identified knowledge (Abel, 2015).

Figure 2.

An example of a knowledge map where k is the knowledge keyword (Hao et al., 2014).

Related work has adopted KR within e-learning. Melis et al. (2003) proposed “ACTIVEMATH,” an open Web-based learning environment for mathematics, where the KR was of the content structure of mathematical learning resource documents. It used “OMDoc” (Kohlhase, 2000), an extension of the OpenMath XML standard, and contained a grammar representation of mathematical objects and sets of standardized symbols (the content dictionaries). Marshall et al. (2003) proposed “GetSmart,” a tool to allow individuals to create and share knowledge, where users could construct concept maps and synthesize their ideas into personal KRs. XML was applied to enhance modularity for concept map sharing. Mendes et al. (2002) used a fuzzy clustering algorithm and “TopicMaps,” a tool for modelling and managing knowledge structures which are in the form of XML documents, to discover and represent knowledge. The relationships between learning materials were identified by fuzzy clustering and later used within adaptive link documents.

Personalized search within educational system

Gordon and Pathak (1999) discussed four different methods for locating information on the Web. Of direct interest is the use of search engines to find and then furnish information that hopefully relates to the search term. Griffiths and Brophy (2007) suggested that learners use search engines to find learning resources from the internet as a self-learning activity. Currently, there are a number of search engines, for example, Google, Bing, Yahoo, and Alta Vista. Google is often used for searches in a learning context, since it gives a high probability that the first result is relevant (Hawking et al., 2001). In addition, Google offers the largest index, useful services, and relatively good performance and usability (Mayr and Tosques, 2005; Pan et al., 2007).

There is an argument, however, against Google’s search engine based on the PageRank algorithm, which is that PageRank is not effective for identifying the best webpages in a university system because of its domination by internal links (Thelwall, 2003). Normally, Google search results contain various kinds of webpages such as blogs, forums, electronic books, and electronic files. While some of these results could contain pages with academic purposes, others may contain internal links with non-academic purposes. Nevertheless, a Google search remains an effective way to gather all resources from the Web which may be related to a learner’s search using competence terms.

Tang and Ng (2006) used Google as a diagnostic aid for both doctors and patients. Their results showed that web-based search engines such as Google are becoming the latest tools in clinical medicine, and doctors in training need to become proficient in their use. Some studies were conducted to assess the effectiveness of Google as a tool for personal learning. Griffiths and Brophy (2007) investigated user searching behavior and information-seeking strategies. Google was rated well for ease of use, success, and time taken to search, and was found to be the search engine of choice, although students found it difficult to locate information and resources and may trade quality of results for effort and time spent searching. Liaw et al. (2006) investigated individual attitudes toward search engines as a learning assistance tool. Their results suggested that experience with high-quality search engines positively influenced user perception of individual enjoyment and self-efficacy. Tsai and Tsai (2003) explored students’ strategies in searching for information via Web-based activities. They concluded that high internet self-efficacy students had better searching strategies and learned better than those with low internet self-efficacy. While Google helped learners find information and study resources, there were some limitations in the quality of found resources and in the students’ self-efficacy skills.

Constructing MyTeLeMap knowledge structures

Some methods and approaches of how to design and construct knowledge structures were reviewed in “Knowledge Representation and Applications” section. In this study, the “MyTeLeMap” knowledge structure of fundamental programming conforms to the requirements of a directed acyclic graph, in which case it is also known as a dependency graph, and a number of theorems of graph theory apply on reachability and path uniqueness. This knowledge structure is a particular instance of Sowa’s (2000) definition of KR as “a multidisciplinary subject that applies theories from three fields: logic, ontology, and computation.” Logic identifies the formal structure and rules of inference. Ontology refers to the kinds of things that exist in the application domain. Computation distinguishes the application of KR from pure philosophy.

Data on subject matter and learning outcomes was obtained from school teachers, followed by a task analysis to give a diagrammatic representation of the subject matter (Gilbert and Gale, 2008). Knowledge structures were built as follows, adapting the process described by Nitchot et al. (2011):

Step 1: Choose knowledge domain, identify ILOs, list subject matter

The construction of a pedagogically informed knowledge structure starts by identifying the ILOs that a student’s knowledge will support. For example, the ILO “define HTML” might be relevant for a knowledge domain of “Web Technologies.” The ILOs are analyzed to provide a list of subject matter content. The earlier example of an ILO suggests “HTML” as an item of subject matter to be added to the list.

Step 2: Undertake task analysis of subject matter

The subject matter content is categorized intofour types based on Merrill’s CDT (Gilbert and Gale, 2008): fact, concept, procedure, and principle. The task analysis provides the relationships and structures inherent in each type of subject matter, using a diagrammatic approach where the type of subject matter has a characteristic notation and representation.

A “fact” is represented by two elements which make a fact pair. Each element is notated as a circle. For example, the fact of “HTML” is represented as a pair of two facts, “standard markup language” and “tag elements,” as shown in Figure 3.

Figure 3.

Task analysis of the fact ‘HTML’ as a standard markup language.

Comprising tag elements

A “concept” involves the concept name, its superordinate class, and a number of attribute-value pairs which appropriately characterize the concept. The relationship between class and superordinate class is “a kind of” or “type of.” The concept is notated as triangle, and its components are shown as facts. For example, the concept of “CSS” is illustrated in Figure 4, showing that the concept “CSS” is a kind of “style” characterized by its “compatibility” with “HTML” and its “composition” comprising “property” and “value.”

Figure 4.

Task analysis of the concept ‘CSS’.

Figure 5.

Task analysis of the procedure ‘Setting the Website Online’.

A “procedure” is represented as a set of steps (and optionally has associated facts and concepts such as the procedure name, the situation in which it may be appropriately applied, and the goal which it achieves). A step is notated as a square (or, more elaborately, the procedure may be notated as a UML activity diagram). For example, the procedure of “setting the website online” is shown in Figure 5.

A “principle” involves the specification of cause and effect. The principle itself is notated as a pentagon with something of a direction suggested by two sides. For example, the principle of CSS definition is shown in Figure 6. Causes are shown on the left side of pentagon and the right side shows the effect or result of the principle. Here, the set of causes and effects are represented as facts.

Figure 6.

Task analysis of the principle ‘CSS Definition’.

Step 3: Completing the subject matter list

The task analysis of the subject matter is reviewedand the list of subject matter enhanced to ensure completeness that all required facts, concepts, procedures, and principles are present. For example, “HTML” comprises “tag elements,” and consideration may be given to enumerating the various tag elements in the subject matter list if these are not already incorporated. This might be appropriate if an ILO such as, “list the major HTML tags” is to be added to the knowledge domain.

Step 4: Structure the subject matter

Given the complete list of subject matter and the associated task analysis which identifies the relationships between the subject matter items, a knowledge structure is created by representing each subject matter item as a node, and by connecting nodes where the items are connected in the task analysis diagrams. The relationship between subject matter nodes is parent–child, and by convention is notated by an arrow pointing to the child node. Which is the parent and which the child is given by the relationship shown in the task analysis, and, pedagogically, identifies the prerequisite. This knowledge structure represents the domain subject matter. In order to develop a competence structure, each node of subject matter requires tagging with a corresponding capability (from the associated ILO) and a context (implied by the ILO). Figure 7 shows a sample of a knowledge structure of a mathematical subject (at high school level).

Figure 7.

Knowledge structure of web technologies, where arrow indicates the prerequisite and circle indicates the subject matter.

Tools for suggesting learning resources’ links

A tool for suggesting links to learning resources based on knowledge structures has been implemented within the prototype “MyTeLeMap.” The current tool incorporates the designed knowledge structures and their associated learning resources (mainly html links). A graph visualization library (such as Graphviz (REF) and Microsoft Automatic Graph Layout (https://www.graphviz.org/)) display the graph nodes and edges from the knowledge database. The Google API is used to gather links from the web. The tool infrastructure is shown in Figure 8. Currently, a recommender system and a learning path service are under investigation (rounded rectangles with strong line).

Figure 8.

MyTeLeMap tool infrastructure.

Figures 9 and 10 show screenshots of the prototype MyTeLeMap tool in use. In Figure 9, the chosen knowledge structure is shown, and links associated with the selected node are suggested. The links are obtained from a Google search using the Google API, where the search keywords are extracted from the knowledge keywords in the selected node. The search results can be filtered by website (e.g. YouTube, Wikipedia). In Figure 10, the nodes are suggested based on the current selected node and previously visited nodes. The teacher role can manage all designed knowledge structures, as shown in Figure 11, and can create new knowledge structures as shown in Figure 12.

Figure 9.

Screenshot of the prototype MyTeLeMap suggesting study materials links.

Figure 10.

Screenshot of the prototype MyTeLeMap giving some suggestions.

Figure 11.

Screenshot of the Prototype MyTeLeMap listing All designed knowledge structures.

Figure 12.

Screenshot of the Prototype MyTeLeMap for managing a designed knowledge structure.

Questionnaire and interview: Design and results

A preliminary survey on teachers’ experiences in using and designing knowledge structures was conducted, comprising a questionnaire and an interview. For the questionnaire study, participants were 173 school teachers from Songkhla province, Thailand. The results showed that the teachers have used some tools for building such structures, such as a mind-mapping tool and the Cmap tool. Most of them (136 school teachers) recognized and used mind maps during their teaching activities. None of them represented or designed pedagogical structures with linkage among the knowledge components. They were interested in using a tool for building pedagogical knowledge structure (interested = 106, highly interested = 61) and were willing to use the structures and their applications as educational aids (interested = 101, highly interested = 62).

The interview study was conducted with 10 school teachers from the schools mentioned. Even though knowledge structures have been recommended for many years, school teachers still did not report using them in their teaching. However, the teachers felt that knowledge structures were useful and could benefit education more. For example, such structures could help learners think about and understand the relationships among the knowledge shown. On the teachers’ side, they thought that designing knowledge structures could be another way of sharing their tacit knowledge with other teachers.

Experiment: design and results

An experiment was conducted to compare whether learning using the prototype MyTeLeMap was better than learning by freely browsing. The pictorial representations of these two learning modes are shown in Figures 13 and 14.

Figure 13.

MyTeLeMap learning mode.

Figure 14.

Freely browsing learning mode.

The experiment was concerned with the second, “learning,” level of Kirkpatrick’s four levels of evaluation (Kirkpatrick, 2007). The participants were assigned to one of two groups: one group experienced the MyTeLeMap learning mode and the other group experienced the freely browsing learning mode. All participants were required to take a pre-test and a post-test, before and after experiencing the respective learning modes. The pre-test and post-test were the same for all participants, being a multiple choice test consisting of 10 questions as shown in Table 1. The scores obtained from the pre-test and post-test were compared for each learning mode.

The required sample size of this experiment was 12 according to G*power, using an effect size f = 1, an alpha error probability = 0.05, power = 0.8, the test family as F-test, the number of groups = 2, and the statistical test as ANOVA repeated measures. The actual number of participants was 40 as shown in Table 2.

Table 1.

Examples of questions in pre-test and post-test.

Knowledge	Questions in pre-test/post-test
Function	The keyword used to transfer control from a function back to a calling function is ___ ?
Operators	Suppose a, b, c are integer variables with values 5, 6, 7 respectively. What is the value of the expression NOT((b+c) > (a+10))?
String	How would you declare a variable to store the city of birth of a person?
Variable	Which of the following is a valid variable name?

Table 2.

Means and standard deviation of test scores.

Test_Type	Learning_Mode	Mean	Std. Dev.	N
Pre_Test	Freely_Browsing	3.8	0.83	20
	MyTeLeMap	3.9	0.93	20
	Total	3.8	0.87	40
Post_Test	Freely_Browsing	5.5	1.10	20
	MyTeLeMap	7.5	1.10	20
	Total	6.5	1.50	40

Figure 15.

Profile graph of mean ratings of test scores of pre-test and post-test for two learning modes (where Group 1 = MyTeLeMap and Group 2 = Freely Browsing).

The questions in the pre-test/post-test were based on selected knowledge of fundamental programming. The chosen knowledge was as follows:

Functions

Operators

String

Variables

A two-way repeated measures ANOVA was used to analyze the obtained test scores, in order to determine the significance of any differences in learning mode outcomes. “Learning mode” comprised two levels, freely-browsing and MyTeLeMap. “Test type” comprised two levels, pre-test and post-test.

Tables 2 to 4 show the descriptive statistics, the tests of within-subjects effects, and the tests of between-subjects effects. Figure 15 displays the profile graphs.

Table 3.

Tests of within-subjects effects.

Source	Type III sum of squares	df	Mean square	F	p.
Test_Type	140.45	1	140.45	154.48	<0.01
Test_Type × Learning_Mode	20.00	1	20.00	22.00	<0.01

Table 4.

Tests of between-subjects effects.

Source	Type III sum of squares	df	Mean Square	F	p.
Learning_Mode	22.05	1	22.05	20.36	<0.01
Error	41.15	38	1.08

The results from the statistical data obtained (as shown in Tables 2 to 4) were as follows.

There was no significant difference between the two groups of students at pre-test, suggesting that the students in both groups had similar knowledge of fundamental programming.

Both learning modes helped students significantly increasing their knowledge.

There was a significant interaction between learning mode and test types, such that MyTeLeMap helped students improve their learning significantly more than freely browsing.

From the experiment, it can be concluded that both learning modes helped learners in learning, however, MyTeLeMap helped significantly more than freely browsing.

Conclusions and future work

A knowledge-based system (MyTeLeMap) for suggesting study material links from the Web has been proposed in this research. The aim of the approach is to assist learners to achieve their desired knowledge. The system provides learners with suggestions based on their current and previously selected nodes in a given knowledge structure. A method of constructing the knowledge structure was also proposed. An experimental study showed that the MyTeLeMap system can support learning better than free browsing. The system depends on a search engine API which tends to change regularly, and this may affect the long-term usability of the application. Future work includes a personalized search engine, improvements to the system’s user interface and user experience, and some management features to allow teachers to attach their own study materials.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by Thailand Research Fund under contract number MRG5980096.

Author Biographies

Athitaya Nitchot is a lecturer and Vice director of Academic at Prince of Songkla University International College. She completed her PhD degree (Computer Science) in July 2012 from University of Southampton. Athitaya obtained a Bachelor of Science degree in Computer Science from Bangalore University, and a Master of Science degree in Web Technology from University of Southampton. She has got a scholarship from Thai government since her bachelor till PhD study. Her research interests are e-learning, self-study, web-based learning and competency model.

Wiphada Wetthayaprasit is current Director of Prince of Songkla University International College. She is alos a full-time lecturer under PhD in Computer Science program at Faculty of Science, Prince of Songkla University.

Lester Gilbert after graduating from the University of Cape Town, Lester Gilbert spent about 10 years as an academic in departments of Psychology in South Africa. He left to avoid the draft, and spent about 10 years in business in the UK working as a programmer, project leader, and IT manager. Lester re-entered academia in 1992 as a lecturer in IT subjects. He currently lectures on the “IT in Organisations” degree programme, jointly presenting introductory and advanced modules on eLearning and Technology Enhanced Learning. His book (co-authored with Veronica Gale) “Principles of e-Learning Systems Engineering” (Chandos, 2008) integrates my business-oriented practical experience of Information Systems development with Multimedia and Computer Aided Instruction development.

References

Abel

M-H

(2015) Knowledge map-based web platform to facilitate organizational learning return of experiences. Computers in Human Behavior 51(Part B): 960–966.

Bench-Capon

TJM

(2014) Knowledge Representation: An Approach to Artificial Intelligence. UK: Academic Press Inc.

Chiu

D-Y

Pan

Y-C

(2014) Topic knowledge map and knowledge structure constructions with genetic algorithm, information retrieval, and multi-dimension scaling method. Knowledge-Based Systems 67: 412–428.

Gilbert

Gale

(2008) Principles of E-learning Systems Engineering.

Gordon

Pathak

(1999) Finding information on the World Wide Web: The retrieval effectiveness of search engines. Information Processing & Management 35(2): 141–180.

Griffiths

Brophy

(2007) Student searching behavior and the web: Use of academic resources and google: Graduate School of Library and Information Science. University of Illinois at Urbana-Champaign, USA.

Hao

Yan

Gong

et al . (2014) Knowledge map-based method for domain knowledge browsing. Decision Support Systems 61: 106–114.

Hawking

Craswell

Bailey

et al . (2001) Measuring search engine quality. Information Retrieval 4(1): 33–59.

Kirkpatrick

(2007) Implementing the Four Levels: A Practical Guide for Effective Evaluation of Training Programs. 1 ed. San Francisco: Berrett-Koehler Publishers.

10.

Liao

S-H

(2003) Knowledge management technologies and applications – Literature review from 1995 to 2002. Expert Systems with Applications 25(2): 155–164.

11.

Liaw

S-S

Chang

W-C

Hung

W-H

et al . (2006) Attitudes toward search engines as a learning assisted tool: Approach of Liaw and Huang’s research model. Computers in Human Behavior 22(2): 177–190.

12.

Mayr

Tosques

(2005) Google Web APIs – an Instrument for Webometric Analyses?

13.

Nitchot

Gilbert

Wills

(2011) Designing XML schema of a competence structure from course intended learning outcomes. In: The 14th IASTED International conference on computers and advanced technology in education, Cambridge, UK, 11–13 July 2011.

14.

Pan

Hembrooke

Joachims

et al . (2007) In Google we trust: Users' decisions on rank. Position, and Relevance Journal of Computer-Mediated Communication 12(3)

15.

Sowa

(2000) Knowledge Representation: Logical, Philosophical, and Computational Foundations. Pacific Grove, CA: Brooks/Cole.

16.

Tang

JHK

(2006) Googling for a diagnosis – Use of Google as a diagnostic aid: Internet based study. BMJ (Clinical Research ed.) 333(7579): 1143–1145.

17.

Thelwall

(2003) Can Google's PageRank be used to find the most important academic Web pages? Journal of Documentation 59(2): 205–217.

18.

Tsai

M-J