Creative AI: A new avenue for the Semantic Web?

Abstract

Computational Creativity (or artificial creativity) is a multidisciplinary field, researching how to construct computer programs that model, simulate, exhibit or enhance creative behaviour. This vision paper explores a potential of the Semantic Web and its technologies for creative AI. Possible uses of the Semantic Web and semantic technologies are discussed, regarding three types of creativity: i) exploratory creativity, ii) combinational creativity, and iii) transformational creativity and relevant research questions. For exploratory creativity, how can we explore the limits of what is possible, while remaining bound by a set of existing domain axioms, templates, and rules, expressed with semantic technologies? To achieve a combinational creativity, how can we combine or blend existing concepts, frames, ontology design patterns, and other constructs, and benefit from cross-fertilization? Ultimately, can we use ontologies and knowledge graphs, which describe an existing domain with its constraints and, applying a meta-rule for transformational creativity, start dropping constraints and adding new constraints to produce novel artifacts? Together with these new challenges, the paper also provides pointers to emerging and growing application domains of Semantic Web related to computational creativity: from recipe generation to scientific discovery and creative design.

1. Introduction

The seminal paper by Tim Berners-Lee et al. [2] describes a vision of the Semantic Web with its main building blocks and enabling technologies: knowledge representation (KR) and automated reasoning, ontologies, agents. The motivating scenario of this paper, described from its first sentences, concerns automated, intelligent services delivered by intelligent (artificial) agents. These agents are capable of carrying out sophisticated tasks for users such as making an appointment with a physical therapist, taking constraints on schedules and routes into account. This is possible thanks to adding explicit, machine-readable semantics to the content of the Web for reasoning and interoperability.

From its early days, the Semantic Web has been largely related to KR, but also more broadly to artificial intelligence (AI). Semantic networks [42], as a form of knowledge representation, dating back to early days of AI, gained new attention (kind of ‘AI summer’ w.r.t. KR) by adding Web technologies (such as URIs) to them and mechanisms of inference based on formal semantics, which resulted in standards like RDF [53] (with its graph interpretation), OWL [24], and leading to nowadays knowledge graphs (KGs) [38] and the Semantic Web. Not only then it is linked data constituting the Semantic Web, but also linked semantics, linked knowledge, and linked services, enabling reasoning on the Web and intelligent applications.

1.1. From analysis to synthesis

As applications and services are the key to adoption of a given technology by users, what intelligent applications and services might be then facilitated and enabled with Semantic Web and semantic technologies in the next several years?

It comes with no surprise that Semantic Web research evolves influenced by major shift changes in knowledge engineering and AI. Early Semantic Web used mainly deductive reasoning, employing logic-based reasoning services. With growing amounts of data, this has later shifted to an increased interest in applying statistical approaches, i.e., inductive reasoning [11,32]. Both may be classified as analytic tasks, but lately, there is an increasing interest in other types of reasoning. Consider for instance generating justifications and explanations (for explainable AI), which may serve for debugging purposes, and which are closely related to abductive reasoning. Some of recently popular tasks deal with synthesis rather then analysis and aim to generate rather than only analyse artefacts. Domains previously reserved for humans, such as making creative designs and scientific discoveries, are increasingly being addressed by AI [13].

Can thus the Semantic Web, semantic technologies and resources facilitate creative AI, i.e. computationally creative systems that use AI techniques?

1.2. What is creativity?

Creativity, creative reasoning and creative problem solving have been researched in cognitive [6] and computational sciences [15]. Cognitive psychologists aim to understand the human creative process. In her influencial works, Boden [5,7] describes creativity as the ability to come up with ideas or artifacts that are new, surprising, and valuable. The former ones may be concepts, musical compositions, poems but also cooking recipes, or even scientific theories. The latter ones may be paintings, pottery, but also vacuum cleaners, engines, etc. Moreover, many researchers use the term ‘concept’ to refer to a range of things such as abstract ideas in arts, science, and in everyday life [6], and we will also use this term throughout the paper.

1.3. Computational creativity

Can machines be creative? Some time ago it was hardly believable. Ada Lovelace, arguably referred to as the first computer programmer, was reflecting on the Charles Babbage’s mechanical general-purpose computer, the Analytical Engine that it “has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform”. However, with the development of machine learning, it is not needed anymore to explicitly program machines that apparently have begun to reveal creative behaviours [1,13].

The computational creativity research area has emerged concerned with “computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative” [9]. Computationally creative systems perform various ‘generative acts’ that create exemplars, concepts, or provide an aesthetic evaluation for the generated artefacts. Such systems have already been built in various domains including not only art (fashion, entertainment), but also design and engineering (drugs, devices, processes, e.g. in software composition [37] or program synthesis [30]), and scientific discovery disciplines [27] or even for inventing recipes [48].

1.4. Motivating use cases

In the following, we briefly introduce illustrative use cases for the research in the Semantic Web for computational creativity, while Table 1 breaks them down to: their relation with the Semantic Web topics, (computational) creativity types (described in Section 2) and sample solution methods (a selection of them is elaborated more in Section 3).

Table 1

Three illustrative use cases, their relation to Semantic Web topics, (computational) creativity types and sample solution methods

Use case	Recipe generation (e.g. generating culinary recipes)	(Collaborative) Scientific discovery (e.g. drug discovery)	Creative design (e.g. product design such as furniture, home applicances or video game design)
Web	Websites providing recipes, recommender systems, recipe ratings in social media	Scientific papers, social media (side effects)	Social trend analysis (fashion), current events, composition and interoperation of Web services
Knowledge resources	Ontologies and knowledge graphs (of ingredients, their types, functions etc.), procedural knowledge	Ontologies and scientific KGs (on compounds, genes and diseases etc.), Research Objects	Ontologies and knowledge graphs (of parts and components, their types, functions etc.)
Reasoning	Constraint-based reasoning (which ingredients are not incompatible with dietary recommendations?), inductive reasoning, deductive reasoning	Abductive reasoning (hypothesis abduction), analogical reasoning, inductive reasoning, deductive reasoning	Constraint-based reasoning (constraint solvers), Cased-Based Reasoning, inductive reasoning, deductive reasoning
Data integration	Chemical databases (fragrance), medical KGs (dietary recommendations), nutritive value, units of measure etc.	Chemical, pharmaceutical, medical KGs etc.	Material databases for product design or multimedia and cultural heritage KGs for video game design etc.
Provenance & trust	Won’t this food cause an allergic reaction?	Won’t this drug cause adverse effects?	Is this material non-toxic? won’t this game offend a player?
Multi-agent systems	Constraint elicitation and negotation	Human and robot scientists	Human and robot designers
Creativity type	Exploratory, combinational (conceptual blending)	Exploratory, combinational (bisociation discovery), transformational	Exploratory, combinational, transformational
Sample methods	Generative models (machine learning against Web or Semantic Web recipes), querying remote resources, constraint programming, evolutionary computation	Bisociation discovery, graph mining, inductive logic programming, structure prediction	Generative models (generative design), evolutionary computation, constraint programming

Recipe generation The goal is to generate a recipe given the list of desired and excluded ingredients. This goes beyond simply retrieving a recipe based on the specified conditions (ingredient lists), as existing databases may not contain one meeting the conditions. In case of culinary recipes, some ingredients may be desirable, for instance, because they are in the user’s fridge, while others may have to be excluded because they are allergens. The recipe cannot be completely random, it must be plausible also regarding taste and smell. A new recipe may have to be aligned to a particular cuisine or a chef, mimicking his or her style. Techniques that may be used for performing this include data mining/machine learning against Web or Semantic Web recipes, querying remote resources or applying constraints regarding the resulting recipes, and finally blending (mixing) available recipes.

Collaborative scientific discovery Another scenario concerns a creative process of generation of plausible hypotheses from observations. Scientists in their work use to invent hypotheses to explain phenomena, and then design and apply methods to either verify or falsify them. Imagine a computer system not only able to verify hypotheses (which is now increasingly being done with machine learning predictive models), but also to abduce novel hypotheses or to find novel, surprising connections between domains to facilitate scientific discoveries. A seminal example of a system capable of abducing and testing novel hypotheses is Robot Scientist, which originates novel hypotheses in functional genomics and has been shown to make scientific discoveries [27] with use of expressive ontologies to describe the domain of interest and use this formal, machine-readable description for automated reasoning to automate scientific experiments.

Creative design Imagine that as a designer you are able to just tell a computer what you want to design, e.g. that you want to design a table, which uses material X and costs no more than Y, and has weight Z, and tell about the style you like or that is compliant with your company’s aesthetics. In response, the computer produces thousands of new solutions, meeting your requirements and that are also easy to manufacture as they consolidate parts. So called generative design makes it increasingly possible and enables designers and engineers to collaborate with machines to co-create new products, which are not only novel but also more effective in terms of time to produce or impact on the environment. To achieve this, evolutionary computation may be used in computational design, where the generic approach is first to parameterize the topology of an underlying knowledge structure and then use a genetic program to modify it. More challenging example use case than designing a table is video game design as it also must integrate various multimedia data and complex artifacts such as, for instance, a monster or a narrative and must incorporate social and cultural context. This case may involve a team of human and robot designers who exchange creative ideas and solutions using languages shared between humans and machines.

2. Three types of creativity: Opportunities for the Semantic Web

The best known categorization of creativity types is by Boden [5], where three types of creativity are defined: (i) exploratory, where new ideas are generated by exploration of a space of concepts, (ii) combinational, which concerns new combinations of familiar ideas, and (iii) transformational, where the space is transformed what facilitates new kinds of ideas to be generated. Other formulations have also been proposed, including extending the Boden’s categorization to also include approaches for extraction and induction of concepts as additional ways of concept creation by Xiao et al. [54]. In particular, Wiggins [52] proposes a unifying formalization of creativity as search, which unifies the categorization of Boden and from [54]. Combinational and exploratory creativity are defined there as search at the concept level, and transformational creativity as search at the meta-level.

2.1. Exploratory: Generation of new ideas by exploration of a space of concepts

Exploratory creativity refers to search within a pre-defined search space (limited by rules, constraints etc.). It is often modeled as an objective-driven search, using techniques such as constraint satisfaction, evolutionary algorithms, and data mining [36].

Regarding data mining, one may notice that its definition as the nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data [20] has commonalities with definitions of computational creativity. Indeed, various techniques of data mining have found their applications in computational creativity, for tasks such as concept creation [47].

Potential for the Semantic Web Ontologies and knowledge graphs may provide conceptualizations for the given domain, including its constraints. As such, they serve to define the search space for generating novel concepts. Use cases such as generating novel recipes also concern procedural knowledge, thus one may pose research questions such as: Are existing ontologies and knowledge graphs sufficient to effectively support creative computing? or What other semantic resources are needed to fuel computationally creative systems?

Regarding methods, concept induction [18,35] and pattern mining [33] have been active areas of research in data mining in the Semantic Web context [32]. Many of these approaches use so-called refinement operators, i.e. functions that ‘traverse’ the search space and generate specializations or generalizations of concepts. For instance, an operator may add a primitive concept as the new conjunct to a complex concept (being an intersection of concepts), replace a primitive concept with its (primitive) subconcept, or add an existential restriction. Those refinements are further evaluated regarding their quality. To assess the quality of generated candidate concepts various measures can be used, not only based on frequency or predictive quality but also such that promote diversity [41] or novelty. The further interesting research question to study would be then: What properties should have refinement operators to support exploratory creativity on the Semantic Web? and What research is needed to define quality measures and evaluation procedures for concept creation with use of Semantic Web technologies that promote novelty?

2.2. Combinational: Novel combinations of familiar ideas

Creativity, understood as unfamiliar combinations of familiar ideas, dates back to the notion of bisociation by Koestler in 1964 [28], who describes creativity as a result of combining distinct frames of reference. The work of Koestler was followed by a subsequent cognitive theory of conceptual blending [19].

2.2.1. Conceptual blending

Conceptual blending is a process of inventing a novel concept (the blend) by combining two familiar input concepts. The framework of conceptual blending proposed by Fauconnier and Turner [19] concerns so-called mental spaces that connect schematic knowledge and frames representing the organization of elements and relations of the familiar knowledge. In the center of the conceptual blending theory, there is a conceptual integration network, which contains such elements as: (i) input spaces, (ii) a generic space (with a structure being an abstraction of commonalities of all the spaces of the system), (iii) a blended space, containing chosen aspects of the structures from the input spaces and its own, created structure, (iv) a partial mapping, connecting chosen aspects of the models in the input mental spaces. This basic framework may be also extended to include additional structure in the blend, that is not copied from the input spaces, via composition (which involves, possibly partial selection of elements), completion, and elaboration.

One of the classical examples of conceptual blending concerns the concepts of house and boat (e.g. [16,19,22]. Figure 1 illustrates one of the possible results, which is a house-boat concept (another example could be a boat-house concept).

Fig. 1.

The houseboat blend (adapted from [16,19,22]).

Various formalisms have been used to represent input spaces, including concept maps, frames, rules and constraints by Pereira [39], Prolog and micro-theories as in the system Divago [40], semantic networks used by Veale and Donoghue [50], description logics by Confalonieri et al., [10], Distributed Ontology Language by Kutz et al. [31], and algebraic specifications by Eppe et al. [16]. Not only concepts may be blended, but also ontologies, as proposed in [31].

The computational challenges associated with conceptual blending are: (i) to compute a generic space (representing what is common to the two input spaces), which can be later specialised to produce meaningful blends with elements from the input concepts and (ii) to ensure that there are no inconsistencies by combining concepts in a too naive and arbitrary way.

Genetic algorithms and neural networks can be used to generate blended concepts, capturing a combination of the inputs.

Potential for the Semantic Web There are multiple options for Semantic Web research in the area of conceptual blending. There have already been proposals to use the Web as a source of background information to generate blends, such as ‘conceptual mash-ups’ proposed by Veale [49]. Can thus these ideas be taken further, and can Linked Data and knowledge graphs be sources of vast amounts of (already structured) knowledge for producing blends? How can we combine or blend existing concepts, semantic frames, and other constructs, and benefit from cross-fertilization? Could we exploit (and how) Ontology Design Patterns [ 4 , 23 ] to represent a generic space?

When input spaces are being combined, another challenge is to compute a generic space automatically, especially for expressive representation languages, and many proposed blending approaches are not capable of it. Can thus automated approaches be developed for computing a generic space automatically, leveraging of reasoning services developed within the Semantic Web area and aimed to compute a most generic concept or of generalisation refinement operators? Even after the generic space is identified, there remains a challenge of a large number of possible combinations to generate blends. Some of them need to be pruned, and, besides using quality measures, can also consistency checking be used and how to prune blends?

2.2.2. Bisociation

The term bisociation was introduced by Koestler [28] to describe the creative act in humor, science and art. It stands for a blend of bi- + association. An association represents a relation between concepts within one domain, and bisociation fuses the information from multiple domains. Elements that are blended are taken from two (previously) unrelated ‘matrices of thought’ (or domains) (see: Fig. 2) to form a new matrix of meaning, applying processes such as abstraction, categorisation, analogies and metaphors, and comparisons.

Fig. 2.

A concept of bisociation, illustrated with a solid line connecting concepts ${\mathit{c}}_1$ , ${\mathit{c}}_2$ viewed in two different matrices of thought or from two different knowledge bases, versus an association, illustrated with a dashed line, which connects concepts from one matrix of thought or a knowledge base (adapted from [14,28]).

Fig. 3.

A BisoNet with a bridging concept (a concept connecting dense sub-graphs from different domains).

Since bisociative thinking occurs when a problem, idea etc. are viewed in two (or more) ‘matrices of thought’ or domains, to find bisociations it requires to integrate data from different domains. Bisociative Networks (BisoNets) [14] have been proposed as a method to compute Koestler’s bisociation, and to semantically integrate information. BisoNets are based on a k-partite graph structure with nodes representing units of information or concepts and with edges representing their relations. Each partition of a BisoNet contains a certain type of concepts or relations (terms, documents etc.). Kötter et al. [29] discuss three different kinds of bisociations: bridging concepts that connect dense sub-graphs from different domains (see: Fig. 3), bridging graphs that are sub-graphs linking concepts from different domains, and bridging by graph similarity, where bisociations are represented by structurally similar sub-graphs of different domains.

Once a BisoNet is formed, it can be mined for novel, and interesting information, and patterns (bisociations) to support creative discoveries. So-called creative information exploration aims to explore large volumes of heterogeneous information to discover new, surprising and valuable relationships in data that would not be mined by conventional information retrieval and data mining approaches [14]. The discovered links represent non-obvious connections and domain-crossing links, where concepts from various domains are not commonly related. One ‘classic’ example from literature of such non-obvious link regards connecting magnesium and migraine. There had been a body of articles on how migraine can be treated with calcium blockers, and another body of articles (not connected with previous ones) describing how magnesium works as a calcium blocker, but the potential to treat migraine with magnesium had not been realized [26].

Such connections may be discovered by graph mining and analysis techniques.

Potential for the Semantic Web The Semantic Web and Linked Data coincide with the model of BisoNets as heterogeneuos information networks, integrating concepts. How then the modeling choices of Linked Data may impact creative information discovery? Could thus the ideas and methods developed for creative information exploration be used to mine multi-domain Linked Data and vice-versa, i.e. can link discovery approaches developed within Semantic Web research be applied to creative information exploration? Mining potential novel associations between Linked Data [51] have already been explored within Semantic Web research. Can this be taken further? Is research on ontology mapping for bridging domains also relevant here? How to compute which nodes in the Semantic Web would bridge domains in creative ways?

2.3. Transformational: Transforming the search space

Transformational creativity may be seen as meta-search, i.e. search not only for concepts, but also for rules, that is modifying rules and constraints and the search method. Transformational creativity takes place when the search space itself is also modified. The result are novel concepts in the modified space. This is the most difficult type of creativity to implement in a computational system, as one may argue that if the system is not equipped with some autonomy to change the rules, constraints or even the goals, nor it can be influenced by external information (beyond what a programmer equipped it with) then it only expresses the programmer’s creativity [25]. Therefore, some form of creative autonomy, i.e. when a system not only evaluates creations, but also changes its standards without being given an explicit direction [25], is required for transformational creativity.

But what inputs can a creative system receive to modify its standards? Toivonen [46] points that those may be: i) introspection, and ii) social interaction. As an example for the former take a system able to write songs, which modifies its own goals and operation [45]. It uses constraint programming, where constraints are used declaratively to define a search space of songs. Consequently, a standard constraint solver can then be used to generate songs. A meta-level control component manipulates the constraints at runtime based on self-reflection of the system. Regarding the latter, social interaction, it can be a source of new influences, ideas and feedback, and for developing creative autonomy it can be more plausible if the system is embedded in a broader society of other creators [25].

Potential for the Semantic Web Semantic Web technologies serve well to describe domain and cross-domain knowledge with making explicit constraints existing in a domain. Can we use ontologies and knowledge graphs, which describe an existing domain with its axioms and constraints and, applying a meta-rule for transformational creativity, start dropping constraints and adding new constraints to produce novel artifacts?

One transformatory assumption regarding reasoning in OWL and on the Web versus relational databases is to assume ‘open world’ rather than ’closed world’. Can we also change some other assumptions underlying reasoning on the Web to obtain novel problem settings and surprising and useful results?

The Semantic Web, envisaged as a multi-agent system with all of its technologies provides an opportunity for developing autonomous, creative agents that socially interact to gather new influences and ideas. They need to communicate using common languages and conceptualizations, shared between humans and machines to maintain common conceptual spaces. Due to this setting, the area of transformational creativity provides a big research opportunity to the Semantic Web specifically. How thus we should model and incorporate into a common conceptual space influences from other agents, e.g. other designers and customers, their preferences and aesthetics?

3. Research directions

In the previous section we discussed research questions regarding the potential of the Semantic Web with respect to three types of creativity. In this section, we gather and indicate promising research directions incorporating Semantic Web technologies with respect to particular areas of artificial intelligence.

Bisociation discovery Bisociation discovery requires development of methods for cross-domain link discovery that go beyond simply linking a pair of single resources in that they should also discover bridging concepts (connecting dense sub-graphs), bridging graphs (sub-graphs linking concepts from different domains) or find structurally similar sub-graphs of different domains. This may require detection of domain-crossing sub-graphs. Such connections may be discovered by graph mining and analysis techniques, and development of similarity measures to compare sub-graphs of knowledge graphs.

Evolutionary computation So far, the use of evolutionary computation techniques within Semantic Web is rather scarce with some exceptions like [17,34]. Genetic programming requires defining operators such as mutation, crossover or selection according to a given fitness measure. Hence research on adequate genetic operators, that exploit domain knowledge and are semantics-aware, is an interesting research direction. Here research results on refinement operators for knowledge structures may be of interest as a starting point for developing mutation operators, and on generating (conceptual) blends useful for developing crossover operators.

Generative models Since creative artefacts should be both novel and useful, creative computational systems commonly work in two phases (conforming to psychological models of creative generation by humans [21]): generation of novel constructs and their evaluation. Useful constructs may be produced by so-called generative models, i.e. models learned from observed data and capable of generating samples sharing similar properties with those of the dataset on which they were learned. For instance, if such data mining/machine learning would be applied against recipes found on the Web then it should enable generation of new recipes with similar properties.

Consider models learned from a dataset of knowledge graphs. Such models can prove useful in many applications, e.g. in drug discovery where sampling may help to discover new configurations or chemical design. However, the research on generative modeling from observed data even of arbitrary graphs is scarce [55]. The problem is challenging due to non-local dependencies that exist between nodes and edges in a given graph which make it hard to model distributions over graphs and their complex relationships, and it becomes even harder when semantics of nodes and edges should be taken into account. Especially deep generative models (i.e. that use deep learning) of knowledge graphs constitute an interesting topic for future research.

Analogical reasoning and case-based reasoning Analogical reasoning consists of transferring and using knowledge learned in one situation to another one, which was not an original target. It commonly focuses on cross-domain structural similarity. The Case-Based Reasoning (CBR) is a related paradigm, but here the solutions are transferred between semantically similar cases within one domain. The idea behind CBR is to use previous problem situations to address new problems, with an assumption that similar problems have similar solutions. The CBR approach consists of four phases [12]: retrieve (similar experiences: situations and cases), reuse (past experiences in the context of a new situation), revise (producing new solution) and retain. Cases may be retained as concrete examples, or a set of similar cases may constitute a generalized case. A sample CBR system which uses ontologies published as Linked Data interlinked with its case model is a tool called myCBR [44].

Though CBR is mostly concentrated on instance analogy and design patterns are abstractions, CBR has commonalities with Ontology Design Patterns, and more generally with Semantic Web patterns in aiming at reuse of knowledge and experiences. The CBR viewpoint has already been combined with the use of patterns in the OntoCase approach to ontology construction [3]. For the applications in computational creativity (e.g., in design, creative problem solving), the area of ODPs and Semantic Web patterns requires further research to introduce more automation at all of the phases, e.g. extracting more structured knowledge representation of a pattern (case), finding and matching similar patterns, automated revision and merging.

Evaluation measures Another research direction is development of new measures for evaluating creative artefacts. Various measures have already been proposed such as novelty, interestingness, surprise, usefulness, elegance (see [8,43] for a starting point). In the context of the Semantic Web, not only such measures are interesting that are local to the system and involve so-called P-creativity or personal creativity (concerning artefacts new to the system) [6], but also such that evaluate creative artefacts in the social and global context (and involve so-called H-creativity or historical creativity [6], i.e. concerned with creating artefacts recognized as novel by society).

4. Conclusions

The intention of this paper was to point to under-explored and rising opportunities for Semantic Web research in the growing area of creative AI. We have briefly surveyed the domain of computational creativity, with specific focus on aspects relevant to the Semantic Web research: Web, knowledge resources, reasoning, data integration, provenance and trust, and multi-agent systems.

We conclude that there is a lot of potential for future research in the Semantic Web for creative AI. This includes: (i) knowledge representation languages to represent concepts in a broader sense (e.g., procedural knowledge to represent ideas such as culinary recipes), (ii) cross-domain mapping discovery (bisociations), (iii) machine learning (generative models) and data mining approaches, including their building blocks such as refinement operators, (iv) evolutionary computation techniques and their buliding blocks (genetic operators) (v) reasoning services beyond deduction (e.g., Cased-Based Reasoning), (vi) metrics for assessing creatively computed artefacts, (vii) knowledge resources in domains such as art and design, scientific discovery and others.