Abstract
Many cyber-physical systems face the challenge of appropriately integrating domain-specific human expert knowledge into the cyber part to create a shared sphere of knowledge and intelligent interactions between humans and the semi-autonomous technical system. Cognitive engineering contributes methods and insights into higher-order cognition that help to embed human knowledge in an appropriate way. The original research introduces a novel transdisciplinary framework called Human-CoMo, which demonstrates a systematic modelling process, different human perspectives, and the integration of expert knowledge at multiple hierarchical levels. Fundamental principles inspired by human cognition, such as conceptual chunking and knowledge precision, are characterised. Furthermore, it is shown how knowledge hierarchies can be methodically reflected in appropriate data analysis and modelling levels for small and big data applications including artificial intelligence approaches. Combined knowledge- and data-based modelling approaches offer more flexibility to integrate the strengths of humans and technology in a complementary way. The cognitive foundations and their computational reflections are outlined for the technical example process electroplating from the field of materials and surface engineering. The possibilities and limitations of integrating human knowledge through formalisation and implications for future forms of human-machine interaction are discussed.
Keywords
Introduction
Why Human-Cyber-Physical Systems (HCPS)?
The conceptualization of technical systems that use powerful capabilities for integrating calculations and real physical processes has been discussed for more than 25 years under the term “cyber-physical systems” (CPS; Lee, 2008). CPS is a technology-oriented concept that differentiates technical systems into a cyber part (C) and a physical part (P). CPS should contribute to overcoming global challenges related to human society such as energy shortages due to global warming or providing healthcare service for aging world's population (Sha et al., 2008). Hence, the pure technical development stepped forward (Horváth, 2022a) to integrate human and social issues in a human-part (H) as well leading to human-cyber-physical systems (HCPS) (e.g. Bocklisch et al., 2022; Liu & Wang, 2020; Zhou et al., 2019) or cyber-physical-social systems (e.g. Wang, 2010). At the same time, fields of application were differentiated, including, for instance, medicine and healthcare, mobility and transportation, energy systems and electric power grid, smart home, agriculture as well as intelligent manufacturing and robotics (Babris et al., 2019; Khaitan & McCalley, 2014). Whenever humans are to work together with cyber and physical elements, the fundamental differences between man and technology must be considered in the design of HCPS. This transdisciplinary endeavour has to overcome several gaps. First, many different interests, viewpoints and specialist areas of those involved must be integrated (see Figures 1 and 2 for examples of different human roles and viewpoints related to HCPS design). Second, the current global challenges, such as the transformation to more sustainable societies (cf. United Nations sustainability development goals) both favour and hinder this integration process. On the one hand, the problems can only be overcome together; on the other hand, agreeing on common goals and approaches – even if it is “only” about scientific questions – is a difficult process. The tendency to see one's own point of view as the (only) right one and to optimise one's own advantage must be replaced by a cooperative way of thinking that is based more on the appreciation of diversity and mutual help. Otherwise, the overarching objectives cannot be achieved.
Transdisciplinary framework
Brief characterisation of examples of different human viewpoints on HCPS-challenge (left) and specification of human roles concerning the model building process.
The multiple perspectives are available in many application areas, for instance, in the field of engineering and intelligent manufacturing. Due to its importance in the production of a wide range of technical products, including high-tech innovations, materials science and engineering (MSE) is selected here as a good example of an application area. Additionally, MSE is currently undergoing dynamic changes in order to digitally capture the entire life cycle of materials. As this is a transdisciplinary field, involving experts from physics, chemistry, crystallography, engineering, etc., this integration is very challenging (Bayerlein et al., 2024). For this reason, an exemplary coating technology (electroplating) from the field of MSE is chosen here to concretise the new transdisciplinary framework (see Figures 1 and 2) and the objectives of the work based on experimental investigations and data (see below). There is a similar need for research in other manufacturing processes such as welding, machining, or forming. The same applies to product development, for example in the design phase. These are not dealt with here.
Due to HCPS challenges such as in MSE, digitisation and the change of mind-set towards cooperative thinking must now take place under great time pressure. Furthermore, the transdisciplinary concepts and methods agreed upon for technology design and implementation must not be merely theoretical or too abstract in nature. Instead, they should have sufficient conceptual strength to be effective in reality (practical and problem-driven approach). Success needs to be measured by (long-term) changes, for instance, regarding HCPS functionality and security. Impact assessment should take place at various levels in the early stages of technology development and explicitly address aspects of human (psychological) well-being including understandability of, trust in and threat or expansion of human identity through CPS-functionalities (Selenko et al., 2022). HCPS development need to emphasise psychological facets because the development of cyber parts based on artificial intelligence (AI) will augment and challenge human cognitive abilities in a yet unprecedented way. The positive and negative consequences of new forms of human-technology interaction (e.g. human-machine teaming: Bocklisch & Huchler, 2023; Christopher Brill et al., 2018; Madni & Madni, 2018) for human integrity and society are currently difficult to assess but need to be part of basic research (Eich et al., 2023) and applied technological development (Lu et al., 2022; Schmid & Wiesche, 2023). Cognitive engineering provides aspects and methods important to support the human-centred design and development processes of HCPS.
Why Cognitive (Systems) Engineering for HCPS?
Cognitive engineering (CE) – or “human problem solving with tools” (Woods & Roth, 1988) – is “a type of Cognitive Science, trying to apply what is known from science to the design and construction of machines.” (Norman, 1986, p. 32). It includes the understanding of fundamental principles behind human performance (e.g. cognitive information processing capabilities) to devise systems pleasant to use. Specifically, CE emphasises the consideration of the human user and his tasks carried out with the help of technical systems, as central to the specification of the system design. Cognitive systems engineering (CSE) is a variant of CE offering a broader systems perspective to the analysis and design of human-machine systems (Endsley et al., 2007) that may help to bridge the transdisciplinary gaps for HCPS development (see above). In this paper, CE and CSE are used synonymously. The CE/CSE design approach addresses research questions that are of interest to many areas and that are raised in parts by “technical” systems engineers as well (Denno, 2024). These include, for instance, the analysis and control of complexity and risk (Nilchiani, 2023) but from a human-centred perspective. Example questions of a cognitive (systems) engineer are “What knowledge and cognitive strategies do practitioners use to cope with complexity?” or “How should automated tools look like that facilitate human work tasks?” (Roth et al., 2002). The traditional focus of CE/CSE needs to be broadened. For the development of HCPS the user is crucial, but not the only relevant human perspective that needs to be considered for HCPS design (cf. Figure 2 for details concerning other human viewpoints). In the following, the previously limited CSE perspective is further developed into a transdisciplinary framework model (Human-CoMo, see below). The most important innovations of the presented work are:
the clear representation of several human roles and perspectives that are important in the development of HCPS (transdisciplinary focus). It represents a new theoretical issue in next-generation CPS research. the remit of the CSE expert is expanded by using cognitive knowledge engineering strategies and cognitive principles for the design of the C-part. In this way, human-centred modelling is achieved, and methodological issues are addressed.
The innovations are demonstrated using the example of MSE electroplating.
Paper Outline
The paper is organised as follows: Section 2 introduces the transdisciplinary framework named
Transdisciplinary Framework Human-CoMo of Human-Centred Modelling for HCPS
Modelling plays a very important role in the overall design process of HCPS as models of the real technical process and human cognition are to be implemented in the C-part. Depending on its maturity level, assistance, advanced automation, or even human-machine teaming can be created. The model building process is usually structured in different phases. Schlesinger (1979) as well as Dullen and Verma (2023) proposes three main model-building phases starting with (1) the real-world challenge followed by (2) a conceptual model and (3) a computational model.
Figure 1 presents these basic phases within the Human-CoMo framework for HCPS design (middle) and relates the different human roles and viewpoints. The illustration in the middle shows the three phases (real-world challenge, conceptual model, computational model) and their interrelations relevant for HCPS design and model building. The focus of Human-CoMo is on the cognitive perspective of the H-part, not on the physical challenges. For example, it can include higher order cognitive processes such as memory, decision-making or problem solving that are relevant in the real HCPS challenge. The C-part can be based on various computerised representations such as AI algorithms (see Section 5.1 for examples). The performance and behaviour of the P-parts can also be simulated or modelled, but this is not the subject here.
Using Human-CoMo, the cognitive knowledge engineering process as well as support of human-centred modelling can be planned and carried out more effectively. Figure 2 (left) briefly characterises the human roles (no. 1–4) based on examples for the mentioned application area “engineering”/”smart manufacturing”. Other roles such as no. 5 (entrepreneur) and no. 6 (customer) are not described in detail here. Nevertheless, these aspects can significantly influence the development of HCPS in practice (e.g. regarding cost-benefit considerations).
Domain experts (no. 1) are crucial as they have relevant basic knowledge (e.g. in the fields of engineering, manufacturing technology, natural sciences such as physics or chemistry). Therefore, cognitive system engineers (no. 4, CSE experts) work together with them in the first phases of the cognitive knowledge engineering process (see Figure 2, middle: “knowledge elicitation”, “knowledge analysis and visualization”, “knowledge verification”). The result of the cognitive knowledge engineering process is the conceptual model (see Figure 1, bottom left). This process differs from other knowledge engineering approaches (e.g. de Hoog, 2019; Liou, 2019) in that it more explicitly integrates knowledge about human cognition (cf. the principles described below in Section 4) and the human-centred design of technical systems. More precisely, the CE/CSE expert looks at the different phases of the HCPS-design process from “outside the box” (see Figure 1) with a focus on general human cognitive processes and their implementation in computational models and human-machine interaction. He helps to select and utilise the right methods and evaluation criteria – not only for knowledge elicitation but for knowledge formalisation, modelling, and transparent result presentation as well (cf. Figure 2, right: “knowledge formalization” and “modelling and computational result analysis”). For instance, computational models should be cognitively understandable as potentially suboptimal but explainable algorithms are to be preferred to black-box optimisations (Madni & Jackson, 2009). Hence, a close cooperation between CSE expert and computational modelling expert (human role no. 2) is necessary as well. Modelling experts (cf. Figure 2, left) possess knowledge related to effective data modelling but usually are neither domain experts for the technical process (role no. 1) nor for human cognition (role no. 4). They also do not use the systems they develop. The HCPS user (role no. 3: user) is not necessarily a qualified expert for the technical process considered nor a modelling specialist. Often, he just operates and monitors the technical system in order to complete the real-world working task in a good way.
Unfortunately, the user-centred design (UCD) approach is often not sufficiently successful because it lacks a differentiated view of the many human roles by focussing only on the user perspective (cf. Figure 2, right part: USD phases “user needs and preferences” as well as “transfer and evaluation”). Self-evidently, the user is of enormous importance but in the design process itself a variety of different technical experts need concrete design advice for human-cyber-integration that UCD methods cannot provide in depth. Even purely technical approaches sometimes fail to achieve the desired goals in a short space of time because they lack the other facets (e.g. modelling expertise). Where there is already a good understanding of the problem with regard to domain expertise and digitisation, such as in the field of MSE, important cognitive aspects are missing. For reasons of cost and because suitable alternative digitisation approaches are lacking, some approaches to creating formal knowledge representations (ontologies) are limited to the description of intermediate knowledge levels (see Figure 3 levels “subordinate categories” or “measurable variables”) that can be more easily standardised (Bayerlein et al., 2024). This can lead to a loss of the depth of expert knowledge (see Section 4) and the ability to customise the C system to the individual human user, resulting in a formal uniformity that is not very realistic or human friendly.
Basic principles to be considered when structuring expert domain knowledge for the design of HCPS: (I) hierarchical organisation of knowledge, (II) coping with knowledge complexity through conceptual chunking and (III) knowledge precision.
The CE/CSE perspective is important for the holistic integration of the complex HCPS design process. Principles and methods origin from cognitive psychology and human factors. If this perspective is not explicit enough – which is still the case in many design processes – transdisciplinary and human-centred integration is insufficient. Undesired consequences in HCPS implementation may appear such as low HCPS performance, poorly calibrated user trust in automation (Kohn et al., 2021), inability to master system complexity (Nilchiani, 2023) and the occurrence of automation ironies (Strauch, 2018). Brainbridge clearly highlighted the field of tension in which both, system designer (including domain and computational experts) and operator (user) find themselves: “
In real HCPS applications, there is undoubtedly a high degree of complexity. Weaver (1948) differentiates between problems of (1) simplicity (low number of variables), (2) organised complexity (substantial number of interrelated variables) and (3) disorganised complexity (many variables showing individual behaviour; relations are unknown, stochastic or “erratic” in nature). Nilchiani (2023) points out that the type of system complexity– here HCPS – needs to be established before system characterisation or even modelling (cf. Figure 1: conceptual characterisation and computational modelling). An HCPS can be specified as system with parts showing organised and others showing disorganised complexity and emergent behaviour that is not easy to understand and explain. Theoretically, it is desirable to engineer systems that possess high levels of organised complexity because this enables system self-adaptation (Nilchiani, 2023). The approach of explicitly describing principles to structure human expert knowledge in an adequate way is chosen in this paper to describe a part of organised complexity. It is also an objective to transfer this into modelling (see below). From a CE/CSE overarching point of view, the cognitive knowledge engineering process helps to develop the conceptual model. Thereafter, knowledge formalisation follows to establish the computational model. Hence, the human perspectives no. 1 (domain expert), 2 (computational expert) and 4 (CSE expert) are outlined in the following in more detail with the help of the example use case. The user (role no. 3) and the UCD-process are not the focus here (for more specific information concerning UCD-principles see ISO standard 9241-210:2019).
The Research Challenge of Integrating Human Expertise into Cognitive Knowledge Engineering and Data-Based Modelling Processes
Expertise is domain-specific and expert skills cannot be transferred easily to other contexts (Phillips et al., 2004). Therefore, the intended general applicability of research models, methods and techniques in the context of HCPS (see research questions below) must always be concretised in order to achieve a clear practical impact. The importance of “context” (= given CPS field of application, real-world situations, tasks, goals and constraints) is one key aspect highlighted by CE/CSE research (Woods & Roth, 1988). The challenge to combine both, general applicability, and specific usefulness, is evident and there is only little research with this objective. Bocklisch and colleagues (2024) present a complementary bottom-up and top-down framework that integrates abstract and concrete levels for multi-criteria decision making related to HCPS research. They show that CE/CSE methods to elicit human expert knowledge are clearly beneficial for designing AI-based hybrid decision-support. For the example coating technology “atmospheric plasma spraying”, conceptual as well as computational model-building is shown (Bocklisch et al., 2024). Wang and colleagues (2020) illustrate the evolution of HCPS from HPS for intelligent welding. The framework presents different steps towards modelling and reviews different AI-methods including their advantages and types of tasks (e.g. complex problem solving, monitoring, classification). Although the human operator (role no. 3) is mentioned as an important part of the system, the relationship between expert knowledge and modelling (human roles no. 1 and 2) is not described in detail.
The multi-layered domain expert knowledge has not yet been sufficiently considered in modelling. The advantage of a CSE-related approach is that the two model building steps “cognitive knowledge engineering process” and “human-centred modelling process” (cf. Figure 2) can be addressed much more clearly in theoretical and methodological terms:
Cognitive knowledge engineering process:
Which cognitive principles of human domain expertise need to be considered for HCPS design? (see Section 4.1.) How can expert knowledge be appropriately prepared and structured for a practical HCPS example? (see Section 4.2.) Human-centred modelling process:
How can elicited expert knowledge be combined with data-based modelling approaches to implement domain expertise in the cyber part? (see Section 5.1) How can the computational model be established based on technical experiments and data for a practical HCPS example? (see Section 5.2)
Cognitive Knowledge Engineering Process
Cognitive Principles for Structuring Expert Knowledge for HCPS
Experts are individuals that achieve exceptional skills in one particular domain (Phillips et al., 2004). The definition is closely related to seniority (e.g. more than 10 years experience) and peer nomination. In CSE approaches based on cognitive information processing paradigm and memory research, knowledge stored in long-term memory comprises two main facets: (1) declarative knowledge vs. (2) procedural knowledge (Roediger et al., 2014; Squire, 1987). While the first refers to “what is known” and is related to proper problem representation, procedural knowledge refers to “how things are done”. For instance, routines and strategies used by experts or the ability to perform mental simulations and make predictions.
In the following, the focus lies on declarative expertise because it is the foundation on which experts can excel. Novices are in a “hypocognitive” state in which they lack concept-based knowledge (Wu & Dunning, 2019). Therefore, their performance in information processing differs significantly from that of experts. In the following, three basic principles of concept-based declarative knowledge formalisation are discussed. These principles need to be considered in a general CSE-related HCPS design in the cognitive knowledge engineering process (Figure 2, right) in order to create a good conceptual model (Figure 1, left).
Expert knowledge is supposed to be stored in long-term memory in an organised, hierarchically structured and highly interrelated manner. Expert knowledge is characterised by: (1) a large breadth and depth leading to (2) a deep level of problem representation and functional understanding, (3) elaborated and explicit mental models, (4) understanding of dynamics of processes and (5) complex information elements (e.g. conceptual chunks which are high-dimensional knowledge patterns) (see Phillips et al., 2004 for a comprehensive overview). In order to establish shared knowledge for new forms of human-machine interaction such as human-machine teaming in HCPS (Bocklisch & Huchler, 2023), domain expertise needs to be implemented (at least partly) in the C-part.
Given the enormous richness of expert knowledge and the major challenge of dealing with this complexity, support from CSE expertise is needed. Using the basic cognitive characterisation of expertise (above) three principles relevant to declarative knowledge structures are selected. Figure 3 presents a summarised overview of the principles and their relations:
the role of hierarchical knowledge organisation (see 4.1.1. Knowledge hierarchy) conceptual chunking as complexity reduction method (see 4.1.2. Conceptual chunking) the consideration of precision or vagueness of knowledge concepts (see 4.1.3. Knowledge precision).
Knowledge Hierarchy
Expert knowledge is organised in hierarchies. CSE related knowledge engineering methods such as cognitive maps or work domain analysis considers this fact, for instance, by representing complex sociotechnical systems and work domains in abstraction-decomposition matrices at different levels of abstraction (Hoffman & Lintern, 2006). Figure 3 (left side) shows examples of knowledge levels relevant for many manufacturing technologies ranging from general (level no. 1) to more specific concepts (level no. 5). The first level (see Figure 2, left) represents the entire process including different phases such as initial stage, technical process, and target state, followed by the level of superordinate categories defined for a (sub)process phase (level no. 2). General examples can be input, process and output variables as well as contextual information, for instance, for the metal deposition phase. These categories can be further specified (level no. 3) such as shown for the superordinate category “output variables” here. Depending on the research question and the breadth and depth of expert knowledge in a specific domain, it may make sense to differentiate between more or fewer levels. Some technical processes consist of many steps, others of only a few. This does not necessarily mean that one process is more complex than the other is, as the underlying technical and natural science-based relationships (causal mechanisms) may be comparable or even more difficult although fewer phases are distinguished. For the knowledge hierarchy, however, the more elements are differentiated at a superordinate knowledge level, the more are also present at the subordinate levels. In addition, the knowledge concepts in one level can vary in number and complexity as well.
Conceptual Chunking
From the CSE perspective, one facet of organised knowledge complexity is the principle of chunking, which is closely related to the limits of human information processing capacity and the complexity reduction strategies of professionals (Klichowicz et al., 2023). “Miller's (1956) concept of a chunk may be defined as a unit of information of arbitrary size” (Halford et al., 1998, p. 809). Chunking can be seen as a form of learning in which concepts are recoded into fewer dimensions, such as “velocity” (one-dimensional conceptual chunk), that is formed from an originally two-dimensional representation of the relationship between “distance” and “time”. In this respect, conceptual chunks are high-dimensional knowledge patterns that can be modelled using pattern classification methods (see Section 5 as well as Bocklisch & Hausmann, 2018; Bocklisch et al., 2024).
Halford and colleagues (1998, p. 810) point out that “chunked concepts can be combined with further dimensions to represent higher level concepts” which relates to the principle of hierarchical knowledge (see above). Chunks thus place the differentiated levels of knowledge (horizontal orientation) in a meaningful context (vertical orientation). Chunking is closely associated with semantic richness of concepts or categories that is defined by the number of features or dimensions. “A major function of expertise is to provide ways of chunking that permit the important features of concepts to be represented without imposing excessive processing demands.” (Halford et al., 1998, p. 810). Murphy and Wright (1984) showed for the medical domain that the number of features per category increases systematically with expertise. In addition to the hierarchical organisation of knowledge and the complexity of the knowledge units, another important characteristic is the precision or vagueness of knowledge concepts.
Knowledge Precision
Natural human language and decision making includes imprecision. The same applies to the underlying knowledge concepts: They are characterised by a varying degree of vagueness or fuzziness (Bellman & Zadeh, 1970; Zadeh, 1965). Vagueness is associated with terms that do not show a sharp transition from belonging to an object class to not belonging, such as measured temperatures (objects), which can be described by the linguistic expressions “very high” or “high” temperature (classes). This means, that concepts overlap if they are vague and are separate if the concept or category is distinctive (Bocklisch, 2019). In contrast to the widespread inner aversion to uncertainty and imprecision as such, which is reinforced rather than alleviated by many current digitalisation approaches such as machine learning, the existence of vagueness is highly functional for human society. Erev and colleagues (1991, p. 321) point out that “It is widely believed that clarity of expression is always desirable […] in contrast with this belief is the widespread use of ambiguous and vague expressions […] one explanation for the inconsistency […] is that language serves multiple conflicting goals […] and the production cost of precision (in terms of time, effort, and loss of expressiveness) often exceeds its benefits.”. They also argue that precision is not always desirable in social situations; the same evidence was provided by Du et al. (2011) in the context of economic investment forecasts. The relation between level of expertise and knowledge precision was shown in the medical domain (Bocklisch et al., 2011; Feltovich et al., 1984; Murphy & Wright, 1984). Tabacchi and Termini (2011) quote the physicist Günther Ludwig, who points out that not even measurements are completely precise - a fact that is also well known in technology. Therefore, knowledge precision or vagueness should be consciously included in the design of HCPS by describing and determining the extent of vagueness rather than denying it in principle or trying to eliminate it.
Figure 3 (right) relates the principle of varying knowledge precision to the knowledge hierarchy levels: Normally, knowledge concepts at higher hierarchical levels are more abstract and complex and therefore vaguer than those at a concrete level. In addition, the vagueness of knowledge concepts can also play a role for conceptual chunking because the basic variables that are combined in a chunk may be precise.
Cognitive Knowledge Engineering for the Example Use Case “Electroplating”
The process of cognitive knowledge engineering is now described in broad outline for the use case of electroplating; including a brief description of the methods and results, which are differentiated according to cognitive principle “knowledge hierarchy” (the other principles are shown in Section 5 below). This process part refers to the left-hand side of Figure 1 focussing on the development of the conceptual model. The “holistic user requirements analysis” was not part of the work.
Methods
The proposed CSE-based methodological approach (Figure 2, middle) is separated into three main parts that were addressed as follows:
Knowledge elicitation: After a short analysis of standard literature in the subject area of electroplating (e.g. from textbooks and overview papers such as Kanani, 2004 or Leiden et al., 2020), the main knowledge hierarchy levels including declarative knowledge concepts were specified. An illustration of this preliminary specification was used as basis for the Knowledge analysis and visualisation: In an iterative process using structured interviews, the first presentation was discussed with two electroplating experts (scientists that work more than 10 years in the electroplating field). Their feedback was used to further develop the knowledge visualisation. Knowledge verification: The visualisation was verified in an interview with two specialist electroplaters from the industry (more than 15 years of expertise). Furthermore, the visualisation was presented and discussed several times at industry-related conferences.
Results
Parts of the results of the cognitive knowledge engineering process are now presented exemplarily to provide a first impression how the proposed Human-CoMo framework and the CSE-related approach can be specified. This presentation is far away from being exhaustive. A more complete version of expert knowledge systematization for electroplating is currently being prepared for publication in a specialised electroplating journal. The feedback from the electroplating experts (scientists, specialist electroplaters from industry and conference participants, see above) revealed that the hierarchical systematization of knowledge and the specific knowledge concepts are considered to be correct and of outstanding importance for the knowledge-based exchange between experts. It was noted that there is more declarative knowledge than could be presented as an example in the survey, but that this can also be classified in the hierarchy presented. Furthermore, the experts pointed out that the structuring of knowledge into hierarchical levels could also be used for other processes similar to electroplating and that the clear knowledge systematization (= result of cognitive knowledge engineering phase) could have advantages for knowledge transfer as well (e.g. training of novices).
In Figure 3 and 4 (left sides), a small part of the elicited expert knowledge is shown. Electroplating can be structured into five hierarchical levels (Figure 3, left). The first level (entire process) comprises three states. The process phase (grey arrow) can be further separated into several pre- and post-treatment phases as well as the most relevant “metal deposition phase”. This is where the actual layer deposition takes place. In other coating processes, such as atmospheric plasma spraying (an example thermal coating technique), the process phase is usually not divided into discrete sub-processes. The metal deposition phase can be further specified on a second knowledge hierarchy level (superordinate categories). This level comprises four main groups of variables: Context, input, process, and output variables, which again can be differentiated at the third, subordinate category level. This is shown for the output variables (Figure 3, left side, green). This category includes “resource efficiency criteria”, “microstructural properties” and “coating properties”. The latter can be described on the next following lower level: the “measurable variables” level. Here, corrosion and wear resistance as well as hardness or roughness can be distinguished. The fifth and most specific level shown in this example is the level where concrete technical measurements and observations can take place. This includes, for instance, roughness values (e.g.
Exemplary compilation of human perspectives and tasks important for HCPS-modelling (left) as well as illustration of data analytics and modelling levels (right; not exhaustive) and CSE integration (top).
The results are a part of the conceptual model specification (see Figure 1, left bottom) that is used for the human-centred modelling process (Figure 2, right).
Human-Centred Modelling for HCPS
Combining Expert Knowledge with Experimental Data for Human-Centred Modelling
Once a conceptual model has been created for the real-world task based on the declarative human domain knowledge the next step is to implement parts of the explicit knowledge in the C-part of the HCPS using appropriate modelling methods. Knowledge modelling presupposes data acquisition because the computational formalisation procedure and programming is based on numerical data (cf. Figure 1; middle bottom as well as Figure 2; right). With regard to the transdisciplinary Human-CoMo framework, Figure 4 shows three relevant human viewpoints and their example specification (not exhaustive):
domain expert (example knowledge hierarchy for electroplating example on the left), modelling expert (example data analytics-/modelling levels and methods differentiated for small/smart vs. large/big-data on the right) and CSE expert (supporting the human-centred elicitation, structuring and modelling of domain knowledge based on cognitive principles from an overarching viewpoint at the top).
The most effective modelling would be achieved if the knowledge hierarchy levels (left) could be directly mapped in the computational model (right) using suitable methods. Due to the complexity of the knowledge system and the multitude of other influencing factors, this cannot be fully realised, but should always be aimed for. Figure 4 suggests the desired relationship between knowledge and modelling levels, although this is not to be understood 1:1. The mapping objective is closely related to the empirical research processes. Usually, research questions and hypotheses are formed at higher levels of human knowledge (H), such as the subordinate category level (cf. Figure 4, left, level 3) followed by a specification of measurable variables. Thereafter, it is possible to carry out experiments, obtain data and incorporate this into data evaluation and modelling (C; bottom middle and right part of Figure 4).
This is where the complementary strengths of the C-part of the HCPS become clear. For example, the qualified analysis of complex experiments and scientific contexts and their modelling can contribute results that the domain expert alone cannot achieve (e.g. due to the limited cognitive capacity to evaluate in multidimensional feature spaces or to consider numerous interactions between variables in high-resolution and large data sets). However, all knowledge content must be correctly represented in the computational model and the amount/type of data must be considered using suitable methods for the real-world task/objective (e.g. generation of a hybrid decision aid for the temperature process control of the electrolyte). For large/big data sets statistical methods and purely data-based modelling such as machine learning can be appropriate (cf. Figure 4, right part, levels b and c). Their results are of great importance for many areas of application. For example, patterns can be recognised from large amounts of data, but in contrast to the human conceptual chunk (= multidimensional pattern), these cannot always be interpreted semantically. The transparent interpretation of results (see Figure 4 from “C” bottom centre to “H” top centre) and the transfer of findings from lower to higher knowledge levels is therefore not always possible. In addition, many big data and machine learning approaches are black-box models that can be compared to unconscious processes such as skill-based behaviour in analogy to human levels of consciousness (Bocklisch et al., 2024; Rasmussen, 1983). Strictly speaking, they therefore do not correspond to the level at which expert knowledge is relatively explicit in the form of mental models. There are a few powerful alternative methods that can be used for large and small data sets and can combine knowledge-based and data-based procedures in a transparent way (explainable AI). Fuzzy pattern classification (see below) is one of them. This method is able to take conceptual chunking and knowledge precision into account as well and is therefore suitable for human-centred digitalisation in HCPS (Bocklisch, Bocklisch et al., 2024; Bocklisch, Paczkowski et al., 2022).
The human-centred modelling process for the electroplating use case is described below, including a brief methods and results part. The cognitive principles of “conceptual chunking” and “knowledge precision” are addressed.
Human-Centred Modelling for the Example Use Case “Electroplating”
Technical Experiments and Modelling Method
A series of technical experiments were carried out using a robot-assisted electroplating system (see Figure 5, top left). The sustainability-related objective was to (1) investigate the influence of metal concentrations and other input variable variations on resulting coating quality and (2) to identify a multi-criteria optimised solution using AI modelling. The experiments were planned with the help of the conceptual model specification from the cognitive knowledge engineering process. Due to the clear hierarchical structure of the domain knowledge visualisation including concepts (see above) and their causal relations (not shown here), the most relevant variables were identified. For instance, electrolyte-related, current-related and other input-related variables were varied in combination (3 × 2 × 6 factorial experiment; multiple repetitions; in total more than 100 coating trials). The coatings were characterised using specialised methods. Among other things, corrosion resistance and the roughness were determined.
Robot-assisted electroplating system used for technical experiments and excerpt from data-based model specification (left). The illustration of three conceptual chunks (high-dimensional knowledge patterns; right) shows three classes of coatings that are of great importance for domain experts. Chunks are modelled using the explainable AI method “fuzzy pattern classification”.
Fuzzy pattern classification (FPC, see Figure 4, left side, e) was chosen for a combined knowledge- and data-based modelling. This method is especially suitable for multi-criteria optimisation tasks and HCPS model building (Bocklisch et al., 2024). The cognitive principles “knowledge precision” and “conceptual chunking” are directly addressed. The computational model describes high-dimensional knowledge patterns (= conceptual chunks) using parametric potential membership functions. The patterns can be interpreted in terms of content (grey-box model approach). Additionally, this modelling method allows implementing imprecision in varying degrees from very precise to very vague based on the experimental data and/or expert estimations.
A more detailed report on the experimental conditions, technical and modelling results is currently being prepared for publication in a specialised electroplating journal. Here we report parts of the results to explain the transdisciplinary Human-CoMo framework and the cognitive principles. The results reflect a first version of the computational model (Figure 1, bottom right). Model verification, validation and implementation in practice are not shown.
Results
Figure 5 shows the electroplating system and an excerpt of the data-based fuzzy pattern model specification (parametric description of membership functions) on the left-hand side. On the right-hand side, three conceptual chunks with different precision/vagueness are shown. The chunks were modelled based on experimental data. They represent results from three different experimental variations and describe single and multi-criteria optimised coatings based on the measurable variables “corrosion resistance” and “roughness” (cf. Figure 3; left side; knowledge hierarchy level 4).
One general objective of the technical experiments was to identify the experimental condition with optimised roughness and corrosion resistance (Figure 5, left, green chunk). This outcome (experimental condition III) combines very high corrosion resistance properties with very low coating roughness. The resulting chunk is rather crisp compared to the other two classes that represent experimental conditions showing only single-criteria optimisations: Optimised roughness or corrosion resistance coloured in pink (1) vs. blue (2). The model parameters and their visual representation in Figure 5 clearly show that even chunks at low hierarchy levels (such as corrosion resistance and roughness on “measurable variables” level, cf. Figure 3, left) can be more or less vague. This is a well-known fact that goes hand in hand with uncertainty in non-deterministic systems. In contrast to statistical uncertainty (stochasticity), which is based on chance, vagueness is a form of epistemic uncertainty (Booker & Meyer, 2002). It arises from a lack of knowledge (e.g. about the variables that influence a production process or the interactions that result from the concatenation of production steps). It is systematic in nature and can be made explicit and used by FPC in the sense of systems with organised complexity (Nilchiani, 2023).
Future Prospects, Conclusions and Limitations
If human-technology interaction is to develop into a more intensive connection such as human-machine teaming (HMT; Christopher Brill et al., 2018; Madni & Madni, 2018), one of the essential criteria is the existence of shared knowledge between humans and cyber-systems (Bocklisch & Huchler, 2023). Figure 6 shows this shared knowledge sphere, which is located between the human sphere and the human-centred cyber sphere. For (industrial) HCPS, this sphere focusses on real physical (technical) systems. It should be structured and implemented considering the cognitive principles “knowledge hierarchy”, “conceptual chunking” and “knowledge precision” and may be modelled using hierarchical, multidimensional fuzzy pattern classifier nets (see Figure 6, scheme in the centre). Based on the proposed Human-CoMo framework and the cognitive principles, future research should elaborate the hierarchical network character for application examples and implement the complete modelling cycle. This could only be shown in part for the coating example here. For instance, an exhaustive model validation is still necessary and it would also be important to include the user perspective (role no. 3). At this point, it would also become clear that hierarchical knowledge networks must remain capable of change and learning and should provide opportunities for adaptability to individual people.
Shared knowledge sphere (based on cognitive systems engineering principles) between human and cyber part of HCPS as one prerequisite for human-machine teaming.
“The primary distinction that separates experts from novices appears to be the breadth and depth of their domain-specific knowledge.” (Phillips et al., 2004, p. 299). Therefore, the present work concentrated on the description of declarative human expert knowledge. The classification of knowledge can be extended considerably. This paper is primarily concerned with the presentation of principles that are important for the domain knowledge of experts and its formalisation. An extension to other forms of knowledge is possible and useful (Horváth, 2022b). Furthermore, the (causal) relationships between knowledge elements (procedural knowledge) and dynamic understanding were not focused on here but are clearly important for the modelling and design of HCPS. There are conceptualisations from an evolutionary perspective that interpret declarative knowledge as part of procedural knowledge (ten Berge & van Hezewijk, 1999) and even consider it more important for behaviour.
The principle of conceptual chunking orients declarative knowledge towards the knowledge connections between horizontal hierarchy levels and their vertical relations. Implicitly, it also contains procedural approaches, as variables such as the hardness and roughness of surfaces (see electroplating example above) are causally linked to the material properties and are therefore not considered in isolation within the chunk.
In view of the large body of research and literature that raises questions related to CPS/HCPS and shows results from multiple disciplines, both technical and cognitive sciences, as well as the rapid dynamics in which new digital technologies are currently developing (e.g. generative AI; Lu et al., 2022), it is hardly possible to keep a complete and comprehensive overview. Nevertheless, overarching issues related to human-machine-integration in HCPS-applications are unresolved to date. These include, for example, the questions of how:
the relation between holistic human understanding and data-based information processing by the technical system can be meaningfully established although they are totally different in nature? a reality-adequate ability to act flexibly in different situations can be maintained or even be enhanced despite formalised representation and the tendency to consolidate knowledge abstractions in the C-system?
Regarding the first question and the principles presented in this paper, the semantic content of the conceptual chunk alone does not determine whether outstanding expert performance is achieved. This means that human-centric digitisation is not just about the most comprehensible, cognitively compatible knowledge representation in the C-system. The overall contexts, in which the individual knowledge elements are interwoven, as well as their changeability and malleability (which are also associated with vagueness) are essential. This is because a rich, well-structured declarative knowledge base is only helpful for an expert if he can use it flexibly and adapt it to the situation at hand. Future research in the field of CSE that deals with HCPS design and the modelling of human expert knowledge should therefore focus on methods for better conceptual capture and description of the context (see Figure 3, second knowledge level “higher-level categories”). Using the example of the electroplating chunk shown in Figure 3, an “excellent” coating specified by hardness and roughness can mean one thing in one application context and something completely different in another. Rapid contextualisation is an advantage of human expert performance, so Madni (2023) points out that contextual interpretation should remain with the human and not be assigned in the C-part. Even if this is the case, for successful communication and joint work, it is necessary to create a shared mental model or knowledge space between the human and the cyber system. At present, it is still unclear how this can be achieved in MSE or industrial HCPS. The presented scheme in Figure 5 is a first suggestion how the integration of all presented cognitive principles could be done using FPC-modelling. More research is needed to develop this idea and validate it for real-world HCPS issues.
Second, the C-system's ability to adapt or learn can refer either to situational adaptation or to adaptation to the individual user (e.g. depending on the level of knowledge). Although current machine learning methods enable impressive performance for individual task areas, they do not yet penetrate sufficiently into the area of real HCPS applications and are largely non-transparent. This means that they have not yet reached the level of awareness and understanding that mental models and declarative expert knowledge show (Bocklisch et al., 2024; Rasmussen, 1983). To clarify this question, a theory of human knowledge and conceptualisation would be important that enables the application of general principles to a concrete area of application, for instance, HCPS for MSE. Thereafter, CSE requirements could more effectively be addressed by modelling methods such as evolving intelligent system approaches (Angelov et al., 2010; Bocklisch et al., 2017). For instance, by proposing modelling principles for adaptation of fuzzy conceptual chunks based on new information or different context (e.g. creation of new coating property assessment classes such as “good” or “acceptable” or adaptation of vagueness based on human learning; cf. Figure 3). This paper aims to take a first step towards this direction with Human-CoMo, the specification of three cognitive principles and the outline of modelling possibilities.
Footnotes
Acknowledgments
Thanks to Steffen F. Bocklisch, Marcel Todtermuschke, Paulin Wels, Roy Morgenstern, Dominik Höhlich and Frank Benner for the joint discussion of the transdisciplinary framework and for providing domain expertise for the example coating technology.
Funding
The work was supported by the Fraunhofer internal programmes under grant: Attract 40-799 06107.
Declaration of Conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.