Abstract
Ontologies are a means to codify our knowledge of existence, representing concepts and their relationships. Applied ontology is the use of ontological approaches for practical purposes, including representing and solving problems within specific domains. This article retrospectively reflects on the applied ontology aspects of developing a computer-aided security modelling tool. It discusses the evolution of the ontology that underpins the modelling tool, in accordance with the incremental development of tool features to address new domain problems and feedback from concrete applications. The reflection is supported by a new OntoUML interpretation of the tool's evolving metamodel, which originally codified domain knowledge and practices. The retrospective analysis of the iterative and incremental tool development demonstrates the applied ontology aspects of a metamodel-driven development approach and provides empirical groundings of ontological unpacking, multi-level modelling, and a newly suggested technique: ontological packing. Making metamodel evolution aspects explicit highlights the significant role of metamodels and other applied ontology techniques in system development, and it suggests that they could be more systematically used for development-related decision making. A comparison with state-of-the-art ontologies further highlights aspects of applied ontology design and validation attributed to the metamodel-driven development approach.
Introduction
Ontologies allow to capture and represent knowledge. They are often employed to master complexity when discussing specific domains of discourse. Creating an ontology involves identifying the relevant domain's concepts and the relationships between them. When these concepts and relationships are properly identified and codified using an ontology, existing or desirable situations of the domain can be specified as instantiations of the ontology.
Ontologies, and particularly formal ontologies, often remain at the type level, i.e., the level of identifying concepts and relations, disregarding their instantiation. Such ontologies are referred herein as type-level ontologies. Contrastingly, metamodeling is an applied ontology approach that aims to provide structural foundations for building models (Bork et al., 2020). A metamodel allows for practical projections of ontological definitions, by designing the conceptual identities and relations that elements of a model can have. This results in models that are aligned and consistent with the metamodel representation of the ontology. Such models are sometimes referred to as models that are “correct by construction,” to signify that they are constrained to comply with the underlying metamodel, reflecting an ontological commitment (Bunting & Sprinkle, 2019; Kezadri Hamiaz et al., 2016; Mordecai et al., 2022).
In this article, we argue and illustrate that metamodel-driven information systems and tools can be used as an applied ontology approach to codify knowledge. We reflect on the evolution of a metamodel and demonstrate how it is aligned with applied ontology techniques. This viewpoint on model-driven engineering is rare. For example, a recent report on the use of model-driven engineering for real-world projects in different application domains discusses important lessons learnt, with one of the categories being “User communication: involvement of non-experts, prototyping”, yet it does not discuss any aspects of ontology (Buschhaus et al., 2024). Furthermore, retrospective reflections on the development process of systems are rarely offered, as, typically, research articles only discuss the final artifacts and not the evolution that involves better understanding domains and users. For example, all pertinent articles referenced throughout this article – i.e., articles showcasing metamodels in the context of model-driven engineering/development – only present the final resulting metamodels and not their evolution throughout the development process. This undermines the ability to understand the role of metamodels and applied ontology techniques in domain knowledge codification and in model-drive engineering/development. By making the evolution of the metamodel explicit and reflecting on it using applied ontology techniques, we illustrate their relevance to system development and knowledge codification, and suggest – based on the empirical observations – that a more systematic use of applied ontology could play a role in design-related decision making.
We exemplify some of the tension between type-level ontologies and metamodel-based ontologies – a tension that originates in their primary objectives. We also highlight the practical aspects of the latter approach. We do this by retrospectively reflecting on the evolution of a security modelling tool, TRADES. This reflection is a significantly extended take on a presentation for the Joint Ontology Workshop Semantic Shields (Shaked, 2024a). TRADES is a metamodel-driven tool, i.e., it allows to create information models and their visual representations based on an explicit metamodel, which is embedded within the tool. In the context of discussing the tool, security modelling is a non-exhaustive collection of security-related problem domains. This collection evolves as new capabilities are introduced into the tool, to model new security problem domains. We examine aspects of the TRADES metamodel's evolution to identify applied ontology techniques that were exercised de facto – in response to application-related challenges – to codify knowledge and design a modelling system. This provides empirical grounding for the observed applied ontology techniques: ontological unpacking, multi-level modelling, and a newly proposed technique, ontological packing.
The rest of the article is organised as follows. In Section 2, we provide appropriate background on ontology, metamodelling, and model-driven development and their role in information systems design; as well as on TRADES. In Section 3, we provide an overview of the methodology used for this research. In Section 4, we analyse the TRADES metamodel – as it evolved during the tool's development – from an ontology perspective, with respect to some of its evolving capabilities, in tandem with the codification of the problem domain's knowledge. In Section 5, we situate the TRADES metamodel-derived ontology with other security-related ontologies. In Section 6, we discuss the results of the reflection on TRADES and its comparison with the related works, by focusing on generalisable findings and applied ontology aspects for future consideration.
Background
Aspects of Ontology Use in Real-World Systems and for Knowledge Codification
Ontologies aim to explicitly capture the fundamental knowledge and essence of a domain, by specifying concepts and relationships and revealing real-world semantics (Guizzardi & Guarino, 2024). Paradoxically, while a primary concern in specifying ontologies is to capture the world as-is, validation of ontologies – as part of research efforts – does not necessitate exercising the specified ontologies to instantiate real-world scenarios/situations, even though such exercise is considered by some to be the preferred method of evaluating ontologies (Quinn & McArthur, 2021). Many ontologies often stay at the type level of identifying a specific domain's concepts and relationships. While this may not take away from the conceptual, theoretical contribution of specific ontologies, it may lead to questioning their accurateness with respect to capturing the real world and, even more so, their practicality.
A possible root cause of avoiding instantiations is that type-level ontologies can be perceived as novel contributions based on their completeness (Abu-Salih, 2021). Completeness has been identified as a prominent ontology quality characteristic (Wilson et al., 2023). The research culture often encourages exhaustive exploration and uncovering of knowledge, and this occasionally takes the form of a detailed ontology. Ontological completeness is also an established major aspect of desirable ontological expressiveness in information systems (alongside ontological clarity) (Wand & Weber, 1993). This is yet another paradox with respect to a primary motivation of using ontologies, which is to master complexity by bounding the domain of discourse (Wand & Weber, 1993). The detailed nature of many ontologies may hinder their practical application as an underlying model for information systems.
Another impediment in instantiating ontologies is the lack of support for this in ontology tools. The conceptual ontology modelling language OntoUML considers the definition of individuals outside of its scope. 1 Other ontology editing tools (e.g., Protégé 2 ) may support importing data entities into an ontology workspace, yet they do not provide mechanisms that allow users to effectively instantiate the type-level ontologies. They often require the user to tediously set up instances and their relations. Furthermore, while the expression of individuals can be seen as a form of instantiation, it is typically very limited in exercising the type-level ontology. In fact, individuals can exist without conforming with any type-level definitions (other than a being a “thing,” befitting of open world assumptions), and Wand et al. (1999) even argue that individuals should be conceived independently of types.
Metamodels’ Role in Model-Driven Development
Metamodels aim to provide the structural foundations for information models. By specifying concepts and relationships between them, a metamodel provides the syntactic rules for models (Bork et al., 2020). These models would then include instantiations of the concepts and relations that are explicitly specified in the metamodel. In the context of model-driven development tools, metamodels can be effectively used to drive the creation of models. When metamodels are employed as foundations for modelling under a closed world assumption – which is a typical model-driven development scheme – this facilitates association of ontological concepts with the model elements (a function maps every element in the model to its metamodel element and its conveyed ontological concept). The Meta Object Facility (MOF) standard provides the basis for metamodel definition based on a simplification of the Unified Modeling Language's class modelling (Object Management Group, 2019). In this article, we use the term “metamodel-driven development” to refer to model-driven development that relies on an explicitly specified metamodel. This term differentiates approaches that explicitly exercise a metamodel for closed-world modelling from other approaches that interpret “model” more widely (e.g., as any abstraction of reality) and/or use open-world modelling that allow models that do not conform to a metamodel (e.g, Hili and Sottet (2017)).
When a metamodel is used by a modelling tool to create models (e.g., by translating the metamodel into code that can then be run by the modelling tool and users to generate metamodel-compliant models), we consider the metamodel as executable. The use of an executable metamodel allows the rapid prototyping and development of information systems (Shaked, 2021) as well as avoiding the potential discrepancy between the design of an information system and its actual implementation (Paniagua & Caso, 2024).
Furthermore, complementary tool capabilities, such as model-driven representations, can provide a user experience that facilitates the creation of metamodel-compliant models. For example, a representation can provide a toolbox that allows a user to create new elements in specific contexts and by prompting the user to input required information, so that the creation of the new elements within the model automatically creates associated elements and relations with other existing elements, based on metamodel specifications.
TRADES Security Modelling
TRADES is a security modelling methodology and tool. It allows to model aspects of systems security and also serves as an information system that holds knowledge bases about aspects of security design (Shaked, 2023, 2024b). TRADES features are designed incrementally with a strict metamodel-driven approach, relying on an executable metamodel. According to this approach, all necessary information is to be stored as a model that conforms with an explicit metamodel, and all representations of the information are based on queries of the information model with respect to the explicit metamodel's specification. The representations and metamodel coevolve, to provide functionality to the user.
Figure 1 shows an excerpt from TRADES's underlying metamodel, which is modelled using the MOF-compliant Eclipse Ecore (Kezadri Hamiaz et al., 2016). The metamodel excerpt is taken directly from a TRADES release 3 , and it includes all elements that are pertinent to the work discussed in Section 4. For orientation purposes, the concepts’ names are coloured according to where they are first discussed in this article (which is also the chronological order of their introduction into the tool for specific capabilities): black font denotes concepts introduces in Section 4.1 (Threat and mitigation specification feature), blue font denotes concepts introduced in Section 4.2 (Risk scoring feature), and green font denotes concepts introduced in Section 4.3 (Vulnerability management feature).
A TRADES metamodel excerpt, showing concepts and relations discussed in the reflection (Section 4).
The next section (Section 4) provides a reflection on some of the TRADES development effort – using a metamodel-driven information system development approach – from an ontology development perspective. For this, we retrospectively analysed several TRADES features/capabilities that were developed during the last five years, with each of these presented as a separate subsection. The selected features/capabilities were curated by the author. This curation involved his personal discretion to offer a reflection on significant capabilities that rely on codification of knowledge and engineering practices (as opposed to pure visualisation, for example). As the current work is a retrospective reflection, it is beyond the scope of this article to justify TRADES design decisions (e.g., by comparing with alternative approaches). Some of the design decisions are explained in previous articles describing TRADES (Shaked, 2023; Shaked et al., 2024). Still, whenever a design decision is pertinent to the analysis, its motivation is explicitly mentioned.
Each subsection of the TRADES analysis results (Sections 4.1–4.3) provides a brief description and motivation for the analysed capability; followed by a reflection on the evolution and design of the metamodel fragment that underlines the implementation as well as on the contribution to understanding and codifying the domain ontology. Whenever appropriate, the reflection addresses practical aspects related to both usability (of the information system while exercising the metamodel) and the established practice and domain knowledge, e.g., risk management representations and knowledge codified in knowledge bases respectively. Knowledge bases commit to some conceptualisation (see, for example, Guarino et al. (2009) quoting Genesereth and Nilsson), and an attempt was made to build on such knowledge, make it explicit (in case where the ontological commitment is only implied), abstract or refine it.
OntoUML is used to reflect on the TRADES metamodel and on the domain knowledge. OntoUML is a language for ontology-driven conceptual modelling, based on Unified Foundational Ontology (UFO) (Guizzardi et al., 2015). Using OntoUML to capture and reflect on the metamodel is done in order to communicate ideas more clearly – and graphically – with the ontology community. Relying on a foundational ontology is expected to contribute to the reflection, by grounding some of the metamodel concepts in ontological foundations. For example, modelling the ontology using OntoUML allows us to identify and reflect on foundational class stereotypes and relationship stereotypes that are absent from the tool's Eclipse metamodel. As a default, the OntoUML “roleMixin” stereotype is used for classes/concepts. This approach is feasible – and to some extend inspired – by roleMixin being an anti-rigid, non-sortal stereotype in OntoUML. Whenever an insight about the nature of the class emerges, the appropriate stereotype to represent it is used (e.g., by using the OntoUML “relator” stereotype to denote a mediation concept and/or for ontological unpacking, as explained later). OntoUML's graphical modelling and viewing and its by-design reliance on a foundational ontology (UFO) make it a powerful tool to design and communicate ontologies (as opposed, for example, to native Protégé, which lacks a by-design reliance on a foundational ontology and falls short in graphical modelling). Furthermore, to our knowledge, OntoUML is the only tool that explicitly and visually communicates the ontological unpacking technique.
The OntoUML translation of the TRADES metamodel is to remain loyal to the original, implemented intention, yet some cosmetic changes may manifest to allow for better readability and interpretation. For example, these changes may include inversing the direction of relations, so that an ontological statement will be read more naturally or concisely (e.g., “A control mitigates threat” as opposed to “A threat is mitigated by a control”), or slight variation in wording to better convey the real-world semantics of the ontology (e.g., using “Likelihood” instead of “Difficulty” to covey the wider risk assessment related concept, as it is intended and used in TRADES). In case significant changes are made, these are explicitly stated.
The translation of the TRADES metamodel into OntoUML is a novel contribution of this paper. It provides a new, explicit type-level ontology that codifies knowledge about security modelling as used in TRADES. Since this newly unveiled ontology is rooted in a previously developed and validated methodology and tool, it has been validated de facto in real-world situations.
The new codification of the knowledge embedded in TRADES – as an ontology modelled with OntoUML – facilitates its comparison with other ontologies. Section 5 offers an illustrative comparison of the TRADES ontology with two other security related ontologies. These ontologies were published in the last three years as refereed articles in respected venues and, therefore, they represent the pertinent state-of-the-art. The comparison is not meant to be exhaustive but rather intends to further highlight the applied ontology aspects of using metamodel-driven development as a vehicle for ontology discovery and modelling. Accordingly, the two other ontologies – Security Threat Analysis (STA) ontology (Alanen et al., 2022) and Reference Ontology for Security Engineering (ROSE) (Oliveira et al., 2022) – were selected as they were developed in parallel to TRADES and aim to cover similar domains of security design and threat assessment. STA was used as an underlying theory for an information system that shares some similar problem domains with TRADES. ROSE was developed as a theory and has an OntoUML representation. Comparing the three ontologies allows us to spotlight aspects of applied ontology when driven by information system application (TRADES, STA) and when driven by theory (ROSE), as well as how specific problems/features translate into different/complementary ontologies (all three).
Developing Domain Ontologies: TRADES
Threat and Mitigation Specification
TRADES was originally envisioned as a tool that allows to specify threats to systems, based on their design, and specify design-level mitigations. Accordingly, a prominent feature – which was the significant part of the original Minimum Viable Product (MVP) – was the specification of threats to systems. Specifically, as a design decision, threats were to be allocated to any system constituent at any level of the system design hierarchy. Constituents are an instantiation of the Component metamodel element (see Figure 1). A threat is an instantiation of the Threat metamodel element. At first, the allocation of a threat to a constituent was specified by the straightforward way: instantiating a relation – allocated to – between a Threat element and a Component element. This is illustrated in Figure 2, using OntoUML. Similarly, a design-level mitigation of the threat was captured as a direct relation – mitigates – between a Control element – signifying the planned mitigation – and the Threat element. This is also illustrated in Figure 2.
An ontology fragment, based on the original TRADES metamodel, depicting core threat assessment concepts and their relationships.
The revised threat assessment ontology fragment, with a reification of the allocated to relation. The allocated to relation was removed from the metamodel, but remains in the OntoUML reproduction for reflection purposes and explicit visualisation of the ontological unpacking technique.
The revised threat assessment ontology fragment, with the corrected “mitigates” relationship that reflects an ontological insight about Threat Allocation being a Risk concept.
The simple allocated to relation was quickly evaluated – based on feedback from users – as insufficient when it comes to the threat assessment (Shaked & Reich, 2021b). On a basic level, the tool needed to provide threat modellers the option to specify whether a threat allocated to a specific component is assessed as an acceptable risk or whether it remains a gap. This required reifying the relation, which, in retrospect, was an application of the ontological unpacking technique (Guizzardi & Guarino, 2024). This resulted in a new conceptual element – Threat Allocation – being introduced into the metamodel (appears as ThreatAllocationRelation in Figure 1). Figure 3 shows the revised ontology with the reified relation. A direct outcome of the reification is the ability to rate each individual threat allocation as an acceptable risk or as a gap, by specifying the “Assessment” attribute. This formed a basis for a nominal scoring system, i.e., the ability to categorise threat allocations without any order. In turn, this led to the understanding that it is the threat allocation that needs to be assessed, and not the threat itself.
Subsequently, threat allocations could then be easily identified as equivalent to risks. Accordingly, actual mitigations should also be captured with respect to the risk, as represented by the Threat Allocation element and not by the Threat element. This resulted in an additional revision of the metamodel – with its ontology captured in Figure 4 – in which the mitigates relation was updated to relate Control elements to Threat Allocation elements. We persist in using the term Threat Allocation to remain faithful to the original metamodel and the intended, original usage of the concept, although it could have been replaced with “Risk” to reflect their equivalence. The reader is encouraged to keep in mind that the Threat Allocation concept is equivalent to Risk, as our reflection continues by exploring risk-related aspects of this concept.
The reification of the allocated to relation manifested the evolving understanding of the threat assignment and assessment domain, with Threat Allocation now taking a central role in representing risks. This demonstrates that codifying the domain practice into the metamodel – as the ontological commitment of the conceptual threat assessment model – contributed to a better identification of the risk element that is needed to be assessed and managed. This has gradually proven instrumental in further advancing our domain understanding for risk scoring, as the next subsection exemplifies.
An ontology fragment, based on the original TRADES metamodel, depicting initial risk scoring capability as enumerated attributes of the Threat Allocation concept.
Also, the same motivation that led to the reification of the allocated to relation – namely, the to assess each individual assignment of a Threat element to a Component element – was rather immediately re-exercised for reifying (ontologically unpacking) the mitigates relation. This is the ThreatMitigationRelation concept in Figure 1. This additional reification was deemed – and later validated – a necessary feature to allow for individual assessment of the mitigation that a Control element provides against a specific risk represented by the Threat Allocation element. It demonstrates how the ontology-driven metamodeling expertise can be put into use for improving practices, as, in this case, the resulting tool allowed threat assessment efforts to be more concrete and grounded by assessing the contribution of each control to the risk mitigation.
The risk matrix is a common risk management tool. It allows to represent risks using a concise, visual form that can promote effective decision making. A typical risk matrix is a representation of risks with respect to two dimensions: likelihood (also, feasibility, probability or difficulty) and impact (also, consequence or severity). As part of the TRADES focus on specifying and managing security risks, implementing a risk matrix view of the security risks was desired.
In alignment with the TRADES metamodel-driven development approach, developing the risk matrix view necessitated not only the design of the graphical representation mechanism, but also introducing the conceptual foundations – that support such a view – into the metamodel. Specifically, to automatically represent risks in the form of a risk matrix, the metamodel-compliant information model needed to allow each risk to be associated with a likelihood score and an impact score. The straightforward way of doing this is by specifying attributes for each risk element, and this was the initial approach that was adopted. Figure 5 shows the resulting ontology, with Threat Allocation – as the risk concept – now including two additional, enumerated attributes: Impact and Likelihood.
The fixed enumeration of Likelihood and Impact soon became an undesirable constraint. Users wished to introduce their own ordinal levels for both likelihood and impact, and – since the enumerated values are specified in the metamodel – this required changing the metamodel to support different applications.
Defining ordinal levels for likelihood and impact is, in fact, the definition of an ordinal scoring system. While security assessments typically employ a scoring system to rank security risks, only few of them explicitly consider that the definition of a scoring system should be part of the risk assessment process (Shaked, 2023). Equipped with the latter insight, the TRADES metamodel was revised to accommodate user-specified scoring systems. Figure 6 shows this codification of the state-of-the-art domain knowledge. It explicitly identifies Impact Level and Likelihood Level as concepts composing the Scoring System and as characteristics of Threat Allocation elements, stereotyped as OntoUML “quality.”
The revised risk scoring ontology fragment, allowing for user-specified/application-specific scoring systems.
Throughout the research into risk management state-of-the-art and practice, it was recognized that the heatmap version of the risk matrix can provide the basis for further decision support capabilities. The heatmap version of the risk matrix allows to colour each cell of the matrix – a matching between a Likelihood Level and an Impact Level – to indicate the risk tolerance, i.e., whether a risk located in this cell is acceptable or not and/or how the risk mitigation should be approached (e.g., provide additional mitigation, accept as it is, or plan more exhaustive mitigation in the future). Figure 7 shows the basic support for this. The Score Configuration concept was introduced into the metamodel, as a component of Scoring System (in Figure 1, it is termed ImpactConfiguration and implemented as a component of the Impact Level concept (ImpactScore) due to implementation limitations). This new concept can be interpreted as an OntoUML “Relator” that mediates a single Impact Level element and a single Likelihood Level element in the context of a Scoring System, and it provides the ability to specify the decision policy. Currently, this decision policy is captured by specifying a colour, which is used by TRADES to generate the heatmap version of the risk matrix. In the future, it is planned to further extend this capability by allowing the user to establish a nominal or ordinal decision policy system, and further automate how the risks are evaluated. For example, the nominal Assessment evaluation of a Threat Allocation element can be derived based on the associated Likelihood Level, Impact Level and Score Configuration.
The revised risk scoring ontology fragment, with support for generating a heatmap version of the risk matrix.
Vulnerability management is an important aspect of securing systems (Cybersecurity & Infrastructure Security Agency, 2023). Introducing vulnerability management into TRADES required investigation into the nature of vulnerability management, including analyses of pertinent knowledge bases, such as MITRE's Common Weakness Enumeration (CWE) 4 and Common Vulnerabilities and Exposure (CVE) 5 ; as well as integrating new concepts into the TRADES metamodel (Shaked et al., 2024; Shaked & Messe, 2025). Figure 8 shows the vulnerability management concepts that were initially added to the TRADES metamodel to support vulnerability management capability within TRADES – Component Type, Vulnerability and Vulnerable Asset – as well as their relationships and relationships to some previous, existing metamodel concepts (Component and Control).
An ontology fragment showing vulnerability management concepts and relations, as used in a preliminary TRADES metamodel.
We reflect on the OntoUML stereotypes of the newly added concepts. We start with Component Type, for which we initially considered the OntoUML stereotype “kind” as a rigid type, i.e., an essential characteristic (of a Component). However, the OntoUML specification does not allow for a “kind”-stereotyped concept to have another “kind”-stereotyped concept as its super-type. In TRADES, however, a Component Type entity may be a refinement of another Component Type entity (e.g., a “Linux operating system” component type can be a refinement of the “Operating system” component type). Therefore, the “kind” stereotype was found to be inconsistent with the application (this part of the application is beyond the scope of this paper). Later, we discovered that a revised version of OntoUML offers the “type” stereotype as a rigid type (Fonseca et al., 2022), which better aligns with the applied use of Component Type, and the ontology reflects this (see Figure 8).
As for stereotyping Vulnerability, there was no immediate candidate OntoUML stereotype. Further investigation into the semantics of this concept is described below, and this led to a potential stereotype later.
At the first stage, Vulnerable Asset was designed to associate vulnerabilities (Vulnerability elements) with the component types (Component Type elements) to which they apply. In addition, Vulnerable Asset could be used to associate security controls (Control elements) that can mitigate the associated vulnerabilities in the context of the associated component types (Shaked & Messe, 2025). Accordingly, it was stereotyped using OntoUML's “relator.” After exercising the initial solution (using the metamodel/ontology represented in Figure 8), practical limitations started to manifest. Specifically, each time a vulnerability needed to be associated with a component type, this required instantiating a Vulnerable Asset element. This was problematic as a specific component type could have many associated vulnerabilities (hundreds or even more). A more scalable approach was in order.
Figure 9 shows the revised, more scalable design. In retrospect, this redesign can be perceived as an ontological packing, i.e., the inverse of ontological unpacking. The reader is reminded that, at the time, OntoUML was not used and no stereotyping of the metamodel classes or relationships was available to support this ontological reflection. The ontological packing is only now made explicit, by comparing the designs in Figure 8 (based on (Shaked & Messe, 2025)) with the design in Figure 9 (based on (Shaked et al., 2024)). The ontological packing was strictly a matter of application, i.e., it is a design consideration for applied ontology. Its “cost” is the loss of ability to attribute additional information to the relationship between a Vulnerability element and a Component Type element, yet this was not deemed as essential or required. In vulnerability management practices, it is typically sufficient to understand that a vulnerability can affect a component of a specific component type, and that was considered a satisfactory basis for the TRADES vulnerability management feature.
An ontology fragment showing the simplified affects relation between vulnerability and component type.
The design in Figure 9 was still incomplete. While it packed the Vulnerable Asset's mediation of vulnerabilities and component types, it did not provide for the required functionality of associating security controls as potential mitigation. For this, a new concept was introduced: Rule. This concept allows security designers to specify mitigation rules (Rule elements) that suggest controls to mitigate vulnerabilities in the context of specific component types. Figure 10 shows the improved design (based on (Shaked et al., 2024)).
A revised ontology for the vulnerability management functionality.
Rules are not relators for the affects material relationships (and this is why there is no derived relation between Rule and the affects relationship; as opposed, for example, to the derived relation – represented as a dashed line in OntoUML – between Threat Allocation and the allocated to relationship in Figure 4). Accordingly, assigning a vulnerability to a component type does not require the instantiation of a rule. Instead, and as an outcome of this design, the existence of a rule signifies the availability of knowledge to mitigate vulnerabilities in the context of specific component types, i.e., it is a mitigation rule. Furthermore, a rule codifies this mitigation knowledge by associating pertinent model elements. This results in the creation of a knowledge base to support design decisions. In retrospect, the separation of the original “Relator”-typed conceptual element Vulnerable Asset into two conceptual mechanisms – the affects relationship and the Rule meditation – reflects two requirements for the vulnerability management functionality: 1) ability to associate vulnerabilities with the component types they affect; and 2) ability to specify potential mitigation to solve vulnerabilities in specific contexts. These two requirements were – at first – treated using a compound structure yet later distilled into two distinctive, complementing mechanisms.
An additional challenge we were facing when dealing with vulnerability management was related to the real-world semantics of the Vulnerability concept. More specifically, the lack of standard, widely accepted interpretation of Vulnerability leads to diverse user perceptions when it comes to specifying and using vulnerabilities. Addressing this effectively was of high importance as there was a need to integrate information from existing knowledge bases into TRADES and use them within the design decision support information system.
Without going into an elaborate, cybersecurity expert discussion (as this is beyond the scope of this article), we wish to illustrate how the vulnerability concept is open to multiple interpretations and how this can affect the design of information systems. We do so by reflecting on the differentiation that MITRE – a nonprofit organisation which is a global leader in security – makes between vulnerability and weakness. In its CVE knowledge base, MITRE considers vulnerability as an “instance of one or more weaknesses in a Product that can be exploited, causing a negative impact to confidentiality, integrity, or availability; a set of conditions or behaviors that allows the violation of an explicit or implicit security policy”. 6 A straightforward OntoUML representation of “instance of one or more weaknesses in a Product” is shown in Figure 11.
OntoUML representation of vulnerability as an ‘instance of one or more weaknesses in a product’.
In MITRE's CWE knowledge base, a vulnerability is defined as “A flaw in a software, firmware, hardware, or service component resulting from a weakness that can be exploited, causing a negative impact to the confidentiality, integrity, or availability of an impacted component or components” 7 . This definition corresponds and somewhat expands the first part of the CVE definition. It establishes an equivalence between a “product” and a “component” as being the characterised end of the vulnerability. In CWE, a weakness is defined as a “condition in a software, firmware, hardware, or service component that, under certain circumstances, could contribute to the introduction of vulnerabilities.” Interestingly, the CWE definition of weakness – as a form of condition – corresponds to the second part of the CVE vulnerability definition (as a set of conditions). This suggests that the distinction between the weakness and the vulnerability concepts is not straightforward, and this is further evident by a range of different interpretations of vulnerability in literature.
Using the CWE definitions, Figure 12 shows a possible incorporation of the vulnerability and weakness concepts into the TRADES vulnerability management ontology (Figure 10). It illustrates that both weaknesses and vulnerabilities can be perceived as qualities (OntoUML stereotype “quality”) characterizing a component, which leads to question the differentiation between Vulnerability and Weakness as concepts. Also, according to OntoUML, the cardinality of characterized end of a characterization relation can only be “1”. This leads to a scalability problem when applying the ontology: each Vulnerability/Weakness element (instance) can only be associated with one Component element. Practically, this means that multiple instances of the same specific vulnerability/weakness record – such as the ones specified in the MITRE knowledge bases – should be created. For example, when one analyzes a system with respect to the “Improper Validation of Specified Quantity in Input” weakness (CWE record CWE-1284) or with respect to the associated implementation vulnerability (CVE record CVE-2022-21668), multiple instances of both CWE-1284 weaknesses and CVE-2022-21668 vulnerabilities might be instantiated to indicate each instance of these weaknesses and vulnerabilities in various components. This is cumbersome and essentially unscalable. It is not practical to manage, and, in fact, leads to a design flaw of the vulnerability management system. This implementation also does not take into consideration any knowledge about the affected component types, and this makes a Rule-based reasoning system unusable.
A possible integration between the TRADES vulnerability management ontology and MITRE definitions.
Furthermore, in vulnerability management it is the descriptions of weaknesses and vulnerabilities that are exercised, to reason about their existence and mitigation. These descriptions are not the weaknesses or vulnerabilities themselves, but the vulnerability/weakness records that contain structured data describing the vulnerabilities/weaknesses and their metadata, e.g., to which components they apply, what are the possible mitigations, etc. Actually, the records are the concepts that are represented in the MITRE knowledge bases. To illustrate, CVE-2022-21668 is vulnerability record and not a vulnerability. In full alignment with the MITRE definition (and OntoUML), a vulnerability would be an actual manifestation of the CVE-2022-21668 record as a quality of one, specific component, with another manifestation appearing as another quality (associated, again, with one, specific component).
Figure 13 captures the distinction between the actual vulnerabilities/weaknesses and their records. The Vulnerability Record and Weakness Record concepts mediate between the Vulnerability and Weakness concepts, respectively, and the other associated concepts. Further ontological analysis reveals that the two record concepts relate to component types and not components (in CVE these are the affected products/configurations and in CWE these are the applicable platforms). This helps to restore the association of vulnerabilities with Component Type, as in Figure 10.
Further elaboration of the integrated vulnerability management ontology, with concepts representing the knowledge bases’ records.
The mediatory role of both record concepts earns them their OntoUML “relator” stereotype, with a small caveat, though: OntoUML recommends a constraint on relator concepts, so that the sum of the minimum cardinalities of the opposite ends of the mediations connected (directly or indirectly) must be greater or equal to 2. In our case, the constraint is not satisfied. This can be justified by the willingness to support unknowns within the implementation: there can be a description of a vulnerability or weakness and of the affected component type that was yet to materialize and/or be found (and once they would be found, the cardinality would be at least 2–1 Vulnerability/Weakness element + 1 Component type element). Ontologically, the OntoUML constraint remains valid, and – if used for evaluating the resulting models – it may indicate a missing piece whose existence a theory may support/speculate but it is yet to be documented (as an analogy, think of an elementary particle in physics).
An alternative design can be achieved by unifying the Vulnerability and Weakness concepts. This design reflects that the nature of these two concepts is similar, as they both – according to MITRE – depict a condition (quality) in a component. The alternative design is, in fact, the original design of Figure 10, with the Vulnerability concept representing an applied, purposeful abstraction that unifies the Vulnerability and Weakness concepts as well as the Vulnerability Record and Weakness Record concepts. In terms of applications – at least for vulnerability management – these concepts are all closely related, and their semantic distance does not justify significant differentiation. However, acknowledging that certain professionals may wish to distinguish Vulnerability from Weakness, a possible extension was made to allow to indicate whether a Vulnerability element is an implementation level vulnerability (as in CVE) or a conceptual vulnerability (as in CWE), using an enumerated attribute. Figure 14 shows the extended design, and how it relates to our ontological observations of vulnerabilities, weaknesses and their knowledge base records. The implemented Vulnerability concept (bottom left in the figure) is a unification of four identified concepts – Vulnerability (MITRE), Weakness (MITRE), Vulnerability Record and Weakness Record. The unification of concepts and their associated relations is another form of ontological packing. A similar rationale to using the OntoUML “type” stereotype for Component Type applies to the packing Vulnerability concept (as in TRADES applications a vulnerability is rigid and may refine a more abstract vulnerability), and, therefore it is “type” stereotyped.
The TRADES vulnerability management ontology, with the vulnerability concept packing the knowledge bases’ concepts (top four concepts).
Figure 15 shows the ontology that underpins the
Security threat analysis ontology, adopted from (Alanen et al., 2022) and annotated to indicate concepts that were directly used (blue star), omitted (red star), or absorbed into others (green star) when translated into a metamodel (and eventually an information system) by the original authors (Alanen et al.). Colored versions of the figures are available online.
Figure 15 shows some annotations we made about STA when analysing the resulting publication. These are shown on top of the original diagram (unchanged from its original publication) as star symbols. When translating the STA ontology into a metamodel, several ontological concepts – such as Risk, Threat description, and Misuse action – were left out. These concepts are annotated with a red star above the class box in Figure 15. Furthermore, some ontological concepts were merged. In Figure 15, the concepts that explicitly appear in the metamodel as a class are annotated using a blue star, and those that were absorbed into or merged with the class entities are annotated using a green star. System and System element appear as a single concept, echoing the TRADES’ Component abstraction. Threat includes some components of Threat Description – Threat type (and its associated Origin), Threat Circumstance, Security incident, Attack, Threat consequence, and Recommended countermeasures. This unification echoes the TRADES unification of Vulnerability (MITRE) and Vulnerability Record, as an abstraction mechanism that unites a concept with a relator. Threat also absorbed Risk level and includes new attributes that are missing from the ontology: Risk severity and Risk likelihood. Feedback from the implementation to the ontology – as our incremental, metamodel-driven development approach encourages – would have likely resolved this omission. By comparison, Risk Severity and Risk Likelihood exist as the Impact Level and Likelihood Level qualities of Threat Allocation in TRADES, and Risk Level exists as a Score Configuration in TRADES. Furthermore, while the STA article acknowledges that “Risk level… is derived from the severity and the likelihood…” the STA ontology and the STA metamodel do not codify this knowledge, whereas the TRADES metamodel explicitly communicates that Score Configuration is the combination of specific Impact Level and Likelihood Level elements.
Also, in the STA metamodel, Existing countermeasure was absorbed with Security Requirement, as evident by a new “Existing risk control evidenced by” relation from Threat to Requirement in the metamodel. This is well aligned with the suggested interpretation – in TRADES – of Control as a desirable function that should be elaborated as requirements (Shaked, 2023). Anecdotally, treating recommended countermeasures (absorbed into Threat in the STA metamodel) differently than existing ones (absorbed into Requirement in the STA metamodel) might lead to problems.
The assignment of requirements/functionality to a system or its component is somewhat more complicated in STA than in TRADES. In STA, a Security Requirement is expected to be “satisfied by” an Engineering Artefact, of which System and System Element are subtypes. In comparison, TRADES directly associates the Control – as a security function – with the Component.
Figure 16 shows the
Reference ontology for security engineering ontology, reproduced from deposited artifact supporting (Oliveira et al., 2022) and annotated to indicate concepts explained by example (green triangle) and by evaluation (blue triangle). Colored versions of the figures are available online.
As opposed to STA and TRADES, ROSE was not designed to underpin an information system implementation. Instead, it offers a theoretical, referential framework to analyse security engineering cases. The original ROSE article provides an example to justify its design (in its Section 4) and a brief evaluation with respect to pertinent standards and practice (in its Section 5). Based on our analysis, the coverage of ROSE by the example and by the evaluation appears in Figure 16, by annotating concepts that were exercised in the example with green triangles; and concepts that were reported to be exercised in the evaluation with blue triangles.
An in-depth analysis reveals that the Threat Event, Loss Event and Attack concepts (of ROSE) are used with very minor differentiation in the example, suggesting a potential redundancy instead of highlighting the potential value of using the separate concepts. The same applies to the Vulnerability and Risk Enabler concepts and to the Loss Event and Loss Event Type concepts. In the latter case, the example makes use of relating a Loss Event Type element to an Object At Risk element, and this relation is not supported by the ontology, which only allows Loss Event elements to relate to Object At Risk elements. By comparison, the TRADES Vulnerability concept holds a similar interpretation to the ROSE Vulnerability concept, and the association of a Vulnerability with a Component element based on its Component Type (Figure 10), results – de facto – in establishing the Component as a ROSE Risk Enabler. Both TRADES and STA communicate explicitly that a system or its component are associated with a vulnerability, whereas in ROSE this highly practical attribution is missing. Also, TRADES uses a single concept – Threat Allocation – to represent the somewhat redundant ROSE concepts Threat Event and Attack, and does not seek to address the log-oriented Loss Event concept, which remains out of scope for the current TRADES functionality (for example, the Loss Event can be useful when investigating security incidents; but this is beyond the TRADES focus on designing systems for security).
Also, while the original ROSE article identifies – in its Section 4 – “the severity scale of risk matrix” as a potential OntoUML “quality”-stereotyped concept, the ROSE ontology lacks this concept. This concept appears explicitly in TRADES as Impact Level, and it was acknowledged in Figure 7 as an OntoUML “quality”-stereotyped concept.
As for the ROSE evaluation, apart from the limited exercise of the ontology, it is noteworthy that while the Triggering Likelihood and Causal Likelihood concepts are explicitly acknowledged, none of these concepts appear in ROSE. These concepts – which are concepts of the Common Ontology of Value and Risk (COVER) that ROSE extends – were not explicitly adopted or extended (as they should be) in ROSE. UFO and COVER concepts that are acknowledged as pertinent by ROSE yet are not integrated with other ROSE concepts appear in Figure 16 within an isolated pink container. In TRADES, Likelihood appears explicitly, as the Likelihood Level concept associated with the risk scoring capability (Figure 7).
Ontologies are a significant tool to codify knowledge and communicate real-world situations. This article reflects on the development of a metamodel-driven information system as an applied approach to developing ontologies. The development of three features of a security modelling tool – TRADES – was surveyed in retrospect, and the underlying ontological foundations of these features – extracted from the tool's metamodel and following its evolution – were reimagined using OntoUML, as examples of codifying domain knowledge.
The first feature example (Threat and mitigation specification, Section 4.1) shows how reifying a relation in correspondence with the ontology of practice contributed to the ontological discovery of Threat Allocation elements and their interpretation as risks. This further shaped our understanding and our consequent development effort. The example demonstrates ontological unpacking as a useful applied mechanism. While Guizzardi and Guarino (2024) describe the technique as an explanatory device, it was exercised several times in the TRADES metamodel design to provide additional functionality.
The second example (Risk scoring, Section 4.2) – in which the previous ontological insight was extended to include an ordinal risk scoring system – illustrates the importance of abstracting away from concrete enumerations, not only to provide user flexibility but also to provide sustainable ontological foundations that can be adopted and adapted to accommodate specific challenges or techniques. This is aligned with some of the concepts behind Multi-Level Modelling, which relates to conceptual modelling approaches that support “representing multiple classification levels within one model” (Frank, 2022). Specifically, it allows to represent concrete instantiations – e.g., Threat allocation model elements – alongside typifying them, e.g., using Impact Level and Likelihood Level model elements (“representing objects on level 0 together with classes/objects on higher levels”).
In the third example (Vulnerability management, Section 4.3), the metamodel was extended to support the modelling of a new problem domain. New concepts and pertinent relations were plugged into the metamodel. The relations between new and existing concepts hint at the possible commonality and interface between the threat and risk assessment problem domain and the vulnerability management problem domain, and it places both problem domains in the context of the grander security body of knowledge (and ontology). This can be easily corroborated by examining domain-specific knowledge bases and other state-of-the-art works (see, for example, (Shi et al., 2024)). The third example includes another manifestation of the Multi-Level Modelling paradigm, by allowing to specify Component elements alongside their characterising Component Type elements, and to reason about the vulnerabilities found in the former based on the latter. More importantly, the example suggests ontological packing – the somewhat reverse mechanism to ontological unpacking – as an applied ontology mechanism.
Collectively, the three examples demonstrate that tracking the evolution of metamodels used for systems development can be a valuable applied ontology tool to codify knowledge for better understanding domains, and, in turn, make informed design decisions about applications that exercise such knowledge. Furthermore, our retrospective reflections illustrate that applying applied ontology techniques – such as ontological unpacking, multi-level modelling and the newly articulated ontological packing – to metamodels can promote codification of knowledge. Further research could explore how to systematically use metamodeling with such techniques and evaluate the benefits of their integrated use in various design situations.
While ontological unpacking allows for better understanding and theorising, ontological packing allows for scalable and applicable use of ontology within information systems. Ontological packing is a way to avoid complexity whenever it is not justified (or not yet justified), by unifying concepts whenever there is no clear incentive to separate them. While, at first, ontological packing may seem contradicting to Wand and Weber's notion of ontological completeness, this contrast can be resolved by noting the context of the latter as the ability to “represent all real-world phenomena that might interest them” (with them referring to the users of the information system grammar) (Wand & Weber, 1993); and this is reaffirmed in a later work, which explains incompleteness as the inability of the information system to represent the states of a real-world (Wand & Wang, 1996), and, by that, relates to the situations that the system (and its designers) intend to represent. Wilson et al. (2023) cleverly acknowledge this by specifying Functional completeness as an ontology quality characteristic that is separate of the Coverage/Completeness quality characteristic. Furthermore, the tension between using ontological packing and using ontological unpacking exemplifies the tension between the two aspects of Wand and Weber's ontological expressivity: ontological unpacking relates to ontological completeness whereas ontological packing relates to ontological clarity. Whereas in ontologies that seek to theorise and exhaustively codify knowledge the former (ontological unpacking, in service of ontological completeness) is preferred, in applied ontology there is no definitive preference. The selection between the two approaches is a design decision. This design decision, in turn, affects the design of visual notations to follow the principle of semiotic clarity (Moody, 2009). Future work should seek ways to systematically approach the tension between ontological unpacking and ontological packing and perhaps even benefit from using them in a unified manner. For example, consider a formalism that allows tracking packed concepts to packing concepts (as illustrated informally in Figure 14, with the use of unspecified relations – represented as dashed edges), so that design decisions of information systems can be mapped to a unifying ontology. This could be beneficial for theorising and further codifying domain knowledge, as well as for understanding the landscape of pertinent information systems in an organisation, with respect to a domain of interest, and how these systems could be used integratively.
The reflection on TRADES development uses OntoUML to represent and reflect on the uncovered knowledge in the pertinent problem domains. The use of OntoUML was beneficial for identifying and executing ontological unpacking and ontological packing. One limitation of OntoUML – with respect to applied ontology design – is the constraint on the characterised end of a relation with a “quality” stereotyped concept to be exactly one (1), i.e., a quality element can only characterise a single element. This is violated in Figure 7. While the constraint may be philosophically valid, in information systems it is often desirable to avoid an explosion of elements as well as use quality attributes effectively. For illustration, consider a model of a car inventory system. If two Car elements have the colour green, OntoUML requires the use of two separate Colour elements, even though both may have the value “Green.” This will hinder the ability to search for green cars within the inventory, for example.
On the metamodeling side, current metamodeling approaches do not allow to explicitly record ontological packing and ontological unpacking. Allowing this could, potentially, contribute to the ability to document overloaded ontological semantics as means for knowledge discovery and codification, and future work should introduce such metamodeling features. Such overloading can also contribute to the interoperability of information models used in various information systems. For example, the TRADES Threat Allocation concept was identified as a risk related concept (Section 4.1). Having an information modelling infrastructure that explicitly states the Risk nature of Threat Allocation could allow each instantiation of the latter (a Threat Allocation element) to be picked up and treated by risk management systems (and their users) as a Risk element.
This article also offers a new perspective on the STA and ROSE ontologies and their practical implementation. The critical analyses of STA and ROSE reveal a potentially generalisable observation: some ontologies are only limitedly validated. Both the STA ontology and ROSE were not fully exercised, and this means that some concepts and relations of the ontologies were not validated in terms of applicability and/or theoretical contribution. The lack of theoretical contribution validation is a significant gap in explaining the potential redundancy of concepts and the comprehensiveness of the ontologies. By comparison, the metamodel of TRADES is exercised within the metamodel-driven modelling tool, in full and as an evolving effort to support each new feature/capability. This not only confirms the technical soundness of the TRADES ontology – as codified in the metamodel – but also facilitates the validation and revision of the ontology to align it with applications and empirical observations. Properly hedging the validation of an ontology can lead to better framing of follow-up research and its contribution (Shaked & Reich, 2021a). For example, by explicitly marking which ontological concepts have yet to be validated, additional research can seek to clarify, demonstrate, prove and provide evidence for the theoretical contribution of these concepts within the ontology's intended domain.
Future efforts could explore introducing theoretical ROSE concepts into TRADES. Guizzardi and Guarino have correctly noted that “ontological unpacking must be complemented by complexity management tools…, e.g., model abstraction/summarisation, modularisation, and viewpoint extraction” (Guizzardi & Guarino, 2024). Accordingly, expanding TRADES with additional concepts requires careful design that would allow human designers to effectively use the tool and the underlying concepts, to model various security problem domains. This is likely to involve the use of a variety of modularisation, abstraction and viewpoint representation techniques. Also, explicitly relating TRADES metamodel concepts to the more refined ROSE ontology concepts – using ontological unpacking and ontological packing – may result in better explanations of cyber situations and potentially even to developing further scenarios for cybersecurity.
Footnotes
Acknowledgements
The author wishes to thank Prof. Tom Melham and Dr. Nan Messe for their contribution to the original TRADES work. The author also wishes to thank Dr. Ítalo Oliveira for his valuable advice on modelling using OntoUML and ROSE.
Funding
This work is supported by Innovate UK, grant #75243.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.