Abstract
Access to persistent computer-generated virtual worlds may provide a powerful tool for conceptualising the mining cycle and managing the domains of exploration, feasibility, planning, design, construction, operations, rehabilitation, decommissioning and closure. Each domain presents a significant challenge to mine operations. Realisation of persistent virtual worlds that can be accessed by many simultaneously may be possible by leveraging Metaverse technologies to produce an ‘always on’ Mining Metaverse based on International Standards and industry collaboration. The realisation of a Mining Metaverse is a complex task because the Metaverse itself has many components and domains that must be managed effectively for it to be sustainable. This article introduces the complexity of the Metaverse components as a taxonomy and was inspired from collaborative work completed by the Standards Australia IT-031 Modelling and Simulation Committee and International Standards Organisation ISO/IEC JTC 1/SC 24 Committee. It is intended as a starting point for the mining industry towards understanding what the Mining Metaverse may be, and effectively embracing and managing this complex emerging technology in the future.
Keywords
Introduction
Access to persistent computer-generated virtual worlds may provide a powerful tool for conceptualising the mining cycle and managing the domains of exploration, feasibility, planning, design, construction, operations, rehabilitation, decommissioning and closure. All these issues present significant challenges to the mining life cycle.
The goal is a completely sustainable mining cycle that provides the most benefit to society, maximises resource recovery, is profitable to investors, and leaves no negative legacy to future generations. That is, ‘meeting the needs of the present without compromising the ability of future generations to meet their own needs’ (Brundtland, 1987).
To leave no negative legacy is a significant industry challenge, and one that may benefit from the adoption of mixed reality simulation via the application of computer-generated virtual worlds as a novel communications tool and deployed as a Mining Metaverse.
The Metaverse itself is not a new concept, especially in science fiction. A comedy science fiction show called Red Dwarf (Naylor, 1993) was set aboard a Jupiter Mining Corporation's spaceship and provided a futuristic view of the use of immersive virtual worlds. ‘Better than life’, was accessed via advanced technologies that tapped into the human consciousness and transported the characters into another location, or dimension. This fiction, except for the direct connection to the human consciousness component, has now mostly become reality and virtual worlds are commonplace, even in the conservative mining industry. For example, researchers at the National Institute for Occupational Safety and Health (NIOSH) developed VR Mine, a VR platform that allows the rapid generation of Virtual Reality, and virtual environment software applications (Hoebbel et al., 2023).
There are numerous examples of virtual worlds developed for mining applications and a Web of Science database search using the keywords ‘virtual, reality, mine, training’, yields some 143 relevant publications since 1994, with the earliest being presented by Schofield et al. (1994), Bise (1997), Squelch (1997), Denby et al. (1998), LeBlanc-Smith et al. (1998), Filigenzi et al. (2000), LeBlanc-Smith et al. (2000), Squelch (2001). These virtual worlds allowed users to interact with a computer-generated virtual environment to complete some task or activity. These were pioneering projects, and essentially precursors to a Mining Metaverse concept.
Since those publications emerged, hardware and software technologies have advanced significantly, making virtual worlds accessible to all, and enabling their integration into mine operations more routinely and across a wider spectrum of applications. Increased network bandwidth, high-performance computing (HPC), and ‘always on’ devices combined with increased interest from social media companies have enabled a resurgence of the Metaverse with the result that it will eventually impact the mining industry itself. More recently, Stothard (2023) explored the Mining Metaverse in more detail and whether it may be applied to the mining industry.
This article aims to provide a taxonomy and overview of what a Mining Metaverse may be and how it may be realised. The topic is broad and hence the article primarily introduces the Metaverse concept and the domains that enable it. Past examples of early Virtual Reality project formats are discussed, and how they may form mining metaverse use cases for the mining industry to use as a concept in any subsequent articles.
The article was inspired from collaborative work completed by the Standards Australia IT-031 Modelling and Simulation Committee and International Standards Organisation ISO/IEC JTC 1/ SC24 committee. It is intended as a starting point for the mining industry towards understanding and effectively embracing and managing this complex emerging technology.
Need for taxonomy
A taxonomy is needed for the Mining Metaverse to provide a structured and systematic way to categorise and arrange its diverse elements. Taxonomies establish a standardised framework that promotes consistency and accuracy, and this would be invaluable in the early stages of development of the Metaverse concept.
A standardised Mining Metaverse taxonomy would enhance communication and shared understanding among the content creators and Metaverse participants. Such a taxonomy would also enable efficient search and retrieval of Metaverse components and would also act as a bridge between the different disciplines that make up the Metaverse such as virtual reality, augmented reality and blockchain by providing a common language and framework for experts from these various fields to collaborate and share knowledge.
In essence, a Mining Metaverse taxonomy can aid with order and understanding, providing the foundation for effective communication, research and decision-making in this new field of human technology and exploration.
Metaverse, what is it?
While there is no formal definition of the Metaverse as of 2023, broadly speaking, it can be described as an infinite and persistent set of computer-generated virtual worlds that may be accessed via some human–computer interface (HCI). It is controlled by the creator's imagination, society, HCIs, computer code and graphics. It can be accessed simultaneously by many people, agents, or machines (Figure 1). There does not necessarily have to be a single Metaverse without a boundary there may in fact be many.
A sketch of the metaverse concept.
The focus of this article is the presentation of the Metaverse concept viewed from a mining industry perspective as a Mining Metaverse which would be an Industrial Metaverse similar to that described by Althoff (2022), who commented that ‘Businesses and Technologists around the world are struggling to define the Metaverse and what their role will be within it’. This is still true in 2023 and certainly the case in the mining industry. Althoff (2022) further stated that a Metaverse experience would include, enterprise collaboration, engineering and design, use of non-fungible tokens, serious games, acquisition of knowledge, understanding and empathy, social media interactions, e-commerce and mentoring. This presents a diverse collection of terms, components, methods and technologies, all of which can potentially be leveraged by mining industry personnel.
Despite still struggling for a formal definition, the Metaverse is not a new concept and was first imagined in science fiction, in the novel Snow Crash (Stephenson, 1992). It has been revisited in recent years as proprietary companies have realised that a global market existed for Head Mounted Displays and Metaverse technologies, (Althoff, 2022; Anon, 2022; Ball, 2021, 2022; Mileva, 2022; Novak, 2022; Smart et al., 2007). However, it should be noted that it may not be Head Mounted Displays that prevail as the most common access portal to the Metaverse, due to human factors and design issues related to interactive 3D and games-based training systems, as discussed by Schofield (2018) and Stone (2023).
ISO (2023) defines the Metaverse as a universal, immersive virtual world that is facilitated by virtual reality and augmented reality. It is a persistent and immersive simulated world that is experienced in the first-person perspective by large groups of simultaneous users who share a strong sense of mutual presence (Rosenberg, 2022).
Smart et al. (2007) describe the Metaverse as, ‘The convergence of a virtually enhanced physical reality and a physically persistent virtual space. It is a fusion of both, while allowing users to experience it as either’. Hence, the Metaverse could be broadly thought of as a persistent, collaborative computer-generated virtual world that may or may not augment the real world, a place where people and agents can interact and exchange ideas, goods, and experiences. A quite similar concept to mixed reality (MR).
The feeling of presence is subjective and difficult to quantify. Presence was defined as the subjective experience of being in one place or environment, even when one is physically situated in another by Witmer and Singer (1998) who commented that the effectiveness of virtual environments was linked to the sense of presence reported by users of virtual environments. Witmer and Singer (1998) suggest that presence is a normal awareness phenomenon that requires directed attention and is based on the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion. Presence also depends on a person's internal tendencies to become involved (Witmer and Singer, 1998). That is, presence does not depend on one single factor but several.
Considering the issue of presence, from a mining industry perspective, and referring to Figure 2, to suspend disbelief that a person is at location ‘A’, say a mine site office, but can see and interact with a virtual environment at location ‘B’, say a Mining Metaverse, that is a computer-generated model or digital twin of location ‘C’, a real mine, is a significant challenge to realise.
Example Mining Metaverse concept.
Digital twin has become an all-encompassing term in recent years within mass media however, there are different ‘levels’ of digital twin, or in this case a ‘digital mine twin’. These are summarised by Stothard (2023).
Digital mine site model—physically isolated
A digital mine site model is a computer-generated digital representation of some or all aspects of a mine site. The physical mine site may be planned or already exist, but the digital mine site model does not have any form of automated data exchange between itself and the physical mine. A digital mine site model may provide a description of the physical mine site that is more, or less, comprehensive. Such models may include geology, orebody data, mine planning, mine simulations, geotechnical models or any other models of the physical mine site that do not use data integration via any form of automation. That is, the digital mine site model is traditionally generated via processes instigated and maintained by human interaction. If that interaction ceases, the model remains unchanged. Should the physical mine site change, the digital mine site model will not alter until updated by a person and vice versa (Stothard 2023).
Digital mine site model—adumbration—unidirectional data
Based on the definition of a
Digital mine twin—bidirectional data
A digital mine twin manifests when mine data is exchanged between an existing physical mine site and digital mine model in both directions and is fully integrated. In this case, the digital mine model may act as a controlling representation of the physical mine. Within the physical mine or digital model, there may be objects, real or virtual, that induce changes of state within the digital model. Hence, a change in the physical mine almost immediately leads to a change of state in the digital mine or virtual environment, and vice versa. The physical mine is connected to the Digital Mine Site Model with bidirectional automated data flow where digital mine updates reflect a change in the physical mine and vice versa (Stothard 2023).
Digital mine twin—bidirectional data—metaverse connected
The digital mine twin is expanded even further when mine data is exchanged between an existing physical mine site and digital mine model in both directions, is fully integrated and is connected to the Mining Metaverse. Again, in this case, the digital mine model may act as a controlling representation of the physical mine, which importantly, may be controlled externally via some collaboration via the Mining Metaverse. Hence, within the physical mine, digital model or remote Mining Metaverse, there may be objects, opinions and actions, real or virtual, that induce changes of state within the digital model and subsequently the mine operation. Hence, a change in the physical mine almost immediately leads to a change of state in the digital mine or virtual environment, or Metaverse and vice versa. The physical mine is connected to the Digital Mine Site Model with bidirectional automated data flow where digital mine updates reflect a change in the physical mine and vice versa the digital mine site twin is also connected to the Mining Metaverse (Stothard 2023).
The above descriptions show that the concept of combining 3D models to form a digital twin, the real world and a Mining Metaverse add another layer of complexity to the term digital mine twin when it is combined with bidirectional data exchange and Metaverse connectivity. The first two are ‘simplistic’ mine site models. The second two are much more sophisticated and hence more powerful as they may allow real-time interaction and collaboration (Stothard 2023).
If the latter two examples could be realised, it would constitute a very powerful capability, especially for achieving the goal of a sustainable mining operation, because it may improve collaboration and understanding of the short- and long-term risks to mining operations. There may also be significant financial benefit derived from creating a Mining Metaverse because it will allow ‘what if’ scenarios and simulations to be run in a safe and forgiving environment as part of staff training and operational risk management processes (Stothard 2023).
In Figure 2, the computer generates representation of location ‘C’ may be a 3D model representing a digital twin (DT) that in this context would be where a physical mine is connected to a digital mine site model that facilitates bidirectional, automated, data flow where any digital mine updates reflect a change in the physical mine and vice versa.
Commerce, gaming, healthcare, education, training, knowledge sharing and socialisation in very near real-time are all a part and could also be leveraged for mine site operations. Essentially, the Mining Metaverse would provide a comprehensive, integrated, online communications tool that brings people together who may be remotely located offsite, or even at the same location but remote from each other (e.g., underground or on the surface). It will thus allow people to interact to complete some activity or objective. Figure 2 shows a generalised layout of a Mining Metaverse example integrated into operations. In this situation, the Mining Metaverse would be and industrial tool and hence only accessible by people who work for the various entities that will perform some service or operations relating to a mine site. For example, it would be a restricted access, collaborative tool, for assisting business and mine-planning decisions. A consumer metaverse would be significantly different in that it would be more open to consumers and the wider public for entertainment, social activities, commerce, etc.
Metaverse research
A Web of Science keyword search using ‘Metaverse’ and ‘Taxonomy’, produced some fifteen research articles that cover the Metaverse concepts in detail. Interestingly, Metaverse Taxonomy articles have only appeared since 2022 with the number of publications doubling from 5 to 10 and citations from 100 to 180. This sudden interest may be a result of the rebranding of Facebook to Meta in 2021 (Meta, 2023), signifying the technology giant's commitment to realising the metaverse concept.
Researchers and industry are possibly rediscovering the Metaverse due to more easily available and powerful hardware and software, all while trying to gauge what its impacts on industry may be. As industry and commercial hype increases and descriptors become confused and interchanged, understanding what is being discussed and delivered becomes confusing to the uninitiated. For example, Zyda (2022) discusses how everything from terms such as virtual reality, synthetic environments, visual environments, second life and so on are now being renamed as the Metaverse as part of a hype cycle.
Park and Kim (2022) comment that the current Metaverse is based on the social value that a person's online and offline self are the same and that the integration of enhanced social activities and neural-net methods requires a new definition of Metaverse suitable for the present iteration. Park and Kim (2022) report that ‘always on’ and access to virtual currency would be essential for a Metaverse so that it can be accessed 24/7 and allow commerce to be undertaken. Park and Kim (2022) also divide the concepts and essential techniques necessary for realising the Metaverse into three components. These are hardware, software and content; with three approaches including user interaction, implementation and application and three key limitations identified as being social influences, restrictions and open challenges (Park and Kim, 2022).
In the case of a Mining Metaverse, the same three components would certainly apply, because, anecdotally, in an early virtual reality project example (Stothard and Galvin, 2005), there was a perception of the need for high resolution and fidelity in mining simulations and virtual environments. Stothard and Galvin (2005) reviewed numerous virtual reality simulation facilities against a needs analysis of the NSW coal industry for virtual reality training. Stothard and Galvin (2005) noted that most positive industry engagement was realised when the facilities had large screens, with high-resolution images shown upon them.
During a subsequent project, it was found that as resolution and fidelity improved, so did industry expert engagement (Stothard, 2008). This was a key and enabling component for building believable virtual worlds and fits with the experiences of Witmer and Singer (1998), with regard to presence within the virtual environment. The industry experts who helped develop the virtual worlds may have begun to feel presence, because they were prepared to be involved in the VR world, more so, as the display resolution and the level of detail of the virtual worlds increased. Such acceptance and believability would be fundamental in any Mining Metaverse developed for the mining industry.
Zhao et al. (2022) describe the Metaverse as a visual world that blends the physical world and digital world and comment that the development of the Metaverse is still in the early stage with a lack of a framework for the visual construction and exploration of the Metaverse. According to Zhao et al. (2022), the Metaverse is still in its infancy of development, no work has been done to systematically summarise the technical framework for its complete visual construction and exploration, nor have graphics, interaction and visualisation been explored separately from the context of the Metaverse.
From a mining industry applications perspective, this would certainly appear to be the case as there are currently almost no case studies or examples of how people will interact with a Mining Metaverse. However, digital twins have been explored in detail by Qu et al. (2023) who demonstrate the complexities of developing digital twins and hence virtual worlds suitable for mining operations.
Ritterbusch and Teichmann (2023) provide a systematic literature review for defining the Metaverse and comment that the Metaverse could become the next generation of the internet and establish Web 3.0 or at least part of it. Ritterbusch and Teichmann (2023) comment that with the announcement of Meta's development of its own Metaverse in 2021 (known as Horizon Worlds), Meta was trying to combine its market position in mixed reality, and also its market leadership in social media. Ritterbusch and Teichmann (2023) mention that potentially, a Metaverse could be used with conventional flat screen displays, but one of the core ideas behind the Metaverse is the concept of immersion, where mixed reality excels.
According to Ritterbusch and Teichmann (2023), this refers primarily to system immersion, which is characterised by a simulated reproduction of human perception via multisensory display and tracking technologies in the form of feedback and feedforward. This system immersion is brought about in particular with virtual reality, mixed reality and augmented reality technologies.
Ritterbusch and Teichmann (2023) present some 28 definitions of what a Metaverse is but it is difficult to quantify exactly what it provides other than an interactive virtual world where the user develops content in isolation or collaboratively as shown in Figure 1. The only real difference from the Virtual Reality worlds described by early researchers (Bise, 1997; Denby et al., 1998; LeBlanc-Smith et al., 1998; Milgram and Kishino, 1994; Schofield et al., 1994; Squelch, 1997) is that these would now be persistent as in the case of second life (Linden, 2023) or Fortnite (Fortnite, 2023), for example.
Guan et al. (2022) comment that the Metaverse encompasses technologies related to the internet, virtual and augmented reality, and other domains towards smart interfaces that are hyper-connected, immersive and engaging. However, Guan et al. (2022) remark that Metaverse applications face inherent disconnects between virtual and physical components and interfaces. Guan et al. (2022) also explore how an Extended Metaverse framework can be used to increase the seamless integration of interoperable agents between virtual and physical environments and present an early theory and practice towards the synthesis of virtual and physical smart environments anticipating future designs and their potential for connected experiences.
As can be seen from the above discussion, in recent years, many scientific articles have been published on the various Metaverse concepts (Althoff, 2022; Anon, 2021, 2022; Ball, 2021, 2022; GlobalData, 2022; IEEE, 2023; Koziol, 2022; Liu et al., 2023; Metamandrill, 2022; Mileva, 2022; Mining-Technology, 2022; Novak, 2022). However, taxonomy, components, applications and challenges that enable the realisation of the Metaverse still require some considerable investigation and structure for it to be realised sustainably.
A whitepaper developed by Standards Australia, by Wallace et al. (2023), suggests there is a need for an industry-led analysis to establish whether the technology is truly feasible and sustainable because according to Wallace et al. (2023), there are many associated risks such as Human Risks, Societal Risks, Regulatory Risks, Legal Risks, Information Risks, and Financial Risks. More specifically Wallace et al. (2023) remark that there are also user on user risks, bad actors risks, corporate abuse risks and psychological wellbeing risks. Those at risk from the Metaverse include children and young people, marginalised and disadvantaged communities, elderly, individuals with disabilities or health conditions and businesses and organisations (Wallace et al., 2023). Clearly this is a new area, and some formal structure or initiative is required to manage Metaverse a development and integration. Their observations of Wallace et al. (2023) are relevant to any Mining Metaverse too, especially if used for community engagement or mine planning activities in remote locations. This highlights the important roles played by cybersecurity, standards and regulation.
Metaverse realisation
Smart et al. (2007) report on the Metaverse Roadmap (MVR) that covers a near-term and longer-term speculation horizon to 2025. According to Smart et al. (2007), the MVR provides a broad definition of what the Metaverse may be, but admit there is no one single unified entity but rather there are multiple mutually reinforcing ways in which virtualisation and 3D web tools are becoming commonplace. Such technologies will provide benefits, investments and customer interest and be subject to drawbacks and unintended consequences. It is these unintended consequences that would be of concern to the mining industry because allowing too broad a user base may introduce extreme negative elements into the Mining Metaverse that may not provide positive experiences or outcomes. Smart et al. (2007) believe that in time, Internet activities will migrate to the 3D spaces of the Metaverse. Perhaps their most relevant comment by Smart et al. (2007), is that the emergence of a robust Metaverse will shape many technological realms that presently appear non-Internet related. This may be true during the realisation of digital twins that incorporate bi-directional data from disparate data sources, and which form a Mining Metaverse.
The Metaverse is generally considered to be a persistent computer-generated virtual world that may be accessed via a HCI boundary layer that sits between the real world and the Metaverse as shown in Figure 1. The boundary layer between the real world and the Metaverse is complex because it must access and connect to the enabling layers or pillars of the Metaverse successfully, to provide entry to the Metaverse experience itself. Figures 3 and 4 and Table 1 show that there are approximately eight interconnected domains (or pillars) and numerous components within each that enable the full experience of the Metaverse concept to be realised. The process of development and delivery of a persistent Mining Metaverse straddles Domains 1, 3 and 5 (Figure 3) and any user interaction will span all eight domains.
Mining metaverse domains with a governing foundation layer and capability pillars supporting the metaverse.
Example framework and governance process for producing a collaborative and sustainable mining metaverse—domain 1.
The foundational components, tools, user interfaces and experiences of the metaverse (modified after GlobalData, 2022).
Holograms suitable for the Metaverse may not yet exist, however, this future technology may for a key unintrusive user interface.
Neural implants are described by (Waltz, 2020) however, they are likely to be a future technology.
Importantly, each domain is connected to every other and hence as a result may be present across all domains. For example, governance inputs from AusIMM, Standards Australia and Engineers Australia, would straddle not only the foundation domain, Domain 1 (Figures 3 and 4), but will likely extend into each other domain. This would also be the case for the other domains (Pillars). It is important to understand that everything is interconnected and interdependent. If one domain fails, then all the others will fail. Without some system of standardisation and collaboration during their development, the task of Mining Metaverse realisation may be unsustainable for any useful and sustainable industrial application. The components of each domain are shown in Figure 5(a)–(j) and discussed below. The list is not exhaustive, and some components such as artificial intelligence (AI), may not fit in or occur at only one location but may extend across all domains.
(a) Domain 1—governance, ethics, regulation, standards and moderation (foundation layer). (b) Domain 2—Infrastructure and communications (Pillar). (c) Domain 3—Example HCI devices (Pillar). (d) Domain 4—Decentralisation components for Metaverse realisation (Pillar). (e) Domain 5—Components for spatial computing. (f) Domain 6—Collaborative realisation. (g) Domain 7—Discovery. (h) Domain 8—Experience (Layers concept after Metamandrill (2022)).
Domain 1—governance, ethics, regulation and moderation
Domain 1 (Figures 3–5(a)) should be considered as the foundation. Domain 1 includes governance, ethics, regulation, standards and moderation and professional industry bodies such as the International Standards Organisation, Standards Australia, Engineers Australia and Australian Institute of Mining and Metallurgy. State and Federal Governments should be included in this domain to ensure that any Mining Metaverse is safe and compliant for people to interact with and use. It will be essential to have some accountability, which governments can provide.
An important aspect of Domain 1 is ‘feed forward’ and ‘feedback’ between peak professional industry bodies, and end user's experiences, which will govern the user experience and allow it to evolve safely within international standards and guidelines. Hence, it would be a process of continual improvement that would be present and essential in all domains and would facilitate the creation of new local and international standards from user's feedback.
Domain 2—infrastructure and communications
Domain 2 (Figure 5(b)) is a pillar and includes infrastructure and communications such as WIFI, 5G, WIFI 6 Intel (2023d), Cloud, 7 nm and 14 nm advanced silicon CPUs, Micro-Electro-Mechanical Systems, graphics processing units (GPUs) and immerging materials, all of which allow the persistent connection and display of the Metaverse and hence its virtual worlds (Metamandrill, 2022).
WIFI is a group of wireless networking protocols based on the IEEE 802.11 network standard (Intel, 2023d). It has been around since the late 1990s and has improved dramatically in the last decade (Intel, 2023d) with better upload and download speeds due to increased bandwidth achieved by Wi-Fi 6 (Intel, 2023d) and WIFI 6E for wireless VR.
Cloud computing is on-demand delivery of IT resources over the Internet. Instead of buying, owning, and maintaining physical data centres and servers, users access technology services, such as computing power, storage and databases, on an as-needed basis from a cloud provider (Amazon, 2023b). Microsoft (2023) describes the Cloud as a vast global network of servers, each with a unique function, that are connected to operate as a single ecosystem and are designed to either store and manage data, run applications, or deliver content or a service such as streaming videos, webmail, office productivity software or social media. Files and data are accessed from any Internet-capable device. Information is available anywhere and at any time (assuming there is a connection).
An issue for the Mining Metaverse connectivity may be that it is likely to be too large an entity for smaller vendors to operate and manage effectively. It may be an opportunity for larger corporations to collaborate to provide a common platform for users to access common experiences. Where a specific or unique application is required, then individuals or corporations can leverage a baseline Mining Metaverse and build upon it for their specific requirements. Such a platform would also be powerful because, via a collaborative development and integration approach, the issue of corporate data protection could be managed effectively. In its simplest form, this is because firstly, everyone would use the same platform with well-developed security tools and data management based on international standards. Secondly, users and companies would be registered to use the capability and hence data can be accessed collaboratively where appropriate and securely where some intellectual property or sensitivities are present.
Kan (2021) and Fox (2022) discuss the impact of CPUs and how chip-makers try to continually miniaturise their technology and processors and pack more transistors on to a single silicon chip. This can create more energy-efficient and faster processors that give smartphones and laptops a longer battery life, thermal efficiency and reduce energy consumption by data centres, while maintaining high clock speeds. Intel's current-generation chips are described as 10 nm chips, and its next-generation will be 7 nm chips.
From a Mining Metaverse perspective and within the spirit of sustainability (Brundtland, 1987), energy consumption for data centres that maintain the persistent virtual worlds for possibly millions of users would be a significant issue that must be managed.
Electro-mechanical systems include electrical and mechanical devices combined to provide user interaction with a computer system, inputs, and output from sensors. The simplest example for the Mining Metaverse would be the keyboard and mouse from a personal computer (Figures 6 and 7). Domain 2 devices such as Omniwalkers (Suryajaya et al., 2010), used for navigation within virtual worlds, maybe electromechanical for example.
(a–e) Various iterations of virtual reality simulation systems that could potentially be an access point to the Mining Metaverse.
Potential methods and apparatus for accessing the mining metaverse.
According to Intel (2023c) graphics processing units, or GPUs, are one of the most important computing technologies for personal and business computing. They are used in a wide range of applications, including graphics and video rendering, and are best known for their capabilities in gaming. GPUs are popular for use in creative production and AI and were originally designed to accelerate the rendering of 3D graphics. GPUs have since become more powerful and efficient, enhancing their capabilities to allow graphics programmers to create more interesting visual effects that make a virtual scene look real. This high fidelity is considered important for acceptance of Mining VR (Stothard and Galvin, 2005; Stothard et al., 2001), and would apply to the Mining Metaverse. Other developers have also begun to leverage GPUs to dramatically accelerate additional workloads in HPC, deep learning and more. GPUs are an essential component of Domain 2 as they allow the Metaverse virtual 3D worlds to be presented.
Nvidia has contributed considerably to the development of high-resolution virtual worlds via, real time ray tracing (NVidia, 2024a), cloud gaming (Alarcon, 2049) and AI computation (NVidia, 2023), all of which will be significant components for any Mining Metaverse. Nvidia describes its omniverse as, ‘an extensible platform for virtual collaboration and real-time, physically accurate simulation. Creators, designers, researchers, and engineers can connect tools, assets, and projects to collaborate in a shared virtual space. Developers and software providers can also build and sell Omniverse Extensions, Apps, Connectors, and Microservices on the Omniverse platform to expand its functionality’ (NVidia, 2024b).
Domain 3—HCI devices
Domain 3 (Figure 5(c)) covers the HCI, and Figure 1 shows the boundary between the real world and the Metaverse where a HCI is required. In its simplest form, the HCI could simply be a computer monitor or screen where a user interacts with the graphics on the monitor via a touchscreen, joystick, gamepad, keyboard and/or mouse as shown in Figure 6(a)–(e). There are many more HCIs that enable users to engage with real and virtual worlds and hence the Metaverse. Table 2 lists such devices which are discussed in detail by Stothard et al. (2008). Such devices may enable users to access the Mining Metaverse and range from Class 1: Monitor-based video displays with capability to show video and computer-generated graphics simultaneously, through Class 6: Large screen(s), completely graphic environments that are partially immersive and use real physical objects or haptic devices to play a role in the computer-generated scene, and on to futuristic, Class 10: True holographic devices. Some devices are more complex than others and some, for example, ‘Class 10—True holographic devices’, are very uncommon and not really in the commercial domain as of 2023.
Classification of mixed reality display systems for engaging with the metaverse (after Stothard and Squelch et al (2008)).
Figure 7 shows various options including neural implants, which would be a significant milestone in HCIs and are unlikely to be commonplace for some time yet. An important consideration for this domain is human factors (HF), an area that is often ignored by technology and system developers alike. HF can impede the adoption of Virtual Reality, Augmented Reality and hence Metaverse technologies if not considered and managed correctly. HF is discussed in detail by Stone (2018) who emphasises the importance of adopting HF techniques and knowledge when developing VR, AR and mixed extended reality (MxR) and presents relevant MxR case studies in defence and healthcare to support the premise.
In Figure 7, an important aspect and capability is the ability to combine real and virtual worlds as described by Milgram and Kishino (1994). While required for the Metaverse in general, this may not be essential if people are operating within purely virtual worlds. However, for a Mining Metaverse, having the ability to overlay computer-generated images to produce a mixed reality experience would be very valuable for community engagement, mine planning, safety or compliance monitoring, for example. The key would be the ability to collaboratively design, assess and make decisions based on past, present and predicted data. The new generation of head-mounted displays such as Meta Quest 3, HTC XR Elite and Apple VisionPro, allow seamless transition between virtual reality and mixed reality.
Domain 4—decentralisation
Domain 4 (Figure 5(d)) includes decentralisation as a key component for the Metaverse and includes edge computing, AI, microservices and blockchain, for example (Metamandrill, 2022).
Edge computing
Edge computing is a relatively new field and is essential for digital twins. Its architecture, benefits and limitations are described by many authors, for example, Talebkhah et al. (2020), Kaur and Batth (2021), Xue et al. (2021), Wu et al. (2022). According to Xue et al. (2021), rapid development of 5G and AI technology has resulted in intelligent devices being connected to the Internet with the result that demand for computing services has increased. Traditional combinations of cloud computing and Internet of Things (IoT) systems have revealed limitations and network bandwidth pressure. Edge computing places the service near the physical edge of the end users and due to its close proximity, dense distribution and low latency, edge computing can reduce system latency, improve transmission speed and relieve bandwidth pressures.
According to Azure (2023), edge computing allows devices in remote locations to process data at the ‘edge’ of the network, either by the device or a local server. When data needs to be processed in the central datacentre, only the most important data is transmitted, thereby minimising latency. For remote mine operations as shown in Figure 1, this would be a very important capability and function especially as people run scenarios or data acquisition and analysis within a Mining Metaverse. Azure (2023) comments that, ‘without edge computing, the massive volume of data generated by edge devices would overwhelm most of today's business networks, hampering all operations on an affected network. Valuable machinery could be damaged or simply be less productive but most importantly, workers’ safety could be compromised in industries that rely on intelligent sensors to keep them safe’.
Amazon (2023c) describes edge computing examples and benefits as being high speeds and low latency of data transfer, combined with the relative ease of installing edge devices. Examples include,
Manufacturing where the IoT devices such as sensors and gateways have made edge computing systems prevalent (Amazon, 2023c). Edge computing enables automation, collects data on-site, improves production efficiency and allows rapid machine-to-machine communication (Amazon, 2023c). Autonomous vehicles, fitted with IoT sensors collect large amounts of data requiring real-time data processing for instant response and cannot rely on a remote server for split-second decision-making (Amazon, 2023c). According to Amazon (2023c), autonomous vehicles interact more efficiently if they communicate with each other first, as opposed to sending relevant data to a remote server. Amazon (2023c) comment that Edge computing is critical for ensuring safety and ability for autonomous vehicles to accurately judge road conditions. The car manufacturer Mercedes-Benz has partnered with NVIDIA to develop software-defined vehicles (Shapiro, 2023). Amazon (2023c) report that energy companies use edge computing to collect and store data on oil rigs, gas fields, wind turbines and solar farms. Rig operators commonly deploy edge AI to detect hazards and optimise and inspect pipelines. Edge computing helps the industry improve operational efficiency, keep its workers safe and forecasts when maintenance work needs to be undertaken (Amazon, 2023c). In Healthcare, edge devices monitor critical patient functions such as temperature and blood sugar levels (Amazon, 2023c). Apparently, edge computing allows storage of patient data locally and improves privacy protection. Also, medical facilities reduce the data volume they send to central locations and reduce the risk of data loss (Amazon, 2023c). Within the mining industry, edge devices may contribute to data management for devices that are monitoring and controlling the mine environment, such as sensors used for gas monitoring, rock mechanics instrumentation, machinery and personnel location and fatigue management, for example. Data from IoT and edge devices can be stored locally and analysed automatically. Analysed, relevant and critical data can then be transmitted to central locations for integration into the mining metaverse and operational decision-making. Essential data, flagged by the system can then evaluated by a person. A challenge for edge devices located in the mining environment will be cooling, water ingress, dust and other environmental effects as well as ethernet access. Within underground coal mining, there will be a need for intrinsically safe and flame proof equipment that can house the various required sensors.
Artificial intelligence
AI is the ability of a computer-controlled system to conduct tasks with minimal human intervention in an intelligent manner. According to Russell and Norvig (2022) AI would likely span many domains, as would other components and may help decentralise information by distributing computing processes to various external locations globally.
AI promotes machines and robots to mimic human brain activities, such as perception, decision-making, and feedback. Robust AI algorithms, within the mining industry, have been utilised for operational decision-making, performance evaluation and automation (Shirani Faradonbeh et al., 2022). AI plays a critical role in the fields of simulation within mining operations due to image processing algorithms and it is possible to extract valuable information from images resulting in innovative functions, such as autonomous mapping, object detection, motion analysis, etc.
Generative AI is described by Intel (2023b) as a solution that generates content, whether it is a demand generation email, a fantastic landscape or a dynamic chatbot reply, in response to a user prompt. Generative AI may have significant value in a Mining Metaverse for generating scenarios, determining best practice solutions based on case studies, or optimising mine planning procedures. The advancements in GPU technologies allow photorealistic virtual environments to be generated, potentially by generative AI in semi-real time, thus allowing people to become immersed in a computer-generated image of the real world as suggested by Figure 2 that is automatically generated by the system as mining progresses.
Microservices
Microservices are described by Amazon (2023a) as an architectural and organisational approach to software development where software is composed of small independent services that communicate over well-defined APIs and are owned by small, self-contained teams.
Microservices perform a single function and because they are independently run, can be updated, deployed and scaled to meet demand for specific functions of an application. For remote mine sites such independent microservices would be invaluable for data analysis and other mining operations.
Blockchain
According to IBM (2023), Blockchain is a shared, immutable ledger that facilitates the process of recording transactions and tracking assets in a business network. An asset can be tangible (a house, car, cash, land) or intangible (intellectual property, patents, copyrights, branding). Virtually anything of value can be tracked and traded on a blockchain network, reducing risk and cutting costs for all involved. Intel (2023a) presents Blockchain as a technology that has the potential to enable everyone to own much of the digital content and services they create. Blockchain fundamentally disrupts the way digital assets are stored, processed, and transacted as the era of Metaverse and Web 3.0 become established. For security in mining projects within the Mining Metaverse this is a very powerful capability and will be an essential component.
Blockchain is important because mining operations run on information and the quicker it is received and the more accurate it is, the better the outcome. Blockchain provides immediate, shared and completely transparent information stored on an immutable ledger that can be accessed only by permissioned network members, an essential aspect for mine site data. Blockchain networks can track orders, payments, accounts, production and much more. Because members share a single view of the truth, all details of a transaction can be seen end to end, providing greater confidence, as well as new efficiencies and opportunities (IBM, 2023).
Domain 5—spatial computing
Domain 5 (Figure 5(e)) includes spatial computing components such as 3D game engines or similar 3D presentation software, VR/AR/MR software, multitasking user interfaces and geospatial mapping capabilities such as Extensible 3D (X3D), Google Earth and Digital Earth Australia. It is worth noting here that VR, AR and MR sit under an encapsulating umbrella term of extended reality (XR) and according to Standards Australia (2023), XR refers to all real-and-virtual combined environments and human-machine interactions generated by computer technology and wearables.
There are several 3D game engines and ‘Metaverse’ like platforms of varying capabilities, such as Castle Game Engine, Unity, Unreal Engine, WebXR, VR Chat, Rec Room, Horizon World, Fortnite, Roblox, Decentraland, Second life and Minverso. Many are commercial and can generate high-resolution virtual worlds that have become expected of modern computer games, and they can produce very realistic-looking environments. Not all are persistently evolving (e.g., Fortnite). A brief interaction with Unity, Unreal, Fortnite, Decentraland and Second Life, soon reveals the commercial intent of these tools and users are quickly asked to purchase options and subscriptions, etc. This is an area that would be managed by the foundation domain of Figures 3 and 4 (Domain 1).
X3D is a spatial computing and simulation environment that is a royalty-free, platform-independent, open-standard file format specification and run-time architecture to represent and communicate 3D scenes and objects on the Web (Web3D, 2023) The power of X3D is that it is based on international standards and is interoperable with many other systems. For development of a mining industry Metaverse, this approach should be considered, albeit in collaboration with larger service providers and stakeholders as discussed earlier and shown in Figure 4.
Spatial computing is an essential component of any Mining Metaverse and the need for collaboration and standardisation that enables access for users at all levels and interoperability between systems cannot be understated. One of the barriers to adoption of Virtual Reality simulations during the 2000s was that proprietary software was usually required to realise spatial 3D virtual environments. This often came with associated licencing fees that become prohibitive as projects moved into the commercial domain and quantities of scale were reached where many systems require many licences and so on. This can prove unsustainable for some projects over time.
Domain 6—collaborative realisation
Collaborative realisation (Figure 5(f)) includes aspects such as networks, social interaction between operations and sites, knowledge and asset curation, consensus, ratings, decolonisation, agents and repositories (Metamandrill, 2022).
Modern mining operations are complex ventures, with many aspects, including those within the domain of collaborative realisation, that must be managed effectively for the mine to remain profitable and maintain its social licence to operate. The nature of the industry in some locations is such that people move from position to position and site to site during their careers and there is an opportunity for corporate knowledge and experience to be lost and hence this must be relearned by new incumbents. Collaboration within a Mining Metaverse may help alleviate this issue by allowing projects to be designed, developed and delivered via best practice obtained through social interactions and networking of site staff. Ideas and options can be recorded or archived, previous knowledge and experience considered and applied to new ore bodies and mine designs. Past experiences and future predictions can be made in a safe and forgiving environment and transparency of mine design and long-term outcomes can be achieved for mine operators and governments alike.
The power of a mine model built within a Mining Metaverse would be that it is a dynamic model that can be used to achieve consensus, improve sustainability rating of projects. It could promote decolonisation by including local stakeholders within mine operational decision and can form a repository for past, present and future knowledge. Agents can be used within the Mining Metaverse to help users interact with data, assess and analyse it, and achieve sustainable outcomes for all. The potential and power of collaborative realisation of mine operations via a Mining Metaverse should not be underestimated and requires further academic research.
Domain 7—discovery
The Discovery domain (Figure 5(g)) would include components such as serious games, simulations, visualisations, team building and case studies (Metamandrill, 2022) that are designed for users to be able to explore and discover information, experiment with data or experiences and learn, use and apply that acquired knowledge in a new situation or environment. Serious games allow users to interact with a mine site environment, for example, a mine construction site and learn about the site, its associated risks and how to avid and mitigate those risks. An example is presented by Stothard and Hengel (2009).
Domain 8—experience
Domain 8 (Figure 5(h)) covers experience, that from a mining engineering perspective, would include: in-depth mine site experience gained via interaction with the Mining Metaverse; establishing empathy with all stakeholders so that all points of view are understood and respected and informed compromise reached during negotiations; improved stakeholder engagement via providing users with experiences that show past present and future data sets and outcomes in context; and improved mine sustainability derived from improved simulation of mine site operations and management of short term, intermediate and long term risks. The Mining Metaverse has the potential to provide a rich knowledge base of experiences gained from collaboration among industry experts, academics, government bodies, local and global stakeholders, and importantly people drawn from a broad cross-section that can add debate, discussion and consensus on whether a mine site fits within the spirit of sustainable mining. Within mining engineering, there has been much interest in VR and AR, but it has not matured; the experiences that a Mining Metaverse could provide may add value and improve VR and AR industry acceptance and engagement.
Discussion and conclusions
In its simplest form, the Metaverse appears to be a set of persistent, ‘always on’ virtual worlds that can be accessed by many people, agents and entities at any time. The Metaverse may or may not represent the real world and is only limited by people's imagination. Access to the Metaverse can be made via HCIs of varying sophistication.
The Metaverse could generally be thought of as a continually improving and persistent computer-generated environment. In this article, an industrial metaverse is described which would be significantly different to a consumer metaverse, where anyone can gain access to participate in commercial, social, or other activities. The industrial, Mining Metaverse, would be much more controlled and probably restricted to those working for an associated mining company or service provider.
The Metaverse is not a new concept, but it has gained a lot of attention in recent years. This resurgence in interest is largely due to major global IT companies and service providers tapping into the market for Head Mounted Displays (Classes 1–4 and 8–9 of Table 2) and the like. Also, the relatively easy access to cloud computing and high-resolution imagery generated by advanced computing has had a significant impact on the state of the art.
Many prototype Metaverse online platforms have emerged that claim to provide a fully immersive Metaverse experience however, the technology maturity is unclear and access to such technologies requires the payment of licences and subscriptions in most cases. Development of these ‘Metaverses’ does not appear to be led by an international standard or collaboration. The collaborative international standards approach taken by the Web3D Consortium may be a more sustainable and professional approach for developing a Mining Metaverse. The key will be an industry-led initiative that is steered by industry professional bodies that guide commercial entities to develop interoperable systems that are open to all.
The concept of the Mining Metaverse was introduced that would contain computer-generated virtual worlds that can be accessed simultaneously by many people whether they be onsite, remote, underground or even in the future, as an historic case study.
To begin to quantify what the Metaverse would provide be from a mining perspective, the concept of a Mining Metaverse was developed and a Taxonomy that introduces the various domains that facilitate such a Mining Metaverse was introduced. The concept of a digital mining twin was also introduced.
The Metaverse concept described here is based on a set of underpinning domains, each of which is essential to its realisation. Some of the complexities within these domains are also discussed and the need to have collaborative approaches to their development and integration via standardisation is raised. This would help reduce risks such as cybercrime, misuse, commercial exploitation and so on since the Metaverse has many interdependent components that must interoperate.
Case studies from earlier virtual reality projects were introduced as examples to demonstrate the complexity of developing a persistent Mining Metaverse. It must be understood that it is not a trivial exercise to realise this technology sustainably within the mining industry, or any other industry for that matter.
The very complexity of such a large system suggests that its development must be led by an industry/government/research collaboration that produces a Mining Metaverse capability that leverages the capabilities within service providers like Amazon, Google, Microsoft, Meta, Nvidia and international collaborations such as ISO, IEEE, etc. Working collaboratively between these groups will allow a baseline Mining Metaverse to be established that meets the needs of all the mining industry stakeholders and which can be customised and tailored to meet the needs of a specific mining organisation or associate group as a ‘microverse’ within the larger Mining Metaverse.
The approach taken by the Web 3D Consortium and the Naval Postgraduate School (NPS) is a powerful initiative. Building upon that work and the work of the International Standards Organisation (ISO) JTC 1/SC 24 (Computer graphics, image processing and environmental data representation) and Standards Australia IT-031 Modelling and Simulation Committee, would be a good course to follow for development of a Mining Metaverse project. Partnering with large commercial entities would also be beneficial provided that the spirit of collaboration is maintained.
One of the key pillars is Domain 2—HCIs. At present, the hype that promotes the Metaverse suggests that head-mounted displays will be the principal portal to the Metaverse. However, the nature of head-mounted displays is such that there is a cost associated with ownership. This may present a barrier to access for people in locations where they cannot afford such items. Hence, a better access method may be a simple personal computer, a smart TV or tablet. Such devices are commonplace globally and do not necessarily have the human factors issues associated with them such as hygiene, motion sickness, isolation, etc. The governance layer of Domain 1 should help promote safe access via local and international standards.
Finally, the previous sections also allude to the potential fragility of the Mining Metaverse because it has at least eight domains (Figure 3) comprising of many interconnected components and systems and if one fails or is restricted, it will lead to issues within the others. The system could be held to ransom if not designed correctly. The Mining Metaverse is an opportunity for mining to develop another complementary tool to enhance existing and future mining operations and improve mine site sustainability.
Footnotes
Acknowledgements
The authors would like to acknowledge the activities of Standards Australia IT-031 Modelling and Simulation and ISO/IEC JTC 1/SC 24/WG 11 ‘Health, safety, security and usability of Augmented & Virtual Reality (AR/VR)’ for their support and inspiration for this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
