Interactive immersive experience: Digital technologies for reconstruction and experiencing temple of Bel using crowdsourced images and 3D photogrammetric processes

Abstract

This paper investigates the potential of dense multi-image 3D photogrammetric reconstruction of destroyed cultural heritage monuments by employing public domain imagery for heritage site visitors. This work focuses on the digital reconstruction of the Temple of Bel, one of the heritage monuments in Palmyra, Syria, which was demolished in the summer of 2015 due to armed conflict. This temple is believed to be one of the most significant religious structures of the first century AD in the Middle East and North Africa (MENA) region with its unique design and condition before destruction actions. The process is carried out using solely one source of images; the freely available visitors’ images collected from the social media platforms and web search engines. This paper presents a digital 3D reconstruction workflow for the collected images using an advanced photogrammetry pipeline and dense image matching software. The virtually reconstructed outputs will be managed and implemented efficiently in Unity3D to create an entire 3D virtual interactive environment for the deconstructed temple to be visualised and experienced using the new Oculus Quest VR headset. The virtual Palmyra’s visitor will be offered an enhanced walk-through off-site interactive, immersive experience compared to the real-world one, which is non-existing and unobtainable at the site in the current time.

Context

Palmyra contains the colossal ruins of a great city that was one of the most significant cultural centres of the ancient world. From the first to the second century, its art and architecture standing at the crossing of several civilisations, combining Graeco–Roman techniques with local traditions and Persian influences. This city is one of the Middle East’s most impressive heritage sites in terms of the quantity and quality of its monuments. It is in the heart of the Syrian desert and could be considered a part of ancient Syria’s heritage identity. It represents a symbol of the Syrian archaeology and one of the UNESCO world heritage sites; it was registered on the UNESCO World and National Heritage Lists in 1980, and to the List of World Heritage in Danger in 2013 because of the Syrian civil war. ¹

After 2015, Palmyra came under the control of ISIS, and many artefacts and monuments were deliberately sabotaged by the group or destroyed during the armed conflict. ¹ One of the well-known destroyed monuments, the Temple of Bel, was regarded as one of the most significant religious monuments of the first century AD in the Middle East for its exceptional design, quality of details, and physical condition (before being destroyed). Since 2011, the archaeological site of Palmyra was closed, and it has not been opened for visitors for more than 8 years. The physical heritage site was inaccessible due to the continuous clashes. The BBC reported ISIS-published video footage revealing that some people were killed in the site of the Roman Amphitheatre, which was considered as the first time a heritage site was used as a place for killing, which may induce negative and traumatic associations for many people. ²

In 2015, many monuments were blown up or destroyed by ISIS in the areas under its control in the North-East of Syria. Between 2015 and 2016, vast archaeological areas have had many monuments removed during a short period. Based on extreme ideology, the radical Islamic group ordered their militia to destroy all the physical assets. In Palmyra, the most celebrated monuments were destroyed, including the following: Arch of Triumph, Baths of Diocletian, Nymphaeum, Tetrapylon, The Roman Amphitheatre, Agora, The Grand Colonnaded Street, Temple of Nebu, Temple of Baalshamin, and Temple of Bel. A massive collection of the most magnificent art pieces depicting early Greco–Roman architecture and art that survived for 19 centuries was pulverised into rubble for religious and political motivations. The temple of Bel is considered as one of the most valuable religious structures of the first century AD (around 32 AD) in the levant because of its distinctive design and intact physical condition before destruction.

The main temple mass was set down on a platform in the centre of an open square, approximately 175 × 185 m. It was well outlined by an arcade of around 30 m depth, which means the external dimensions of the frame were enlarged to 205 × 210 m. The external wall had Corinthian columns, and the porch had pair rows of the same style columns. However, the western side was only had a single row. The temple complex included various facilities necessary for the rites of Bel. The temple is asymmetrically situated in the heart of the courtyard, fronting the north and the central axis of the temple, separated the courtyard into two main parts with a proportion of 3:2. The eastern and western parts are roughly 70 m and 105 m, respectively, and the main axis tilts by 5° from the north-south axis, the central mass of the temple is placed over a rectangular platform of 29.52 × 54.6 m. The Cella also had a rectangular shape of 15.8 × 39 m, walled by a single row of Corinthian columns. ³ The columnar arrangement was Pseudodipteral. ² There were eight Corinthian columns at the northern and southern elevations, 12 at the western, and 14 at the eastern elevation. The height of the Cella was 33.14 m (at the apex of the roof ornaments), with columns of 15.81 m in height, with a 1.33 m base diameter. ³

The contribution of digital technology in cultural heritage domain

Middle East and North Africa region is an ancient cradle of many civilisations that have witnessed numerous traumatic conflicts and upheavals that have caused catastrophic damage to many heritage sites, as well as human lives. Recently, UNESCO confirmed that several cultural heritage sites in this region face different and various considerable threats due to different reasons, that is, armed conflicts, mass tourism, and natural disasters. ³ Today, more than ever before, the world is losing these icons faster than they can be recorded and documented. Therefore, the necessity arises to find new propositions for documenting and preserving all these treasures, lest they are lost to us forever. Virtual Heritage (VH) is one of the computer-based interactive technologies in a virtual reality where it produces a visual representation of artefacts, monuments, structures, and culture to deliver openly to global audiences.^4,5 El-Hakim et al., ⁶ introduced several motivations for any VH reconstruction, such as recording monuments and artefacts for reconstruction in case of disasters or unexpected events. Reconstructing digitally historical monuments that no longer or only partially exist, would allow visitors for interacting with artefacts without risk of damage, visualising scenes from viewpoints impossible in the real world due to size or accessibility issues and finally offering virtual tourism in the cases of restricted humans’ mobility and accessibility, for instance, the travel ban that recently imposed in different countries over humans due to the COVID-19 pandemic.

The last two decades have witnessed an unprecedented transformation in generating 3D models, and digital replicas, as many artefacts, monuments, and whole heritage sites have been documented and transformed into various digital versions. ⁷ The significance of adopting digital and virtual reconstruction is to protect, preserve and interpret culture and history, ⁸ bringing back some elements – either physical or nonphysical assets exist and non-exist – from the past to our real-time life. ⁹ Such created digital models are flexible and variable in including different added graphical and textual information, such as dimensions, colours, textures and some data related to the artefact’s historical period. ¹⁰

3D photogrammetric reconstruction processes

The most common used practise of 3D photogrammetric reconstruction in the field of cultural heritage is capturing, documenting and representing digitally different historical assets based on multi-images matching. ¹¹ By enhancing digital tools and photogrammetric reconstruction technology and the capabilities of new digital cameras, this method has become more effective and less costly. ¹² Employing such technology of generating 3D models from 2D images allows to understand the monument and its environment virtually, with less need for any direct physical contact. ¹³ Digitising heritage assets offers new approaches not only to preventing such valuable monuments from degradation and destruction but also bringing back some non-exist artefacts virtually through employing 3D photogrammetric modelling and remote sensing utilising Structure from Motion (SfM) point cloud and followed by Multi-View Stereo (MVS) densification processes.^14,15

The 3D reconstruction of physical heritage assets using either laser scanning or 3D modelling software is expensive and involves a high level of expertise. On the other hand, the multi-image 3D remote-based reconstruction technique proposes a cheaper alternative, carrying out this task with relative ease. ¹⁶ In recent years, such virtual construction methods have gained attraction in the advanced computer graphics realm under the name of image-based modelling. ¹⁷ This process involves creating 3D models from an assortment of input images and supported with particular developed computer-vision algorithms, that is, SfM and MVS. ¹⁸ Such algorithms are being implemented extensively in the cultural heritage domain and efficiently have been employed to create accurate digital replicas for threatened or even destroyed monuments and artefacts. ¹⁹ Also, it offers the possibility to visualise and interact with VH assets without any physical contact using different immersive technology applications. ²⁰ This technology employs several photos with a fair amount of overlap to create 3D digital replicas. The recent advancement of robust computer-vision algorithms has facilitated the standard photogrammetric pipeline democratisation for 3D reconstruction purposes. ²¹ Lately, numerous free, open-sourced, easy-to-learn and user-friendly software packages have emerged to support implementing this technology in different applications. The user can effortlessly convert a set of 2D images into 3D models, that is, ARC3D, MICMAC, Regard3D, Insight3D, COLMAP and Visual SfM. Thanks to such technology where these images can be imported easily into software followed by some automated actions, that is, align photos and camera calibrations, build sparse point cloud and densification by SfM and MVS algorithms, then potentially the workflow can be accomplished by 3D mesh or creating a fully textured model.

Mainly photogrammetric pipeline generates a 3D point cloud with camera poses obtained from uncalibrated images. Agisoft Metashape software determines the geometric properties of the object from several photographic images. The process of this method requires comparing matching pixels or reference points throughout a series of images. The quality and a certain number of images are considered necessary to allow the surface to process, match and triangulate visual features and generate a 3D point cloud. The computer-vision algorithms of SfM and MVS are among the most popular techniques in image-based modelling in various software packages. This technology enables non-expert users to swiftly and efficiently capture high-quality digital replicas through uncalibrated photos taken from affordable setups, without needing any specialist hardware or artificially designed illumination conditions. Typically, the workflow follows six steps to produce 3D reconstructions/3D digital replicas:

• Image acquisition (or adding photos).

• Feature detection, matching, triangulation (or align photos).

• Sparse reconstruction (or point cloud generation through SfM).

• Dense correspondence matching (or dense cloud generation through MVS).

• Surface/mesh generation.

• Texture generation.

The internet and crowdsourced materials in digital cultural heritage reconstruction

The internet, particularly broadband facilities, have allowed all kinds of opportunities and access to an abundant amount of information, including significant access to an uncountable number of images to be captured, shared, uploaded and disseminated easily between people. Simultaneously, the advent of developed digital imaging and digital photography has completely changed the photogrammetric reconstruction progress in becoming ‘digital’. ¹⁷ Previously, it was a hurdle to capture hundreds, or even thousands, of digital photos that can be directly displayed, uploaded and analysed. At present, time and cost matters have been greatly reduced, or any practical constraints regarding the number of captured photos that can be taken for creating a 3D model. The latest smartphones provide outstanding results in capturing instant digital images close to the quality of compact SLR cameras, with high-resolution quality of up to 4K in some prototypes.

The growth in the number of captured images and their obtainability and accessibility through the internet from search engines and different social media platforms has also assisted in adopting the public domain imagery and crowdsourced photogrammetry method, particularly to reconstruct different historic sites and monuments.^22,23 For instance, in the project Build Rome in a Day, ⁴ the main project objective is reconstructing a virtual 3D model of ancient Rome based on two million public images taken from Flickr image-based platform. ²⁴ Also, the Buddhas of Bamiyan in Afghanistan have been brought back to life after being reconstructed virtually, utilising several touristic digital images taken before the statues were destroyed in 2001. ²⁵ Snavely et al. ¹⁷ discussed the 3D reconstruction of the Great Wall of China from different visitors’ images collected from the internet. Additionally, historical images were used to reconstruct lost heritage in several projects, sometimes with wide-ranging recognition, for example, the digitally reconstruction of Palmyra’s monuments in Syria by combining public domain images and professional panoramic imagery. ²³

The internet has become a prime source of visual information about our physical world, with billions of people uploading photos and videos every day to media-sharing sites at astounding levels. Almost all the world’s well-known attractions, landmarks and heritage sites (and many not-so-famous ones) have been photographed thousands or millions of times, and many of these photos can be found on photo-sharing websites and different social media platforms. For instance, a Flickr search for ‘Petra, Jordan’ results in nearly 740,000 photos (as of June 2022), showing the city monuments and Al-Khazneh from almost every possible viewing position and angle, different times of day and night, changes in season, weather, period and during different events. Meanwhile, several authors proposed the photogrammetric solution to reconstruct entire 3D architectural environments, artefacts and monuments by creating 3D photorealistic virtual environments for different heritage sites and the possibility to experience them using different extended reality applications, that is, AR and VR.^26–28

Methodology of digital reconstruction

Visitors’ photos of monuments are very beneficial to record geometric information in such case s. As conflicts have systematically destroyed the heritage of vast areas in MENA, the necessity is increasing to find a way to document those destroyed monuments, to reconstruct digital models as a basis for 3D visualisations, developing immersive reality experiences or even a base for an ultimate future physical reconstruction. 3D virtual reconstruction is also an essential statement that organised destruction and war crimes might devastate physical heritage assets, but not their virtual representation in collective human memory.

The researcher aims a methodology for digitally reconstructing the destroyed Temple of Bel in Palmyra through employing the 3D photogrammetric reconstruction pipeline and crowdsourced images that collected from the internet. This research proposes a structured workflow that relied solely on massive number of public images taken from various social media platforms to reconstruct and create 3D digital model for the Temple of Bel. 3D photogrammetric reconstruction is one of the proposed available solutions in this special case, as there is less possibility of using another method to reconstruct assets that no longer exist or are no longer accessible. The process was carried out through three main phases, as the pipeline illustrates Figure 1:

Pre-processing phase: Collecting crowdsourced materials.

Processing phase: 3D photogrammetric reconstruction using Agisoft Metashape software. This phase comprises the following main sequential steps: align photos and camera calibrations, build sparse point cloud and densification by SfM and MVS algorithms, create the main mesh and finally generate texture.

Post-processing phase: The initial 3D textured model usually contains some problems; therefore, which can be cleaned to remove unwanted parts, and be edited, refined and optimised.

Figure 1.

The primary digital reconstruction methodology. Source: Author

Pre-processing Phase: Collecting crowdsourced images

The internet has allowed all kinds of opportunities and access to an ample amount of information, including significant access to an uncountable number of images to be captured, shared, uploaded and disseminated easily between people. The growth in the number of taken images and their accessibility through the internet from search engines and social media platforms has also assisted in adopting the crowdsourced photogrammetry method, particularly to reconstruct historic sites and monuments. For instance, more than 10,000 images have been found on Google Images and Flickr for the Temple of Bel in the period between 2008 and 2015. This could make the internet a significant source for digital reconstruction for other cases in the future. For the digitally reconstruction of the Temple of Bel, one image source was available: public domain visitors’ photos collected from the web. The crowdsourced materials (images) were searched and collected from different websites and platforms, as most of them were downloaded from social media platforms such as Facebook, Twitter, Instagram and Flickr using different hashtags (#Palmyra, #Temple_of_Bel, معبد_بعل#). The search was carried out looking for tags written in several different languages (Arabic, English and French), which might indicate the place and function of the monument. Different search engines such as Google Images and Bing were also helpful, particularly the image search by similarity called ‘Reverse image search’, which yielded several untagged photos.

The researcher in this phase collected the key components and crowdsourced materials based on an extensive search on different social media platforms and search engines. The collected images are filtered to select the most appropriate photos for the digital reconstruction process.

To avoid the copyright issues, it should be explicitly clarified that the researcher only used open domain images for which the owners have granted the use of their images. Therefore, the researcher adopted the Public Domain Dedication license option (CC0) available on some social media platforms and image search engines such as Google Images to avoid copyrighting people’s images (Figure 2). This license is one of several public copyright licenses that enable the free distribution of an otherwise copyrighted work. ⁵

Figure 2.

Crowdsourced images collected from different social media platforms and search engines for the Temple of Bel. Source: Google Images.

Image filtering excluded images without Exchangeable Image file Format (EXIF) data, where metadata is embedded in the image file, as well as those with low-resolution and small dimensions, in order to improve the accuracy or resultant images. One of the problems with which researchers such as myself are faced is that many social media images contain potentially identifiable and, as such sensitive content of people, such as their faces, bodies, belongings and other personal-related information. Therefore, those images that could contain people’s faces or sensitive issues such as vulnerable people and children have been removed as a part of the filtration process.

This left a total of 319 appropriate visitors’ crowdsourced images. The collected images comprised 166 images for the exterior and 153 images for the temple’s interior, with a wide range in focal length distributed as follows: Main western façade 81, back eastern façade 43, northern façade 23, southern façade 19, Northern Adytum 56, Southern Adytum 63, western inner façade 16, and eastern inner façade 18 photos. The minimum image size was 1200 × 1600 pixels, and the period of the available imagery ranged from 2009 to 2015. Figure 3 illustrates the distribution and numbers of crowdsourced images for the exterior and interior façades.

Figure 3.

Collected crowdsourced images for temple exterior and interior showing distribution and numbers of visitor images on all façades. Source: Author.

Most collected crowdsourced photos show that visitors were more attracted and interested to capture images of specific parts of the monument, such as the main gate, the Southern Adytum and Northern Adytum inside the Cella. The collected images show a dearth of coverage for some other major parts inside and outside the Cella, such as the eastern and western flat blank internal walls. This reflects the lack of visitors’ interest to capture photos of these flat undecorated walls, due to the poor visual value (i.e. photogenic nature) of such architectural parts and components for tourists.

In terms of external façades, the western and eastern might be more attractive for visitors to be captured. Both contain fascinating architectural parts with decorated details such as the main gate, openings and large ornamented Corinthian columns. On the two side façades, we can see a kind of redundancy with high symmetrical balance between two identical Ionic capital pillars, without openings or decorated parts. As a result, the side façades present less visual attractiveness to the visitor from the perspective of visual value and the image quality, reducing visitors’ chances of being attracted to capture a photo of these views (Figure 4).

Figure 4.

Architectural components of Temple of Bel’s external façades (A) Eastern elevation. (B) Western elevation. (C) Southern elevation. (D) Northern elevation. Source: Social media platforms and edited by Author.

In some cases, the topography of the heritage site might play a vital role to block and constrain the visitor from continuing the experience and capturing photos. It could be a significant reason for the lack of images for specific main parts (façades and walls) of the monument. For instance, the northern and southern external façades have the least number of collected photos; this dearth refers to the destroyed surrounding columns on the ground, which blocked visitors to capture photos for these two façades. Such obstacles (physical barriers, i.e. dilapidated structures) might prevent visitor access either physically or visually, to capture appropriate photos for the monument from different viewpoints.

Processing Phase: 3D photogrammetric reconstruction process

A set of 319 crowdsourced photos collected from the web and different social media platforms was filtered from the previous phase and imported to Metashape software to photogrammetrically reconstruct the destroyed Temple of Bel (Figure 5).

Figure 5.

Screenshot from Agisoft Metashape software shows the cameras calibration and the parameters of different visitors’ cameras. Source: Author.

This phase mainly involves six main sequential steps. In the first trial, the 319 crowdsourced photos have been imported simultaneously, but the results were unsatisfactory and below expectations. The 3D-generated model was distorted, containing many problems in the structure, geometry and textures, and it lacked form rationalisation. This might happen due to crowdsourced photos being captured by different visitors using different cameras with different settings and parameters during different periods, making the 3D output nonhomogeneous perfectly. Therefore, as an alternative plan, the reconstruction methodology has been changed by dividing the collected 319 photos into six main groups. The temple was divided into six main parts for reconstruction, as illustrated in Figures 6 and 7, and described below:

• Part one contains the images of the western and northern façades.

• Part two includes the images of the eastern and southern façades.

• Parts three and four comprise the images of the inner space, particularly the internal façades of the holy chamber inside the Cella.

• Parts five and six involve mainly two zones: the internal decorated stone carved ceiling of the Northern Adytum (the zodiac pattern), and the Southern Adytum (the floral pattern).

Figure 6.

Part one and part two: The western and northern façades (Left) and second: the eastern and southern façades (Right). Source: Social media platforms and edited by Author.

Figure 7.

(Top) Northern and (Bottom) Southern Adytum façades (parts three and part four) and their decorated ceilings (parts five and six). Source: Social media platforms and edited by Author.

Part one: Western and Northern Façades

A set of 104 crowdsourced photos were imported in Agisoft Metashape to reconstruct the western and northern façades. Figure 8 illustrates the process of reconstruction and the final digital output of the model.

Figure 8.

3D photogrammetric reconstruction process, part one (A) Photo alignment and image triangulation. (B) Spars of point cloud. (C) Densification of point cloud. (D) Mesh generating. (E) Wireframe of the mesh. (F) Texture mapping phase using UV mapping method. Source: Author.

Part two: Eastern and Southern Façades

A set of 62 crowdsourced photos was used to reconstruct the eastern and southern façades. Figure 9 shows the process of reconstruction and the final digital output of the model.

Figure 9.

3D photogrammetric reconstruction process, part two (A) Photo alignment and image triangulation. (B) Spars of point cloud. (C) Densification of point cloud. Source: Author.

Part three: The Northern Adytum

A set of 25 crowdsourced photos was imported in Agisoft Metashape to reconstruct this part (Figure 10).

Figure 10.

3D photogrammetric reconstruction process, part three (A) Photo alignment and image triangulation. (B) Spars of point cloud. Source: Author.

Part four: The Southern Adytum

Twenty-eight crowdsourced photos were used to reconstruct the Southern Adytum. Figure 11 shows the process of reconstruction and the final digital output of the model.

Figure 11.

3D photogrammetric reconstruction process, part four (A) Photo alignment and image triangulation. (B) Spars of point cloud. Source: Author.

Part five: The decorated stone carved ceiling of the Northern Adytum (The Zodiac Pattern)

Thirty-one crowdsourced photos were used to reconstruct this part (Figure 12).

Figure 12.

3D photogrammetric model for the decorated stone carved ceiling of the Northern Adytum (the zodiac pattern). Source: Author.

Part six: The decorated stone carved ceiling of the Southern Adytum (The Flower Pattern)

Thirty-five crowdsourced photos were employed to reconstruct this part (Figure 13).

Figure 13.

3D photogrammetric model for the decorated stone carved ceiling of the Southern Adytum (the flower pattern). Source: Author.

Post-processing Phase: Mesh editing, optimisation and assembling 3D generated parts

The initial 3D photogrammetric model usually contains various problems; therefore, this phase is vital for removing the unwanted parts, editing, refining and optimising the 3D generated models. Also, this phase involves assembling the created 3D models generated from the six parts of the preceding phase using different applications and software packages. Autodesk Meshmixer was used in this project to modify and edit the generated 3D models through applying different commands, such as mesh simplification and optimising surfaces, ⁶ mesh smoothing, hole filling and joining the disconnected meshes, and assembling and combining the photogrammetric geometries. Figure 14 illustrates the process of optimising the surfaces of one of the generated models using Autodesk Meshmixer.

Figure 14.

Surface optimisation using Autodesk Meshmixer for the main elevation of the Southern Adytum (A) Reduction of 25%, (B) 50% and (C) 75%. Source: Author.

Mesh editing and fixing the missing parts

This case presents a particularly challenging case through using crowdsourced images collected for this temple from different visitors during different periods for the purpose of 3D photogrammetric reconstruction. Consequently, this could offer incomplete and imperfect 3D digital models with missing components and unfinished meshes. Several defects and problems appeared in the 3D generated models, and these defects are possible to fix using Meshmixer software (Figure 15).

Figure 15.

Fixing digitally the missing parts using Autodesk Meshmixer. Source: Author.

Assembling 3D generated models

All six generated models created in the previous phase can be exported in different digital output formats, such as (*.obj) and (*.fbx), using both Metashape and Meshmixer. After fixing and editing the missing and incomplete parts of all the digital models, the same software can be used to combine and assemble all the six photogrammetric models to obtain a comprehensive 3D model of the monument. Figure 16 explains the process of assembling and combining the exterior and interior parts.

Figure 16.

Assembly process of six temple parts. Two main parts for the exterior and four parts for the interior. Source: Author.

Limitations

The visitors’ contribution in reconstructing the 3D model of Temple of Bel through their photos and experiences reflects a significant and useful source of data – regardless of the quality of generated models – that is instructive for future reconstruction of destroyed or inaccessible monuments in heritage sites like Palmyra. Although visitors’ experiences and their photos might provide a good source for reconstructing heritage assets digitally, this was inadequate to reconstruct the whole mass of this monument with all the required details, as the process of such project faced many challenges and limitations, as summarised below.

1. Crowdsourced images are created by normal visitors for non-professional (not for research) purposes; consequently, many of the available images on social media platforms and the web do not apply the requirements of 3D photogrammetric reconstruction due to several reasons, as discussed earlier. Many people do not pay attention to capture high-quality photos of such monuments, as their primary concern is the image and the moment itself, rather than their incidental photographic capabilities. In addition, digital images could lose a high degree of their accuracy and quality through uploading and sharing on some social media platforms and websites (e.g. due to the use of filters etc.). Therefore, the number of collected photos is no guarantee of creating a suitable 3D digital model using photogrammetric reconstruction methods.

2. This digital reconstruction method relies significantly on the visual diversity of monuments and their current physical condition within the heritage site. Some monuments remain relatively intact and have many details in good condition that carry aesthetic and historical value, including entire façades, openings, gates, decorated columns and statues. Conversely, other sites do not contain visual attraction points or historical values that could fascinate visitors to stay longer and capture more photos. Even within the same heritage, monument visitors tend to focus on photographing certain features prolifically while ignoring others, related to their interests and the visual quality of the monument.

3. One of the main limitations that faced the reconstruction process was the dearth of images covering the external and internal walls, which is related to intrinsic interest and physical accessibility and visibility issues. Consequently, the generated 3D models of walls appeared in 2D single layer meshes without depth, not as one complete 3D object (Figure 17). Therefore, when creating the temple sides from external walls and internal walls, the upper parts of these walls (vertical parts) could not be generated, as visitors’ cameras did not capture these parts.

4. Figure 18 shows how certain parts of the temple could not be digitally reconstructed, as visitors did not show any attention to take photos for them, such as the area behind the columns in the eastern façade, the lower part of openings (the sill), the floor and the internal wide flat walls (the eastern and western walls) of the Cella. It is likely that visitors did not consider these relatively plain walls worth of photographing (due to their poor visual value, from a tourist perspective). In ideal scenarios, one of the most significant requirements for reconstructing 3D photogrammetric models is offering the maximum amount of overlapping for different architectural parts in the captured photos. However, in this case, visitors’ photos did not show the required overlapping to obtain the required results.

5. Some areas of the internal parts inside the Cella were not captured by visitors’ cameras due to some physical restrictions; some areas might be hard to be access due to the massive scale of the temple. For instance, the hidden corners inside the Northern and Southern Adytum affected the created mesh reconstruction of the 3D model (Figure 19), illustrating that the missing parts of the upper and lower corners of the Northern Adytum were due to visitors’ cameras inaccessibility to reach these areas.

6. Visitors sometimes tend to capture photos of the monument from the typical touristic paths of the experience and the views from these paths. Therefore, their photos were taken from particular viewpoints from the path. For this reason, on social media platforms, we can see a lot of repetitive images, taken by visitors who took photographs from usual viewpoints, depending on their movement through the site and their personal preferences.

7. The colour variation of the 3D generated model can be obviously seen in several architectural parts in the temple, and this is a frequent problem in crowdsourced digital reconstruction. This difference in colours might happen due to crowdsourced photos being captured by different visitors using different cameras (mobile phones and digital cameras) during different periods (summer and winter, sunny, overcast, morning and evening). This limitation can be addressed by neglecting odd photos or editing photos and adjusting colours to match the palette of other photos using the colour correction or texture mapping phase.

Figure 17.

3D generated model for the Southern Adytum; a 2D mesh without depth. Source: Author

Figure 18.

Unreconstructed parts in the eastern and western façades of the temple due to the dearth of crowdsourced images covering these areas. Source: Author

Figure 19.

Unreconstructed upper and lower corners of the Northern Adytum. Source: Author

Results of 3D Photogrammetric Reconstruction Method using Crowdsourced Images

This project has shown the workflow of generating a 3D digital photogrammetric model of the destroyed Temple of Bel by employing several public domain touristic and crowdsourced images collected from different social media platforms as a primary source of reconstruction. Although the number of images was theoretically adequate, it was insufficient to reconstruct the entire mass of the temple. The quality and quantity of collected images was insufficient to cover the exterior and interior parts of the temple completely (Figure 20).

Figure 20.

An external and internal view of the digitally reconstructed temple. Source: Author.

The proposed reconstruction methodology led to a large number of point clouds and constructed meshes for different exterior and interior parts, based on the quantity and quality of collected photos. However, several areas have not presented satisfactory results, particularly the upper part of the walls, the floor and some hidden areas. The visitors did not show interest to capture images for these areas, such as the undecorated flat walls of the Cella, the back part of the main gate and the back of the peristyle area (behind the columns). On the other hand, the visitors, through the collected photos and the constructed 3D models, showed more attention to the inner space of the Cella, particularly the Northern and Southern Adytum and their decorated ceilings, more than other parts – this reflects the intrinsic attraction of areas that have richer visual value for visitors capturing photos. The point cloud model and 3D generated meshes for these two parts produced photogrammetric models with great details, making them able to be used for 3D visualisation purposes, in contrast to the incomplete, distorted external parts. Figure 21 illustrates the entire workflow and the main phases of reconstructing the Temple of Bel using crowdsourced materials collected from social media, employed to generate 3D photogrammetric models able to be used for developing engaging visitors’ experiences in the future using immersive technology.

Figure 21.

Entire workflow and pipeline for the digital reconstruction of the Temple of Bel using crowdsourced images and 3D photogrammetric reconstruction process. Source: Author.

Disseminating and visualising the digitally reconstructed temple of Bel

The last phase in the entire process discusses how to disseminate and visualise the 3D digital output reconstructed in the previous part of this paper. It is vital to find a compelling digital platform able to accept a big range of file formats, which can be accessible and available for different users across the world. There are common points of overlap between AR and VR, and recently many development platforms have been suitable for both experiences, whereby the user can customise experiences based on the level of immersion, heritage content and used devices (HHDs or VR headset). The generated digital heritage assets for the temple have been incorporated and disseminated in the following kinds of interactive visualisation scenes.

3D online digital platforms

The two photogrammetric models of the Northern and Southern Adytum were uploaded into Sketchfab to be available for different users for visualisations. Users can experience reconstructed models through AR and VR technologies using any consumer smartphone and VR cardboard. Also, more advanced VR headsets can be used for increased user engagement and interaction with virtual annotations using the joystick (Figure 22).

Figure 22.

VR experience via Sketchfab 3D online platform. Source: Author.

AR allows users to explore any virtual monument without going to the actual (physical) heritage site where the monument belongs. This feature is handy, especially in inaccessible heritage sites like Palmyra; it offers those who already have some mobility difficulties an AR ex-situ interactive experience using simple technology (Figure 23). Heritage dissemination has changed significantly, thanks to such platforms that offer non-skilled users an excellent opportunity to upload their digital heritage assets, allowing many users worldwide to experience and interact with them in their physical environments.

Figure 23.

Testing Northern and Southern Adytum augmentation using Sketchfab mobile application for AR experiences. Source: Author.

VR remote off-site interactive immersive experience

The recent technological revolution has assisted in the emergence of new affordable, easy-to-use solutions that can replace older expensive systems to efficiently disseminate heritage assets. New VR headsets offer interactive, immersive experiences where visitors can move freely and naturally in a life-size virtual space. The visitor will be allowed to get certain limited interactions with the virtual environment of Temple of Bel, with an emphasis that no two visitors will have the same 3D virtual experience. Such VH applications and experiences could be significantly developed in the future to work as a temporary or permanent solution – according to the state of the heritage site and monument – that allow many visitors to explore the destroyed city of Palmyra using more advanced immersive and interactive methods that put them in the centre of engaging experiences. ⁷

The VR experience setup and implementation

For creating the VR experience, all the digital heritage assets were imported to Unity3D to build the virtual setting and manage all the imported assets to develop the experience. The photogrammetric models’ polygons were reduced from 4 million to fewer than 1.5 million, to avoid any possible glitches that may occur during the experience. Game engines, in general, are responsible for managing all the digital files (audio-visual, i.e. 2D files, 3D models, sound and video clips), rendering the virtual scenes, collision detection, adding animation and interaction. The two photogrammetric models of the Northern and Southern Adytum were transferred to Unity3D to create the required virtual setting. Before starting the virtual tour, the visitors will be offered with the instruction sheet that developed to show them how to use the User Interface (UI) and VR joysticks during the experience for moving, interacting and displaying the audio-visual data (if available) and virtual annotations (Figure 24). Then, they will be requested to wear the VR headset to start viewing the virtual environment. ⁸

Figure 24.

Instruction sheet to assist visitors to move and interact during the experience. Source: Author.

The virtual tour offers the possibility of experiencing the temple from a real person perspective, with some simple interactive functions to attract the visitor to proceed the experience and evoke the sense of exploration. Inside the virtual environment, the user can walk and explore the space easily, while to bridge long distances the locomotion fly-through or teleportation technique can be used to walk in and outside the temple, to allow visitors to observe the external architectural details closely. Through this experience, the visitors will be offered a fully interactive immersive voyage to explore the temple from the inside by walking inside the inner space of the temple the Cella by viewing the Northern and Southern Adytum. A number of predefined virtual hot points and interactive annotations have been placed in the 3D environment to offer visitors further visual data, which appear by clicking with the joystick (Figure 25).

Figure 25.

Visitor’s interaction with the Northern and Southern Adytum inside the Cella viewing visual data through proposed interactive virtual hot points and annotations. Source: Author.

Conclusion

This paper presented an approach and scenario for a 3D photogrammetric reconstruction pipeline of ruined cultural heritage monuments using the example of the destroyed Bel Temple in Palmyra. It investigates the potential of exploiting public domain visitors’ images collected from social media platforms and the web to reconstruct this remarkable monument digitally. Essentially, the crowdsourced materials were not created for research-based purposes but were captured by visitors for touristic purposes. Although voluminous quantities of photos were found on social media capturing various views in Palmyra, most of them were inefficient for high-quality 3D digital reconstruction and visualisation purposes.

Visitors of Palmyra revealed significant appreciation and attention to the destroyed Temple of Bel, as evinced by the large volume of collected photos for this monument, which was the locus for the majority of collected photos. It was formerly the most visited attraction on the heritage site due to its previous intact condition before destruction. The crowdsourced photos for this monument were collected in a limited period, precisely from 2008 to 2015; after the latter year, no new updated photos were uploaded online, which made the process more challenging. This research employed solely crowdsourced materials based on two main reasons: firstly, the heritage site of Palmyra is inaccessible at the current time due to the current instability situation and security reasons. Secondly, the required monuments were already destroyed and no longer existed. Therefore, photogrammetry is one of the only available solution s in this case, as there is no possibility to use another technique for the reconstruction of monuments which no longer exist or are no longer accessible.

From the 3D digital outputs, this method was incapable of representing the entire model of the temple due to the quality and quantity of collected photos since not all the collected images were used in the process. At the same time, the results revealed that many visitors were interested in capturing the inner space of the temple (the Cella) more than the exterior. Although adopting this method in such a case was not efficient for representing the entire temple, the two Adytum (the northern and southern inner chambers) were photogrammetrically digitised and effectively represented using the collected crowdsourced data with excellent quality obtained; therefore, this makes them suitable for documentation purposes or to be used as input data to develop immersive experiences. Furthermore, the proposed reconstruction pipeline from these free materials provided a good relative accuracy and very high levels of detail for such prominent interior parts of the temple, particularly in such extraordinary cases. It is also particularly advantageous in times of COVID-19 and complies with the world ’s new social distancing etiquette and the restrictions on humans’ mobility.

The reconstructed digital replicas might provide a valuable source for the physical restoration of this temple in the future. It can also be used efficiently to develop new interactive, immersive experiences, whereby the user can get the opportunity to travel and explore several heritage sites without facing the barriers and dangers of the physical world. Furthermore, these environments are flexible enough to accommodate digital humans or personal virtual assistants who can tell stories about these structures and hidden stories regarding our threatened intangible heritage within one unified interactive virtual experience. In conclusion, we hope to demonstrate how this research can find similar applications in other locations worldwide and offer new methods, for instance, to reconstruct lost heritage sites. Expectantly, this research can offer a practical and helpful workflow for other researchers to digitise different heritage monuments through 3D photogrammetric reconstruction based on utilising crowdsourced materials.