Mixed Reality-Enhanced Digital Twin for Adaptive Photovoltaic Shutter Operation: A Pilot Study

Abstract

Adaptive Photovoltaic Shutter (APS) is an innovative shading device that integrates photovoltaic materials into its slats, enabling it to convert sunlight into electricity while adaptively adjusting frame and slat angles to optimize multiple performance objectives. Due to the complexity of its mechanical system and operation, an advanced digital twin (DT) is essential for effective monitoring and control. Current research integrates mixed reality (MR) with DT to provide a more immersive and intuitive management experience. An Arduino-based IoT system collects real-time data from the APS, uploads it to the cloud, and enables visualization through a MR application. Within the MR interface, users can observe the status of the physical APS and control it by interacting with its digital counterpart. A survey was conducted to evaluate the DT's usability and user engagement. The results suggest that the MR-DT can enhance user engagement and support anomaly detection tasks, although it is associated with longer task initiation time and perceived interaction latency. These findings highlight both the potential and limitations of MR-enhanced DT in APS operation.

Keywords

Mixed reality digital twin data visualization shading device interaction design

Get full access to this article

View all access options for this article.

References

Adu-amankwa

N. A.

Rahimian

(2025). Blockchain-Enabled digital twins in the built environment : Socio-technical barriers to post- construction integration. Proceedings of Smart and Sustainable Built Environment Conference Series, Lille, France (pp. 106–115).

Aheleroff

Zhong

R. Y.

(2021). Digital twin as a service (DTaaS) in industry 4.0: An architecture reference model. Advanced Engineering. Informatics, 47(October 2020), 101225. https://doi.org/10.1016/j.aei.2020.101225

Alizadehsalehi

Yitmen

(2021). Digital twin-based progress through reality capture to extended reality technologies (DRX). Smart and Sustainable Built Environment. 12(1), 200–236. https://doi.org/10.1108/SASBE-01-2021-0016

Angelova

D. D.

Fernández

D. C.

Godoy

M. C.

(2024). A review on digital twins and its application in the modeling of photovoltaic installations. Energies, 17(5), 1227. https://doi.org/10.20944/preprints202401.1585.v1

Arowoiya

V. A.

Moehler

R. C.

Fang

(2024). Digital twin technology for thermal comfort and energy efficiency in buildings: A state-of-the-art and future directions. Energy and Built Environment, 5(5), 641–656. https://doi.org/10.1016/j.enbenv.2023.05.004

Artetxe

Uralde

Barambones

Calvo

Martin

(2023). Maximum power point tracker controller for solar photovoltaic based on reinforcement learning agent with a digital twin. Mathematics, 11(9). https://doi.org/10.3390/math11092166

Baumgartner

Brägger

Koebel

Scheidegger

Çöltekin

(2022). Visually annotated responsive digital twins for remote collaboration in mixed reality environments. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, V-4-2022, 329–336. https://doi.org/10.5194/isprs-annals-v-4-2022-329-2022

Biagini

Bongini

Marzi

(2024). From bim to digital twin. Iot data integration in asset management platform. Journal Of Information Technology in Construction, 29(May), 1103–1127. https://doi.org/10.36680/j.itcon.2024.049

Blanco-novoa

Fraga-lamas

Vilar-montesinos

M. A.

(2020). Towards the internet of augmented things : An open-source framework to interconnect IoT devices and augmented reality systems. Proceedings, 42(1). https://doi.org/10.3390/ecsa-6-06563

10.

Böke

Denz

Suwannapruk

Vongsingha

(2022). Active, passive and cyber- physical adaptive façade strategies : A comparative analysis through case studies. Journal of Facade Design and Engineering, 10(2), 1–18. https://doi.org/10.47982/jfde.2022.powerskin.01

11.

Böke

Knaack

Hemmerling

(2020a). Automated adaptive façade functions in practice - case studies on office buildings. Automation in Construction, 113(June 2019), 103113. https://doi.org/10.1016/j.autcon.2020.103113

12.

Böke

Knaack

Hemmerling

(2020b). Prototype of a cyber-physical façade system. Journal Of Building Engineering, 31(April), 101397. https://doi.org/10.1016/j.jobe.2020.101397

13.

Caiza

Sanz

(2022). Digital twin for monitoring an industrial process using augmented reality. In 17th Iberian Conference on Information Systems and Technologies (CISTI) (pp. 22–25). ITMA.

14.

Cantoral-ceballos

J. A.

(2022). Extended reality applications in industry 4 . 0 . – A systematic literature review. Telematics and Informatics, 73(September 2022). https://doi.org/10.1016/j.tele.2022.101863

15.

Carvalho

W. O.

(2021). BIM and AM to Manage Critical and Relevant Water and Wastewater Utilities Assets vol 3. Springer International Publishing. https://doi.org/10.1007/978-3-030-51295-8

16.

Catalano

, et al. (2022). A digital twin-driven and conceptual framework for enabling extended reality applications: A case study of a brake discs manufacturer. Procedia Computer Science, 200(2019), 1885–1893. https://doi.org/10.1016/j.procs.2022.01.389

17.

Coupry

Noblecourt

Richard

Baudry

Bigaud

(2021). BIM-Based digital twin and XR devices to improve maintenance procedures in smart buildings: A literature review. Application Science, 11(15). https://doi.org/10.3390/app11156810

18.

Davari

Shahinmoghadam

Motamedi

Poirier

(2022). Demystifying the Definition of Digital Twin for Built Environment. The 9th International Conference on Construction Engineering and Project Management.

19.

de Barros Lima

(2020). A web-based application to integrate building management system sensor data and building information model data to support facility management tasks . Eindhoven University of Technology.

20.

Dias

J. A.

Campos

N. P.

Boavida

(2025). Towards an IoT framework for the new generation of XR applications. IEEE Access, 13, 68726–68752. https://doi.org/10.1109/ACCESS.2025.3560547

21.

Donkers

(2022). Parallel Intelligence in Semantic Digital Twins : An Interactive Decision-Support System for Indoor. IEEE 2nd International Conference on Digital Twins and Parallel Intelligence (DTPI). IEEE.

22.

Dorian

E. G. R.

Björn

Henrik

Christof

(2020). A genetic algorithm approach as a self-learning and optimization tool for PV power simulation and digital twinning. Energies, 13. https://doi.org/10.3390/en13246712

23.

Edington

Dervilis

Ben Abdessalem

Wagg

(2023). A time-evolving digital twin tool for engineering dynamics applications. Mechanical System and Signal Processing, 188(November 2022), 109971. https://doi.org/10.1016/j.ymssp.2022.109971

24.

Eneyew

D. D.

Member

G. S.

Miriam

A. M.

(2022). Towards smart building digital twins: BIM and IoT data integration. IEEE Access, 10, 130487–130506. https://doi.org/10.1109/ACCESS.2022.3229370

25.

Fawad

Salamak

Chen

Koris

(2024). Development of an immersive digital twin framework to support infrastructure management : a case study of bridge asset health monitoring. 41st International Symposium on Automation and Robotics in Construction (ISARC 2024), Lille, France. https://doi.org/10.22260/ISARC2024/0105

26.

FILT3RS, “Solar diaphragms at IMA,” filt3rs.net.

27.

García

Á.

Bregon

Martínez-Prieto

M. A.

(2022). Towards a connected digital twin learning ecosystem in manufacturing: Enablers and challenges. Computers & Industrial Engineering, 171(September 2022). https://doi.org/10.1016/j.cie.2022.108463

28.

Georgiou

Kyza

E. A.

(2017). The development and validation of the ARI questionnaire: An instrument for measuring immersion in location-based augmented reality settings. International Journal Of Humuan Computer Studies, 98(February 2017), 24–37. https://doi.org/10.1016/j.ijhcs.2016.09.014

29.

Gordo-Gregorio

Alavi

Edwards

D. J.

Forcada

Guéna

(2025). An occupant-centric approach on digital twins for building management. Building Research & Information, 54(February 2025), 59–78. https://doi.org/10.1080/09613218.2025.2463676

30.

Grieves

(2005). Product lifecycle management: The new paradigm for enterprises. International Journal Of Product Development, 2(1–2), 71–84. https://doi.org/10.1504/IJPD.2005.006669

31.

Haag

Anderl

(2018). Digital twin – proof of concept. Manufacture Letter, 15(Part B), 64–66. https://doi.org/10.1016/j.mfglet.2018.02.006

32.

Haroon

Yucel

(2026). Design and evaluation of a mixed reality system for facility inspection and maintenance †. Buildings, 16(2). https://doi.org/10.3390/buildings16020425

33.

Hartmann

, et al. (2025). Digital-Twin-Based management of sewer systems: Research strategy for the KaSyTwin project. Water (Switzerland), 17(3), 1–13. https://doi.org/10.3390/w17030299

34.

Hasan

S. M.

Lee

Moon

Kwon

Jinwoo

Lee

(2022). Augmented reality and digital twin system for interaction with construction machinery. Journal Of Asian Architecture Building Enginerring, 21(2), 564–574. https://doi.org/10.1080/13467581.2020.1869557

35.

Havard

Jeanne

Lacomblez

Baudry

(2019). Digital twin and virtual reality: A co-simulation environment for design and assessment of industrial workstations. Production & Manufacturing Research, 7(1), 472–489. https://doi.org/10.1080/21693277.2019.1660283

36.

Liu

Chang

(2026). Integrating digital twin technologies for maintenance 4.0 in the building industry: A review and conceptual framework. Building Environment, 288(January 2026). https://doi.org/10.1016/j.buildenv.2025.113997

37.

Jain

, et al. (2020). A digital twin approach for fault diagnosis in distributed photovoltaic systems. IEEE Transactions on Power Electronics, 35(1), 940–956. https://doi.org/10.1109/TPEL.2019.2911594

38.

Jeršov

(2020). Development of Digital Twin in Extended Reality With Unreal Engine 4. Tallinn University of Technology.

39.

Jesudian

J. J. S. S.

(2018). An Integration of IOT and Mixed Reality Using Fog Computing Concepts. Texas State University.

40.

Jesús

Nuria

M-C.

Carlos

S-S.

(2022). BIPV modeling with artificial neural networks: Towards a BIPV digital twin. Energies, 15(11). https://doi.org/10.3390/en15114173

41.

Jiang

Tobia

Lawther

Branchaud

Bednarz

(2020). Double vision: Digital twin applications within extended reality. ACM SIGGRAPH 2020 Appy Hour, SIGGRAPH 2020 (pp. 10–12). https://doi.org/10.1145/3388529.3407313

42.

Jiang

Shi

Wang

Pan

(2023). Intelligent control of building fire protection system using digital twins and semantic web technologies. Automation Construction, 147(March 2023). https://doi.org/10.1016/j.autcon.2022.104728

43.

Jiang

Liu

Chen

Liu

Ding

(2024). Early-warning of unsafe hoisting operations: An integration of digital twin and knowledge graph. Development Of Built Environment, 19(October 2024). https://doi.org/10.1016/j.dibe.2024.100490

44.

Juaristi

Loonen

Isaia

Gómez-Acebo

Monge-Barrio

(2020). Dynamic climate analysis for early design stages: A new methodological approach to detect preferable adaptive opaque façade responses. Sustainable Cities and Society, 60(April), 102232. https://doi.org/10.1016/j.scs.2020.102232

45.

Xiang

Zhang

Zuo

(2019). A enhanced interaction framework based on VR, AR and MR in digital twin. In Procedia CIRP (Vol. 83, pp. 753–758). Elsevier B.V. https://doi.org/10.1016/j.procir.2019.04.103

46.

Khajavi

S. H.

Motlagh

N. H.

Jaribion

Werner

L. C.

Holmstrom

(2019). Digital twin: Vision, benefits, boundaries, and creation for buildings. IEEE Access, 7, 147406–147419. https://doi.org/10.1109/ACCESS.2019.2946515

47.

Khan

L. U.

Saad

Niyato

Han

Hong

C. S.

(2022). Digital-Twin-Enabled 6G: Vision, architectural trends, and future directions. IEEE Communications Magazine, 60(1), 74–80. https://doi.org/10.1109/MCOM.001.21143

48.

Khelifi

Morris

(2024). Mixed reality IoT smart environments with large language model agents. 2024 IEEE 4th Int. Conf. Human-Machine Syst. ICHMS 2024 (pp. 1–7), https://doi.org/10.1109/ICHMS59971.2024.10555610

49.

Krayem

Younes

Ghali

Jradi

(2026). A digital twin solution for optimizing productivity and energy performance in hybrid ventilated office space. Energy Conversion Management, 349(November 2025), 120880. https://doi.org/10.1016/j.enconman.2025.120880

50.

Kunnen

Adamenko

Pluhnau

Loibl

Nagarajah

(2020). System-based concept for a mixed reality supported maintenance phase of an industrial plant. In Procedia CIRP (pp. 15–20). Elsevier B.V. https://doi.org/10.1016/j.procir.2020.03.096

51.

Kuts

, et al. (2022). Digital twin as industrial robots manipulation validation tool. Robotics, 11(5), 1–24. https://doi.org/10.3390/robotics11050113

52.

Zheng

Pang

Lee

C. K. M.

(2022). AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop. Robotics and Computer Integrated Manufacturing, 76(July 2021), 102321. https://doi.org/10.1016/j.rcim.2022.102321

53.

Q. A.

Fang

(2025). An automatic ontology construction approach for BIM-IoT semantic integration in intelligent operation and maintenance of high-speed railway stations. Engineering Research Express, 7(4). https://doi.org/10.1088/2631-8695/ae1c92

54.

Yin

Xie

Pan

Wang

Brilakis

(2025). Development of A trustworthy AI-supported digital twin framework for road operation and maintenance. Smart Construction, 2025(December 2024), 1–28. https://doi.org/10.55092/sc20250001

55.

Luu

Nguyen

Tran

Ding

Kua

Hoang

(2025). Enhancing real-time robot teleoperation with immersive virtual reality in industrial IoT networks. The International Journal of Advanced Manufacturing Technology, 139, 6233–6257. https://doi.org/10.1007/s00170-025-16236-w

56.

Macci

Carf

Mastrogiovanni

(2022). Mixed reality as communication Medium for human-robot collaboration. 2022 International conference on robotics and automation (ICRA). IEEE (pp. 2796–2802).

57.

Mahmoodian

Shahrivar

Setunge

Mazaheri

(2022). Development of Digital Twin for Intelligent Maintenance of Civil Infrastructure. Sustain., 14(14). https://doi.org/10.3390/su14148664

58.

Mohammadi

Assaf

Assaad

R. H.

(2024). Real-time spatial-temporal mapping and visualization of thermal comfort and HVAC control by integrating immersive augmented reality technologies and IoT-enabled wireless sensor networks : Towards immersive human-building interactions. Journal Of Building Engineering., 94(February), 109887. https://doi.org/10.1016/j.jobe.2024.109887

59.

Natephra

Motamedi

(2019). Live data visualization of IoT sensors using augmented reality (AR) and BIM. ISARC 2019 (pp. 632–638).

60.

Noroozinejad Farsangi

Shehata

A. O.

Rashidi

Ghassempour

Mirjalili

(2024). Transitioning from BIM to digital twin to metaverse. Frontiers In Built Environment, 10(December), 1–7. https://doi.org/10.3389/fbuil.2024.1486423

61.

Orhon

(2016). A review on adaptive photovoltaic facades. In: Solar TR2016 International Solar Conference & Exhibition, pp.463–470.

62.

Oyekan

J. O.

, et al. (2019). The effectiveness of virtual environments in developing collaborative strategies between industrial robots and humans. Robotics Computer Integrated Manufacturing, 55(June 2018), 41–54. https://doi.org/10.1016/j.rcim.2018.07.006

63.

Pincheira

Vecchio

Antonelli

(2023). Smarttwin: A blockchain-based software framework for digital twins using IoT. In BLOCKCHAIN 2022 (pp. 67–77). Springer International Publishing. https://doi.org/10.1007/978-3-031-21229-1_7

64.

Pizzagalli

Kuts

Bondarenko

Otto

(2021). Evaluation of virtual reality interface interaction methods for digital twin industrial robot programming and control, a pilot study. ASME International Mechanical Engineering Congress and Exposition, Proceedings (IMECE), (pp. 1–9). https://doi.org/10.1115/IMECE2021-69408

65.

Ravi

K. S. D.

M. S.

Medina Ibáñez

Hall

D. M.

(2021). Real-time digital twin of on-site robotic construction processes in mixed reality. Proceedings of the International Symposium on Automation and Robotics in Construction (pp. 451–458). https://doi.org/10.22260/isarc2021/0062

66.

Saadatifar

Sawyer

A. O.

Byrne

(2024). Occupant-Centric digital twin: A case study on occupant engagement in thermal comfort decision-making. Architecture, 4(2), 390–415. https://doi.org/10.3390/architecture4020022

67.

Schmidt

Yigitbas

(2024). Development and Usability Evaluation of Transitional Cross-Reality Interfaces. Proc. ACM Hum.-Comput. Interact, Honolulu, USA, 8. https://doi.org/10.1145/3664637

68.

Shukla

A. K.

Shukla

K. N.

Dixit

(2026). IoT-enabled solar PV systems: Real-time monitoring and efficiency optimization. Results In Optics, 22(November 2025), 100949. https://doi.org/10.1016/j.rio.2025.100949

69.

Siegele

Penna

Riedl

(2021). Visualizing building energy measurement data in mixed reality applying B.I.M. AVR 2021 (pp. 247–253). https://doi.org/10.1007/978-3-030-87595-4_18

70.

Stark

Haffner

Drahoš

Leskovský

(2020). Using augmented reality and internet of things for control and monitoring of mechatronic devices. Electronics, 9(8). https://doi.org/10.3390/electronics9081272

71.

Stavropoulos

Papacharalampopoulos

Siatras

Mourtzis

(2021). An AR based digital twin for Laser based manufacturing process monitoring. Procedia CIRP, 102, 258–263. https://doi.org/10.1016/j.procir.2021.09.044

72.

Sun

Jayasinghe

Sidiq

Shahrivar

Mahmoodian

Setunge

(2025). Approach Towards the Development of Digital Twin for Structural Health Monitoring of Civil Infrastructure: A Comprehensive Review. Sensors, 25(1). https://doi.org/10.3390/s25010059

73.

Suzuki

Karim

Xia

Hedayati

Marquardt

(2022). Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces. Conference on Human Factors in Computing Systems - Proceedings, New Orleans, LA, USA. Association for Computing Machinery. https://doi.org/10.1145/3491102.3517719

74.

Tabadkani

Roetzel

H. X.

Tsangrassoulis

(2021). Design approaches and typologies of adaptive facades: A review. Automation in Construction, 121(November 2020), 103450. Elsevier B.V. https://doi.org/10.1016/j.autcon.2020.103450

75.

Tabadkani

Valinejad Shoubi

Soflaei

Banihashemi

(2019). Integrated parametric design of adaptive facades for user’s visual comfort. Automation In Construction, 106(May), 102857. https://doi.org/10.1016/j.autcon.2019.102857

76.

Autiosalo

Ala-laurinaho

Yang

Salminen

Tammi

(2023). TwinXR: Method for using digital twin descriptions in industrial eXtended reality applications. Frontiers In Virtual Reality, 4(1019080). https://doi.org/10.3389/frvir.2023.1019080

77.

Autiosalo

Jadid

Tammi

Klinker

(2021). A mixed reality interface for a digital twin based crane. Applied Science, 11(20), 1–19. https://doi.org/10.3390/app11209480

78.

Walczyk

Andrzej

(2024). Building information modeling and digital twins for functional and technical design of smart buildings with distributed IoT networks — Review and new challenges discussion. Future Internet, 16(7). https://doi.org/10.3390/fi16070225

79.

Wang

Song

Bao

Xiang

(2024). From BIM to digital twin in BIPV : A review of current knowledge. Sustainable Energy Technology. Assessments, 67(May), 103855. https://doi.org/10.1016/j.seta.2024.103855

80.

Xie

, et al. (2020). Interactive multi-user 3D visual analytics in augmented reality. International Symposium on Electronic Imaging 2020 (pp. 1–6). https://doi.org/10.2352/ISSN.2470-1173.2020.13.ERVR-363

81.

Xie

, et al. (2025). AIoT-powered building digital twin for smart firefighting and super real-time fire forecast. Advanced Engineering. Informatics, 65(Part A), 103117. https://doi.org/10.1016/j.aei.2025.103117

82.

Yang

, et al. (2022). Extended Reality Application Framework for a Digital-Twin-Based Smart Crane. Applied Science, 12(12). https://doi.org/10.3390/app12126030

83.

Yang

Zheng

Feng

Ala-Laurinaho

Tammi

(2025). A digital twin-driven industrial context-aware system: A case study of overhead crane operation. Journal Of Manufacturing. System, 78(2024), 394–409. https://doi.org/10.1016/j.jmsy.2024.12.006

84.

Yigitbas

Engels

(2023). Enhancing Robot Programming through Digital Twin and Augmented Reality. Proceedings of the 56th Hawaii International Conference on System Sciences. on-line.

85.

Yigitbas

Karch

(2025). Immersive smart home planning and automation with mixed reality and digital twin technologies. In PETRA ‘25 (pp. 137–145). Corfu Island. https://doi.org/10.1145/3733155.3733198

86.

Yildiz

Møller

Bilberg

(2020). Sciencedirect virtual factory : Digital twin based integrated factory simulations. In Procedia CIRP (pp. 216–221). Elsevier B.V. https://doi.org/10.1016/j.procir.2020.04.043

87.

Zhao

Dong

Tian

Chen

(2023). Comparing three XR technologies in reviewing performance-based building design: A pilot study of façade fenestrations. Computer Animated Virtual Worlds, 34(6), 1–20. https://doi.org/10.1002/cav.2139

88.

Zhao

Pan

Gao

Cheng

(2022). Integrating internet of things and mixed reality to teach performance-based architectural design: A case study of shading devices. Education Information Technology. 27, 9125–9143. https://doi.org/10.1007/s10639-022-10998-6 Integrating

89.

Zhu

Sun

(2025). A digital twin-based green construction management method for prefabricated buildings. Digital Twin, 2(3). https://doi.org/10.12688/digitaltwin.17830.1