Enhancing human-building interaction and spatial experience in cultural spaces

Abstract

In the realm of architectural evolution, the paradigm of static built environments has progressively shifted towards adaptive, responsive, and intelligent spaces. This paper presents a pioneering government-funded applied research project, the ‘Spatially Intelligent Arts Centre’, which exemplifies the convergence of multiple disciplines in the pursuit of redefining the traditional notion of a static arts centre. This endeavour sought to transform an existing arts centre building into a dynamic and intelligent space, thereby redefining its agency to foster novel user experiences and optimize operational efficiency. The project transcended conventional boundaries by merging the expertise of architects, computer scientists, user experience designers, spatial computing specialists, technology developers, and interface designers. Through transdisciplinary collaboration and adopting a design-thinking methodology, the project not only challenges the conventional limitations of architectural design but also offers a tangible illustration of the burgeoning field of human-building interaction research and practice. This paper details the conceptual underpinnings, technical implementations, and experiential outcomes of the Spatially Intelligent Arts Centre project, underscoring its significance as a model for implementation of human-building interaction in the context of a cultural building.

Introduction

The evolution of architectural design has historically been shaped by the intricate interplay between functional utility and aesthetic appeal. However, the dawn of ubiquitous computing, sensor networks, and artificial intelligence has introduced a new era of possibilities for the architectural space. This transformation is manifesting as a departure from viewing buildings as static artifacts towards conceiving them as adaptable entities, responsive to their surroundings and inhabitants. This shift has engendered an emerging research domain, often referred to as human-building interaction (HBI), ¹ which advocates for architecture to be more than a passive backdrop for human activity but an active participant that senses, responds, and adapts to the needs and behaviours of its occupants.

Central to the development of human-building interaction is the concept of ‘agency shift’. Traditionally, architectural spaces have occupied a passive role, serving as fixed enclosures encapsulating human activity. In contrast, an ‘agency shift’ is all about empowering people to have a say in how the building serves their evolving needs and preferences. This concept involves introducing dynamic elements, such as content and interactive features, into what used to be a static, unchanging structure. By doing so, we can engage people in a more active and personalized way. The building can communicate with users through real-time interactions, providing them with information, options, and opportunities to influence how spaces are utilized.

The realm of human-building interaction (HBI) explores the intricate interplay and engagement between technology, the built environment, and humans. This exploration encompasses two interconnected realms: the interactions between humans and physical spaces, and the synergistic interchanges between humans and digital technology, enhancing the experiential dimensions of physical environments. This inquiry raises fundamental questions regarding the seamless integration of technology and information into our tangible ‘physical space’, cultivating novel dimensions of physical, social, and spatial encounters.

The trajectory of this emergent field aspires to evolve into a self-adaptive paradigm of design research, harmonizing insights from diverse domains, including Human–Computer Interaction, Ubiquitous Computing, Architecture, and Urban Design. The aim is to create a comprehensive framework that acknowledges, celebrates, and empowers the intricate relationship between technology, human presence, and built environments.

This paper presents the outcomes of an applied design-research project, referred to as the ‘Spatially Intelligent Arts Centre’. The primary objective of this project was to dynamically transform an Arts Centre building, traditionally regarded as a static physical structure, into an intelligent and adaptable environment. Funded by the Victorian State Government, the project was centred on comprehensive research, design, development, and testing of scaled prototypes within the actual spaces of the existing building. The core aim was to generate fresh insights and practical guidelines in the domain of HBI, ultimately enhancing the spatial, physical, and social encounters of visitors at the Performance Arts Centre in Geelong (Victoria, Australia) referred to throughout this paper as GAC (Geelong Arts Centre).

This paper is organized to offer an extensive overview of the project and its results. Section 2 will offer an overview of diverse research approaches in HBI, situating our project within this context and aligning it with the concept of intelligent place-making. In Section 3, we will delve into the project itself, highlighting its distinctive methodology – a design thinking approach inspired by Dan Nassler. ² This section will delve into the four distinct stages of the Design Thinking methodology, elucidating the project journey through problem space exploration, use case development, and prototype creation. Within Section 4, we conducted a comprehensive evaluation of two of the pivotal installations, namely, GAC Spatial Kiosk and Memory Lane. This assessment was grounded in user testing outcomes and augmented by an exploration of the potential for future implementations. Lastly, the conclusion and discussion section will offer additional insights, including reflections on project outcomes, prospects for the future, challenges encountered during the design research process, and the promising frontiers of HBI research within the realm of the built environment.

Human-building interaction

Distinguishing product/process improvement from user experience

From a broad perspective, the infusion of technology into the architecture and built environment can be categorized into two principal directions. The first direction centres on product and process improvement through the integration of advanced technologies and systems. This approach aims to enhance the efficiency and suitability of building design, construction, and maintenance operations. Research within this realm primarily seeks to leverage digital and intelligent technologies to optimize various stages of the construction lifecycle, from conceptualization and design to fabrication, construction, and ongoing maintenance. This optimization seeks to streamline workflows, minimize errors, reduce costs, and optimize resource allocation. However, as underscored by Ahmadi-Karvigh, ³ the predominant emphasis on technology integration into buildings has centred around top-down technological advancements, often overlooking the intricacies of diverse building users with varying preferences and requirements.

The transition from Industry 3.0 to Industry 4.0 has redirected attention from mere product and process enhancement to a focus on augmenting interactions between users and the built environment. This evolution has catalyzed the development of novel technologies and systems, empowering buildings to be more responsive to the needs and behaviours of their occupants. This is a more recent approach focusing on the infusion of information into the physical environment to create novel spatial, physical, and social experiences for users within buildings and/or urban spaces. HBI, as an emerging domain, represents an interdisciplinary field bridging Human–Computer Interaction (HCI) with Architecture and Urban Design.^4,5 It is dedicated to fostering innovative forms of interactivity between users and buildings, contingent upon the nature of the structures and their social contexts.

Human-building interaction can also be defined as repositioning and repurposing the intricate interplay between humans, buildings, and computing (Figure 1), encompassing three distinct yet interconnected interactions:

• Human-Space Interaction: This facet of HBI focuses on how individuals interact with physical spaces, encompassing aspects such as navigation, comfort, accessibility, and sensory experiences within architectural environments. It considers the ways in which the design and arrangement of spaces influence human behaviour and well-being.

• Human–Computer Interaction (HCI): In the context of HBI, HCI addresses the interaction between individuals and computing technologies integrated into architectural spaces. This includes touchscreens, sensors, augmented reality interfaces, and other digital elements that facilitate user engagement and responsiveness within the built environment.

• Building–Computer Interaction: Building–Computer Interaction explores the dynamic relationship between architectural structures and computing systems. This interaction involves the use of sensors, automation, data analysis, and artificial intelligence to enable buildings to sense, respond to, and adapt to the needs and behaviours of their occupants in real-time.

Figure 1.

HBI as an interplay between people, building, and computing.

These three dimensions of HBI collectively contribute to the transformation of traditional static architectural spaces into dynamic and intelligent environments that enhance human experiences, improve efficiency, and foster new possibilities in the realms of architecture, technology, and urban design.

The literature on human-building interaction in existing buildings showcases diverse and impactful applications of this evolving paradigm. From adaptive reuse and cultural centres to smart offices and healthcare facilities, these cases underscore the transformative potential of HBI to redefine user experiences and enhance operational efficiency in a myriad of ways. Whether initially planned as an intelligent built environment or later enhanced or retrofitted to incorporate technological elements, the physical structure and its operational aspects remain pivotal components of HBI. Within this sphere of HBI, researchers and practitioners investigate the reciprocal influence between buildings and humans. They observe how users interact with built environments and design technologies to facilitate novel interactions in these spaces. ⁶ The features relevant to HBI are diverse in design and multifaceted in function, for example, focusing on the drivers of human behaviour ⁷ ; impact of social behaviour on building performance, such as energy and comfort management^8–10; indoor way-finding in complex buildings ¹¹ ; and real-time rescue operations in emergency situations.^12,13 The majority of prior research integrates diverse sensing technologies for data collection, which can then be amalgamated to construct activity patterns. This process assists in establishing context and occupant awareness, ultimately enhancing building operations and/or human engagement and experience.

Human interactions with buildings, either individual or collective, exhibit a range of dynamic possibilities along the passive-to-active interaction continuum. ⁶ Through innovative and adaptable multimodal interfaces enhanced by sensing and computing, people and buildings can engage with each other in novel ways. The built environments can learn, adapt, and evolve through these interactions at different scales (individual building, community level, and city level). Alavi et al. ¹⁴ highlighted the exciting possibilities that HBI research offers when it goes beyond solely studying individual aspects of the built environment, such as the physical, spatial, and social elements. Instead, they proposed a model that explores the intersections of these elements, promising a deeper understanding and innovative insights within HBI. For instance, in the socio-spatial aspect of this model, there are opportunities for HBI research to improve the communal aspects of public spaces. Likewise, in the physical-spatial dimension, there are further possibilities to explore, like the creation of novel interactive systems for indoor comfort, which can influence aspects like health, well-being, cognitive performance, productivity, and learning. Similarly, a third intersection highlights opportunities for exploration focused on using personal data to generate personalized innovative services.

Intelligent building versus intelligent place-making

Data-driven practices serve as a pivotal link connecting previously distinct design disciplines. Historically, design for cultural and social change and digital and interaction design functioned in isolation from other design realms like interior design, architecture, or experience design. Moreover, urban planning remained detached from the design of individual user experiences. Nevertheless, the emergence of spatial digitization is reshaping this scenario. Data equips us to study and grasp the influence of alterations in human interactions, cultural dynamics, and sustainability requirements on the built environment. It allows us to contemplate human engagement across the dimensions of people, space, services, and interactions.

Richard Tyson ¹⁵ brings forth a crucial perspective that gains relevance as advanced technologies become integral to our design tools and daily lives. He emphasizes that our criteria for success should extend beyond the mere adoption of technology; instead, it should revolve around our capacity to adeptly orchestrate experiences and human interaction. Tyson introduces the concept of an ‘intelligent place’, underscoring that a technologically advanced building, on its own, doesn’t ensure an enriched human experience. An intelligent place transcends being merely a structure filled with technology; it’s a space that skilfully integrates design, experience, operational efficiency, and performance intelligence to foster meaningful human engagement. Its objectives encompass simplicity, well-being, discovery, learning, and the creation of a welcoming, comfortable atmosphere.

It’s also important to define ‘intelligence’ in the context of an arts centre. Arts centres host diverse activities that change over time. Spaces serve different functions, and user occupancy varies. To address these dynamics, effective monitoring, coordination, planning, and communication are necessary for improving program management and direct people flow at all times. In alignment with these principles, the Spatially Intelligent Arts Centre project, discussed in this paper, represents a dedicated pursuit of intelligent place-making. In the context of this project, by using ‘intelligence’ in the context of ‘place-making’, we refer to the idea of reimagining, revitalizing, and improving awareness in existing spaces, making them more functional, engaging, and relevant to the communities they serve. This was one of the reasons for the GAC staff to reach out to the project team about the need to revitalize and transform the ordinary or underutilized spaces into vibrant, people-oriented places. In this regard, the project actively sought to transform an existing building by integrating data, artificial intelligence, and intelligent systems, all aimed at cultivating spaces capable of producing meaningful results, especially within the unique setting of a cultural institution. This research endeavour was guided by the overarching objective of devising novel methodologies and solutions to enrich spatial experience and human-building interaction within the existing framework of a Performance Arts Centre building.

The project

The ‘Spatially Intelligent Arts Centre’ project has been developed by an interdisciplinary team of design researchers at Deakin University, in close collaboration with GAC to respond to some of the most pressing challenges faced by cultural institutions today, namely: How does an Arts Centre building stay relevant in the age of digital technology? How can an arts centre building attract and increase the number of diverse audiences?

The project team embarked on a highly mission-oriented design research with a primary focus: how can we transform an Arts Centre Building from a static physical structure into a dynamic and engaging space? The initial phase of this project concentrated on research, ideation, design, development, and testing a number of scaled prototype applications (installations) within the actual spaces of the existing building. The installations were intended to sense and understand human activities and support them with timely information and suggestions while offering new experiences that would define new services and operations for the GAC.

The project interpreted and explored ‘spatial intelligence’ through two distinct yet complementary lenses. One focused on the operational side of an existing Arts Centre Building, and explored opportunities to sense, capture, and analyse patron activity and movement, to provide practical safety messaging and other opportunities to program the building systems to respond to changing conditions in real-time. The other focused on the creation of new social, spatial, and experiential interactions through the creation of alternative (digital) content creation, distribution channels, and consumption modalities (e.g. participation and immersion).

The project specifically delved into how the responsive interfaces could affect the human-building interactions in an existing building, while the changing interaction patterns of the users with the building would continuously altered the information on the displays in real-time. In other words, the information displayed (or otherwise communicated) has been continuously altered by the different movements and interactions of the people within the building spaces. This served as an illustrative instance of how the amalgamation of real-time sensing and data visualization fostered a dynamic and responsive rapport between individuals and the architectural spaces they inhabit.

Research design and methodology

The project employed Design Thinking as its core methodological approach. Design Thinking^16–18 is distinct from conventional empirical research methods in its approach to problem-solving. While traditional research often involves gathering facts and data and then formulating a comprehensive, abstract solution concept, Design Thinking takes a different path which is characterized by an epistemology that emphasizes action. It operates on the premise that, once we’ve collected and understood all available information and research, the most effective way to explore potential solutions is by promptly creating tangible, well-defined experiments. This approach stands in contrast to the conventional method of trying to assemble all interpretations and hypotheses into a detailed yet abstract solution concept from the beginning. Design Thinking encourages team members to maintain open-mindedness, consider multiple options and pathways, and be consistently willing to explore alternatives.

In essence, Design Thinking is a dynamic and iterative problem-solving approach that prioritizes experimentation and action over prematurely committing to a single, abstract solution. It promotes adaptability, creativity, and a willingness to explore various avenues to arrive at innovative and effective solutions.

For this project, we have adapted Dan Nassler’s ² interpretation of the UK Design Council’s Double Diamond Diagram ¹⁹ which has been specifically tailored for Human-Centered Design and UX (user experience) principles. Consequently, the subsequent phases of our project have been formulated in response to the different stages of this diagram, denoted in Figure 2 as follows:

1. Part 1: Discover – Research (insight into the problem – diverging).

2. Part 2: Define – Synthesis (the area to focus upon – converging).

3. Part 3: Develop – Ideation (potential solutions – diverging).

4. Part 4: Prototype/implement (solutions that work – converging).

Figure 2.

Dan Nassler’s interpretation of UK design council’s double diamond methodology for human-centred design and UX principles. ²

In this context, the design thinking process was guided by alternating phases of Divergent and Convergent Thinking. Divergent Thinking, guided by research, broadened the team’s perspective, fostered new insights, and encouraged the exploration of alternative options and ideas. Convergent Thinking narrowed the team’s perspective, necessitated decision-making, and involved prioritizing or eliminating options. Distinguishing between the problem space and solution space is imperative in preventing a team from prematurely generating solutions while still heavily reliant on assumptions about the problem. Neglecting exploration of the problem space often yields an over-engineered, costly solution with minimal added value to the user(s). It was, therefore, essential for the team to delve deeply into the problem space without preconceived bias. This laid the groundwork for discovering and developing solutions that were not only inspired by but also addressed real-world problems pertinent to the context.

Discover – Research

The research part explored key questions regarding the role of the physical building in connecting to audiences and the barriers to overcome in encouraging attendance to cultural activity. Additionally, we have conducted an in-depth study of the context of application, to develop a good understanding of technology, people, and the needs that the design is intended to meet.

Desk research and qualitative user interviews

The research team created a set of semi-structured interviews with GAC staff. It’s worth mentioning that this phase of the project occurred during COVID lockdowns, preventing real-time observation of visitor movement and direct interactions with the visitors. As a result, we relied on existing data from GAC staff about visitor movement and behaviour, which was sufficient for this project phase. This led to our focus on interviewing GAC staff exclusively. The interviews were between 40 and 80 min long, all depending on the participants’ experience and knowledge related to users’ and visitors’ experience of GAC Ryrie Street Building. The interviews focused on issues related to the usage of the building by the arts centre visitors, services provided by the building, technology usage, and movement through the building. The research team collected data from 9 GAC staff, comprising close to 11 h of audio recordings, including the written notes by the interviewer. We used inductive analysis of verbatim transcribed audio data to gain an understanding of the experiences of staff and visitors in the building. Two researchers worked independently from each other to complete an open coding of a representative sample of 10% of the data to develop an initial coding tree of common behaviour patterns. The research team then used this initial coding to code the remaining material, providing us with a list of insights that characterize the building usage. In our analysis, we employed a combined approach of ‘content’ and ‘thematic’ analysis, ²⁰ with a focus on both the frequency and repetition of events within individual narratives and across the entire study group. We also paid particular attention to the unique stories, sentences, conclusions, and remarks made by participants, especially those that deviated from the prevailing views on the utilization of the GAC space. To initiate our analysis and categorize the most salient themes, we utilized NVivo software. This process involved consulting interview notes and integrating them into the analytical framework. The initial themes identified as problematic areas are outlined below:

• Physical distribution of building users throughout building spaces

The design of complex buildings often necessitates separate vertical communication routes for distinct user groups, such as in-house employees and occasional visitors. However, in the GAC building, all vertical communication routes are merged into one, leading to various utilization challenges stemming from the initial building design. This issue notably impacts the visibility and communication between staff and visitors during events, hindering the organization of performances, corporate events, and children’s activities. It also leads to confusion among visitors regarding floor navigation, necessitating additional staff to guide them. One of the comments that came from a GAC staff member emphasized this issue in the following quote:

‘...The whole building is designed vertically, in a sense. There’s no flow across the building (horizontally), the flow is only up and down. And then it's completely restricted by one access route...’ (Participant #4).

• Distribution of information throughout the building

The role of employees in guiding people within the building during events is crucial. However, the absence of well-designed signage, consistency in information distribution, and poorly positioned signage contribute to visitor confusion and, at times, frustration. While staff members emphasize the importance of human interaction in a theatre environment, the limited number of front house staff can lead to operational challenges when dealing with a large audience. Front house staff, particularly the Box Office staff, often provide information about upcoming shows. One of the comments that came from a GAC staff member emphasized this issue in the following quote:

‘...They are a nightmare. The BrightSign are a nightmare to manage. The management of them is time consuming. Time consuming, they are not great system. I have a team, between marketing and my team, we have to actually do them ourselves, and it just takes a long time to do them. It's not an easy process...’ (Participant #6).

• Utilization of open building spaces/building activation

Participants recognize the aesthetic appeal of spacious lobby and foyer areas, featuring high ceilings, openness, natural lighting, and excellent interior design. However, these areas are deemed underutilized or not used effectively. While they serve as gathering spaces before and during intermissions, they lack a clear purpose and can be unwelcoming. Poor wayfinding signage further complicates visitor engagement with these open interior areas. One of the comments that came from a GAC staff member emphasized this issue in the following quote:

‘...For example, when you walk into the building, the wayfinding sign is only at the lifts. Another wayfinding sign is outside, you can see it on the pole, but you don’t know what studio spaces are there, you just know that you’re at the Art Centre. You walk in, you see the Box Office, you have somebody there to talk to you. But then where do I go...? If I was new to the space, where do I look when I don’t feel up to social interaction? And I don’t, and I just want to go enjoy the space for myself. I want to get to the event myself. So where is...? I can’t find that until I get to the lift. And before you get to the lift, you can see the stairwell. Do I go up the stairs first?...’ (Participant #2).

• User-building interaction

Discussions and workshops with GAC staff members revealed the significance of understanding how visitors and staff interact with the GAC building on Ryrie Street. However, there is currently no systematic methodology in place to comprehensively measure and establish behaviour patterns related to building utilization. All nine study participants expressed the need for ‘movement tracking’, ‘understanding where people are when in building’, and ‘the need for direct feedback’. They agreed that in order to improve the quality of building utilization, it is necessary to understand how and why people move around as they do and what can be done to make that experience more efficient, meaningful, memorable, and exciting.

In summary, our analysis utilized both content and thematic analysis methods to explore various challenges related to the utilization of the GAC building spaces.

Define – Synthesis

Stakeholder workshops

The stakeholder workshops involving the design-research team and GAC staff provided a valuable resource of background information and highlighted issues to be addressed, in connection with the problems identified during the interviews. Following the key stakeholder workshop, several key insights emerged:

• Understanding User Behaviour: Examining how visitors at the Arts Centre utilize various services and physical spaces within the building. Analysing how user behaviours and movements inside the building impact different operations.

• Management’s Motivation: Investigating the management’s objectives for enhancing space utilization. Identifying their primary motivations, such as resource efficiency, attracting more visitors, or improving business outcomes.

• Engagement Strategies During COVID and Beyond: Exploring the strategies planned by the management to maintain public engagement with the Arts Centre. Assessing measures taken during COVID-related lockdowns, current strategies, and those intended for an unforeseen future.

Conceptual frameworks – potential pathways for development

The initial workshops illuminated potential pathways for ideation. As a result, three conceptual frameworks were formulated, to guide our initial desktop research and technology appraisal:

• Interactive Co-creation: This framework emphasizes enabling building users to co-create and engage with digital content both inside and outside the building. Interactive information screens, for instance, allow users to access information and provide feedback.

• Intelligent Navigation: This framework involves the analysis of visitor movement patterns to facilitate easy and informed navigation within the building. The goal is to help visitors find activity locations effortlessly and without delays.

• User/Location-Specific Content Activation: This framework focuses on the real-time activation of selected digital content based on user profiles and location. For instance, visitors can receive personalized information about upcoming shows or interviews with their favourite actors while waiting at the cafeteria, outside the building, or while browsing the Arts Centre website.

It’s important to note that the COVID period introduced limitations to our methodology and required additional measures to address these challenges. For instance, observing users in their natural context posed a particular challenge during the COVID period. Nevertheless, conducting interviews with the staff proved to be a valuable alternative, yielding useful insights into the various movements and space utilization patterns of visitors.

Develop – Ideation

The design-research team developed numerous use cases, as potential pathways for the project, categorized under three overarching themes: ‘Breaking the 4th Wall’, ‘Time Travel’, and ‘Sensory Interaction’ in response to the conceptual frameworks identified earlier. These themes were employed to explore innovative possibilities that could be offered by existing technologies for enhancing various experiences of users in a performance arts centre, including visitors and the administrative staff. The following section provides a snapshot of some of the pathways explored by the team during the ideation process.

Theme 1: Breaking through the 4th wall

This theme revolves around transcending traditional boundaries between the audience and the performance at the GAC. It envisions use cases that provide audiences with access to information beyond the confines of standard performances and activities.

Use case 1: Talking screens

Talking screens involves digital screen installations (Figure 3) that enable GAC patrons to engage in conversations with digitized versions of professional GAC members. These digital conversation partners can include performers, members of the creative team, or GAC management personnel. Patrons can initiate conversations by asking questions from a predetermined list, prompting pre-recorded responses. This application activates the foyer space, fostering connections and knowledge-sharing between patrons and GAC professionals.

Figure 3.

GAC foyer with three large screens that are strategically positioned to engage visitors in the expansive space, each with a lifelike digitized representation of a GAC actor.

Use case 2: See-through walls

See-through walls theme offers a potential pathway to overcome the physical limitations of the GAC spaces. The concept builds on the idea to allow users to observe events in different areas of the GAC building, virtually, through digital representations superimposed on physical spaces. The added value would be to enhance visitors’ experience by offering a comprehensive awareness of the various activities taking place inside the building (Figure 4)

Figure 4.

See through walls. A visual representation of the idea of a visitor glimpsing at a performance taking place behind a wall through dynamic digital media overlays that augment the space and reveals hidden activities (in real-time) and the details of architectural space.

Theme 2: Time travel

The time travel theme explores opportunities to navigate experiences across different points in time within the GAC context.

Use case 3: Virtual windows

Virtual windows build upon the foundation of the see-through walls augmented reality app by providing glimpses into events from the past, present, and future. The idea revolves around not only delivering visitors information about what is currently happening inside the building, but what has happened in the past and will be happening in the future, creating an almost anachronic experience of the cultural space.

Use case 4: Sliding through time

Sliding through time had been envisioned as a novel sliding screen display to grant patrons access to a database of GAC events and performances (Figure 5). The idea is to build a repository of current and historical data, such as archival footage, images, posters, and relevant content. If designed as an interactive display, it could serve as an educational tool for school groups and a browsing platform for patrons interested in exploring GAC’s archives.

Figure 5.

Patron interacting with the display, browsing a rich database of GAC’s archival footage, images, and posters.

Theme 3: Sensory interaction

The sensory interaction theme explores the collection and visualization of data related to GAC activities, enabling enhanced user engagement and operational optimization.

Use case 5: Sentiment projection

Sentiment projection envisioned a ‘living’ building capable of expressing emotions influenced by various data streams, including user feedback, social media, and external news related to the arts in our city. Sentiment analysis would be conducted to detect positive or negative sentiments in the collected data, which would then be displayed on the building facade through projection to visually convey the ‘mood of the building’ to the public outside the building (Figure 6).

Figure 6.

Sentiment projection concept, with the building facade dynamically illuminated to reflect the mood (from left to right: anger, fear, disgust, sadness, and happiness) of the building determined by sentiment analysis of its inhabitants (source: Data Visualization and Feelings by Paul Eckman overlaid over GAC building façade).

Use case 6: Redirection through activation

This use case investigates how data on space usage can be leveraged to redirect users to other areas in the building, particularly areas with limited usage. Interactive projections in underutilized spaces could potentially motivate patrons to explore different parts of the building, improving overall flow and enhancing the visitor experience. For example, visitors waiting for an elevator may be enticed to take the nearby stairway through engaging sounds and visual effects (Figure 7).

Figure 7.

Redirection through activation concept. Interactive projection in the stairwell near the elevator enticing visitors to take the stairs with captivating sounds and visuals.

In summary, these three thematic areas and their respective use cases have been presented to showcase some of the early ideations of the project team, which led to further investigations of novel human-building interaction scenarios in a cultural space.

Prototype – Implement

Numerous ideas have undergone iterative development, practical experimentation, and rigorous testing. A pivotal factor shaping this work has been the invaluable feedback received from GAC staff during numerous workshops. The selection of ideas to progress into the next developmental phase has been guided by the overarching concept, based on the metaphor of the ‘Building as a Living Organism’. This concept has been developed after iterative testing of the earlier use cases, as a guiding principle to help with the selection, further refinement, and identification of specific features for each use case for further development. This concept builds upon the initial focus of ‘intelligent and adaptable environment’ situated within HBI to empower the interplay between human and the built environment.

The ‘Building as a Living Organism’ metaphor, as illustrated in Figure 8, posits that a building can engage in meaningful interactions, retain and share its memory, and sense user sentiments and movement. This transformative concept suggests that by harnessing relevant technologies, a building can evolve from passive physicality into a perceptive, sensitive, responsive, and intelligent ‘organism’ capable of living and engaging with visitors and users.

Figure 8.

Conceptual representation of the ‘Building as a Living Organism: An Edifice with Memory, Sensory Abilities, and Interactive Features’.

To mark the conclusion of the first stage of the Spatially Intelligent Arts Centre project, we have effectively implemented the following proof-of-concept installations. These installations amalgamate various aspects of the previously developed use cases but have been further refined in alignment with the concept of the Building as a Living Organism:

GAC spatial kiosk

A large touchscreen which uses an interactive 3D model to display; real-time information related to the use of spaces and the activities taking place inside the building at any given time; real-time occupancy information of key public areas; and suggestions for navigating inside the building in response to changing occupancy information in real-time (to avoid crowds). The GAC Spatial Kiosk installation coincides with the ‘interaction’ function of a living organism metaphor.

Talking screens

Talking screens are a digital intelligent building manager displayed on a screen that responds to visitors’ questions in real-time. Based on an AI-based conversational video as a new way to provide users with interactive content enabled by natural language processing and artificial intelligence, the system facilitates the playback of video snippets and creates a question-and-answer dynamic that approximates face-to-face communication. The talking screens installation coincides with the ‘interaction’ function of a living organism metaphor.

Thought bubbles

Though bubbles provides a multi-modal and cross-platform collection of post-performance audience voting experience. Each vote is represented as a bubble, and bubbles of the same category (voting option) merge and form larger bubbles. The bubble reactions gather and animate on the shared display for the audience to anonymously share and view the different sentiments collected about the show/performance. The information can be displayed across multiple venues of GAC. The thought bubbles coincides with the ‘sensing’ function of a living organism metaphor.

Memory lane

Memory lane is a multi-user, interactive, and tactile navigation through GAC’s past, current, and future performances through a digital archive, displayed on a large touch screen. The display goes beyond static image/video screens by incorporating elements of interactivity, dynamic memory, and thought processes into a playful display. The memory lane installation coincides with the ‘memory’ function of a living organism metaphor.

These prototype installations have been designed, developed, and assessed, showing promise in enhancing visitors’ overall spatial, physical, and social awareness inside the building. Implemented together, these installations were intended to provide unique individual and collective experiences for visitors throughout their journey through the building and create a number of intelligent interactions between the visitors and the building.

At this point, it is important to articulate our understanding of ‘intelligence’ within the context of an arts centre as a building. The arts centre, being a multifunctional facility, accommodates a diverse range of activities and events. However, due to the dynamic and ever-evolving nature of an arts centre’s program, the allocation of spaces for various functions is not fixed and continuously change. In other words, these spaces continuously transition to serve different purposes, including performances, rehearsals, workshops, and more. The users of these spaces vary (depending on the target audience), and their occupancy schedules fluctuate throughout the day. As already discussed in the paper, the interviews and workshops conducted with the GAC staff early in the project revealed the necessity to effectively monitor, coordinate, plan, and communicate the highly dynamic program content within the space to improve (1) visitors experience within the building and (2) the operational efficiency of the building. In this regard, the fluid and constantly changing functions, users, and occupancies within an arts centre necessitate a form of intelligence that can assist building managers and staff in effectively coordinating this dynamic environment. For example, for the GAC Spatial Kiosk, the intelligence refers to the capturing (e.g. sensors), communicating (real-time visualization), and purposefully redirecting (intelligent navigation) people’s movement within the space. For talking screens, on the other hand, the intelligence refers to their ability to engage with visitors in real-time using an AI-based conversational video, which leverages natural language processing and artificial intelligence to facilitate interactive, face-to-face-like conversations. Additionally, we also refer to a coordinated intelligence that is activated through the use of four installations all at the same time, across the building at any giving time – which collectively fulfils the sensing, interaction, and memory functions highlighted through the ‘Building as a Living Organism’ metaphor (Figure 9).

Figure 9.

Four proof-of-concept installations which collectively respond to the memory, sensing, and interaction functions of the ‘Building as a Living Organism’ metaphor.

Evaluation sessions were conducted in collaboration with GAC management staff and with the visitors of the performance arts centre. The primary aim of these sessions was to provide GAC staff and visitors with hands-on experiences of these installations. Simultaneously, the project team gathered feedback encompassing usability, general impressions, and suggestions for enhancements, and future directions.

Nine members of GAC’s management staff had the opportunity to engage with and provide feedback on the first four installations. In contrast, the Memory Lane prototype underwent testing with the general public during an event night. Following these interactive experiences, participants shared valuable insights regarding their engagement and interactions with the prototypes. Subsequently, qualitative data was gathered through recorded group discussions and one-on-one conversations with the participants. This rich dataset was further complemented by the administration of a digital survey form, individually completed by each participant. Additionally, video and screen recordings were utilized to observe and carefully document how participants interacted with the prototypes.

In the following section, a comprehensive assessment of the design of two installations will be presented, namely, the GAC Spatial Kiosk and Memory Lane. This evaluation is grounded in user-testing outcomes, while also considering the viability of future implementations and offering potential directions for further development.

User evaluations of prototype installations

GAC spatial kiosk

Geelong Arts Centre Spatial Kiosk has been developed as a large-screen touchscreen interface (Figure 10). This interactive system leverages a dynamic 3D model to provide real-time pertinent information to visitors. The installation aims to introduce a new dimension of human-building interaction by interweaving spatial information with spatial awareness through the dynamic visualization of real-time data. This prototype’s primary emphasis revolved around three specific categories of real-time information and their subsequent visualization, all closely linked to the efficient use of space and the facilitation of movement within the building:

• Activities and Performances: Real-time information about ongoing activities and performances in different sections/spaces within the building at any given moment.

• Space Occupancy: Data indicating the level of occupancy in specific areas, accompanied by estimations of average waiting times.

• Wayfinding: Guidance on the most convenient and efficient routes to various destinations within the facility.

Figure 10.

GAC Spatial Kiosk interactive user interface that presents a 3D spatial model overlaid with real-time event information and crowd data in a visually dynamic manner.

The overarching objectives of the GAC Spatial Kiosk encompass augmenting the building’s operational efficiency, achieved through the provision of a comprehensive overview of spatially connected activities. Simultaneously, it seeks to enhance the experiential dimension for visitors by enabling them to avoid congested areas (wayfinding advice adjusted according to real-time data) and by offering precise guidance to their desired locations, thereby mitigating potential overcrowding concerns.

Data concerning space occupancy is collected through spatial sensors. The system compares the actual headcount to the optimal occupancy for each location and visually represents the level of crowding using colour coding (e.g. red for busy and green for available, Figure 10). Two notable features of the GAC Spatial Kiosk are its intelligence and dynamic information display. Intelligence stems from the presentation of dynamic content, continuously updated in response to real-time data. For example, the system draws real-time program data from the GAC timetable, mapping the locations of each activity, event, or performance on the 3D model. Consequently, users can visually identify the names of each activity within the 3D spaces on the kiosk’s model.

The prototype tested with users (GAC staff) was a work in progress, with certain functionalities not fully operational at the time of testing. Therefore, the evaluation primarily revolved around assessing the general usability of the user interface, particularly the clarity of the displayed content. Eleven users participated, including nine members of the GAC team and two university students who volunteered for the evaluation. After a brief demonstration by one of the researchers, each user had 5 minutes to interact with the kiosk and then provided feedback through a questionnaire, rating ‘ease of use’ on a scale of 1 to 5 and offering written comments (Figure 11). While the connection between crowding data and the spatial information display has been successfully tested in a controlled environment, this prototype version did not undergo testing with real users as the linkage of this data to the physical display was not yet realized during user testing. The user evaluation primarily focused on assessing the general usability of the user interface, particularly the clarity of the displayed content and accessibility.

Figure 11.

User interacting with the GAC Spatial Kiosk display.

User results

The users predominantly reported positive experiences with the kiosk. They expressed satisfaction with the overall visual quality of the content and the clarity of the information presented. Notably, users appreciated the use of QR codes to access additional content and the visualization of crowded areas. However, some issues arose due to the partial implementation of planned functionalities, specifically concerning inactive buttons and some of the real-time information not yet linked to the spatial model.

Feasibility

The feasibility of this installation lies in its portability, making it adaptable for various settings, and it can also be easily replicated on different floors for staff and visitor use. The installation requires portable vertical touch screens and sensors to effectively capture crowd traffic, enhancing its real-time capabilities. From a software perspective, Unity3D served as the pivotal platform for application development.

Limitations

While the prototype utilized sample data for demonstration purposes, it can be effortlessly updated with additional data. The connection to live sensor data, although lab-tested, was not integrated into user evaluation due to a lack of visitors in the indicated areas during testing.

Future direction

Several staff members highlighted the kiosk’s utility also for backstage and back-of-house operations, aiding in the coordination of theatre company activities across various building spaces. One recommendation was the development of a content management system to facilitate efficient management of real-time spatial data by GAC staff. Multiple touch screens could be deployed at different locations and floors within the building for the convenience of employees and other involved parties. This expansion into a multi-screen data management system for backstage use would require minimal adjustments to the existing visualization of the current ‘way-finding’ function for specified locations. Additional future work could involve presenting real-time information about the building and various activities to enhance ease of movement within the facility.

Memory lane

This installation addressed the idea that the building has a ‘memory’ inspired by the overarching theme of ‘building as a living organism’, which has been used as a metaphor in guiding our ideation of the different installations and the impact of their collective use within the building. The conceptualized Arts Centre memory consists of a dynamic visualization of past, present, and future shows and events on a large touch screen through which ‘the memories’ can be accessed and interrogated by staff and visitors (Figure 12). Each memory is defined as a collection of images and video snippets, with a brief textual description.

Figure 12.

An impression of how the Memory Lane interactive display wall could be embedded into the spaces of the new GAC building.

Memory Lane goes beyond static video screens by incorporating elements of interactivity, dynamic memory, and thought processes into the display through the inclusion of the following functions in its design and operation:

• Unlike static video advertisements, Memory Lane allows users to access and interact with a collection of images, video snippets, and textual descriptions. This interaction creates a personalized and engaging experience for both GAC staff and visitors

• The system’s ability to facilitate a flow between the current (past and present) memory and related memories by providing links to related shows (including future shows) sets it apart. It enables users to explore the arts center’s history, discover related events, and delve into a narrative that is absent in traditional video advertisements. It provides a personalized experience to navigate through the collective memory of the arts centre.

• The proposed improvements, such as screen hints, active zone demarcation, and signage, underscore the project’s commitment to optimizing user experience. In that regard, Memory Lane goes beyond passive viewing to create an interactive and immersive environment which also allows content distribution via the users.

• Memory Lane transcends traditional advertising by emphasizing the value of a shared, collective memory the users can navigate through and share with people outside the arts centre (distribution of content via its users).

When the system is not in use by visitors, it will show that it is alive by visualizing a thinking process which involves (selecting) a series of shows and events that light up whilst all other shows are dimmed in brightness. A white highlight behind the tiles of the shows visualizes the firing of synapses and the thought process.

The Memory Lane Installation was developed over three stages. Stage 1 involved the compilation of GAC events and shows the development of a data structure that facilitated the linking of the files with metadata such as performance dates, additional image and video materials, description text, and related shows through a manual process. Stage 2 followed an iterative design process with project team members’ inputs, to develop and test multiple interface designs and idle animation ideas. In Stage 3, the team used Unreal Engine 5 to develop the Memory Lane application and its capability to dynamically read the metadata table and visualize the content from a media archive.

Memory Lane prototype installation was setup on an 85 inch, 8K Samsung Smart TV (Figure 13). The installation was tested at one of the venues of GAC in Geelong during a performance night with the public. The system used an OptiTUIO LiDAR scanner to turn the Smart TV screen into a multi-user touch screen. The project team also developed a mechanism, based on a Lightbridge, to trigger the idle animations when there were no users in front of the screen. Patrons of the performance were invited to trial the test system for 2–3 min and browse the content. A GoPro camera with a wide-angle lens was positioned to record the users and their interactions on the screen. Up to two users could interact with the system at the same time. After using the system, we asked the participants to complete a short, semi-structured interview about their experience.

Figure 13.

Memory Lane Installation setup in one of the GAC performance venues for user testing.

User results

Through the analysis of videos and semi-structured interviews, we derived six notable findings. Firstly, while the OptiTUIO system displayed some input errors, leading to task abandonment in 38% of cases, these issues did not significantly deter participants, primarily stemming from the LiDAR beam’s width. Secondly, participants generally found the touch interface’s interaction design intuitive, although some suggested a clearer call-to-action during idle states for enhanced interactivity. Thirdly, only a small proportion of participants engaged with textual descriptions of past projects (11%), emphasizing a preference for visual content. Fourthly, personal connections to previous shows and events emerged as a strong driver of interaction, fostering a sense of community and place, a recognition also acknowledged by GAC staff. Additionally, participants identified the use of colour as an attractive feature but sometimes found the design overwhelming due to the volume of content. Finally, participants offered valuable recommendations, including categorizing past projects by genre, incorporating multimedia forms, using larger poster sizes, adding more tags for relevant show suggestions, and implementing common touch paradigms like scrolling.

Feasibility

Regarding feasibility, it is worth noting that this installation, while physically larger than some others discussed in this report, remains portable. In terms of hardware, it necessitates a large screen connected to a high-performance computer and smart sensing systems for detecting user hand gestures. On the software front, the research team has developed a custom application for installation on a high-performance computer. In terms of personnel and skills, a person with database management expertise is required for content management and ongoing maintenance. Notably, there are no subscriptions or monthly fees associated with this installation. However, regular content maintenance will require the services of a technician.

Limitations

The Memory Lane prototype trial was limited by the distribution of the content. For the purpose of this installation, the project team did not include all of the very large collection of the image/video data provided by GAC. The prototype was also limited by the screen size (85 inches) – although the final installation is intended to cover full wall surfaces in a special enclave in the new building of the GAC (Figure 12). At the time of testing the application, the research team was unable to find a screen with sufficient resolution to cover an area of that size whilst still being able to provide users with readable images and text at a close range.

Future direction

Following the results of the user test, future versions of Memory Lane should indicate that the system is interactive, use a larger screen for display, and utilize sound and music to assist users in reminiscing about past performances, and making a clear distinction about the past, current, and future performances, with a possibility to update this time-related information. The interactive nature of the screen can be communicated via screen hints, clever environment design, such as the demarcation of an active zone, and signage around the screen. A large screen would provide a more attractive visualization of the content. This could be facilitated by scaffolding multiple 8K displays to form the screen space. A larger screen would have also improved the OptiTUIO tracking issues because interactive elements on the screen would have been larger and spaced further apart. Finally, sound is believed to make up a large part of visitor experience inside this building. The Memory Lane experience could be improved by providing an attractive system sound loop, interesting interaction sounds, sound snippets from the shows, and by playing sounds of video snippets.

Discussion and Conclusion

For buildings like the GAC to become a dynamic and intelligent place, each installation was designed to imitate an aspect of a living organism. These contributions included aspects of memory (Memory Lane), spatial awareness (GAC Spatial and GAC Kiosk), and interaction with visitors (talking screens and thought bubbles). Prior to the introduction of prototypes, the building lacked mechanisms for visitor engagement, relying solely on GAC staff for interactions. The prototypes illuminated the potential to redefine the building’s role, allowing it to function as an additional member of the GAC team. In this capacity, it could offer information (via GAC Spatial and GAC Kiosk and talking screens), convey the collective emotional ambiance of its inhabitants (as demonstrated by thought bubbles), and capture meaningful moments within its lifespan for future reflection and recollection (as seen in Memory Lane).

Central to our research inquiry was the evolving relevance of architectural spaces in the face of changing requirements, programs, content, and dynamic demands. The concept of what constitutes an effective space diverges between its users and those responsible for its management. Our approach recognized the significance of accommodating both perspectives in crafting effective solutions. While our project implementation was applied to an existing building, it is imperative to acknowledge the vast opportunities that lie ahead when these innovative concepts are integrated at the design stage of architectural projects.

This research primarily focused on the post-operational use of technology to enhance user experiences and operational efficiency, emphasizing the vital intersection of technology, architecture, user experience, and related fields. However, we must emphasize that technology, in isolation, is insufficient. The true potential of our proposed solutions can only be realized and intended benefits can only be harnessed when considered within the broader systemic (business, operations, and services) context of an arts centre building. This necessitates collaborative inputs from diverse experts, underscoring that an architect’s programming abilities are no substitute for the specialized knowledge and insights required from subject matter experts.

In conclusion, our academic exploration has shed light on the intricate challenges of translating research into commercially viable technology, revealing the need for additional developmental processes and resource allocations to achieve such viability. Nonetheless, our journey has revealed the multifaceted nature of architecture, extending far beyond the mere physicality of bricks and mortar. In this evolution, technology emerges as a potent interface, facilitating the seamless transition between space, function, and program within an arts center, while simultaneously enriching both the experiential and operational dimensions. As we look ahead, it is evident that the fusion of technology and design will continue to shape the future of built environments, offering boundless possibilities for innovation and improvement.