Using co-designed smartphone apps to cultivate authentic communication and disaster risk resilience in multi-cultural communities

Abstract

This article and research was first inspired by fire personnel attending emergencies that could have easily been avoided if the people living in the dwellings had been able to better understand fire hazards and safety preparedness. In response to this, we explore how smartphone technology can be used to empower culturally and linguistically diverse (CaLD) communities to enhance their disaster risk resilience. It presents the findings of a collaborative research project between Fire and Rescue NSW (FRNSW), a leading state emergency agency in Australia, and researchers from the University of Technology Sydney. The project, funded by the NSW Reconstruction Authority in partnership with the NSW and Australian Governments, addresses the pressing need for cultural competence within emergency and disaster response agencies to mitigate risks and bolster resilience among non-English speaking community members in New South Wales (NSW), Australia. Australia's increasingly diverse population, with nearly 30% born overseas, underscores the necessity for further investigation into the preparedness and empowerment¹ of multi-cultural communities in responding to emergencies and disasters. The article focuses on the development of a proof-of-concept smartphone app prototype Ready, Set, Go! that is aimed at empowering our culturally diverse community members. Through a user-centred and co-design approach, the app was collaboratively designed with CaLD community groups, utilising user experience research methodologies. Recognising that a good user experience transcends cultural boundaries, the study integrated cultural fluency into the design process, emphasising the active participation of CaLD community members as key stakeholders in the design process. This article discusses key findings and considerations around co-design and inclusive practices that elicit learning from lived experiences, the importance of personalising the app experience for optimal benefit of CaLD users, and how focusing on visuals, utilising gamification and simplifying language ensures greater accessibility. This research contributes to the existing literature by addressing the question of how to design a disaster risk resilience smartphone app that is accessible and usable for CaLD communities. By emphasising the role of digital technologies in fostering authentic and critical communication between government agencies and diverse community members, this study endeavours to empower CaLD communities in disaster risk resilience, making strides towards a more inclusive and culturally responsive approach to disaster preparedness.

Keywords

co-design cultural fluency disaster risk resilience multi-cultural communities smartphone apps user-centred design

Introduction

This article focuses on the role of smartphone technology in empowering culturally and linguistically diverse (CaLD) communities in disaster risk resilience.² There is clear inequity in fire safety messaging in New South Wales (NSW), Australia – an equity that literally puts lives in danger. This inequity is not only unjust but relatively easy to fix with a more nuanced understanding of how current communications are failing these communities and how digital inclusion is key to ensuring the safety of these communities. According to the Australian Digital Inclusion Index, our economic and social lives are experiencing an accelerated digital transformation and a key to digital inclusion is ensuring that all Australians have the opportunity to benefit from digital technologies to manage aspects of their lives including health and education (Thomas et al., 2023, p. 3).

This article presents findings from a collaborative, publicly funded research project between Fire and Rescue NSW (FRNSW), a leading state emergency and disaster agency in Australia, and researchers from the University of Technology Sydney, Australia. The broader project, funded by the NSW Reconstruction Authority in partnership with the NSW and Australian Governments, focuses on cultural competence for emergency and disaster response agencies and aims to reduce risk and improve resilience among community members in NSW, Australia from non-English speaking backgrounds.

Australia is an increasingly culturally diverse society. Currently, nearly 30% of Australia's population was born overseas (Crawford et al., 2021). Existing studies indicate that more research is needed to understand whether such multi-cultural communities are prepared and empowered to respond to emergencies and disasters (Crawford et al., 2021; Duckworth, 2022; Edwards, 2020). In emergency management, cultural diversity has posed challenges in communicating messages to members of CaLD communities (Brooks, 2023; Subramaniam and Villenueve, 2020; Young and Jones, 2019). Reports from various cultural settings describe gaps in effective communication to and with multi-cultural groups during the COVID-19 pandemic (Brooks, 2023; Ferdinand, 2021). Governments faced challenges in their messaging about vaccination, self-isolation, lockdown protocols, and other COVID-19 related instructions, especially in communities where English was not the first language spoken in homes (Arlikatti et al., 2014; Ferdinand, 2021; Ogie et al., 2018). It was also during COVID-19 that governments saw the use of smartphone technology as being central to managing reliable communication and messaging around community safety. In Australia, media reports highlighted that during disasters such as bushfires and flooding in regional areas in NSW and Victoria, multi-cultural communities did not receive timely and relevant emergency warnings and response instructions due to language barriers (Arora and Selvaraj, 2022; Brooks, 2023; Cassidy, 2022).

With the increasing frequency and complexity of emergencies and disasters faced by communities, including those in NSW, building community resilience is key to assisting emergency and disaster responders to manage these critical events (Hanson-Easy et al., 2018; Seeger et al., 2021). It is therefore imperative for the emergency management sector to proactively develop strategies for effectively communicating with people from diverse cultural and linguistic backgrounds who may be vulnerable when emergencies and disasters occur. This may be due to a lack of proficiency in English as the default language used in Australia in emergency and disaster communication. Furthermore, the impact of COVID-19 demonstrated the need to integrate inter-agency and cross-sectoral collaboration, with technology as the nexus, to not only support culturally diverse communities, but more importantly to encourage them to be active agents in developing disaster risk resilience mechanisms that are meaningful to them.

One of the goals of the project was to produce and deliver a proof-of-concept smartphone app prototype for FRNSW that demonstrates the effectiveness and viability of the app for its culturally diverse community members. This article contributes to this research gap by posing the question, how might we best design a disaster risk resilience smartphone app to ensure accessibility and use by CaLD communities? In answering this question, this study contributes to the literature by presenting empirical insights on how smartphone apps can facilitate authentic and critical communication³ in disaster risk resilience and management that caters to the linguistic and cultural needs of Australia's multi-cultural societal fabric. We emphasise the role of digital technologies (Stephens et al., 2021) such as smartphone apps in creating avenues for trustworthy, dependable and genuine communication between disaster risk agents (i.e. government agencies) and disaster risk bearers (i.e. culturally diverse community members) as co-creators in disaster risk resilience (Sukhwani and Shaw, 2020). We focused on how to utilise the multi-modal affordances of smartphone app technology to produce a more visual-based communication, eliminating the necessity for language translation, while also providing culturally diverse imagery (i.e. photos, animations, videos reflecting people of various cultures and races) for adoptability.

Why smartphone apps? Research shows that more than 92% of Australians own a smartphone (Deloitte Touche Tohmatsu, 2021), and that apps have become so ubiquitous and integrated into our everyday lives that we often forget we are even using them (Goggin, 2011). Traditional modes of communicating with communities utilised by emergency agencies like FRNSW have relied on translated fact sheets made available online. Additionally, research suggests that digital and gamified approaches to public safety communication have been highly effective. For instance, focusing on a child-friendly disaster awareness smartphone app, Clerveaux et al. (2010) find that this intervention was ‘effective in addressing the multicultural challenges’ (p. 216) as it enabled migrant children to navigate the different stages of disasters from preparedness to recovery. As children often serve as information conduits for their immigrant parents, who may have limited language proficiency, the smartphone app served as a means to disseminate information to these families. Fernando et al. (2019) also discuss the effectiveness of a smartphone app designed to teach users about the different stages of an emergency or disaster. Unlike Clerveaux et al. (2010), whose app was aimed at school-aged children, the app by Fernando et al. (2019) encompassed various learning modules that were intended to take adult users through different disaster stages, while still gamifying learning and preparation for disaster events. This approach made the application accessible to users with varying levels of literacy.

Using a user-centred, co-design approach and user experience research methodologies including card sorting and usability testing, the FRNSW smartphone app prototype Ready, Set, Go! was developed collaboratively between the researchers and members of NSW CaLD community groups.⁴ Over three sessions, community members participated in key activities that would test the feasibility, functionality and usability of the app. User-centred design is an approach to product and service development where every stage of the design process considers the end user and how the product or service will meet their needs (Garrett, 2003). As Kuniavsky (2010) asserts, ‘the user experience is the totality of the end users’ perceptions as they interact with a product or service’ (p. 14) and can include effectiveness, efficiency, emotional satisfaction and the quality of the relationship with the product or service provider. When you can understand and empathise with the people you are trying to reach and design from their perspective, or better yet, with them, you end up building something they will use. The challenge arises when you are trying to ensure your design meets the needs of a diverse group of users from diverse cultures and with varying degrees of English language competence. We know that social and cultural conventions can influence how people interact with a digital product or service and what people will expect from it (Perea and Giner, 2017). Apps, as with other digital and physical products, contain embedded cultural influence, and the more embedded it becomes, the more pernicious its influence (Madden, 2013).⁵

While we want to design with cultural fluency⁶ in mind, we must also adhere, to some degree, with the Western-centric design principles expected by the app stores (primarily Apple and Google). However, as Madden (2013) notes, ‘enjoying a better user experience shouldn’t require acting more American’. Rather, he argues that cultural fluency is required to ensure that the design is compatible with the user's cultural background to meet them halfway. There is a consensus among user experience designers that a good user experience should be culturally compatible (Madden 2013), however ensuring compatibility across diverse cultures and languages present design challenges. To best tackle this, we adopted a user-centred design approach, designing and developing our app prototype with our CaLD community members throughout the entire design process. There was a purposeful intent to embed as many elements of co-design practice as possible. Co-design builds upon user-centred design by including the end-users in the design process as key stakeholders. It also advocates for the active participation of the end-users to ensure that the product or service being designed is relevant and usable for all those affected by the design solutions (Durall et al., 2020a; Sanders and Stappers, 2008) and based on their lived experiences (Sanders, 2002). The goal is for the users to have agency in their experiences and feel empowered (Durall et al., 2020b; Kwon et al., 2014; Matuk et al., 2016; Tissenbaum et al., 2012). Importantly, as Tan et al. (2020) note, well-designed technological solutions can help users make informed decisions in times of crisis, but ‘a disaster app will be of little or no value if a user finds it unusable when interacting with it during crises’ (p. 616). The purpose of the app is to help keep people safe, so if the app is purposefully made to be accessible to those in our communities from diverse cultural and linguistic backgrounds, then by extension, the app is accessible to a greater percentage of our population, increasing safety and resilience.

This article profiles the development of the fire prevention education app, Ready, Set, Go! and discusses key findings and considerations around the co-design and inclusive practices that elicit learning from lived experiences, the importance of personalising the app experience for optimal benefit by CaLD community members, and how focusing on visuals, utilising gamification and simplifying language ensures greater accessibility.

Literature review

Towards better cultural competence for NSW's CaLD communities

This study is guided by theoretical perspectives from existing scholarship in the areas of cultural competence and disaster risk resilience. Specifically, we draw from Campinha-Bacote's (2002) Process of Cultural Competence Model and the disaster resilience framework co-developed by the Australian government's National Recovery and Resilience Agency and the Australian Institute for Disaster Resilience. Of the 8.1 million people living in NSW, more than 2 million were born overseas and more than 1.8 million have a first language other than English (Australian Bureau of Statistics, 2022). NSW is the most populous state in Australia and includes Sydney, the state capital and the nation's largest city. NSW's cultural diversity and expanding population bring many economic, social and cultural benefits, but also create challenges in developing strategies to reduce risk for those who are most impacted by—and often least involved in—emergency and disaster preparedness, response and recovery. It is imperative that the emergency management sector is proactive in developing strategies to more effectively communicate with NSW's substantial CaLD communities. However, Australia is plagued by a monolingual mindset (Clyne, 2005; Grover 2023; Hajek and Slaughter, 2015). To date, FRNSW's initiatives to connect with CaLD communities in NSW are ad hoc and largely limited to printed generic information sheets that are translated into different languages. Despite a genuine commitment to addressing community safety, emergency personnel and volunteers are often restricted by a lack of resources and training.

There are two key dimensions of barriers that impact the vulnerability of CaLD communities in disaster risk. On the one hand, individual-level barriers include socio-economic status, language and trust. Moreover, these barriers can extend to cultural linguistic challenges in communication that are not limited to translation. In the event of a disaster, crises can be compounded by cascading problems such as words that are not directly translatable, cultural incompatibility of concepts and social barriers (Alexander and Pescaroli, 2019). For example, when asked to translate the fire safety messages like ‘get down low, and go, go, go!’ Arabic and Kurmanji speakers were unable to provide a direct translation, and when translated the rhyming is lost, making the message less memorable. Ambiguous and unclear communication (or translation) can exacerbate the ambiguity and uncertainty brought by emergencies and disasters and can lead to weakened trust amongst community members (Cadwell, 2020). On the other hand, there can also be institutional-level barriers to cultural competence, which relate to the inability of organisations to adapt to these challenges. For instance, there may be limited knowledge of CaLD community needs and limited stakeholder collaboration (Andrulis et al., 2011). These barriers highlight the necessity of a reciprocal relationship between emergency management agencies and communities in building trust and resilience in emergencies and disasters (Duckworth, 2022; Ogie et al., 2018). To further this relationship, we draw from Campinha-Bacote's (2002) model for cultural competence, which proposes that cultural competence lies at the intersection of cultural awareness, cultural knowledge, cultural skill, cultural encounters and cultural desires (see Figure 1). By using this framework, we can understand the various components that create cultural competency, including awareness, skills and knowledge, but also encounters and desire. Scholars such as Marlowe et al. (2018) have contributed to this field of inquiry by proposing the 4R Model of Culturally-centric Communication that includes reach, relevance, receptiveness and relationships as factors to consider in multi-cultural emergency and disaster communication. Campinha-Bacote's (2002) and Marlowe et al.’s (2018) propositions are important to this study's designing of the disaster smartphone app. Cultural factors are critical inputs to the development of a disaster smartphone app that aims to raise awareness and encourage community members to work with emergency and disaster responders.

Figure 1.

Campinha-Bacote's process model of cultural competence (Campinha-Bacote, 2002, p. 183).

Disaster risk resilience and the role of technology

Australia's general framework in disaster communication and management (Australian Disaster Resilience Policy Landscape) is aligned with international strategy and policy in managing disasters across four stages: prevention, preparedness, recovery and response (AIDR, 2021). This framework underpins this study's focus on the use of smartphone app technology across all four stages. It highlights that for smartphone app technology to be useful in encouraging community engagement in disaster risk resilience, it should be integrated holistically prior to, during and after emergencies and disasters in consultation with community members. The model reflects specific communication activities that can be developed as content features of the proposed disaster smartphone app prototype. However, there is still more to understand about the role of digital technology in promoting disaster risk resilience amongst CaLD communities. Ogie et al. (2018) highlight the challenges of technology in communicating with CaLD communities and argue that culturally appropriate messaging during emergencies and disasters is key to addressing the unique cultural and informational needs of diverse communities, thus encouraging them to be active co-participants in the communication process. Factors such as a cultural group's experience with emergencies and disasters, their level of knowledge and expectations in disaster communication and shared local values and survival practices require a nuanced understanding of emergency and disaster response for impactful messaging via communication technologies. This is a current gap identified by FRNSW that is addressed by this current project.

While the use of technology in disaster risk resilience is not a new area of inquiry, culturally competent smartphone apps are one form of technology that is yet to be fully examined in disaster communication and management, particularly in the Australian context. At present, there are no culturally competent smartphone apps devoted to disaster risk resilience, constituting a significant gap in the literature. However, there are some comparable applications that have been trialled in the fields of nursing and public health (Bender et al., 2016; Gonzalez et al., 2021; Owens et al., 2020; Victorson et al., 2014). Such studies demonstrate the necessity of culturally competent e-health interventions to facilitate improved patient outcomes, while also demonstrating their utility in the cultural competence training and education of service providers and professionals. This current study draws on these theoretical and methodological approaches used in public health smartphone apps by applying them to a community emergency and disaster risk resilience context.

Fostering more resilient communities using smartphone apps

There is a demonstrated need for research to better understand if innovative technologies can be leveraged to communicate strategies for disaster and emergency prevention and preparedness to potentially high-risk CaLD community members in NSW and increase resilience. In the context of natural disasters, Freeman and Hancock (2017) working off the United Nations Office for Disaster Risk Reduction (UNDRR) policies, define resilience as the ability of communities ‘to resist, absorb, accommodate and recover from hazards to which they are vulnerable, including physical, social, economic and environmental susceptibilities, in a timely manner’ (p. 934). Further, they argue that ‘preparedness, risk reduction, resilience and vulnerability are interlinked and point to the need for public policies that address structural risks and susceptibilities that may be barriers to disaster preparedness, recovery and socio-economic sustainability’ (Freeman and Hancock, 2017, p. 935). This definition highlights scholars’ recent progression from crisis resilience frameworks centred on response efforts to more holistic approaches that include prevention and preparedness. For FRNSW and NSW Reconstruction Authority, the importance of education in community preparedness is demonstrated through both policy and programmes aimed at improving NSW residents’ resilience (Fire and Rescue NSW, 2023; NSW Reconstruction Authority, 2022).

The use of smartphone apps for disaster risk preparedness is not a new phenomenon, and in Australia, state and federal governments have been utilising apps for this purpose since the release of the now defunct DisasterWatch app from 2011, which was designed to improve access to disaster information and reduce the number of non-emergency calls made to triple zero (Australian Emergency Management Institute [AEMI], 2012). Most disaster apps are designed for response and recovery, to provide early warning of emergencies using location-awareness technology such as the Hazards Near Me app (NSW Government, 2023), or information about disaster relief resources and delivery (Chen, 2019). While the affordances of smartphones and applications include the ability of users to not only receive vital information but then share that information among their networks, the lack of digital connectivity during natural disasters has illustrated the limitations of network capabilities in the stages of response and recovery (Marshall et al., 2023). As Marshall et al. (2023) note, by using digital technologies, agencies such as the Bureau of Meteorology can target their messaging to specific regions and communities, yet ‘deficits in connectivity literacy become particularly problematic when they impact on access to critical communication and information channels during times of crisis’ (p. 28). It is globally recognised that disaster and crisis resilience is greatly dependent on telecommunications as essential infrastructure, as well as access and literacy (Marshall et al., 2023, p. 25). Digital connectivity is essential for crisis resilience because it enables communities to use various digital technologies and the Internet to access critical information from authoritative sources, as well as enabling real-time updates and discussions on social media (Marshall et al., 2023; Vargo et al., 2021).

In Australia, the COVID-19 pandemic highlighted the critical role of communication and messaging, often delivered via digital channels, in influencing public behaviour (Williams et al., 2022). Additionally, the COVID-19 pandemic illustrated how Australian citizens have also become more wary of government agency-produced apps, with privacy concerns about data surveillance fears (Goggin, 2020; Mann and Daly, 2019; Wilson, 2020; Wolf, 2021; Yang et al., 2021). Therefore, the willingness to download and use apps developed by government agencies may be reduced. Notably, while disaster apps have the potential for improving the public's preparedness and response to disasters, the usability of these apps from the perspective of the public as end-users is an understudied topic (Tan et al., 2017; Tan et al., 2020). Taking into consideration issues of cultural competence, the use of technology in disaster risk resilience and the use of smartphone apps in fostering community resilience, this article builds on existing knowledge and frameworks to profile the development of the Ready, Set, Go! fire prevention and education app.

Methods

The focus of this article constitutes phase 2 of a broader study, which encompasses the development and pilot testing of a smartphone app prototype using the culturally competent communication model produced in phase 1. During this initial phase, four CaLD community groups from different cultural and language backgrounds (including Arabic-speaking, Chinese-speaking, Vietnamese-speaking, and Kurmanji-speaking) were invited to participate in collaborative and co-designed research activities to develop a fire safety and preparedness smartphone app prototype. Each of these groups had basic to intermediate English-language proficiency and were included in the study as they were participating in adult English-language programmes targeted at newly arrived migrants. Translators were present in each session, providing translation for facilitators, but answers were mostly provided by participants in English. Translators were provided with the instructions and activities prior to the sessions so that they were familiar with the content and confident to provide translation.

In phase 1, focus group discussions were held with these four community groups in their local adult education classrooms or settlement services office. A series of questions around existing fire safety knowledge, effective communication practices, media use and desire and needs for app features and functionality were asked.⁷ Because the literature review revealed both individual and institutional barriers to cultural competence, the qualitative research consists of both focus group discussions with selected CaLD communities in NSW and a series of semi-structured interviews with FRNSW emergency and disaster responders. Key findings from phase 1 included:

Translation is not communication: Research shows that during emergency and disaster events, a range of linguistic challenges can arise, including words that are not directly translatable, cultural incompatibility of concepts, as well as the existence of social barriers. Along with varying English proficiency levels, members of CaLD communities also have differing levels of literacy in their native language. Translations are enacted by different people at different levels: they may be oral or written, provided by a trained professional and sometimes not; sometimes by an adult and sometimes by a child. Translation cannot simply be the conversion of a written message from one language to another; but requires communication strategies that take into account cultural beliefs, practices, needs and goals of communities, and deliver tailored messaging in multi-modal approaches – such as the use of images, infographics and videos to engage users regardless of literacy level.

A matter of trust: Firefighters interviewed told us that some cultures are wary of authority, and that this is a potential barrier to authentic engagement. But on further discussion, it became clear that the firefighters were a little wary too. Many expressed a lack of confidence to engage in a cross-cultural context – a fear of embarrassing themselves, of offending, or of simply failing to convey their message. Trust clearly plays a significant role in the engagement and receptiveness of CaLD communities, and a culture of trust is built when community members and emergency responders become co-creators in culturally competent approaches that reflect the values and norms of community members on the one hand, while aligning with the necessary messaging of government agencies around safety on the other.

Towards cultural competence as the foundation for authentic engagement: At an individual level, cultural competence requires an awareness of our own assumptions, values, beliefs, as well as a recognition that others have different needs because of their own culturally informed worldviews and backgrounds. This is demonstrated at an organisational level through the creation and implementation of systems, policies, and processes to ensure that cultural diversity and difference are considered across the organisation.

The development of the smartphone app prototype in phase 2 builds on the findings of phase 1 discussed above, with a mixed-methods user experience (UX) research design. It follows a user-centred, co-design approach, wherein user experience design methods including card sorting and usability testing with CaLD community members were implemented. Two CaLD community groups from the original four in phase 1 (Arabic-speaking and Chinese-speaking) were selected to participate in the user testing. The two main reasons for this were logistical – we could only obtain ongoing access to two of the four groups as they were part of adult education classes and the Kurmanji-speaking community group was located regionally and it was not feasible to travel multiple times for the testing workshops; and secondly, timing – we only had 3 months to complete the user testing and including more than two groups would have been unrealistic within that timeframe with the resources available. We believe that testing with two groups still provides viable and sufficient data to support the outcomes of the project. The goal was to produce a proof-of-concept smartphone fire prevention and education app prototype (including developing new graphics, icons and videos specifically for the app) that demonstrates the effectiveness and viability of the app for its culturally diverse community members. The extent of participation, based on Hodson et al. (2023), is outlined in Table 1 below and illustrates the key research methods and activities engaged with across both phases of the research project. Importantly, it highlights how culturally diverse communities were engaged in the process from inception and throughout the research and design process.

Table 1.

Extent of participation by CaLD community members (based on Hodson et al., 2023).

Extent of participation	Participants	Key activities
1. Users are actively involved in designing the app prototype	n = 60 (4 × focus groups in phase 1)	Workshopping desired features and functionality
	n = 30 (4 × workshops in phase 2)	User testing for information organisation, functionality and features
2. Information is solicited from key stakeholders	n = 10 (NSW firefighters)	Semi-structured interviews
3. Stakeholders with knowledge of users are informed and given opportunities to provide feedback	n = 8 (members of inter-agency reference panel)	Quarterly meetings to present updates and findings and gain input from expert industry and agency representatives
4. Secondary sources reporting on users are consulted	Chief investigators and research assistants	Literature review and scoping completed
4. Secondary sources reporting on users are consulted	Chief investigator from app design background	Training the research facilitation team

CaLD: culturally and linguistically diverse.

The app prototype design followed Jesse James Garrett's (2003) UX design model, which highlights the need to start the design process by understanding the strategic objectives of the app before moving onto visual design elements (pp. 21–36), as well as Mike Kuniavksy's (2010) UX design model, which focuses on information design, interaction design and interface design (pp. 21–25). Information design is a critical step in the design process as digital systems collect and display information, and it is crucial that the information (or content) is organised, prioritised and presented in a way that meets users’ needs and goals. Information design is supported by a cohesive information architecture and organisational structure that is consistent and coherent across cultural and language groups (Kuniavksy, 2010, pp. 21–25). Interaction design determines how users can interact with the various interfaces and how elements flow and fit together on a screen. As Kuniavksy (2010) notes ‘when using a device or environment, people need to understand how to accomplish their goals. They need to know what choices are available and how to make them’ (p. 21). The final consideration was in the smartphone app's readability and functionality. Interface design is a term used to describe the process of designing and organising the design elements on a screen (Kunivasky, 2010, p. 19). The interface design should emphasise task flow: ‘the sequence in which different interface elements are invoked” (Kunivasky, 2010, p. 19). In our final workshop, participants were asked to open the prototype on their smartphones and complete set of tasks, including completing the ‘onboarding’ process and finding and completing the ‘create a home fire escape plan’, ‘home fire safety checklist’, ‘how to check your smoke alarm’ and ‘practice calling 000’. In this workshop we were testing how participants were navigating through the app, but also how the visual design itself (composition of elements on the screen) was supporting or hindering that navigation.

Findings and discussion

Campinha-Bacote's (2002) model for cultural competence, proposed that cultural competence lies at the intersection of cultural awareness, cultural knowledge, cultural skill, cultural encounters and cultural desires. To improve our own cultural competency, we wanted to better understand the possible cultural nuances when engaging with the content and information in the app prototype.⁸ The first of our community workshops involved participants completing a card sorting activity to inform our information design. Card sorting is a UX research method where participants are asked to group or sort individual notecards with labels according to what makes sense to them (their mental model). This allows us to determine how our target users understand the content and helps us create an information architecture to best meet their needs and expectations (Sherwin, 2018). The findings from the open card sorting activity (examples seen in Figure 2 below) informed the information architecture of the app, giving us a better understanding of how to group information for the majority of our target users.

Figure 2.

Examples of completed card sorting activities where participants grouped content into meaningful categories and provided a label for the groups.

The second workshop involved participants viewing the draft smartphone app prototype on the laptop computer and completing key tasks and scenarios that we would expect users to frequently undertake when using the app. These scenarios tested whether participants were able to locate important information in the app and understand the information when they found it. For example, participants were asked to locate the information on ‘testing your smoke alarm’, ‘completing a winter fire safety checklist’ and play the ‘keep looking when cooking’ game. These scenarios presented information to users in different ways, which allowed us to gauge how the information was being understood by our participants. The ‘testing your smoke alarm’ content was mostly visual, including a video and illustrated step-by-step instructions (as seen in Figure 3 below). The ‘winter fire safety checklist’ was mostly text-based with instructions presented in a written checklist with small icons to support each item (as seen in Figure 3 below). Lastly, the ‘keep looking when cooking’ game enabled participants to explore an interactive illustrated kitchen for hazards (as seen in Figure 3 below). These scenarios gave us a better understanding of how participants navigated around the app and how they were interacting with the content.

Figure 3.

‘Testing your smoke alarm’ screen (left), ‘winter fire safety checklist’ screen (middle) and ‘keep looking when cooking’ screen (right).

The main feedback gained through this round of user testing was that participants were locating information using icons first, rather than words, and if the images were not specific enough, users did not locate the desired information. So, images were going to be incredibly important in translating not just fire safety messaging, but also basic navigation and categorisation of content. Observers noted the steps the participants took to achieve the task and at the conclusion of tasks asked participants to report back what steps they took and what they now understand about fire safety.

Co-design and inclusive practice: Learning from lived experiences

Findings of phase 2 of the research project describe the development of a prototype of a smartphone app for culturally competent disaster communication and management. This app intends to serve as a two-way platform for communication and engagement between state emergency and disaster agencies and members of CaLD communities in NSW, Australia. Therefore, it was an important consideration to include members from those communities in the design process from the start and throughout the project. The workshops provided opportunities to test our assumptions about the inherent risk in CaLD communities. When speaking with our CaLD community members that have immigrated to Australia, we found that many in fact had experiences that made them more resilient to disasters and emergencies. Many had previously experienced disasters triggered by natural hazards and some, conflict and war, while others had lived in refugee camps where fires and other emergencies were commonplace. Learning about their lived experiences through focus group discussions enabled us to better understand the personal attributes they bring to disaster risk resilience, and how we can best leverage those attributes in the communication model and smartphone app prototype development.⁹

The creator of the Inclusive Design Toolkit for Microsoft (2016), Kat Holmes, argues that ‘human beings have amazing capabilities to adapt to different situations, and understanding those adaptations is the key to real insight’ (p. 26). We wanted to ensure that the participants felt they were included in the process and that they were valued as agents in the research. As Holmes and Maeda (2018) note, there are many challenges that stand in the way of inclusion, including ‘treating inclusion as a benevolent mission… [and] believing that it should prevail simply because it's the right thing to do is the fastest way to undermine its progress’ (p. 4). By engaging in active listening, the research team gained invaluable insight into the types of features and functionality that would be utilised more frequently. For example, many participants with basic English language proficiency explained that one of the most frequently used apps in their smartphones was a translation dictionary. When usability testing revealed that users were navigating through the app using images rather than text, many explained that this was because they did not understand the terminology like ‘hazard’ and ‘alarm’. These words are important to learn in English, so a dictionary function was added to the app so when users come across these important keywords in the app, they were able to click on the underlined text to read the pop-up explanations and save it to their in-app dictionary. While having an in-built translation function for the entire app would aid in all words being translated, we cannot count on the reliability and accuracy of the Apple and Google translation. Additionally, understanding these key words in English means that they are understood in other environments, particularly in times of crisis. This design feature would not have been realised without the engagement of community members from CaLD backgrounds.

Personalising the app experience for optimal benefit

An early finding from the focus groups was that the users wanted an app that had relevant information to them. An affordance of the app environment is the ability to obtain input from the user in order to personalise their experience. The app prototype includes an onboarding section, designed to be completed by the user upon downloading. This section requires input from the user, including personal information, the type of house they live in (e.g. single-storey house or apartment/unit), who lives with them in their house (e.g. children, pets, elderly people or people with disabilities) and where they live (i.e. their address to identify if they live in bushfire or flood prone areas). This input will then personalise the app content to provide users with hazard-specific information such as bushfire or flood preparedness content, if they are in disaster-prone areas, how to include their family members or flatmates in home fire safety planning and home fire safety messaging for the type of residence they live in. The app also includes the use of push notifications to remind users to perform important timely tasks like testing their smoke alarms each month and completing safety checklists such as Winter and Bushfire Safety at key times in the year.

Focus on visuals, gamification and simplified language

There are over 280 languages spoken in NSW (Transcultural Mental Health Centre, 2022), so it is impossible to translate the app content to meet the language needs of all residents, although this is a feature enabled in app development from Apple Google translation, though accuracy and reliability of translation is an issue. Furthermore, not everyone has basic written literacy levels even in their first language. In the second workshop when participants first interacted with the draft prototype, most participants with basic English proficiency navigated the app and located information using the icons first rather than the words or labels within the app. This suggested that visuals were going to play an incredibly important role in communicating key information and messaging. The app utilises icons to visually represent key topics, so users are able to find the desired information more easily. The use of images to illustrate key messages was also useful in comprehension and retention. Lastly, important fire safety skills like how to check a smoke alarm, how to create a home fire escape plan and what to do in the event there is a fire in your home were more easily understood when presented as videos. In workshop 2, users were presented with the videos mentioned above and were able to recollect the key steps involved in these fire safety activities and recount them to facilitators. Additionally, when shown the explanatory videos for ‘how to create a home fire escape plan’ and ‘how to call triple zero’, participants were able to recall the required safety steps more accurately then when presented in the text-based explanations provided in the existing fire safety information sheets, which contained mostly text and some images, as seen in the Figure 4 below which shows the two versions of ‘how to test your smoke alarm’.

Figure 4.

The text-based explanation provided in the Fire Safety Module for ESOL students (left) and video explanation provided in the app prototype (right) of ‘how to test your smoke alarm’.

The inclusion of gamified elements including ‘keep looking when cooking’, ‘practice your home fire escape plan’ and ‘practice calling 000’ provided fun ways for users to practice their fire safety knowledge. These mini games can also be played with other members of the home, including children, and help families start conversations about home fire safety.

In workshop 3, participants were asked to play through these mini games and were then asked if they found them to be ‘fun’ and ‘useful’. Each participant shared that they indeed found them fun and useful, with many articulating that the ability to practice calling triple zero would help them feel more confident in the event of a fire incident. The main feedback points from workshop 3 included that some participants were not understanding the task instructions and were therefore missing key information to achieve the task; some would skip quickly through screens missing key videos, trying to complete the task as quickly as possible, rather than comprehensively. Most found the GIFs in the checklist to be effective and emotive, stopping to read when they saw flickering flames for example. And, finally, the language used by FRNSW still required further simplification with words like ‘prone’ in relation to bushfire safety not being understood.

In the first draft of the prototype, the textual information was taken mostly from existing information found on the FRNSW website (https://www.fire.nsw.gov.au/). The static fact sheets and checklists were initially adapted into interactive processes, encouraging users to swipe through each item and check off when they have read and understood the content. In the first workshop, many participants with basic English language competence voiced that much of the text was too difficult to understand. For the second workshop, the language was simplified, and features were added so that key words were able to be highlighted and added to the dictionary. In workshop 2, some participants explained that some of the language was still too difficult and that they were relying purely on the images to understand the content. As mentioned above, the visuals then became especially important in ensuring key messaging was being understood. Having keywords alongside images ensured that those important safety words were being learnt. Participants were able to communicate their understanding of the messages and recall them more easily when they were provided in visual form. Having CaLD participants as part of the project therefore heavily influenced and informed the information, interaction and interface design of the app.

It was through these key UX research activities that we were able to test the design with our target users (CaLD community members) and receive important feedback in the design of the app prototype. Embedding a user-centred co-design approach provided invaluable input and key insights at key stages of the app prototype development, resulting in an app that is more likely to meet the needs of the target user groups, including the CaLD communities of NSW. Additionally, by taking a cultural-competent approach (Campinha-Bacote, 2002) to design a smartphone app prototype that would serve the CaLD community across the four stages of a fire emergency: prevention, preparedness, recovery and response (AIDR, 2021) highlights how digital technology can be used to encourage community engagement in disaster risk resilience, and how it can be used in prevention education, during and after integrated after fire emergencies.

We used Marlowe et al.'s 4R Model of Culturally-centric Communication (2018) to evaluate how the app could effectively engage target communities (reach), resonate with their needs (relevance and receptiveness), and foster ongoing, authentic relationships between the communities and the agency (relationships). If the app is further developed and deployed by the agency, these are areas of desired further study.

Limitations

The project was limited in its capacity for genuine participation in the co-design process. We were limited in our access to the CaLD community groups, with adult education classes providing a pathway to our participants, as having existing partnerships with FRNSW. This meant that we did not always have the same participants in each workshop and were at the mercy of the education organisations’ timetables and availability. Additionally, the participant's English language proficiency varied from very basic to intermediate proficiency, as they were included in the study through their participation in adult English-language programmes targeted at newly arrived migrants. In most cases, the use of translators allowed for clear translation of instructions, however when undertaking 1–1 user testing, some barriers to clear communication were anticipated, which is why the activities were designed to have language translation when necessary and visual directives provided to support translation. We would recommend that alternative CaLD organisations or community groups be consulted to provide broader and more consistent access to participants.

Peters et al. (2018) note that ideally three steps would be followed in the co-design process to better empower traditionally disenfranchised groups such as those in the CaLD communities. These steps include user-led recruitment, user-leader training and user-led workshops. But as they too note, one core challenge in the user-led process is time. It takes much greater time (arguably double) to recruit, prepare and support user-leaders than it does sending professionals (Peters et al., 2018). Ideally, participants would have been consulted more often and earlier to fully optimise the potential for inclusive design.

Finally, the project objective was only to produce a prototype of the app, and so we were unable to develop and test a fully functioning app. As a result, it was not possible to measure the success of the app's ability to provide effective fire safety education for CaLD communities. To understand and determine the effectiveness of the app, we recommend the app be developed and pilot tested with several community groups over time.

Conclusion

This article contributes to existing scholarly discussion in digital interventions in disaster risk resilience using a co-design approach to produce a fire prevention and education smartphone app prototype. Through the development of the app prototype, informed by a CaLD-centric communication model, this study extends understanding of the important intersection of technology, communication and people. These findings contribute to an evidence-based approach to support culturally competent communication that can be used and adapted by emergency management agencies in NSW and other sectors and locations. The app prototype and the design process that informed its development can be adapted across the emergency and disaster response sector in NSW to increase responders’ level of competence in engaging effectively with ‘at risk’ CaLD communities. The project demonstrates the critical role of community members and how they engage with and use technology in meaningful and authentic ways to improve their community resilience to emergencies and disasters. While there were certainly constraints on the project, the adoption of a user-centred, co-design mindset ensures that at every step of the project journey, the research team was considering the end users and how they could best be involved. It is hoped that Fire and Rescue NSW will continue with the app prototype and further develop the app, so that meaningful measurement of the app's purpose to provide fire safety education can be undertaken.

This article profiled the development of the Ready, Set, Go! smartphone app prototype to demonstrate how disaster prevention and education communication could be improved through the adoption of a co-design methodology that engages community members throughout the design process. As a result, the communication design is co-created and more accessible to CaLD communities, keeping our communities safer. It argues that the affordances of app technology to personalise the user experience ensure that community members are receiving appropriate and targeted information. Additionally, it speaks to the importance of multi-modal and visual communication to ensure wider understanding of key messaging and eliminate the need for language translation. Ultimately, it is hoped that by making the app accessible to members of our community from diverse language and cultural backgrounds, those in our communities traditionally at risk of not receiving fire prevention education are given more opportunity to become more risk resilient. And beyond the scope of this project, it is hoped that this co-design method of smartphone app development could be utilised elsewhere in other settings for similar community benefit.

Footnotes

Acknowledgements

The authors would like to thank the many participants who contributed their time and knowledge to the project. Thank you also to Navitas College and TAFE NSW for giving us access to the adult educational classes to run our focus groups and workshops.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded through the NSW Reconstruction Authority Disaster Risk Reduction Fund (Discovery scheme).

ORCID iD

Natalie Krikowa

Notes

References

Alexander

Pescaroli

(2019) The role of translators and interpreters in cascading crises and disasters: towards a framework for confronting the challenges. Disaster Prevention and Management: An International Journal 29(2): 144–156.

Andrulis

Siddiqui

Purtle

(2011) Integrating racially and ethnically diverse communities into planning for disasters: the California experience. Disaster Medicine and Public Health Preparedness 5(3): 227–234.

Arlikatti

Taibah

Andrew

(2014) How do you warn them if they speak only Spanish? Challenges for organizations in communicating risk to colonias residents in Texas, USA. Disaster Prevention and Management 23(5): 533–550.

Arora

Selvaraj

(2022) Are flood-related warnings reaching linguistically diverse communities? SBS News. https://www.sbs.com.au/news/article/are-flood-related-warnings-reaching-linguistically-diverse-communities/tfg9hnm4a

Australian Bureau of Statistics (2022, January 12). Cultural diversity: Census. https://www.abs.gov.au/statistics/people/people-and-communities/cultural-diversity-census/latest-release

Australian Emergency Management Institute (2012) DisasterWatch phone app. Australian Journal of Emergency Management 27(1): 9. https://ajem.infoservices.com.au/items/AJEM-27-01-05

Australian Institute for Disaster Resilience (2021). Systemic Disaster Risk. https://knowledge.aidr.org.au/media/9228/handbook_systemic_disaster_risk_2022-03-17_v11.pdf

Bender

Martinez

Kennedy

(2016) Designing a culturally appropriate visually enhanced low-text mobile health app promoting physical activity for Latinos: a qualitative study. Journal of Transcultural Nursing 27(4): 420–428.

Brooks

(2023). Pandemic, fires and floods prompt new approaches to preparing Victoria's multicultural communities for emergencies. ABC News. https://www.abc.net.au/news/2023-01-14/multicultural-communities-prepare-for-emergencies-in-victoria/101817356

10.

Cadwell

(2020) Trust, distrust and translation in a disaster. Disaster Prevention and Management: An International Journal 29(2): 157–174.

11.

Campinha-Bacote

(2002) The process of cultural competence in the delivery of healthcare services: a model of care. Journal of Transcultural Nursing 13(3): 181–184.

12.

Cassidy

(2022). ‘We’d have lost lives’: absence of translators in Shepparton flood response criticised. The Guardian. https://www.theguardian.com/australia-news/2022/oct/18/wed-have-lost-lives-cultural-officer-decries-absence-of-translators-in-shepparton-flood-response

13.

Chen

(2019, November). A conceptual model of mobile disaster relief system combining social apps. In Proceedings of the 1st International Conference on Advanced Information Science and System, 1–5. https://doi.org/10.1145/3373477.3373481

14.

Clerveaux

Spence

Katada

(2010) Promoting disaster awareness in multicultural societies: the DAG approach. Disaster Prevention and Management 19(2): 199–218.

15.

Clyne

(2005) Australia’s Language Potential. Sydney, Australia: University of New South Wales Press.

16.

Deloitte Touche Tohmatsu (2021). Digital Consumer Trends 2021, Hybrid normal, Australian edition. https://content.deloitte.com.au/au-tmt-digitalconsumertrends2021-rego-inb-nat-pre-reg-24092021

17.

Diversity Council Australia (2023). Culturally and Racially Marginalised Women in Leadership: A framework for (Intersectional) organisational action. https://www.dca.org.au/wp-content/uploads/2023/09/carm_women_synopsis_online.pdf

18.

Duckworth

(2022) Just add trust: implementing diversity and inclusion in emergency management. The Australian Journal of Emergency Management 37(1): 18–20.

19.

Durall

Bauters

Hietala

, et al. (2020a) Co-creation and co-design in technology-enhanced learning: innovating science learning outside the classroom. Interaction Design and Architecture(s) Journal 42: 202–226. http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/42_10.pdf

20.

Durall

Virnes

Leinonen

, et al. (2020b) Ownership of learning in monitoring technology: design case of self-monitoring tech in independent study. Interaction Design and Architecture(s) Journal 45: 133–154. http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/45_6.pdf

21.

Edwards

(2020) Cultural competence and disaster management. In: Knox CC and Brie Haupt B (eds) Cultural Competency for Emergency and Crisis Management: Concepts, Theories and Case Studies. New York, NY: Routledge, 178–196.

22.

Ferdinand

(2021) Overcoming barriers to COVID-19 vaccination in African Americans: the need for cultural humility. American Journal of Public Health 111(4): 586–588.

23.

Fernando

Solomo

Lagman

(2019) iHanda: A mobile application for disaster preparedness. International Journal of Simulation: Systems, Science and Technology 20: 1–6.

24.

Freeman

Hancock

(2017) Energy and communication infrastructure for disaster resilience in rural and regional Australia. Regional Studies 51(6): 933–944.

25.

Garrett

(2003) The Elements of User Experience: User-Centered Design for the Web. New York, NY: New Riders.

26.

Goggin

(2011) Ubiquitous apps: politics of openness in global mobile cultures. Digital Creativity 22(3): 148–159.

27.

Goggin

(2020) COVID-19 apps in Singapore and Australia: reimagining healthy nations with digital technology. Media International Australia 177(1): 61–75.

28.

Gonzalez

Early

Gordon-Dseagu

, et al. (2021) Promoting culturally tailored mHealth: a scoping review of mobile health interventions in Latinx communities. Journal of Immigrant and Minority Health 23(5): 1065–1077.

29.

Grover

(2023) From monolingual mindset to plurilingual ethos: challenging perspectives on language(s). Journal of Multilingual and Multicultural Development 44(8): 751–764.

30.

Hajek

Slaughter

(2015) Challenging the Monolingual Mindset. Bristol, UK: Multilingual Matters.

31.

Hanson-Easey

Every

Hansen

, et al. (2018) Risk communication for new and emerging communities: the contingent role of social capital. International Journal of Disaster Risk Reduction 28: 620–628.

32.

Hodson

Svanda

Dadashi

(2023) Whom do we include and when? Participatory design with vulnerable groups. CoDesign 19(4): 269–286.

33.

Holmes

Maeda

(2018) Mismatch: How Inclusion Shapes Design. Cambridge, MA: MIT Press.

34.

Kelly

Hajistassi

Ramasundram

(2024) Migrant and refugee communities strengthening disaster resilience. Australian Journal of Emergency Management 30(2): 49–58.

35.

Kuniavsky

(2010) Smart things: Ubiquitous Computing User Experience Design. Burlington, MA: Elsevier Science & Technology.

36.

Kwon

Wardrip

Gomez

(2014) Co-design of interdisciplinary projects as a mechanism for school capacity growth. Improving Schools 17(1): 54–71.

37.

Madden

(2013, December 12). “American-Centric UI is Leveling Tech Culture – and Design Diversity”, Wired.com. https://www.wired.com/2013/12/how-american-centered-design-is-leveling-tech-culture-too-much/

38.

Mann

Daly

(2019) (Big) data and the north-in-south: Australia’s informational imperialism and digital colonialism. Television & New Media 20(4): 379–395.

39.

Marlowe

Neef

Tevaga

, et al. (2018) A new guiding framework for engaging diverse populations in disaster risk reduction: reach, relevance, receptiveness, and relationships. International Journal of Disaster Risk Science 9(4): 507–518.

40.

Marshall

Wilson

C-A

Dale

(2023) New pathways to crisis resilience: solutions for improved digital connectivity and capability in rural Australia. Media International Australia 189(1): 24–42.

41.

Matuk

Gerard

Lin-Breitbart

, et al. (2016) Gathering requirements for teacher tools: strategies for empowering teachers through co-design. Journal of Science Teacher Education 27(1): 79–110.

42.

Microsoft (2016). Inclusive design toolkit. https://download.microsoft.com/download/b/0/d/b0d4bf87-09ce-4417-8f28-d60703d672ed/inclusive_toolkit_manual_final.pdf

43.

Multicultural NSW (2022). Guide: what is CALD? https://multicultural.nsw.gov.au/wp-content/uploads/2022/11/P3-GUIDE-What-is-CALD.pdf

44.

NSW Government (2023). Hazards near me app, NSW Rural Fire Service. https://www.nsw.gov.au/emergency/hazards-near-me-app

45.

Ogie

Rho

Clarke

, et al. (2018) Disaster risk communication in culturally and linguistically diverse communities: the role of technology. Proceedings 2(19): 1256–1263.

46.

Owens

Smith

Beer

, et al. (2020) A qualitative cultural sensitivity assessment of the breathe easier mobile application for lung cancer survivors and their families. Oncology Nursing Forum 47(3): 331–341.

47.

Perea

Giner

(2017) UX Design for Mobile: Design Apps that Deliver Impressive Mobile Experiences. Birmingham, UK: Packt Publishing.

48.

Peters

Hansen

McMullen

, et al. (2018). “Participation is note enough”: towards Indigenous-led co-design. OzCHI ‘18: Proceedings of the 30th Australian Conference on Computer-Human Interaction. December 2018, 97–101. https://doi.org/10.1145/3292147.3292204

49.

Sanders

EBN

(2002) From user-centered to participatory design approaches. In: Frascara J (ed) Design and the Social Sciences. London: CRC Press, 18–25.

50.

Sanders

EBN

Stappers

(2008) Co-creation and the new landscapes of design. CoDesign 4(1): 5–18.

51.

Seeger

Islam

Seeger

. (2021). Emergency preparedness, response, and strategic communication for natural disasters. In: Botan CH (ed) The Handbook of Strategic Communication. NJ, USA: John Wiley & Sons Inc., 208–221.

52.

Sherwin

. (2018). Card sorting: uncover users’ mental models for better information architecture. Nielsen Normal Group. https://www.nngroup.com/articles/card-sorting-definition/

53.

Stephens

Tich

Quist

L-M

(2021) The official emergency responders had infrastructure: we had iPhones. In: Lieberman C and Wright K (eds) Casing Mediated Communication. Dubuque, IA: Kendall Hunt Publishing, 1–14.

54.

Subramaniam

Villeneuve

(2020) Advancing emergency preparedness for people with disabilities and chronic health conditions in the community: a scoping review. Disability and Rehabilitation 42(22): 3256–3264.

55.

Sukhwani

Shaw

(2020) Operationalizing crowdsourcing through mobile applications for disaster management in India. Progress in Disaster Science 5: 1–9.

56.

Tan

Prasanna

Stock

, et al. (2017) Mobile applications in crisis informatics literature: a systematic review. International Journal of Disaster Risk Reduction 24: 297–311.

57.

Tan

Prasanna

Stock

, et al. (2020) Modified usability framework for disaster apps: a qualitative thematic analysis of user reviews. International Journal of Disaster Risk Science 11: 615–629.

58.

Thomas

McCosker

Parkinson

, et al. (2023) Measuring Australia’s digital divide: Australian digital inclusion index: 2023. Melbourne: ARC Centre of Excellence for Automated Decision-Making and Society, RMIT University, Swinburn University of Technology, and Telstra.

59.

Tissenbaum

Lui

Slotta

(2012) Co-designing collaborative smart classroom curriculum for secondary school science. Journal of Universal Computer Science 18(3): 327–352.

60.

Transcultural Mental Health Centre (2022). NSW demographic fact sheet 2022. https://www.dhi.health.nsw.gov.au/ArticleDocuments/197/NSWDemographicFactSheet%202022.pdf.aspx

61.

United Nations Office for Disaster Risk Reduction (UNDRR) (2024). Terminology: resilience. https://www.undrr.org/terminology/resilience

62.

Vargo

Zhu

Benwell

, et al. (2021) Digital technology use during COVID-19 pandemic: a rapid review. Human Behavior and Emerging Technologies 3(1): 13–24.

63.

Victorson

Banas

Smith

, et al. (2014) Esalud: designing and implementing culturally competent eHealth research with Latino patient populations. American Journal of Public Health 104(12): 2259–2265.

64.

Williams

Haines

Williams

, et al. (2022) Age differences in preferred methods of obtaining and understanding health related information during the COVID-19 pandemic in Australia. Frontiers in Public Health 10: 912188.

65.

Wilson

(2020). The politics of privacy protection: an analysis of resistance to metadata retention and encryption access laws. [PhD Thesis, Queensland University of Technology, Australia.] https://eprints.qut.edu.au/199696/1/Michael_Wilson_Thesis.pdf

66.

Wolf

(2021, April 15) Robodebt was an Algorithmic Weapon of Calculated Political Cruelty. Canberra Times. https://www.canberratimes.com.au/story/6775350/robodebt-was-an-algorithmic-weapon-of-calculated-political-cruelty/ (accessed 13 October 2023).

67.

Yang

Heemsbergen

Fordyce

(2021) Comparative analysis of China’s health code, Australia’s COVID Safe and New Zealand’s COVID tracer surveillance apps: a new corona of public health governmentality? Media International Australia 171(1): 182–197.

68.

Young

Jones

(2019) Effective diversity in emergency management organisations: the long road. Australian Journal of Emergency Management 34(2): 38–45. https://search.informit.org/doi/10.3316/informit.379652494202411