Use of Remote Focus Groups to Improve Inclusive Design Practice

Abstract

This study examines the use of remote focus groups to obtain wheeled mobility device user’s ideas regarding Shared Autonomous Vehicles (SAVs). Given the global increase in the aging population and disabled people, inclusive design is essential for accessible transportation. Using remote focus groups, this research collected statements from nine wheeled mobility device users about their design preferences, needs, and challenges with SAVs. Participants attended four virtual sessions featured with presentations and videos of SAVs. Discussions were video recorded in Zoom, with transcripts reviewed for accuracy. Data analysis included cleaning, coding, and assessing interrater reliability using the ICC. The findings not reported here highlighted the diverse needs and preferences of participants, emphasizing the importance of collecting information from this group early in the design process. This study shows well-planned remote focus groups can be an effective tool to gather human factors system design information from user populations that face transportation barriers.

Introduction

Importance of Inclusive Design: The functional limitations of the growing populations of older adults and people with disabilities is a critical global concern, predicted to become one of the foremost social shifts of the 21st century (United Nations, 2015). The UN anticipates a 54% increase in individuals over 60 years of age by 2030. Concurrently, the WHO highlights that over one billion people live with some form of disability (World Health Organization, 2011). Designers play a crucial role in addressing the design challenges experienced by those with limitations by creating usable and accessible products for a diverse and expanding market. To effectively meet the needs of user populations, particularly people with disabilities, it is crucial for designers to have the tools and knowledge to conduct thorough user needs testing. This approach ensures that design solutions are informed by accurate and comprehensive user insights, leading to more inclusive and accessible products.

Use of Focus Groups to Inform Design: Focus groups are a well-established method in market research when designing new products (Bruseberg & McDonagh-Philp, 2002). They are also commonly used in human factors research and usability evaluation (Jordan, 1998), whereby individuals are brought together at a physical location to discuss a specific topic or issue. The discussion is guided by a moderator who leads the group through various subjects and activities. One of the key features of focus groups is the interaction among participants, which allows for the sharing and comparison of ideas, thereby stimulating and encouraging each other. This method is ideal for exploratory research, as it allows for the examination of open-ended questions. The data collected is qualitative, encompassing experiences, opinions, ideas, and motivations rather than quantitative data like statistics (Morgan et al., 1998).

One of the challenges of using focus groups in inclusive design is recruiting individuals with disabilities to participate because they are more likely to face difficulties in getting to the focus group site. These individuals confront persistent transportation obstacles as traditional modes, like private vehicles and public transit, lack sufficient accessibility, creating significant challenges in transportation or reducing mobility options altogether (Bezyak et al., 2017).

This exploratory study used carefully designed remote focus groups involving wheeled mobility device users to collect new information about shared autonomous transportation design needs and preferences along the initiation of the journey to final disembarkation. The objective was to pilot test an inclusive design approach to obtain information from users often excluded in the design process due to transportation challenges.

Methods

For this study we developed and tested a focus group methodology that concentrated on a procedure for obtaining new information from wheeled mobility device users on their design needs and preferences about shared automated vehicles (SAVs) transportation systems. This study was approved by our university’s institutional review board (IRB).

Participant Selection: The organization and execution of focus groups can vary depending on the specific research and design objectives. However, there are fundamental principles and both methodological and theoretical considerations that must be considered. The selection of the appropriate participants is crucial for the success of the study. Research suggests that discussions on sensitive or controversial topics can be more effective when participants share key characteristics (Hoppe et al., 1995). For this study, nine users of wheeled mobility devices were each asked to participate in one of two different focus group sessions: one discussing 1 to 7 passenger vehicle transportation systems and a second discussing 8 to 12 passenger vehicle transportation systems.

Having too few participants per group (fewer than four) can limit the depth and breadth of the discussion, while having too many (more than eight) can restrict the time available for each participant to contribute meaningfully. In this study, nine wheeled mobility devise users participated. To obtain the most accurate results, we divided them into two groups: one group with four participants and another with five. Additionally, since we needed to gather feedback on two different types of vehicles, we held separate sessions for the mid-size and small-size vehicles to ensure the opinions did not get mixed. As a result, each participant attended two sessions. Given that the participants were divided into two groups, a total of four sessions were conducted. To gain insights from diverse social and cultural perspectives among wheeled mobility device users, we included both women and men of different ages, life stages, and ethnic backgrounds.

Sensitivity and Controversial Topics: It is crucial to foster an environment where participants feel at ease, particularly in social interactions like focus groups (Krueger, 2014). Given the personal nature of mobility challenges, it was essential to create an environment conducive to open and honest discussions. We required all individuals to be video activated throughout the remote focus groups so that they could personally connect as they do in on site focus group sessions. Participants had the option to identify themselves using their real first name, a nickname, or a fictitious name for anonymity. Names were displayed to facilitate discussions throughout the focus group.

Additionally, participants were allowed to provide insights and preferences of not only themselves but their understanding of others’ preferences and needs based on their experiences. They could even present hypothetical examples of why a system’s component should be designed in a specific way. Fictional examples can stimulate discussion without requiring participants to disclose their personal challenges when participants do not wish to disclose them. This strategy, employed by Goss and Leinbach (1996), allows participants to discuss hypothetical scenarios, thereby reducing the pressure to share personal information. Ensuring confidentiality was a crucial aspect of the focus groups (Goss & Leinbach,1996). Participants were reminded to share only what they felt comfortable disclosing publicly, acknowledging that absolute confidentiality could not be guaranteed.

Questions and Topics: In preparing for the focus group sessions’ questions, the emphasis was on employing conversational style questions to create a comfortable and inclusive atmosphere. The questions were carefully crafted to be direct, clear, and straightforward, ensuring they were easily understandable and articulable.

The aim was to keep the questions concise, devoid of unnecessary complexity or jargon, and to maintain clarity and engagement throughout the discussions. This approach aimed to enable participants to express their statements effectively, regardless of their familiarity with technical terms or industry-specific language (Krueger, 2014).

Conducting Focus Groups: In academic research, the researcher, who is well-versed in the research aims and the purpose of the focus groups, is often best suited to the role of facilitator or moderator (Morgan, 1993; Stewart & Shamdasani, 1990). In this study, the moderator was experienced in the human factors’ considerations of SAV and public transportation design.

The focus groups began with an introductory presentation and video demonstrations of small (1–7 passengers) and mid-size (8–12 passengers) SAVs to acquaint participants with the vehicles’ operations. The videos developed by the research team including were 2 to 3 min in length and closed captioned. Questions were posed to participants about their preferences across various elements of the journey, including trip planning, vehicle identification, boarding and exiting, passenger riding location, trip safety measures, and handling emergencies.

Zoom was used for recording focus groups, capturing both audio and video. Transcripts were automatically generated through Zoom, with subsequent manual review and editing by two researchers to ensure accuracy and clarity. Unclear statements were marked and revised collaboratively. Each session lasted approximately 90 min.

Results

After the data were cleaned, two independent coders collaborated to establish general themes for the responses to the focus group questions. They then jointly coded 20% of the transcripts to test the thematic coding. Following this, each coder independently coded the remaining focus group transcripts using seven coding categories: need, preference, challenge, general experience, clarification, question, and others. During the analysis, the categories of “clarification,” “question,” and “other” were merged, as they primarily served to distinguish relevant statements from those unrelated to the main topics. Definitions and examples for each category are provided in Table 1.

Table 1.

Definition and Examples of Coding Categories for Focus Group Analysis.

Category	Definition	Example from focus group transcripts
Need	A physical or service-oriented design element or characteristic that is required for the individual to complete the journey.	. . . they [autonomous vehicles] have to be able to park, like, close enough to the curb . . . . So, it has to be able, orientation wise, to know, like where it’s parking or just sort of, how can I say seek out a curb cut-out or something where it’s going to be feasible for me to get on the vehicle.
Preference	A physical or service-oriented design element or characteristic that is favored or ideal for the individual to complete the journey but is not required.	I definitely would like the flexibility of being able to adjust the trip, whether it’s the pick-up times or just a location or even add a location, if I have to make a stop.
Challenge	A specific barrier or difficulty encountered by an individual when attempting to complete a task within the journey.	You have to give them a day’s notice as far as where you want to go and schedule the ride. Now, as far as changing plans is concerned, you cannot do that with paratransit because they will not allow you to do that. You have to stick with the plan that you originally had, and that’s the only place they’re going to take you. They will not change the point of address.
General experience	An example of what the individual has done in the past related to the completion of journey, including descriptions of challenges the individual experiences when traveling. “need” and an “example.”	There were times when I booked a trip with paratransit and didn’t realize that the time I requested they were actually going to pick me up 15 min early.
Other	Statements/Questions that are not related to needs, preferences or travel experiences within a journey that requires the use of a self-driving vehicle.	How [is the vehicle going] to be able to sense where the ramp is going to be deployed to make sure deployed in a safe area?

After independently coding the transcripts, the researchers evaluated inter-rater reliability to assess the consistency between coders and the accuracy of categorizations. While the percent agreement between raters ( $\frac{numberofratingsinagreement}{totalnumberofratings}$ ) is a basic method for calculating inter-rater reliability, it has limitations, such as not accounting for chance agreements, leading to an overestimation of reliability. To address this issue, an intraclass correlation (ICC; Koo & Li, 2016) was utilized for a more accurate evaluation.

Following coding by independent researchers, discrepancies in the codes, indicating statements on which they disagreed, were reviewed by the two researchers to reach a consensus. In cases where consensus was not reached, a third party reviewed the statement and provided a deciding vote. This iterative process continued until the code for each statement was determined.

The intraclass correlation coefficient (ICC) was utilized to gain inter-rater reliability. The ICC was .63 suggesting that while acceptable there was some degree of thematic disagreement. As a result, the themes of challenge, general experience and other were combined because they were not considered central to the study. Furthermore, other disagreements were resolved through discussions between the two researchers who came to consensus on each.

Discussion

Focus Group Methodology: In this study, the use of remote focus groups provided a unique opportunity to gather information from participants with mobility-related disabilities in an inclusive and accessible manner. The methodology employed offered several key advantages and revealed important considerations in research for inclusive design.

Anonymity and Participant Comfort: One of the critical elements in our focus group methodology was offering participants the option to use their real name, a nickname, or a fictitious name. This flexibility promoted a sense of comfort and security, encouraging open and honest communication (Krueger, 2014). Our observations indicated that participants were more willing to share detailed and personal experiences when they felt their anonymity was protected.

Technological Access and Participation: Ensuring that all participants had access to a computer or tablet was a prerequisite for participation in the focus groups. This requirement was essential to facilitate smooth communication and interaction during the sessions. In this study, all participants had access to the technologies, but sometimes lack of robust connections happened through the sessions for some participants. Therefore, it should be considered as a potential barrier for individuals who might lack enough access to these technologies. Future studies should consider providing necessary technological resources or support to ensure broader participation, especially for older adults.

Introductory Presentations and Video Demonstrations: The introductory presentation and video demonstrations of small and mid-size SAVs were instrumental in standardizing participants’ understanding of SAV operations. These preparatory steps ensured that participants were adequately informed, which likely contributed to the depth and relevance of the feedback provided.

Implications in Human Factor Research: The integration of remote focus group methodologies in human factors research provides a powerful tool for exploring user design needs and preferences in depth, particularly when inclusive design is a goal. This approach fosters interactive discussions among participants who might not otherwise participate in on site focus groups, allowing researchers to uncover nuanced insights crucial for inclusive design. By understanding diverse user perspectives through carefully designed remote focus groups, researchers and designers can identify and prioritize inclusive design features that enhance accessibility, usability, and overall user satisfaction in products and systems.

Recording and Transcription: Using Zoom for recording both audio and video allowed for comprehensive data capture. The automatic transcription feature, followed by manual review and editing, ensured accuracy and clarity in the data. The collaborative effort between two researchers in this study helped to resolve unclear statements, which was crucial in maintaining the quality of the transcripts. This methodology highlights the importance of meticulous data handling processes in qualitative research.

Conclusion

The focus group methodology used in this study revealed significant findings about the preferences, needs, and challenges experienced by individuals with mobility-related disabilities regarding SAVs. Key methodological decisions, such as maintaining participant anonymity, ensuring access to necessary technology, and providing comprehensive introductory materials, were crucial in promoting effective participation and generating valuable data. The rigorous process of data handling, encompassing recording, transcription, and coding, was instrumental in ensuring the reliability and accuracy of our findings. The structured and supportive approach helped participants feel comfortable and encouraged them to share their perspectives. In conclusion, this study illustrates that remote focus groups, carefully designed and executed, can be a powerful tool for inclusive and accessible qualitative research. Future research should aim to further refine these methodologies to enhance participation and data quality.