Exploring Learning Paths: Understanding the Learning Strategies of Artificial Intelligence System Users and Their Involvement in Social Forums

Abstract

In the last decade, there has been a significant increase in the use of commercial semi-autonomous vehicles by consumers. This has led to a surge in concerns among users about the limitations of these systems, especially when it comes to safety. In order to address these concerns, users often seek out diverse educational resources to comprehend these constraints, explore alternatives, and determine whether their experiences are representative. This paper examines the viewpoints of users who are using a new AI system that has received minimal training from the vendor. We conducted interviews of the users of the AI to examine the many sources from which users obtain knowledge and developed a learning technology-based framework that utilizes technology to disseminate knowledge among users of the AI.

Introduction

Explainable Artificial Intelligence (XAI) is a crucial component in the field of autonomous vehicles, guaranteeing transparency, and interpretability in decision-making mechanisms. Given the increasing prevalence of intelligent systems in autonomous vehicles, it is essential to identify user perceptions and interactions with these systems to ensure smooth integration and widespread acceptance.

The need for explainability and user understanding is unlikely to go away as self-driving technology improves—especially as the developers target non-drivers who may not understand complicated driving situations. So, an analysis of how inexperienced users negotiate abnormal driving situations while using the AI system can provide vital information for potential AI users, thereby reducing the difficulty of the learning process to a great extent. This benefits manufacturers as well, because improved comprehension can improve the user experience and accelerate the acceptance of AI. Strand et al. (2018) highlighted the crucial significance of the initial engagement with automated vehicle systems. An initial unpleasant encounter can dissuade consumers, impeding potential safety advantages and influencing their opinions. In order to tackle this issue, it is crucial to provide customers with an appropriate initial introduction as well as contextual instruction.

Although one might expect vendors to provide instruction and explanation for AI systems, self-driving manufacturers have been slow to do so, partly because their marketing includes the dogma that the system is so good and so easy to use it doesn’t need user guides. Consequently, users have worked around this lack of support by turning to social media and on-line forums. AI systems’ social forums have evolved into active platforms where users contribute an array of knowledge, producing an enthusiastic community willing to interact with one another when vendor training is limited. In this cooperative setting, individuals exchange their experiences and offer valuable recommendations to tackle the limits of AI (Linja et al., 2022). According to Ibne Mamun et al. (2022), those who use social platforms for collaborative explanations demonstrate more understanding and satisfaction with the AI system, suggesting one way in which user-generated explanations may improve adoption and safety of self-driving vehicles.

Interview Study

To identify the process by which users of Tesla Full Self-Driving (FSD)acquire knowledge about the technology, we conducted an interview study with users of FSD. Interviews have demonstrated efficacy in identifying user expectations and mental models, including in the context of automation. Preusse and Rogers (2016) demonstrated the interview’s efficacy in error management for automated systems, successfully detecting and understanding flaws.

Our interview investigated the sources of learning, such as acquaintances and social media, and examined whether there were other advantages derived from these social platforms. We used a semi-structured technique to investigate a variety of aspects of FSD users understanding and experience (see Mamun, 2023). In this report, we will focus on the responses to two probes that clarify FSD user training experiences:

• Probe 1: How do Tesla Full Self-Driving (FSD) users acquire knowledge about the technology in cases where the availability of official vendor training is restricted?

• Probe 2: Under what circumstances do Tesla Full Self-Driving (FSD) consumers turn to public forums for assistance?

Participants

For this study, a total of 10 Tesla FSD consumers were selected using a combination of word-of-mouth (8) and social forum (2) advertising. The participants had an average of 2.5 years of experience using the technology and had prior expertise with semi-autonomous technologies such as adaptive cruise control and lane assistance. The participants were required to engage in a minimum of 5 hr of driving per week utilizing FSD technology.

Interview Procedure

The participants were queried regarding their experience with semi-autonomous technology, also with FSD, and the duration of their engagement with it. Additionally, they were asked about their training methods provided by the technology provider and their self-education using popular forums such as Reddit, Facebook Tesla groups, Twitter, YouTube, etc. Afterward, the participants were given 10 unusual driving scenarios that they may have encountered while using FSD (refer to Table 1, the compilation of this list was based on data acquired from social forums through a search aimed at identifying atypical driving FSD). Next, they were asked to communicate their personal encounters and perceptions pertaining to these particular driving situations. In addition, the participants were asked to provide further details about any two of the situations we had identified earlier. They were asked to indicate if they were already aware of these instances and if they sought assistance or advice from social forums to address them.

Table 1.

Counts for the 10 Anomalous Driving Situations.

Incidents	Count (range 1–8)
Occurred before a rail crossing	5
Occurred when saw a yellow sign/blinking yellow	5
Occurred when saw no center lane marking in the neighborhood	6
Occurred when saw a STOP sign	6
Did a rolling stop	2
Occurred when saw stopped cars in the lane	5
Occurred saw a road work ahead sign/crew	3
Occurred when saw a bump	6
Occurred when saw a fence/neighborhood gate	0
The vehicle suddenly pulling over to the right	4

The purpose of this procedure was to determine the alternative sources that new AI drivers relied on when the initial training source provided by the vendor is restricted. In fact, all participants reported receiving minimal assistance from the vendor, limited to a brief walkthrough or release notes for the new version of FSD.

Results

We categorized reported incidents about how users learned about FSD into several categories describing the nature of the interactivity in an information source. The categorization was established through bottom-up sorting and consensus coding between two researchers involved in the study. The training for the new AI system is divided into three distinct groups based on the level of collaboration involved. The groups are as follows:

Informal Inter-Personal Learning

Several participants reported learning from others who were familiar with the technology, including close friends or colleagues who were already users as the main sources of training and guidance. Some reports include:

“I always like to try new things, but in this case, nobody explained how this thing works, I wasn’t comfortable sort of giving away control as I used to be more novice, and then I think I went to a colleague or friend who had a Model X, I went and drove with him few times and saw how he was using it, and then I started to become more comfortable.” (p1)

This training covered a range of topics, including introducing the system, providing guidance on whether to activate or deactivate it depending on specific events. This training assisted novice users in grasping tips and techniques for specific situations where they typically used to take control, but now they perform actions that prevent seizing control. For instance, they might learn to gently push the wheel during certain events to avoid triggering a takeover, instead of immediately assuming control as they used to.

“The main thing I sort of learned from my colleague, see this whole thing about keeping your hands on the steering wheel. And making sure you’re applying. A slight pressure on the in the opposite direction. That took me a while to figure out because I used to turn it off.” (p1)

This kind of learning was also done in group sessions. These sessions allow a close-knitted community of new users to interact, share knowledge, and improve their collective comprehension of the functionality and best practices of the technology.

“There was three of us that got the car pretty close together. . . .and three of us bouncing ideas of each other, I only got the FSD, rest of them had autopilot. So, we were talking about autopilot” (p8)

“Sometimes we have discussions (regarding issues about FSD) in our groups as well” (p9)

This training methodology also has the potential to aid users in efficiently managing errors by providing drivers with access to relevant data from various diverse sources.

Self-training is another method new users of the technology apply to become proficient. Curiosity is a common driving force in this process. The next quotation comes from a driver who wants to test the vehicle out of curiosity.

“Beyond the intersection was basically like a core points, you know, concrete divider that so the car found itself heading towards because it deviated from the lane. . . . I was out. At about 2:00 in the morning to make sure that (no traffic was there)” (p2)

Drivers reported actively investigating and evaluating the numerous scenarios in which the AI succeeds or fails as they travel. They were able to gradually identify the AI’s advantages and disadvantages thanks to this hands-on approach, empowering them to decide when to exercise control and when to rely on the technology.

Collaborative Online Training

Collaborative online training comes from several different online social forums, including reddit, Facebook, twitter, and Tesla user forums. Drivers frequently reported reading about an incident they faced, or posting notes if they did not find it in their desired forum.

“Even though I have a microscopic Twitter following, I did post a couple of my experiences like brief notes about those” (p2)

Within these social forums, individuals expressed their perspectives, leaving each driver to make judgments on the correctness of a perspective based on their experience. Social forums also helped drivers to gathers various individuals’ accounts of how they dealt with different iterations of an adverse driving condition.

“I want to see how to handle this (a specific problem). And so, I go (to the forum) and see experiences on how it was handled (different versions) of this problem. . . What it does it gives me a better understanding of the things the car can do and the things the things the car can’t do” (p7)

This methodology has the potential to induce a transformation in the initial perception or comprehension of the system. As individuals accumulate further exposure and familiarity with the system, they may undergo a process of revising and enhancing their initial notions and beliefs pertaining to its functioning and potential accomplishments.

Non-Collaborative Online Training

Primarily, this training occurred via diverse video platforms such as YouTube, although interaction among drivers is constrained on these platforms. However, these video platforms serve as one of the primary online training resources. Here, drivers reported encountering a variety of online profiles that furnished feature lists and offered insights into various challenges.

“Used YouTube mostly. To mostly figure out things that I can’t figure out like how to open the glove compartment or how the phantom breaking works or how the most efficient way to drive” (p9)

While this medium is valuable, it occasionally fell short in addressing tailored issues or getting a second opinion on a specific issue regarding the AI. To address this gap, drivers turned to social forums for more customized problems (R2).

“Yeah, certainly. I mean, YouTube certainly has a lot of information about the functionality of FSD. There are some relatively long-time testers, some of whom have, you know, relatively consistent over the weekend. I guess look to it for a reasonably coherent snapshot of the current state of the system. . . And the ones that I would primarily participate in are Tesla Motors Club forum. It’s a good place to go for sort of a counter reaction.” (p2)

Nonetheless, participants reported that these video platforms excelled in presenting an array of typical problems and corresponding strategies for circumventing them.

Another strategy employed was search via general-purpose search engines to learn generally or for specific situations.

“I’ve done a lot of Google searches and stuff since then about it (FSD). . . . Search most of the time, my search was just Tesla FSD latest version or something like that, I was not looking something specific.” (p3)

Anxiety prompts individuals to seek information or concentrate on specific situations, such as anticipating how the vehicle will respond to an event.

“(Google) searches on how do you turn it on or turn it off and what does it do if I break or don’t break, like just looking at specific situations where I worry about something, then I go do a search and see” (p1)

As drivers begin their journey as beginners, they usually familiarized themselves with the system using the aforementioned methods. These strategies aimed to provide them with knowledge regarding the AI’s characteristics, as well as its advantages and disadvantages. However, after they progressed and attained a core level of technical competency, they transitioned to a primarily case-based approach. To enhance their understanding, they begin focusing on specific real-life conditions and scenarios encountered by others. The modification in the training procedure apparently signified their increasing expertise and understanding of the intricacies of the technology.

Discussion

Tesla FSD users rely not only on social forums but also on several other sources for training: acquaintances, video platforms, self-experimentation, and web searches. When we consider the social forum as an important platform for exchanging knowledge and acknowledge that these other sources are the main ways to get insights about AI, it becomes evident that these sources can actually have a vital role in disseminating knowledge within the social forum. The diagram presented in Figure 1 depicts a social interaction that incorporates several sources of knowledge inside a social forum, inspired by the e-Learning framework which combines different learning methods and uses computer technology to make the learning process easier (Tavangarian, 2004). Our descriptive model (called the Socio-technical learning model) incorporates several general sources for learning, including self-learning, social learning, learning via forums, and other technology-based learning. Gregorc and Ward (1977) categorized individual learners into four categories; Abstract Sequential—Individuals with an abstract sequential learning preference exhibit a high level of proficiency in decoding many forms of symbols, including written, verbal, and visual representations. Individuals who exhibit this preference demonstrate a vast cognitive repository of conceptual pictures, which they utilize to evaluate and synchronize information they come across via reading, hearing, or observing visual and graphical representations, Abstract Random—notable for their strong emphasis on the study of human or system behavior and their outstanding ability to perceive and grasp tiny cues or stimuli. Individuals in this group exhibit an elevated level of perceptiveness toward the subtleties present in their surroundings and the dominant emotional atmosphere, Concrete Random—individual with this learning technique possess this tendency demonstrate a rapid comprehension of fundamental concepts and exhibit exceptional aptitude in effectively navigating unstructured problem-solving scenarios, hence facilitating creative breakthroughs, and Concrete Sequential—characterized by a proficient ability to acquire knowledge through direct engagement in practical activities and firsthand experiences.

Figure 1.

The socio-technological learning model inside a social forum—the model shows a bi-directional communication between different categories of individual learners.

The Socio-Technological Learning Model, which incorporates a Social Forum, caters to diverse learning preferences as outlined in the aforementioned categories. Self-directed learning promotes active participation, akin to the act of operating a vehicle, wherein learners have the opportunity to acquire cues and insights through their activities. This process of self-learning can be a topic of discussion within a peer group, where peers can demonstrate exceptional skills in effectively addressing unstructured problem-solving scenarios based on fundamental concepts. While there is currently a lack of scientific evidence definitively confirming the efficacy of particular learning styles (Bishka, 2010; Deng et al., 2022; Glenn, 2009; Kirschner, 2017; Nancekivell et al., 2021; Rohrer & Pashler, 2012), the analysis shows a pattern in individuals’ learning preferences, which can be important for vendors trying to reach the largest group of customers. Future research might provide valuable insights into the factors influencing their propensity toward one strategy over another to learn about an AI system in autonomous vehicle.

Conclusion

The analysis of Tesla FSD driver self-reports reveals a comprehensive method for learning about the system. Although social forums continue to be a prominent medium for sharing knowledge, our research indicates that users also depend on acquaintances, online videos, and web searches, and self-experimentation to acquire better understanding of AI behavior. Recognizing these alternative sources as the main pathways for acquiring information makes it clear that they have a vital function in spreading knowledge within social forums.