Sage Journals: Discover world-class research

Abstract

In recent years, assisted driving and self-driving have captured the imagination of manufacturers, designers, technology providers, and the general public with the expectation of a sustainable, safer, and intelligent mobility in the near future. Self-driving or assisted driving vehicles are complex systems that integrate environmental perception, intelligent planning and decision-making, tracking, and control. With the increasing intelligence of vehicles, personalized design is an inevitable trend. A design that is in line with the driver’s personality can bring a better driving experience to the driver. Thus, classifying driving types while driving in a self-driving environment may play an important role in the construction of trajectory planning algorithms. This paper uses the motion-in-mind model from game refinement theory to model driver behavior. Further, a classification of the model parameters into three categories helped in distinguishing cautious, aggressive, and average drivers. The results showed that the self-driving environment can be successfully modeled as a game and adaptation to match the riders’ driving skills may improve satisfaction.

Keywords

game refinement theory information technology vehicles autumated reasoning information science artificicial intelligence and advanced computing applications

In the literature there exist many methods and algorithms to classify or identify different driving styles by analyzing driving behavior data. For example, Wang et al. ( 3 ) identified differences in driving styles considering a set of statistics namely, the mean, standard deviation, maximum, and minimum values. Qindt et al. ( 4 ) proposed to estimate driving style recognition coefficients based on impact (rate of change of acceleration) and entropy theory. Ouali et al. ( 5 ) proposed an algorithm for differentiating driving styles (the driving style indicator—DSI) considering the correlation between driving style scores and fuel consumption. Very few studies considered the DSI from the perspective of drivers’ psychology. del Campo et al. ( 6 ) proposed a DSI method focusing on the global comfort of the driver and the passengers. The criterion of global comfort was based on the amplitude and frequency of acceleration but no analysis was carried out considering the drivers’ perception of comfort. The mood of a driver can greatly affect her/his driving behavior. Therefore, further studies are necessary to fully comprehend this factor in DSIs. Further, none of these models or algorithms considered identification of driving style from the perspective of drivers’ psychology, especially treating the act of driving as a game. In this paper, we propose to model the act of driving considering the concepts of game refinement (GR) theory ( 8 ). This enables clustering the measures of drivers’ psychological states during the driving process to more accurately classify their driving style.

Several researchers have studied strategic decision-making within game theory, focusing on mathematical models of conflict and cooperation between intelligent rational game players. Game theory is used in economics, political science, psychology, logic, and biology because of the assumption that two-person zero-sum games have a mixed-strategy equilibrium ( 7 ). A GR theory-based logical paradigm was applied to several board games, including chess variants. A generic GR paradigm based on game progression and game information progression was recently suggested that connected chess and soccer. Iida et al. ( 8 ) suggested a GR metric based on game result uncertainty. GR theory is a game theory focusing on the attractiveness and sophistication of games. The foundation of this direction was made by Iida et al. ( 8 ), in which a measure of GR was proposed based on the concept of information about game outcome uncertainty. This theory provides a perspective to analyze the psychology of a player or participant with respect to the game and determine the degree of their entertainment, rather than focusing on winning or losing the match. A game is enjoyable when its challenge matches the preferences and skills of a player. In GR theory, the uncertainty of the game outcome is described with classical physics (theory of kinematics)-based models. The GR measure reflects the attractiveness of a game from the viewpoint of the players. This theory has been applied to almost all board games ( 9 , 10 ). Later, it has been used not only for board games ( 11 ) but also video games, educational games ( 12 ), business ( 13 ), and riding comfort ( 14 ) as well. In this study, GR theory has been adopted assuming that the action of driving or the experience of a driver when assisted through a smart system is the most comfortable when it is the most enjoyable.

This paper uses the motion-in-mind model ( 15 ) from GR theory to design the act of handling a steering wheel as a game. The motion-in-mind model ( 15 ) defines a player’s feelings during a game by finding the relationship between game-play indicators and the movement of information in the player’s mind indicators. Each of these indicators represents a game-play feeling of the player, such as sense of control, motivation, curiosity, and so forth. A balanced perception of the player’s experience in each dimension of the game can bring satisfaction to the player. As an extension of the motion-in-mind model, a player satisfaction model ( 16 ), which focuses on the reward system using a reward ratio ( 17 ) has been developed, which provided a new method to calculate the motion-in-mind values. In this study, by linking the drivers’ level of satisfaction or enjoyment from handling the steering and subsequent trajectory navigation of the vehicle has been modeled in a motion-in-mind framework. Further, the driving styles are classified applying traditional classification methods, namely k-means clustering, to the designed satisfaction indicators. The good agreement of the results with subjective classification information illustrated the validity of the proposed methodology. Further, this study provides a new way of imagining the act of driving and personalizing assisted driving.

To implement the concept personalization of autonomous driving, the study used end-to-end learning to model autonomous driving. At present, the majority of the industrial design for autonomous driving, assisted driving, or both, is composed of multiple modules, namely detection (of traffic signs, lights, cars, pedestrians), segmentation (of lanes, facades), motion estimation, tracking, and so forth. The results from these components are then typically combined in a predictive planning module, which feeds the control module. This requires detailed scene understanding with the objective of controlling the vehicle for safe and efficient navigation. Furthermore, auxiliary loss functions are required to train each module independently. This often ignores the traditional traffic modeling-based objectives of the driving task, which include optimized travel time, safety, and comfort. As an alternative, many studies are now considering autonomous driving as an end-to-end learning problem. In this approach, the tasks of perception, planning, and control are combined, and a single model is trained end-to-end using a deep neural network. Most end-to-end autonomous driving systems map from sensory inputs, such as front-facing camera images, directly to driving actions such as steering angle. These methods consider a model input and output without having to consider many of the intermediate steps, which is usually more efficient.

End-to-end driving methods are typically trained from expert demonstrations to learn a driving strategy that imitates the behavior of a human expert or using reinforcement learning (RL) to explore the environment by trial and error through simulations. Well-known methods in this area are behavior cloning approaches where the system learns to map sensor observations, such as RGB images, to desired driving behavior by learning to clone the behavior of an expert ( 18 , 19 ) and approaches based on RL where the system learns to drive by training an agent that tries to maximize a user-defined reward that the agent receives while interacting with the environment ( 20 ). This paper is utilizes an end-to-end autonomous driving deep learning algorithm from NVIDIA ( 21 ). This algorithm uses convolutional neural networks (CNNs) to capture images by the car’s front camera and correspond them to the direction of travel of the self-driving car. Because it is a pure CNN, the algorithm works on both normal roads and highways, with and without road markings. This architecture minimizes the mean square error between the directional angle output by the CNN and the directional angle of the car when driven by a real person.

The remaining part of the paper is organized as follows: the next section presents the theory of the proposed methods, the third section describes the design of experiments, followed by the results and discussion and the conclusion.

Methodology

In this section, brief theoretical descriptions of the models used in this study are presented.

Game Refinement Theory

GR theory is a mathematical method that focuses on the attractiveness and the sophistication of games. This method assumes the entertainment of a game by modeling the uncertainty during the process of the game ( 8 ). The GR theory is based on the concept of game progress and game information progress. Game information progress presents the degree of certainty of a game’s results in time or in steps. Having full information of the game progress after its conclusion, game progress x(t) will be given as a linear function of time t. However, the game information progress given by a linear function will remain unknown during the in-game period. The presence of uncertainty during the game, often until the final moments of a game, renders exponential game progress. Therefore, a realistic model of game information progress is given by the following:

x (t) = {(\frac{t}{T})}^{n}

(1)

x^{″} (t) = \frac{n (n - 1)}{T^{n}} t^{n - 2} |_{t = T} = \frac{n (n - 1)}{T^{2}}

(2)

GR = \frac{\sqrt{n (n - 1)}}{T}

(3)

A realistic formulation of game progress with the known outcome is given as Equation 1. The parameter n ( $1 \leq n \in R$ ) is the number of possible options and x(t) is normalized within the range of $0 \leq x (t) \leq 1$ . Deriving x(t) twice at t $\in$ [0, T], given by Equation 2, indicates the change velocity or rate of acquired information (acceleration) of the solved uncertainty of a game. Then, a measure of GR is (root square form) given by Equation 3, which represents the metric for game sophistication. It is assumed in the current model that game information progress in any type of game is encoded and transported in our brains. If the transport of information is assumed to follow the classical theory of kinematics then it is likely that the acceleration of information progress is subject to the forces and laws of physics. Therefore, we expect that the larger the value of x′′(t), the more the game becomes exciting, due in part to the uncertainty of the game outcome. Thus, we use its root square as a GR measure for the game under consideration.

For board and scoring games, the GR measure is determined by Equation 5 using the model of move candidate selection and scoring rate ( 15 ). Here we referred to Xiong and Iida’s ( 22 ) work, where B and G represent the average branching factor and average goals (number of possible options), respectively. Meanwhile, D is the game length (total number of plies) and T is the total points or goals (depth of the whole game tree). One round in board games can be illustrated as a decision tree. At each depth of the game tree, one will choose a move and the game will progress. Figure 1 illustrates one level of the game tree. The distance d, which is shown in Figure 1, can be found by using the simple Pythagoras theorem, a realistic game progress model for board games, which is given by Equation 4:

x (t) = B {(\frac{t}{D})}^{n}

(4)

G R_{board} \approx \frac{\sqrt{B}}{D} or G R_{scoring} \approx \frac{\sqrt{G}}{T}

(5)

These respective variables were collected from the average of the total number of play-testing experiments. The sophistication of games converges to an almost similar sense of thrill (or noble uncertainty [ 23 ]) of GR ∈ [0.07,0.08] (Table 1).

Figure 1.

Learning approaches for automated driving: (a) traditional model and (b) end-to-end model.

Table 1.

Measures of Game Refinement for Various Games ( 16 )

	B/G	D/T	GR
Chess	35	80	0.074
Shogi	80	115	0.078
Go	250	208	0.076
Table tennis ( 24 )	54.86	96.47	0.077
Basketball ( 25 )	36.38	82.01	0.073
Soccer ( 25 )	2.64	22	0.073
Badminton ( 25 )	46.34	79.34	0.086

Note: B/G = average of branching factors/average goals; D/T = average game depth/total points for goals; GR = game refinement.

Variable Ratio Schedule (N) and Winning Hardness (m) in Games

With the development of GR theory, focusing on the reward, we connected the theory with a variable ratio schedule to further research from a reward view. The basic notion of game playing depends on the rate of information representation, which constitutes the speed or “velocity” of a game, and is generally defined as the success rate/winning rate (v). In contrast, the challenge faced or difficulty of reaching such a success, which constitutes the weight or “mass” of a game, is generally defined as the difficulty rate/winning hardness (m).

In Variable Ratio (VR) schedule, the parameter N shows the average reward frequency, where $1 \leq N \in R$ . In this study, winning a game corresponds to obtaining a reward, then it implies the game length, which is D in board games (total number of plies) and T in scoring games (total points or goals). Therefore, N = D or N = T, implying a general form of reward frequency of the game’s winning rate. Based on such a notion, the winning rate v and winning hardness m are defined by Equation 6:

m = 1 - v with v = \frac{1}{N} or v = \frac{1}{T}

(6)

Motions in Mind

Analogical links between motions in physics and motions in mind have been previously established based on the notions of winning rate (or velocity) v and winning hardness m ( 15 ). The winning rate and winning hardness are defined as the velocity and mass, respectively. The various motions in mind quantities can be determined accordingly. The correspondence between the variables from the theory of kinematics and the progress of games (game information) are presented in Table 2.

Table 2.

Analogical Link Between Game and Physics ( 15 )

Notation	Game context	Notation	Physics context
y	Solved uncertainty	x	Displacement
t	Progress or length	t	Time
v	Win rate	v	Velocity
m	Win hardness	M	Mass
a	Acceleration	g	Gravitational acceleration
E_p	Potential energy	U	Potential energy

Previous work by Iida and Khalid ( 15 ) defined F as the player’s strength to move a game or ability in general, where a is the growth rate of the “flow” experience of the player in the game (since a = $\frac{F}{m}$ , then F is the ability and m is the challenge [ 26 ]). In this study, a = $\frac{1}{N}$ can be regarded as the sense of gravity in people’s minds, where it is the source of cultural tendencies of people’s minds in game-playing reflected at a specific time/era. Therefore, the measure of F is given by Equation 7:

F_{1} = ma = \frac{N - 1}{N} a

(7)

The potential energy ( $E_{p}$ ) in the game is defined as the game-playing potential or the expected game information required to finish a game ( 15 ), given by Equation 8. Meanwhile, momentum ( $\vec{p}$ ) in the game refers to the competitive balance of a game, which involves the degree of challenge needed (m) and effort given (v) to drive the game progression ( 15 ), given by Equation 9:

E_{p} = 2 m v^{2} = \frac{2 (N - 1)}{N^{3}}

(8)

\vec{p} = mv = \frac{(N - 1)}{N^{2}}

(9)

Similar to the law of conservation of energy in classical physics, Ep is expected to be conserved, where the momentum of the game-playing motions, while differing in level, contains both objective (in-game) and subjective (in-mind) recognition. Potential energy is transformed into the sum of the momentum from the game’s motion ( $\vec{p_{1}}$ ) and the momentum of the mind’s motion ( $\vec{p_{2}}$ ), that is, Ep = $\vec{p_{1}}$ + $\vec{p_{2}}$ ( 27 , 28 ).

Therefore, it is expected that $\vec{p_{2}}$ is a reliable measurement of engagement. Applying Equations 8 and 10, Equation 11 is obtained. Then, the first derivative of Equation 11 is solved, where m = $\frac{3 \pm \sqrt{3}}{6}$ is obtained and represents high excitement ( $m = \frac{3 + \sqrt{3}}{6}$ ) and high expectancy (m = $\frac{3 - \sqrt{3}}{6}$ ). Therefore, $\vec{p_{2}}$ has two peaks at which play engagement will be maximized. Respectively, the objective winning rate v1 and subjective winning rate v2 are given by Equations 12 and 13. Subjective acceleration $a_{2}$ is given by Equation 14, then the subjective force $F_{2}$ in mind can be considered as Equation 16. Be solving Equations 10 and 11, then Eq is given by the following:

\vec{p_{1}} = mv

(10)

\vec{p_{2}} = E_{p} - \vec{p_{1}} = 2 m^{3} - 3 m^{2} + m

(11)

v_{1} (m) = 1 - m

(12)

v_{2} (m) = 2 m^{2} - 3 m + 1

(13)

a_{2} (m) = 4 m - 3

(14)

F_{2} (m) = m a_{2} = 4 m^{2} - 3 m

(15)

δ E (E_{q}) = {\vec{p}}_{1} - {\vec{p}}_{2} = 2 m^{2} - 2 m^{3} = 2 m^{2} v

(16)

The analogous connections between the physics model and the game progress model were presented in Table 2. In this paper, for the application aspect of motion in mind in a driving environment, Table 3 shows an analogous connection between the motion-in-mind model in the game and driving environments.

Table 3.

Analogical Link Between Motion in Game and Driving

Notation	Game context	Motion in mind in driving
y	Solved uncertainty	Displacement
t	Progress or length	Time
v	Win rate	Frequency of operation
m	Win hardness	Difficulty rate of operation
a	Acceleration	Acceleration
E_p	Potential energy	Objective potential energy(perfect normal comfort)
E_q	Subjective potential energy	Subjective potential energy (driver)
$\vec{p_{1}}$	Objective momentum	Objective momentum
$\vec{p_{2}}$	Player momentum	Subjective momentum
vk	Subjective outcome/reward of a player with ability k	Subjective outcome/reward of a driver with ability k
mk/mx	2-Person and N-person game inequality mass	Driver and rider mass
E0	Objective reinforcement energy	Super standard driver energy

End-to-End Deep Learning-Based Self-Driving Model

This paper utilized end-to-end deep learning rather than the traditional models incorporating several modules (see Figure 1).

This paper is based on an end-to-end autonomous driving deep learning algorithm from Bojarski et al. ( 21 ) (Figure 2).

Figure 2.

End-to-end deep learning model for automated driving ( 21 ).

In this model, a CNN has been used to map the raw pixels from a front-facing camera to the steering commands for a self-driving car. This powerful end-to-end approach means that with minimum training data from humans, the system learns to steer, with or without lane markings, on both local roads and highways. The system can also operate in areas with unclear visual guidance, such as parking lots or unpaved roads. There are five convolutional layers and three fully coupled layers, and the network is very small compared to networks commonly used in image recognition.

Driver Category

Driving style is a characteristic of a driver’s behavior when driving and is reflected in the human input to the car and the overall response ( 5 ). The classification and identification of driving styles can help to improve driving safety and fuel efficiency ( 29 ). By identifying different driving styles, it is also possible to detect potentially dangerous situations and regulate abnormal driving behavior ( 30 ). It helps to enhance safe driving. In this study, the player satisfaction or driver satisfaction is modeled considering the concepts of motion-in-mind theory.

Here, N is an important unit for measuring player satisfaction, and in this paper it is connected with the driver’s comfort. Here, we consider N to be the number of times the steering wheel is operated in each turn, and in general we consider that N will be smaller for novices and larger for the experienced. Similarly, the size of N will vary between drivers with different personalities. Adventurous drivers seeking the thrill of deviating from the curve will have a smaller N, while cautious drivers seeking a smooth driving experience will have a correspondingly larger N for the curve. The validity of N was justified using a simulation-based experiment which showed that N improved with practice and N stabilized for values of 4–6.

Based on this idea, the drivers were divided into three groups (cautious, regular, aggressive).

Each type is defined as follows.

Cautious driver: this type of driver will turn at slow speed and smoothly negotiate corners.

General driver: this type of driver will turn at a moderate speed with moderate cornering/turning fluctuations.

Aggressive driver: this type of driver turns at a higher speed and adjusts the steering wheel with more jitters.

Driver Category Classification

In general, acceleration, speed, and pulling power during acceleration can be good indicators of a driver’s driving habits ( 31 ), but the use of the brake pedal, following distance, and number of lane changes can sometimes be better indicators of a driver’s driving personality ( 32 ). For this reason, the selection of different DSIs may produce different results with respect to driving style classification. The degree to which the selection of DSIs is reasonable will affect the results of the driving style classification, and in this paper six values of motion in mind were selected: E_p, E_q, F₁, F₂, $\vec{p_{1}}, \vec{p_{2}}$ (see Table 3). The F-values represent the force between the game and the player, and in the case of driving activities, the force of driving itself is similarly greater than the driver’s willpower to ensure safety. Nikulin’s ( 33 ) study found that during acceleration the driver’s driving habits can be fully reflected, so we chose F₁, E_p, and acceleration as the classification benchmarks for classifying the drivers into cautious, general, and aggressive categories using k-means clustering.

Design of Experiments

The experiments were designed in three steps: (1) establishing N as the driving style identifier; (2) driving style database creation; and (3) driving style classification (Figure 3).

Figure 3.

Design of experiment process flow.

In this study all driving was recorded in Udacity’s self-driving car, Nanodegree (Figure 4). The Nanodegree project is designed to teach students how to train self-driving cars and navigate road courses using deep learning. All the assets in this repository require free game making engine Unity.

Figure 4.

Simulator from Udacity’s self-driving car, Nanodegree.

Establishing N as a Driving Style Identifier

Following the player satisfaction model, we know that N represents the average number of rewards in the game activity, which is an important indicator to measure user experience. Observation of N is necessary to calculate the variables of the motion-in-mind model. Further, it is necessary to establish that N is linked with turns of the steering wheel and comfort levels.

To investigate the reasonableness of the N setting, we have conducted an experiment. Using the unity simulation environment, we asked the same driver to drive the same mountain road section (with many bends) 100 times in a row and observed the change in the subject’s growth. As a prediction, the subject’s N will slowly increase and then become stable at a value. The result is shown in Figure 5; as the player gained experience, their N will be larger, which means the veteran period players are more inclined to fine-tune the steering wheel with more N. An end-to-end learning environment was used in this experiment.

Figure 5.

Variations in N during the one-player 80-lap growth experiment (where N represents the average number of changes per turn.).

Driving Style Database Creation

The driving style database was created using two approaches. Firstly, experiments were conducted with human players, in which data was collected on the performance of human players during driving on the mountain road in Udacity’s self-driving car, Nanodegree. Secondly, experiments were conducted using artificial intelligence (AI) players, where an end-to-end learning algorithm ( 21 ) was used to train the AI average player and collect data on their performance in the same scenario. The use of AI drivers were necessary as a limited number of human drivers were available and the training data-set needed to be of reasonable size for proper implementation and testing of the methodology.

As for human drivers, 10 candidates of varying ages and personalities who had never used the simulator before and who had real-life driving qualifications were sought as subjects. Each driver did 20 laps of the circuit. We assumed that the first five laps and the last five laps were representative of the novice and expert behavior of each driver. Observations collected from 10 subjects were included in the database.

As it is not possible to rank or classify driving style from the data of 10 candidates/drivers driving on a self-driving simulator, AI was used to generate more driving data. The AI players are trained by the data of each driver’s novice and veteran periods. From 10 human players, the 20 models that were representative of individual styles were trained. In a follow-up experiment, 10 human players will perform manual classification for the 20 models from their performance to validate the results. It was designed to derive steering control commands directly using the end-to-end learning algorithm from the input video from the camera. Data of 200 laps of approximately were simulated using a standard-level AI driver model. As shown in Figure 5, it was found that the trends and values were largely correlated and therefore the simulation data could be approximated as an analogy to the human data.

Analysis and Results

Motion-in-Mind Model

In this section, the values for motion-in-mind model variables of the drivers when driving a car in a self-driving environment are reported in Table 4.

Table 4.

Motions in the Experiment for 10 Players

Player	v	m	E_p	E_q	F ₁	F ₂	p ₁	p ₂
1	0.1644	0.8356	0.0452	0.2296	0.0026	0.2981	0.1374	−0.0922
2	0.1857	0.8143	0.0562	0.2463	0.0022	0.2137	0.1512	−0.0951
3	0.1785	0.8215	0.0523	0.2409	0.0002	0.2406	0.1466	−0.0943
4	0.1736	0.8264	0.0498	0.2371	−0.0010	0.2589	0.1435	−0.0937
5	0.1683	0.8317	0.0471	0.2328	−0.0007	0.2805	0.1400	−0.0929
6	0.1700	0.8300	0.0479	0.2342	0.0002	0.2773	0.1411	−0.0931
7	0.1809	0.8191	0.0536	0.2428	−0.0011	0.2343	0.1482	−0.0946
8	0.1979	0.8021	0.0628	0.2546	−0.0083	0.1732	0.1587	−0.0959
9	0.1853	0.8147	0.0559	0.2460	−0.0062	0.2214	0.1509	−0.0950
10	0.1582	0.8418	0.0422	0.2242	0.0015	0.3195	0.1332	−0.0910
Average	0.1763	0.8237	0.0513	0.2389	−0.0011	0.2517	0.1451	−0.0938

According to the motion-in-mind model, E_p is subjective energy and represents the connection between the player and the game; when the E_p value is large, the player has high subjective energy and a stronger subjective connection to the game. Through the data we can see that the mean value of E_p is 0.0513, ranging from 0.0422 to 0.0628, while the E_q value is much larger than the E_p value, with a value of around 0.2390. The objective game energy is greater compared to the player’s energy connection to the game, and indicates that the driving activity is an activity with low freedom, where the driving process is limited by road settings, safety considerations, and other factors. The objective settings of the activity itself place great restrictions on the driver’s behavior. Thus, the objective energy needs to be greater than the subjective energy. Similarly, the F-value represents the force between the game and the player, and in the case of driving activities, the force of driving itself is similarly greater than the driver’s willpower to ensure safety. In the F₁ values we find that players 4, 5, 7, 8, and 9 are even negative, indicating that in this process the player does not manage the whole process very well and is not able to gain the initiative under the rules. Whereas momentum indicates the intensity with which the player can advance the game, in the driving activity, $\vec{p_{1}}$ is much greater than $\vec{p_{2}}$ , indicating that in the intensity with which a player can move is larger than the driving activity itself can control.

Classification Results

As discussed in the Methodology section, the driving database was classified using the k-means clustering algorithm. Here, F₁, E_p, and acceleration are used as the benchmarks for classifying the 600 driving clips of 10 players, and determined the driving style of the player according to their percentage (Figure 6).

Figure 6.

The k-means classification result.

The number of elements in the three clusters was 129, 278, and 193, and the three centers are [−0.47, 0.49 −0.25], [−0.35, 0.01, 1.37], and [0.91, −0.72, −0.55]. For the cautious driver there is usually a smaller F₁, taking less initiative to follow the rules of the game to the greatest extent possible, and a larger E_p, that is, more energy to interact with the game to keep an eye on it and ensure it is under control. and a smaller realistic acceleration. Aggressive drivers, on the other hand, usually have a larger F₁ and are more focused on taking greater initiative to enjoy the thrill of the act of driving as opposed to respecting the rules. Smaller E_p values indicate less subjective energy interaction during the process, with drivers enjoying the feeling of losing control of themselves more, while requiring less acceleration to ensure safety. Whereas between E_p and F₁ for an average (general) driver, acceleration is approximately equal to 1 to ensure safety and smooth driving.

The driver type was identified based on the highest percentage of driving trips, as shown in the Table 5. No driver was identified as general/average type. Further to the k-means clustering, a subjective evaluation was carried out by voting. In this process, 10 subjects were shown a set of 20 randomly chosen videos of driving from the created database. The subjects then classified the driver of each video into a category. The results of this experiment were compared with the data obtained from Table 5 and there was an 80% match between subjective voting and machine learning-based classification. Machine clustering classification focuses more on the data performance of each segment of samples. In our experiments, we selected three relevant metrics to classify the driving style of each person for multiple segments of samples, whichever accounted for more. However, when classified by human players, the overall atmosphere of driving will be of interest. For example, in divergent player 5, the machine classifies the player as cautious because the number of samples classified as cautious is higher, yet the player also has a fairly high number of aggressive samples, so the majority of human players with respect to overall atmosphere believe that their style should be more aggressive. This resulted in a difference in the data. To eliminate or reduce such discrepancies, we believe that valid indicators for this issue could be added to future studies.

Table 5.

Classification of Driving Trips and Driver Types

Driver	Cautious sample	Driver %	General sample	Driver %	Aggressive sample	Driver %	Type from classification	Type from human evaluation
1	16	26.67%	17	28.33%	27	45.00%	Aggressive driver	Aggressive driver
2	56	93.33%	1	1.67%	3	5.00%	Cautious driver	Cautious driver
3	25	41.67%	20	33.33%	15	25.00%	Cautious driver	Cautious driver
4	49	81.67%	3	5.00%	8	13.33%	Cautious driver	Cautious driver
5	27	45.00%	12	20.00%	21	35.00%	Cautious driver	Aggressive driver
6	9	15.00%	19	31.67%	32	53.33%	Aggressive driver	Aggressive driver
7	20	33.33%	18	30.00%	22	36.67%	Aggressive driver	General driver
8	22	36.67%	10	16.67%	28	46.67%	Aggressive driver	Aggressive driver
9	43	71.67%	10	16.67%	7	11.67%	Cautious driver	Cautious driver
10	11	18.33%	19	31.67%	30	50.00%	Aggressive driver	Aggressive driver

Conclusion

This paper proposes to use GR theory to model the behavior of self-driving cars. The motivation behind this implementation was the idea that a driver would draw the most enjoyment out of a riding experience in a self-driving car environment if the driving style of the driver matches that of the car. Accordingly, the study designed an experiment where a driver’s style was classified based on their steering wheel handling characteristics when negotiating turns in Udacity’s self-driving car, Nanodegree. An end-to-end deep-learning algorithm was used to measure the driving behaviors. The steering wheel handling characteristics were measured and then modeled using the motion-in-mind model of GR theory. The motion-in-mind variables were further classified to identify the driving style of each driver. The study provided the first application of successfully modeling the self-driving car environment as a game and establishing that driving style can be identified from the interaction of a driver with the steering wheel of a Nanodegree vehicle. In the future, a more complex and detailed formulation of the motion-in-mind concept for driver satisfaction modeling can be developed by including more driving variables. Adaptation of assisted or automated driving according to the characteristics of the rider for enhanced satisfaction and driving comfort can be explored further with experimentation in a simulated self-driving environment especially for dangerous conditions. Also, real-world rewards are often not simply a varied schedule but frequently the consequence of a combination of sophisticated reward mechanisms. In such situations, the ambiguity generated by rewards has ramifications for player psychology and game participation that call for additional study.

Footnotes

Author Contributions

The authors confirm contribution to the paper as follows: study conception and design: X. Kang, W. Qiu, H. Iida, B. Ghosh; data collection: X. Kang, W. Qiu, H. Iida, B. Ghosh; analysis and interpretation of results: X. Kang, W. Qiu, H. Iida, B. Ghosh; draft manuscript preparation: X. Kang, W. Qiu, H. Iida, B. Ghosh. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research has been supported by funding from European Union’s Horizon Europe Research and Innovation Program under grant agreement 101056931. This research has also been funded by NexSys which is financially supported by Science Foundation Ireland under the SFI Strategic Partnership Programme (Grant 21/SPP/3756)

ORCID iDs

Xiaohan Kang

Hiroyuki Iida

Bidisha Ghosh

References

Bimbraw

Autonomous Cars: Past, Present and Future a Review of the Developments in the Last Century, the Present Scenario and the Expected Future of Autonomous Vehicle Technology. Proc., 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO), Colmar, France, IEEE, New York, 2015, pp. 191–198.

González

Pérez

Milanés

Nashashibi

A Review of Motion Planning Techniques for Automated Vehicles. IEEE Transactions on Intelligent Transportation Systems, Vol. 17, No. 4, 2015, pp. 1135–1145.

Wang

Chen

Modeling and Recognizing Driver Behavior Based on Driving Data: A Survey. Mathematical Problems in Engineering, Vol. 2014, 2014. https://doi.org/10.1155/2014/245641.

Qind

Zhan

Zeng

Hybrid Vehicle Energy Management Strategy Based on Driving Style Recognition. Journal of Mechanical Engineering, Vol. 52, No. 8, 2016, pp. 162–169. (In Chinese)

Ouali

Shah

Kim

Fuente

Gao.

Driving Style Identification Algorithm with Real-World Data Based on Statistical Approach. SAE Technical Paper 2016-01-1422. SAE International, 2016.

del Campo

Asua

Martínez

Mata-Carballeira

Ó.

Echanobe

Driving Style Recognition based on Ride Comfort Using a Hybrid Machine Learning Algorithm. Proc., 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, IEEE, New York, 2018, pp. 3251–3258. https://doi.org/10.1109/ITSC.2018.8569722.

Neumann

V. J.

Zur theorie der gesellschaftsspiele. Mathematische annalen, Vol. 100, No. 1, 1928, pp. 295–320.

Iida

Takahara

Nagashima

Kajihara

Hashimoto

An Application of Game Refinement Theory to Mah Jong. In Entertainment Computing – ICEC 2004. ICEC 2004 ( Rauterberg

, ed.). Lecture Notes in Computer Science, Vol. 3166, Springer, Berlin, Heidelberg, 2004, pp. 333–338. https://doi.org/10.1007/978-3-540-28643-141.

Iida

Takeshita

Yoshimura

A Metric for Entertainment of Boardgames: Its Implication for Evolution of Chess Variants. In Entertainment Computing. IFIP—The International Federation for Information Processing ( Nakatsu

Hoshino

, eds.), Vol. 112, Springer, Boston, MA, 2003, pp. 65–72. https://doi.org/10.1007/978-0-387-35660-08.

10.

Sutiono

A. P.

Purwarianti

Iida

A Mathematical Model of Game Refinement. In Intelligent Technologies for Interactive Entertainment. INTETAIN 2014 ( Reidsma

Choi

Bargar

, eds.). Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol 136, Springer, Cham, 2014, pp. 148–151. https://doi.org/10.1007/978-3-319-08189-222.

11.

Huynh

Zuo

Iida

Analyzing Gamification of “Duolingo” with Focus on Its Course Structure. In Games and Learning Alliance. GALA 2016 ( Bottino

Jeuring

Veltkamp

, eds.), Utrecht, The Netherlands, Springer, Cham, 2016, pp. 268–277.

12.

Anunpattana

Khalid

M. N. A.

Iida

Inchamnan

Capturing Potential Impact of Challenge-Based Gamification on Gamified Quizzing in the Classroom. Heliyon, Vol. 7, No. 12, 2021, p. e08637.

13.

Zuo

Xion

Wang

Lida

Evaluation of Hotel Loyalty Program with Game Refinement Theory and Analytic Hierarchy Process. Proc., International Conference on Computer, Sanya, China, Electronic Information and Communications (CEIC 2018), 2018, pp. 433–441.

14.

Zhang

Xiaohan

Khalid

M. N. A.

Iida

Bridging Ride and Play Comfort. Information, Vol. 12, No. 3, 2021, p. 119. https://doi.org/10.3390/info12030119.

15.

Iida

Khalid

M. N. A.

Using Games to Study Law of Motions in Mind. IEEE Access, 2020, Vol. 8, pp. 138701–138709. https://doi.org/10.1109/ACCESS.2020.3012597.

16.

Xiaohan

Khalid

M. N. A.

Iida

Player Satisfaction Model and Its Implication to Cultural Change. IEEE Access, Vol. 8, 2020, pp. 184375–184382. https://doi.org/10.1109/ACCESS.2020.3029817.

17.

Ferster

C. B.

Skinner

B. F.

Schedules of Reinforcement. Appleton-Century-Crofts, New York, NY, 1957.

18.

Pomerleau, D. ALVINN: An Autonomous Land Vehicle in a Neural Network. Proc., Advances in Neural Information Processing Systems (NeurIPS), The MIT Press, Denver, Colorado, 1988, pp. 305–313.

19.

Bojarski

Testa

D. D.

Dworakowski

Firner

Flepp

Goyal

Jackel

L. D.

, et al. End to End Learning for Self-Driving Cars. arXiv Preprint arXiv: 1604.07316, 2016.

20.

Dosovitskiy

Ros

Codevilla

Lopez

Koltun

CARLA: An Open Urban Driving Simulator. Proceedings of the Conference on Robot Learning (CoRL), PMLR, Vol. 78, 2017, 1–16.

21.

Bojarski

Del Testa

Dworakowski

Firner

Flepp

Goyal

Jackel

L. D.

, et al. End to End Learning for Self-Driving Cars. arXiv Preprint arXiv:1604.07316, 2016.

22.

Xiong

Iida

Attractiveness of Real Time Strategy Games. Proc., 2nd International Conference on Systems and Informatics (ICSAI 2014), Shanghai, China, IEEE, New York, 2014, pp. 271–276. https://doi.org/10.1109/ICSAI.2014.7009298.

23.

Yicong

Aung

H. P. P.

Khalid

M. N. A.

Iida

Evolution of Games Towards the Discovery of Noble Uncertainty. Proc., International Conference on Advanced Information Technologies (ICAIT), Yangon, Myanmar, IEEE, New York, 2019, pp. 72–77. https://doi.org/10.1109/AITC.2019.8921061.

24.

Jiangzhou

Primanita

Khalid

M. N. A.

Hiroyuki

Analyzing the Improvement Process of Table Tennis Using the Game Refinement Theory. Proc., Sriwijaya International Conference on Information Technology and Its Applications (SICONIAN 2019), Palembang, Indonesia, Atlantis Press, 2020.

25.

Nossal

Iida

Game Refinement Theory and Its Application to Score Limit Games. Proc., IEEE Games Media Entertainment, Toronto, ON, Canada, IEEE, New York, 2014, pp. 1–3. https://doi.org/10.1109/GEM.2014.7048120.

26.

Csikszentmihalyi

Flow: The Psychology of Optimal Experience: Steps Toward Enhancing the Quality of Life. Harper Collins Publishers, New York, NY, 1991.

27.

Iida

Khalid

A Paradigm Shift from Optimal Play to Mental Comfort: A Perspective from the Game Refinement Theory. International Journal of Informatics, Information System and Computer Engineering (INJIISCOM), Vol. 1, No. 1, 2020, pp. 47–78.

28.

Khalid

M. N. A.

Iida

Objectivity and Subjectivity in Games: Understanding Engagement and Addiction Mechanism. IEEE Access, Vol. 9, 2021, pp. 65187–65205. https://doi.org/10.1109/ACCESS.2021.3075954.

29.

Lee

Son

Relationships Between Driving Style and Fuel Consumption in High-Way Driving. SAE Technical Paper 2011-28-0051. SAE International, 2011.

30.

Miyajima

Nishiwaki

Ozawa

Wakita

Itou

Takeda

Itakura

Driver Modeling Based on Driving Behavior and Its Evaluation in Driver Identification. Proceedings of the IEEE, Vol. 95, No. 2, 2007, pp. 427–437. https://doi.org/10.1109/JPROC.2006.888405.

31.

Nikulin

Driving Style Identification with Unsupervised Learning. Proc., International Conference on Machine Learning and Data Mining in Patern Recognition. Springer International Publishing, Cham, 2016, pp. 155–169.

32.

Van Ly

Martin

Trivedi

Driver Classification and Driving Style Recognition Using Intertial Sensors. Proc., IEEE Intelligent Vehicles Symposium (IV), Gold Coast, Australia, June 23–26, 2013.

33.

Nikulin

Driving Style Identification with Unsupervised Learning. In Machine Learning and Data Mining in Pattern Recognition, MLDM 2016 ( Perner

, eds.). Lecture Notes in Computer Science, Vol. 9729, Springer, Cham, 2016, pp. 155–169. https://doi.org/10.1007/978-3-319-41920-6-12.

Application of Game Refinement Theory to Automated Driving

Abstract

Keywords

Methodology

Game Refinement Theory

Variable Ratio Schedule (N) and Winning Hardness (m) in Games

Motions in Mind

End-to-End Deep Learning-Based Self-Driving Model

Driver Category

Driver Category Classification

Design of Experiments

Establishing N as a Driving Style Identifier

Driving Style Database Creation

Analysis and Results

Motion-in-Mind Model

Classification Results

Conclusion

Footnotes

Author Contributions

Declaration of Conflicting Interests

Funding

ORCID iDs

References