Can Proficiency in Chess Predict Situation Awareness in a Novel Task?

Abstract

This study explored the correlation between situation awareness (SA) and proficiency in chess. Eighteen participants were tested, performing a letter factory (LF) task and providing their ELO scores. Each participant received $10 for 30 min of participation. The results showed a negative correlation between the number of games played and the SA score, suggesting that increased gameplay was associated with lower SA in the letter factory task. Additionally, the ELO score was negatively correlated with the number of double attempts to place a designated letter in the loading boxes, indicating that more skilled players made fewer errors.

Situation Awareness (SA), a cognitive process involving the perception, comprehension, and prediction of elements in a dynamic environment, plays a pivotal role in safety-critical operations such as aviation and air traffic control (Durso et al., 2006; Endsley, 1988, 1995, 1999; Jones & Endsley, 1996; Rogers, 2008). Kharoufah et al. (2018) identified SA as a key factor in a sample of 200 commercial air transport accidents from 2000 to 2016. Despite ongoing debates about its representational, operational, causal, and moral status (Dekker, 2015; Salmon et al., 2015), the importance of SA in safety-critical operations remains undisputed within the scientific domain (Pritchett, 2015). Understanding the cognitive mechanisms that give rise to SA as an operational state is of great interest to researchers. Such understanding can enhance the effectiveness of performance assessments used in the selection, training, and evaluation of pilots (Durso et al., 2006; Endsley, 2015).

In this context, the game of chess, with its demands on cognitive abilities, particularly fluid intelligence, and working memory, serves as an ideal domain for studying the relationship between cognitive ability and skill. Elite youth chess players have been found to score higher on the Wechsler Intelligence Scale for Children, which primarily measures fluid reasoning (Frydman & Lynn, 1992). Furthermore, a positive correlation has been observed between full-scale IQ and chess rating, indicating that general intelligence can predict chess ability (Grabner et al., 2007).

Working memory, which is strongly connected to fluid intelligence (gF), plays a crucial role in chess. This cognitive faculty allows players to hold and manipulate information in their minds over short periods, a skill vital in chess where players must anticipate and plan several moves ahead. Brain-imaging studies have shown that tasks involving fluid intelligence and working memory activate similar areas of the prefrontal and parietal cortex, suggesting a shared neural basis (Prabhakaran et al., 1997).

Moreover, working memory is critical for achieving Level 1 SA, the perception stage, which involves recognizing relevant elements in the environment (Endsley, 1995; Endsley & Garland, 2000). This aligns with the demands of chess, where players must maintain awareness of the entire board and the positions of all pieces.

In conclusion, chess expertise is not merely a function of mastering the game’s rules and strategies but also reflects broader cognitive abilities, particularly fluid intelligence, and working memory, integral to SA. The current study aims to investigate the correlation between participants’ ELO ratings (an online chess assessment), the number of games played, and their scores in the letter factory task. By examining this relationship, we seek to identify whether having better ELO ratings can serve as predictor for SA, thereby enhancing our understanding of the cognitive mechanisms underlying SA and improving performance in safety-critical operations. However, it’s important to acknowledge that the sheer quantity of games played may present a limitation in our analysis. Simply playing more games doesn’t necessarily equate to mastery in chess, as proficiency in the game relies on a multifaceted approach encompassing study, practice, and reflection.

Method

Participants

Eighteen students from Embry-Riddle Aeronautical University took part in this study. Each participant received $10 for approximately 30 min of participation, during which they completed a demographic form and played the Letter Factory task (a measure of SA).

Materials and Procedure

Upon consenting to participate, the participants were asked to complete a demographic form requesting information such as age, gender, their ELO rating (Rapid format) on chess.com, and the number of games played on chess.com. Following this, participants engaged in the Letter Factory task, a component of the ATCPrep software (see Figure 1).

Figure 1.

Letter factory task scenario.

The primary objective was to correctly sort moving letters into designated bins. This involved not only the accurate placement of letters but also maintaining quality control. Specifically, any letters not A, B, C, or D were considered part of the quality control process.

Additionally, the task encompassed sorting letters into boxes of matching colors, managing resources by ordering new boxes when current ones were filled, and responding to SA questions. Upon completion of the task, participants were assessed on SA, quality control, and various other performance metrics related to the Letter Factory task. The Letter Factory task is based on a study guide provided by ATC Prep, originally designed to assess SA and performance (Dattel & King, 2006). After participants completed the test, screenshots of the results were taken and later scored on a spreadsheet.

Results

Table 1 includes the means and standard deviations for the Elo score, the number of chess games played, LF Performance score, and SA score for LF. As expected, several of the LF performance measures were correlated.

Table 1.

Means and Standard Deviation for Selected Variables.

Variable	Mean	Standard deviation
Elo Score	1,282.56	398.48
Games Played	2,857.00	3,631.27
LF Performance	60.09	16.10
LF SA	56.24	4.87

A significant correlation was observed between the Elo score and the number of games played, r(16) = .76, p < .001 (see Figure 2).

Figure 2.

Relationship between Elo score and number of games played.

Furthermore, a significant correlation was found between the Elo score and the number of double attempts to place a designated letter in the loading boxes, r(16) = −.523, p = .026 (see Figure 3).

Figure 3.

Relationship between Elo score and number of double letter box errors.

In the context of the LF, only Letters A through D are to be removed from the conveyor belt and placed in a loading box of a corresponding color (e.g., a green letter in a green box). However, each colored box can only contain one unique letter of the same color (e.g., a green box can only contain one green A, one green B, one green C, and one green D). An error is committed when one attempts to add a letter to a box that already contains that same-colored letter. A significant correlation was also found between the number of games played and the SA score, r(16) = −.494, p = .037 (see Figure 4).

Figure 4.

Relationship between number of games played and SA.

A between-subjects ANOVA, using the median split of the Elo score, revealed no significant difference, F(1, 16) = 0.001, p = .974. However, when the number of games was controlled, the equation changed to F(1, 15) = 2.693, p = .122.

Discussion

A positive correlation was found between the Elo score, a measure of player skill, and the number of games played, suggesting that experience enhances skill level. The Elo score was negatively correlated with the number of double attempts to place a designated letter in the loading boxes, indicating that skilled players made fewer errors. This task required placing letters A-D into color-matching boxes, with duplicate placements considered errors. Interestingly, the number of games played was negatively correlated with the SA score, suggesting that more gameplay the lower SA for the LF task.

This finding could be due to the unfamiliarity of the LF for players proficient in chess, as indicated by a higher Elo score. Their extensive game knowledge might not have directly translated to these new tasks. When the number of games played was used as a covariate, the result approached statistical significance, suggesting an effect of gameplay on the Elo score.

These discoveries warrant future investigations into whether individuals with higher Elo scores exhibit greater adaptability to novel tasks. Given that those who excel in chess typically possess a profound comprehension of the game, with adequate time and instruction, they may possess the capacity to swiftly acquire proficiency in new tasks.

Furthermore, the findings indicate that individuals with a greater number of games played demonstrate a reduced likelihood of errors in the letter factory game, suggesting potential enhancements in their performance within dynamic settings. Such insights hold considerable promise for training and advancement across diverse domains.