Workload or Situational Awareness?: TLX vs. SART for Aerospace Systems Design Evaluation

Abstract

Both workload and situational awareness (SA) have been recognised as important considerations in aerospace system design evaluation. This paper considers the use of subjective techniques for the measurement of workload and SA, namely the NASA TLX workload scale and the Situational Awareness Rating Technique (SART), and compares the diagnosticity and sensitivity of each within a computer-graphics flight simulation environment. Both the 10-dimensional and 3-dimensional implementations of SART were tested. Twelve RAF pilots were asked to rate a videotape of an air combat flight simulation sequence, using the TLX and SART techniques. Task difficulty was varied between the flight sequences used. The experimental group was split according to their flying experience. Task difficulty and experience were analysed as the main independent variables in the experiment. Results showed that although both scales were sensitive to task difficulty, only SART showed any differences between experience conditions. This implies that SART may have some added utility where operator capability or domain expertise is a design variable. Intercorrelations showed a substantial overlap between the two scales on a number of factors. These factors are discussed to attempt to clarify both differences and similarities between the scales, and also the degree of commonality between the concepts of workload and SA. The implications of such differences are considered within the application domain of aerospace systems design and evaluation.