Proceedings of the Human Factors and Ergonomics Society Annual Meeting

In a review of flight simulation performance experiments conducted at the U.S. Navy's Visual Technology Research simulator (VTRS), it was observed that individual difference variables accounted for a major portion of the total explained variance, in many cases more than the simulator equipment variables that were deliberately manipulated. This finding underscores the importance of individual differences in performance and training research in support of man-machine systems development and implementation. The identification of the substrates underlying individual differences will impact on equipment design considerations and training program requirements for military and industrial systems.

Recently, there has been a re-emergence of interest in the cognitive ability determinants of individual differences in skill acquisition and skilled performance. First we review some basic characteristics of individual differences in skill acquisition. We next consider the current evidence for the emergent “task-specific” factor, a matter that may have important implications for the utility of ability measures as predictors of individual differences in asymptotic skilled performance. We also review two major factors in determining the relations between abilities and individual differences in skill acquisition, advances in theory and the enlargement of the data base for discussion of the topic. We address these factors, in the context of a discussion of “which” abilities predict individual differences in skilled performance, “when” such predictors are maximally effective, and “how” abilities and information processing demands interact to determine ability-performance associations.

The FAA is concerned that flight-safety could be compromised by undetected cognitive impairment in pilots due to conditions such as substance abuse, mental illness and neuropsychological problems. Interest has been shown in the possibility of adding a brief “mini mental exam”, or a simple automated test-battery to the standard flight medical to screen for such conditions. This paper reports an empirical evaluation of four such tests, focusing upon a prototype version of an automated screening battery, SPARTANS (Simple Portable Aviation Relevant Test-battery and Answer-scoring System).

A promising approach in recent years has been to develop measures of individual differences based upon componential cognitive theory to supplement or supplant traditional measures. Cognitive tests are developed to measure theoretically based mental operations which can be isolated by the computation of derived measures such as slope and difference scores. Along with others, we believe there are impediments to this approach due to unreliability of derived measures and lack of demonstrated statistical independence of tests of cognitive abilities. This paper describes a methodology for examining measures of individual differences in information processing skills that first follows the tenets of psychometric theory and then addresses cognitive theories. The approach is illustrated by demonstrating its application in tests representing four distinct cognitive paradigms which were administered repeatedly to subjects over three weeks. Recommended direct measures and derived scores for the four paradigms were examined in terms of their stabilities, retest reliabilities, and cross-correlations. Use of these procedures revealed that 1) derived scores had reliabilities near zero, and therefore, their correlations with other variables were equally low, rendering them of little use as individual difference variables, and 2) correlations between basic or nonderived scores were as high as their reliabilities would allow, suggesting that one common factor could account for the majority of the variance. The generality of this repeated measures paradigmatic approach to the identification of individual differences in human ability is illustrated by describing its application to the evaluation of a family of video games, tests of episodic memory, and visual contrast sensitivity at different spatial frequencies.

In spite of the critical need to match the capabilities of complex human-interfaced systems to the capabilities and limitations of the human operator, relevant research findings on human perception and performance are seldom given systematic consideration in the design of control and display systems. A major reason is that the costs and risks associated with accessing, interpreting, and applying these data are unacceptably high to designers already overburdened with technical information. To help reduce these costs, the Integrated Perceptual Information for Designers (IPID) program has developed: (1) a procedure for compiling and integrating widely scattered human performance research data with potential application in system design; and (2) a format for presenting these data so they can be used directly by practitioners to support design decisions and trade-offs. This data consolidation procedure and presentation format have been used to produce a full-scale demonstration data resource, the Engineering Data Compendium, that integrates information from over 75 subareas of human perception and performance into a 4-volume reference work for designers.

The Lotus 1·2·3^™ spreadsheet macro language provides a low-cost, readily accessible, and easy-to-use method for rapidly building prototypes of computer interface dialogue systems. This method of prototyping was used to evaluate specifications for the human interface component of a satellite ground control system. The prototype mimics the essential functional components of the interface dialogue, and is easily modified, making it possible to determine whether recommended changes in dialogue specifications produce actual improvements in dialogue design.

Under Federal Aviation Administration (FAA) sponsorship, MITRE's Human Performance Assessment Group is contributing to the design of an expert system to support air traffic control. We are working closely with a team of expert, full-performance-level air traffic controllers to capture the formal and informal rules they use in maintaining flight safety and efficiency. This paper documents our approach to working with these experts, the results of using that approach, and a distillation of lessons learned.

This study represents a first phase in the design of a human factors tool for artificial intelligence (AI) system assessment. Desirable attributes of AI interfaces were identified as a result of a review of the literature. A questionnaire was developed where explicit definitions were presented for 17 selected attributes. Nineteen AI system developers rated the attributes under four different context conditions: (1) no context (i.e., general application); (2) a bomber crew system; (3) a command and control station; and (4) an intelligence analyst position. Examination of the ratings showed that attributes associated with communication and education aspects of AI were given the lowest ratings, whereas attributes pertaining to tasks which impose a high level of time stress received the highest ratings of importance. The ratings data were subjected to Multidimensional Scaling (MDS) analyses where the following dimensions were determined: (1) tasks performed principally by the system versus tasks requiring system-human communication; and (2) system attributes that principally require algorithmic interpretation versus those that require a high level of AI capabilities.

The Reliability & Maintainability Information System (REMIS) is being developed by a team of Litton Computer Services, Tandem Computers, and SofTech under contract to the Air Force Logistics Command (AFLC) Logistics Management Support Center. Its purpose is to make accessible to AF managers worldwide, the data necessary to keep weapon systems combat ready in peace and sustain them in war. REMIS will modernize the collection and use of inventory, status, utilization, operational reliability, maintenance, configuration, mission capability, and awaiting parts data for aircraft, trainers, automatic test equipment, Communications-Electronics (C-E) equipment, and support equipment.

The Government procurement request for REMIS envisioned the need for human engineering of the Human Computer Interface (HCI), but there were no requirements to deliver human factors analysis documentation nor to conduct any formal testing. The HCI design task was eventually assigned to an ad hoc team (Messrs. Lorenzetti, Beck, and Maguire) with no formal human engineering experience, and with severe time constraints. Design of a user-friendly system under these constraints (using available human factors data sources) proved to be a challenging exercise!

This paper presents a description of the informal user surveys and qualitative evaluations that were used as a surrogate to the more formal approaches normally recommended. After the fact, the over 900 guidelines in Smith and Mosier were reviewed, with 75 found to be specifically applicable to REMIS. REMIS conformance to each of the 75 guidelines was then assessed. Although the REMIS design was evaluated as reasonably good, we concluded that specific human engineering requirements, schedule, budget, and documentation should be provided. The accessibility of human factors data supporting design should be substantially improved if better quality HCIs are to be assured.

Inadequacies of existing simulation and modeling tools in supporting development and evaluation of man-machine systems are considered. A new tool is proposed to support the development and use, by the end user, of a base of fundamental human performance models representing alternative factors and levels of detail. The Human Operator Model Management Environment (HOMME) is designed to aid in the construction and management of human performance models that are used as elements of larger man-machine system performance simulations. HOMME aids the user in constructing performance procedures and in defining the control interrelations between procedures, as well as in storing, retrieving, and editing procedures generated in earlier simulation efforts. It is intended to serve as an adjunct tool to the more general simulation shell known as HOS-IV.

HOS-IV is a general purpose simulation tool for modeling human operators, systems, and the environment. To build a simulation, inputs to the model typically include descriptions of the system design, procedures for using the system, human operator characteristics, and a mission scenario. A set of operator micromodels are available to the HOS user to assist in the development of the simulation. These micromodels contain algorithms, based on experimental literature, that can predict the timing and accuracy of basic human cognitive, perceptual, and psychomotor actions. This paper describes the current set of human performance models available in HOS-IV as well as micromodels planned for development.

This paper describes the Manpower Constraints Aid (M-CON) and the Personnel Constraints Aid (P-CON). Both aids are being developed under the Army Research Institute's (ARI's) project to develop improved MANPRINT methods. The M-CON Aid and P-CON Aid will assist Army analysts in producing estimates of manpower and personnel constraints during the earliest phases of the acquisition process. These constraints will be incorporated into requirements documents, system specifications, and MANPRINT target audience descriptions.

This paper discusses two of the six software tools which are being developed as part of the Army Research Institute's MANPRINT Methods development program. The first tool discussed here is known as the System Performance and RAM Constraints Aid or SPARC. This tool permits system designers to determine levels of subfunction performance which are required to achieve function and higher level mission requirements. These levels of subfunction and function performance then serve as requirements which are fed into the second tool, the Manpower Systems Evaluation Aid (MAN-SEVAL). MAN-SEVAL takes as input the system design and then predicts the operator and maintainer manpower required to achieve the required levels of task and function performance. For maintenance manpower evaluation, MAN-SEVAL considers component failure rates, time to perform maintenance, and the mission scenario. For operator manpower and to estimate maintenance task times, MAN-SEVAL conducts an analysis of workload, control/display accessability, and maximum acceptable performance time to allocate tasks across crewmembers. Because all manpower requirements are truly driven by system performance requirements, these two tools are being developed collectively with common data bases and software design. While they are currently being developed for the Army, they will be useful general purpose manpower analysis tools.

This paper describes the Personnel-Based System Evaluation Aid (PER-SEVAL), one of the automated tools being developed under the Army Research Institute's project to develop improved MANPRINT methods. The PER-SEVAL Aid will assist Army analysts in identifying the quality of personnel needed to support a partcular contractor's design.

Four operator workload (OWL) scales were retrospectively applied to crewmembers of a mobile air defense missile system, LOS-F(H), following a candidate-selection field evaluation: NASA TLX, SWAT, Overall Workload (OW), and the Modified Cooper-Harper (MCH). Jackknife factor analysis revealed the presence of only a single factor (explaining 79.6% of the total variation) and indicated a significant (p<.0075) ordering of the mean factor loadings: TLX (.935) and OW (.927) were significantly greater than MCH (.862) and SWAT(.860). Comparison with an earlier field test of a remotely piloted vehicle revealed a significant (p<.00005) interaction of test and ordering of the OWL scales, but TLX and MCH consistently had the respectively highest and lowest loadings across the two field tests. Multiple correlation also revealed a significant (p < .0001) relationship, R = 0.66, between system performance and TLX. These findings and lessons learned are discussed in the context of the development and validation of a methodology for assessing OWL.

The central concern of human factors engineering (HFE) is facilitating a productive relationship between man and machine. A new generation of man-machine systems has arisen in which the machine acts in a relatively intelligent manner to enhance the operator's decision-making capabilities in real-time multi-tasking situations. These systems have been termed “distributed intelligence systems” (DIS) because intelligence is distributed among all system entities, whether they are human or computer. The ability of these systems to aid humans in a flexible, interactive fashion depends on the capability of the machine to predict the human's information needs in a given decision-making situation. Thus, the DIS must incorporate a model that reflects the operator's information processing requirements for the tasks necessary to operate the system. To construct this model, it is necessary to develop a DIS testbed where experimental investigations can occur. The mission environment chosen for simulation is the Naval Air Anti-Submarine Warfare (ASW) mission, whose objectives to search for, find, and attack the enemy submarine involve complex tactical decisions in a real-time multi-tasking environment. In the Air ASW mission, most significant tactical decisions are made by the Tactical Coordinator (TACCO), the main operator of the system. The aspects of the testbed discussed in this paper include those elements of the simulation and responsibilities of the TACCO needed to illustrate the types of information processing tasks involved in the ASW mission. Also, the data collection capabilities of the testbed and how this data will be applied to operator model development will be discussed.

This study evaluated the concept of using electronically delivered technical procedures to support maintenance operations in lieu of paper publications. The concept was tested on the basis of effectiveness, feasibility, and human factors issues. The Electronic Maintenance Publication System (EMPS) was tested on the PATRIOT system, an air defense missile system. No significant difference in maintenance time was found between EMPS or paper manuals. Errors committed while performing the tasks were negligible. Human factors issues were considered primarily to evaluate the concept of an electronic delivery and to guide refinement and future development of the system. Based on this study, it was concluded that an electronic delivery of maintenance information (as tested in EMPS) is an effective and feasible alternative to paper publications.

The U.S. Army Tank Automotive Command (TACOM) has developed a Vetronics Crew Display Demonstrator (VCDD) to aid in the design of crew-system interfaces in future land combat vehicles. One major component of such vehicles will be a combat vehicle command and control (CVC²) system, which will include innovative navigation and communication functions. The VCDD has been configured to simulate a range of CVC² system concepts that vary widely in appearance and method of crew-system interaction. Use of the VCDD has provided TACOM with insights into the potential benefits of alternative CVC² system concepts, and will support continued development of CVC² system requirements.

This paper describes the Man-Machine Interface for a militarized radar system. The interface strives to achieve high reliability in terms of both hardware and operator performance, and allows a single operator the ability to control all aspects of the radar system. To accomplish this, a computer controlled touch input design has been assembled, is being tested, and can be fielded in the early 1990's. Reduction in operator fatigue and increases in operator proficiency are combined with the capabilities to minimize required training time and money, provide a system capable of cost effective updates and growth along with the ability for rapid, real time reconfiguration due to failed electronics or changing battlefield conditions.

This paper describes the state and status of human factors within the Space and Naval Warfare Command (SPAWAR) by focusing on a major effort currently being pursued within SPAWAR, that of developing a standard workstation design concept for Navy applications. Human factors concerns were paramount in the assessment of requirements for a standardized workstation applicable to Navy-wide requirements. The major human factors concern was display usability.

Military standards applied to the private sector contracts have a substantial effect on the quality of Computer Based Training (CBT) systems procured for the Naval Air Systems Command. This study evaluated standards regulating the following areas in CBT development and procurement: interactive training systems, cognitive task analysis, and CBT hardware. The objective was to develop some high-level recommendations for evolving standards that will govern the next generation of CBT systems. One of the key recommendations is that there be an integration of the instructional systems development, the human factors engineering, and the software development standards. Recommendations were also made for task analysis and CBT hardware standards.

The U.S. Navy is developing methods for integrating the disciplines concerned with personnel considerations into the weapon system acquisition process. This integration essentially involves human factors engineering, manpower, personnel and training, and life support engineering. Since the Navy already has the HARDMAN methodology in place to ensure that manpower, personnel and training concerns are addressed early in system development, the process of integration of personnel issues will involve expanding the HARDMAN methods and data to include human factors engineering and life support engineering, resulting in the Enhanced HARDMAN process. This paper describes the objectives of Enhanced HARDMAN.

This paper describes the status of human engineering in the Naval Sea Systems Command (NAVSEA). NAVSEA has pursued four major thrusts in the development and application of human engineering technology: 1) human engineering research and development efforts, 2) human engineering front-end analysis, 3) human engineering audits as part of the Logistic Review Group (LRG) formal review of each program, and 4) ship and ship system human engineering design and evaluation. This paper describes the progress that NAVSEA has made in each area.

For the U.S. Army Human Engineering Laboratory (USAHEL), Carlow Associates recently completed development of the MANPRINT Integrated Decision/Engineering Aid (IDEA) for the Program Initiation Phase of system development. The IDEA includes a standard MANPRINT process describing activities, events and products for the Phase, a decision aid and analyst's workstation to support the application of MANPRINT technology to the Program Initiation Phase, and computer-based MANPRINT tools to support the Program Initiation Phase. Specific tools included in IDEA are: 1) an early comparability analysis tool (ECA) which provides baseline system lessons learned and high MANPRINT drivers, 2) an automated task analysis tool, 3) an allocation of function tool which enables and supports the determination of the required role of the soldier in the system, and 4) a workload and human performance simulation tool based on a task network and probabilistic process variables. Efforts involved in the MANPRINT process development addressed the integration of the activities and products of Human Factors Engineering (HFE), Manpower, Personnel and Training (MPT), and System Safety (SS) and Health Hazard Assessment (HHA) with the activities of the Materiel Acquisition Process (MAP), and identification of MANPRINT technology requirements to support the application of this process.

The objective of this paper is to describe the approach utilized in the development of MANPRINT requirements for the Lighter, Amphibious — Heavy Lift (LAMP-H). LAMP-H is an air cushioned vehicle with a crew of six: a pilot, a navigator, and four stevedores who load and unload equipment from the vessel. The project was initiated during the program initiation phase of development. Several types of Air Cushion Vehicle (ACV) lighter craft were evaluated as baseline comparison systems for LAMP-H. The effort involved insuring compliance with human engineering design criteria and practice, incorporating lessons learned from analogous air-cushioned vehicles lighter craft, and addressing habitability, noise and other design issues affecting crew performance of tasks critical to the operation and maintenance of the LAMP-H. This paper details the analyses and techniques implemented in the early phases of the weapon system acquisition process for designing improved soldier-machine systems, as well as the products of the effort.

HARDMAN II, an automated form of HARDMAN (Hardware vs. Manpower) analysis was applied to two new Army air defense systems for purposes of estimating maintainer workload and maintenance manpower requirements. Estimates showed a shortfall in official manpower allocations. Partially as a result of the HARDMAN II analyses, the Army decided to add more maintainers to the organization supporting both systems.

The Workload Assessment Aid (WAA) is a software tool developed for the Army Research Institute as part of the MANPRINT effort. This software toolkit is specifically designed to predict operator workload at the earliest stages of design. It builds upon a task network simulation tool, Micro SAINT, by incorporating several other predictive workload techniques, in addition to several new dimensions. In its final form the tool will automatically make task reallocation recommendations based on workload profiles, personnel characteristics and display-control accessibility.

This paper proposes allocating to user and developer organizations a portion of the human engineering required to design a user interface of a large scale system. The integration of the human engineering activities and the benefits realized by both organizations are discussed.