A Comparison of Test Methodologies for Air Traffic Control Systems

Abstract

The Federal Aviation Administration (FAA) is currently involved in a tremendous effort to upgrade the nation's Air Traffic Control (ATC) system. Included in this effort is a new communications system called the Voice Switching and Control System (VSCS) for use primarily in the Air Route Traffic Control Centers (ARTCCs). The critical nature of the communications system, the complex Computer Human Interface (CHI) and functionality, and technology differences from the existing communications system caused the FAA to involve an Air Traffic Controller user group early in the design process. This group of subject matter experts was intended to help ensure adequate operational testing which would help secure user acceptance when VSCS was fielded. This paper discusses the methodology developed for the operational evaluation of the prototype VSCS in the Design Competition Phase (DCP) of the program. The methodology exposed problems in the areas of specification compliance, system stability, voice quality, CHI and operational suitability beyond those identified by the formal development test program called Factory Acceptance Testing (FAT). It was discovered that individual system functions that worked properly when exercised in isolation during FAT did not always work properly when exercised in realistic operational scenarios by controllers. The comparison of results between FAT and the operational evaluations show that controller evaluations should be included throughout the development process of FAA programs, and that the methodology used plays a fundamental role in uncovering issues.