Perspectives on Validating Complex Human-Machine System Performance

Abstract

The performance of complex systems, such as power plants and commercial aircraft, is based on the integration of human, software, and hardware elements. They are designed to operate safely under normal and disturbance conditions. Ideally one would validate that safety and productivity goals can be achieved prior to actual operation. While some types of systems can be tested in actual operational environments, validation is especially difficult when safety or economic considerations preclude testing the types of disturbance conditions that the integrated system is designed to handle. Thus the conceptual and methodological challenges to validating such systems are significant and new approaches are emerging. The papers in this symposium will describe approaches taken to validation using examples from commercial nuclear power, civil aviation, military, and medical systems.

Get full access to this article

View all access options for this article.

References

IEEE. (1988). IEEE Guide to the Application of Human Factors Engineering to Systems, Equipment, and Facilities of Nuclear Power Generating Stations (IEEE Std. 1023–1988). New York: Institute of Electrical and Electronics Engineers.

International Electrotechnical Commission, IEC (1995). Nuclear power plants main control rooms: Verification and validation designs (IEC Standard 1771). Bureau Central de la Commission Electrotechnique Internationale, Geneva, Switzerland.

Meister

(1986). Human factors in testing and evaluation. Elsevier, Amsterdam.

O'Hara

(1999). A quasi-experimental model of complex human-machine system validation. International Journal of Cognition, Technology, and Work, 1, 37–46.

Perrow

(1984). Normal accidents. Basic Books, New York.

Rasmussen

(1988). Coping safely with complex systems. Paper presented to the American Association for the Advancement of Science, Boston, MA.

Vicente

(1992). Multilevel interfaces for power plant control rooms I: An integrative review. Nuclear Safety, 33: 381–397.

Vicente

Rasmussen

(1992). Ecological interface design: Theoretical foundations. In IEEE Transactions on Systems, Man, and Cybernetics, 2: 589–606.

Wise

Hopkin

Stager

Harwood

(1994). Human factors certification of systems. In Proceedings of the Human Factors Society 38th Annual Meeting. Human Factors and Ergonomics Society, Santa Monica.

10.

Wise

Hopkin

Stager

(1993). Verification and validation of complex systems: Human factors issues (NATO ASI Series F, Vol. 110). Springer-Verlag, Berlin.

11.

U.S. Department of Defense. (1979). Human Engineering Requirements for Military Systems, Equipment and Facilities (MIL-H-46855B). Washington, D.C.: Office of Management and Budget.