Rule-Based Analysis of Pilot Decisions

Abstract

Rule identification is proposed as an alternative to parameter estimation in the analysis of human performanc data. The relation between the choice of language and identifiable consistencies is discussed. Advantages of production system models for the description of complex human behaviors are examined. Threats to validity posed by the use of flexible languages in data analysis are examined. Contrivedness, defined by Eilbert and Christensen 82) as, “ the tendency of a search procedure to uncover apparent patterns where none exist”, is advanced as the major inferential hazard to rule identification. A Monte Carlo significance testing procedure to deal with this threat is proposed. Nonparametric measures of relation, tau-b and PRE, are presented as appropriate performance measures for rule-based models. A rule-based analysis of data from an experiment (Palmer, 1983) involving pilot decisions in air traffic conflicts is presented. Identified rules indicate satisficing with respect to the secondary goal of maintaining a constant course. Reduction in dimensionality of utilized information led to errors for a subgroup of pilots.

Get full access to this article

View all access options for this article.

References

Eilbert

R.F.

Christensen

R.A.

“Contrivedness: The boundary between pattern recognition and numerology”, Pattern Recognition, 1982, 253–261.

Ellis

S.R.

Palmer

E.A.

“Threat perception while viewinq single conflicts on a cockpit display of traffic information“, (Report No. 81341) Moffett Field, Ca.: NASA-Ames Research Center, 1982.

Forsythe

A.B

Laszlo

Jennrich

May

P.R.A.

“A Stopping Rule for Variable Selection in Multiple Regression”, Journal of the American Statistical Association, 1973, 75–77.

Hunt

R.W.

Rouse

W.B.

“A fuzzy rule-based model of human problem solving”, IEEE Transactions on Systems, Man, and Cybernetics. 14, 1984, 112–119.

Lester

P.T.

Quan

E.E.

“The cockpit display of traffic information and the threat alert and collision avoidance system integration: a review”, Proceedings of the Second Symposium on Aviation Psychology, Columbus, Ohio: Aviation Psychology Laboratory, Ohio State University, 1983.

Lewis

C.M.

Hammer

J.M.

“Significance testing of rules in rule-based models of human problem solving”, to appear in: IEEE Transactions on Systems, Man, and Cybernetics.

Margolin

B.H.

Light

R.J.

“An analysis of variance for categorical data II: Small sample comparisons with chi-square and other competitors”, Journal of the American Statistical Association. 69, 1974.

Michalski

Ryszard S.

“Pattern recognition as rule guided inference”, IEEE Transactions on Pattern Analysis and Machine Intelligence. 4, 1982, 349–361.

Michalski

R.S.

Chilausky

R.L.

“Learning by being told and learning from examples: An experimental comparison of the two method of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis”, Policy Analysis and Information Systems, 1980, 125–160.

10.

Mitchell

T. M.

“Version spaces: a candidate elimination approach to rule learning”, Proceedings of the Fifth International Joint Conference on Artificial Intelligence. IJCAI, Cambridge, Mass., 1977, 305–310.

11.

Newell

Simon

H.A.

, Human Problem Solving. Englewood Cliffs, N.J.: Prentice-Hall, 1972.

12.

Palmer

“Conflict resolution maneuvers during near miss encounters with cockpit traffic displays”, Proceedings of the Human Factors Society-27th Annual Meeting, Santa Monica, Ca.: Human Factors Society, 757–761, 1983.

13.

Palmer

Jago

DuBord

“Horizontal conflict maneuvers with a cockpit display of traffic information”. Proceedinqs of the Seventeenth Annual Conference on Manual Control, Los Angeles, Ca.: UCLA, 1981.

14.

Smith

J.D.

Ellis

S.R.

Lee

C.E.

“Perceived threat and avoidance maneuvers in response to cockpit traffic displays”, Human Factors. Santa Monica, Ca.: Human Factors Society, 33–48, 26, 1984.

15.

Wesson

R.B.

“Planning in the world of the air traffic controller”, Proceedings of the 5th International Joint Conference on Artificial Intelligence. M.I.T., Cambridge, Mass., 1977, 473–479.