The Use of Paired Comparisons for Evaluating Complex Route Matching Performance in a Spatial Awareness Task

Abstract

A method is proposed for evaluating participants’ performance in a global spatial awareness task involving identification of complex winding routes. Rather than using coarse measures of spatial error, the method of paired comparisons employs impartial judges to compare sets of aggregated experimental data generated by the participants with respect closeness in shape to the target route. The method was applied to a set of data in an experimental investigation of the effect of height on a participant’s ability to identify the route he had just flown over (in a 20 second video). Seven participants performed a total of 48 trials in a 4 (heights) X 2 (trial blocks) within-subjects experiment. Results from the paired comparison analysis suggested that height had a statistically significant effect on correct route identification. Of equal importance in the context of the present paper, the method of paired comparison analysis proved to be effective in quantifying performance data that did not otherwise lend themselves to conventional methods of quantification.

Get full access to this article

View all access options for this article.

References

Bolton

M. L.

Bass

E. J.

Comstock

J. R.

(2007). Spatial Awareness in Synthetic Vision Systems: Using Spatial and Temporal Judgments to Evaluate Texture and Field of View. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(6), 961–974.

Burros

(1951). The application of the method of paired comparisons to the study of reaction potential. Psychological review, 58(1), 60–6.

David

(1959). The method of paired comparisons. Proceedings of the Fifth Conference on the Design of Experiments in Army Research Development and Testing (pp. 1–16).

Dunn-Rankin

Knezek

G. A.

Wallace

S. R.

Zhang

(2004). Scaling methods. Lawrence Erlbaum.

Edwards

(1957). Techniques of attitude scale construction. New York, NY: Appleton-Century-Crofts, Inc.

Golledge

(1999). Wayfinding Behavior: Cognitive Mapping and Other Spatial Processes. Baltimore, MD: The Johns Hopkins University Press.

Gridgeman

(1963). Significance and adjustment in paired comparisons. Biometrics, 19(2), 213–228.

Jackson

Fleckenstein

(1957). An evaluation of some statistical techniques used in the analysis of paired comparison data. Biometrics, 13(1), 51–64.

Mosteller

(1951). Remarks on the method of paired comparisons: I. The least squares solution assuming equal standard deviations and equal correlations. Psychometrika, 16(1), 3–9.

10.

Ross

(1934). Optimum orders for the presentation of pairs in the method of paired comparisons. Journal of Educational Psychology, 375–382.

11.

Stager

(1974). Visual search capability in Search And Rescue (SAR). Defence and Civil Institute of Environmental Medicine.

12.

Starks

David

(1961). Significance tests for paired-comparison experiments. Biometrika, 48(1), 95–108.

13.

Tan

D. S.

Gergle

Scupelli

P. G.

Pausch

(2004). Physically large displays improve path integration in 3D virtual navigation tasks. Proceedings of the 2004 conference on Human factors in computing systems - CHI ‘04, 6(1), 439–446.

14.

Thurstone

(1927). A law of comparative judgment. Psychological review, (34), 273–286.

15.

Whiteside

Samuel

(1970). Blur zone. Nature, 225, 94–95.

16.

Woods

D.D.

(1984). Visual momentum: a concept to improve the cognitive coupling of person and computer. Int. J. Man-Machine Studies, 21, 229-244.

17.

Woods

(2010). Statistical analysis of subjective preferences for video enhancement. Proceedings of SPIE - Human Vision and Electronic Imaging XV (Vol. 7527, pp. 1–10).