Predicting User Performance in Augmented Reality User Interfaces with Image Analysis Algorithms

Abstract

This study determines how to apply existing image analysis measures of visual clutter to augmented reality user interfaces, in conjunction with other factors that may affect performance such as the percentage of virtual objects compared to real objects in an interface, and the type of object a user is searching for (real or virtual). Image analysis measures of clutter were specifically chosen as they can be applied to complex and naturalistic images as is common to experience while using an AR UI. The end goal of this research is to develop an algorithm capable of predicting user performance for a given AR UI. In this experiment, twelve participants performed a visual search task of locating a target object in an array of objects where some objects were virtual, and some were real. Participants completed this task under three different clutter levels (low, medium, high) against five different levels of virtual object percentage (0%, 25%, 50%, 75%, 100%) and two types of targets (real, virtual) with repetition. Task performance was measured through response time. Results show significant differences in response time between clutter levels and between virtual object percentage, but not target type. Participants consistently had more difficulty finding objects in more cluttered scenes, where clutter was determined through image analysis methods, and had more difficulty in finding objects when the virtual of objects was at 50% as opposed to other scenarios. Response time positively correlated to measures of combined clutter (virtual and real) arrays but not for measures of clutter taken of the individual array components (virtual or real), and positively correlated with the clutter scores of the target objects themselves.

Get full access to this article

View all access options for this article.

References

Boston

B. N.

Braun

C. C.

(1996). Clutter and Display Conformality: Changes in Cognitive Capture. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 40(2), 57–61. https://doi.org/10.1177/154193129604000211

Bravo

M. J.

Farid

(2004). Search for a Category Target in Clutter. Perception, 33(6), 643–652. https://doi.org/10.1068/p5244

Eriksen

C. W.

Schultz

D. W.

(1979). Information processing in visual search: A continuous flow conception and experimental results. Perception & Psychophysics, 25(4), 249–263. https://doi.org/10.3758/BF03198804

Henderson

J. M.

Chanceaux

Smith

T. J.

(2009). The influence of clutter on real-world scene search: Evidence from search efficiency and eye movements. Journal of Vision, 9(1), 32–32. https://doi.org/10.1167/9.1.32

Lim

J. H.

Tsimhoni

Liu

(2010). Investigation of Driver Performance With Night Vision and Pedestrian Detection Systems—Part I: Empirical Study on Visual Clutter and Glance Behavior. IEEE Transactions on Intelligent Transportation Systems, 11(3), 670–677. https://doi.org/10.1109/TITS.2010.2049843

Duh

H. B. L.

Feiner

Zhao

(2014). Attributes of Subtle Cues for Facilitating Visual Search in Augmented Reality. IEEE Transactions on Visualization and Computer Graphics, 20(3), 404–412. https://doi.org/10.1109/TVCG.2013.241

Mack

Oliva

(2004). Computational estimation of visual complexity. The 12th Annual Object, Perception, Attention, and Memory Conference.

Moacdieh

N. M.

Prinet

J. C.

Sarter

N. B.

(2013). Effects of Modern Primary Flight Display Clutter: Evidence from Performance and Eye Tracking Data. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 11–15. https://doi.org/10.1177/1541931213571005

Moacdieh

N. M.

Sarter

N. B.

(2012). Eye Tracking Metrics: A Toolbox for Assessing the Effects of Clutter on Attention Allocation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 1366–1370. https://doi.org/10.1177/1071181312561391

10.

Moacdieh

Sarter

(2015). Clutter in Electronic Medical Records: Examining Its Performance and Attentional Costs Using Eye Tracking. Human Factors: The Journal of the Human Factors and Ergonomics Society, 57(4), 591–606. https://doi.org/10.1177/0018720814564594

11.

Neider

M. B.

Zelinsky

G. J.

(2011). Cutting through the clutter: Searching for targets in evolving complex scenes. Journal of Vision, 11(14), 7–7. https://doi.org/10.1167/11.14.7

12.

Rosenholtz

Nakano

(2005). Feature Congestion 1.0: A Measure of Visual Clutter. Journal of Vision, 5(1), 1. https://doi.org/10.1167/5.1.1

13.

Rosenholtz

Nakano

(2007). Measuring visual clutter. Journal of Vision, 7(2), 17–17. https://doi.org/10.1167/7.2.17

14.

Rotman

S. R.

Tidhar

Kowalczyk

M. L.

(1994). Clutter metrics for target detection systems. IEEE Transactions on Aerospace and Electronic Systems, 30(1), 81–91. https://doi.org/10.1109/7.250408

15.

St. John

Smallman

H. S.

Manes

D. I.

Feher

B. A.

Morrison

J. G

. (2005). Heuristic Automation for Decluttering Tactical Displays. Human Factors, 47(3), 509–525. https://doi.org/10.1518/001872005774860014

16.

Stedmon

A. W.

Kalawsky

R. S.

Hill

Cook

C. A.

(1999). Old theories, new technologies: Cumulative clutter effects using augmented reality. 1999 IEEE International Conference on Information Visualization (Cat. No. PR00210), 132–137. https://doi.org/10.1109/IV.1999.781549

17.

Wickens

C. D.

Alex

A. L.

Horrey

W. J.

Nunes

Hardy

T. J.

(2004). Traffic and flight guidance depiction on a Synthetic Vision System display: The effects of clutter on performance and visual attention allocation.

18.

Wickens

C. D.

Kroft

Yeh

(2000). Data Base Overlay in Electronic Map Design: Testing a Computational Model. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 44(21), 3–451. https://doi.org/10.1177/154193120004402119

19.

Wolfe

J. M.

Oliva

Horowitz

T. S.

Butcher

S. J.

Bompas

(2002). Segmentation of objects from backgrounds in visual search tasks. Vision Research, 42(28), 2985–3004. https://doi.org/10.1016/S0042-6989(02)00388-7