This article describes progress in providing user models with sufficient visual information and motor control to perform teleoperation with an unmodified, physically realized robot. User models that are built by extending cognitive models to interact directly with interfaces can provide a theoretical basis for predicting user behavior. These models can help in summarizing and explaining usability issues in domains for which conventional user testing is too time consuming, too demanding of other resources, or too dynamic for static models. The user model consists of an ACT-R cognitive model and the SegMan image-processing and interpretation system. ACT-R supports directing simple rover navigation and response-time predictions. SegMan supports interpreting many aspects of HCI interfaces and can now interpret simple aspects of video used in simple navigation tasks and can generate key presses and mouse actions directly. Processing limited amounts of an image as a human fovea helped make this system work in real time. A study in robot teleoperation provides evidence that the cognitive and perceptual-motor model approximates human behavior (based on comparison of task time, learning behavior, and mouse actions) in a simple navigation task. This work demonstrates how user modeling techniques are maturing to the extent that they can be used for assessing interfaces for dynamic tasks by predicting performance during teleoperation, a common human-robot interaction task.
Get full access to this article
View all access options for this article.
References
1.
AndersonJ. R.LebiereC. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum.
2.
ByrneM. D. (2001). ACT-R/PM and menu selection: Applying a cognitive architecture to HCI. International Journal of Human-Computer Studies, 55(1), 41–84.
3.
ByrneM. D.GrayW. D. (2003). Returning human factors to an engineering discipline: Expanding the science base through a new generation of quantitative methods (special section preface). Human Factors, 45, 1–4.
4.
ByrneM. D.KirlikA. (2005). Using computational cognitive modeling to diagnose possible sources of aviation error. International Journal of Aviation Psychology, 15, 135–155.
5.
CasperJ.MurphyR. (2003). Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Transactions on Systems Man and Cybernetics Part B, 33, 367–385.
CradallJ. W.NielsenC. W.GoodrichM. A. (2003). Towards predicting robot team performance. Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (pp. 906–911). Piscataway, NJ: IEEE.
8.
DruryJ. L.HestandD.YancoH. A.ScholtzJ. (2004). Design guidelines for improved human-robot interaction. Proceedings of CHI, Extended Abstracts 2004 (p. 1540). New York: Association for Computing Machinery.
9.
GluckK. A.BallJ. T.KrusmarkM. A.RodgersS. M.PurteeM. D. (2003). A computational process model of basic aircraft maneuvering. Proceedings of the Fifth International Conference on Cognitive Modeling (pp. 117–122). Bamberg, Germany: Universitäts-Verlag Bamberg.
10.
GluckK. A.PewR. W. (Eds.) (2005). Modeling human behavior with integrated cognitive architectures: Comparison, evaluation, and validation. Mahwah, NJ: Erlbaum.
11.
GrantD. A. (1962). Testing the null hypothesis and the strategy and tactics of investigating theoretical models. Psychological Review, 69(1), 54–61.
12.
GrayW. D. (Ed.) (2007). Integrated models of cognitive systems. New York: Oxford University Press.
13.
GrayW. D.JohnB. E.AtwoodM. E. (1993). Project Ernestine: Validating a GOMS analysis for predicting and explaining real-world task performance. Human-Computer Interaction, 8, 237–309.
14.
GunzelmannG.AndersonJ. R. (2002). Strategic differences in the coordination of different views of space. Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society (pp. 387–392). Mahwah, NJ: Erlbaum.
15.
HalelamienN. S. (2004). Visual routines for spatial cognition on a mobile robot. Master's thesis, Department of Computer Science, Carnegie-Mellon University.
16.
HarrisonA. M.SchunnC. D. (2003). ACT-R/S: Look Ma, no “cognitive-map” Proceedings of the Fifth International Conference on Cognitive Modeling (pp. 129–134). Bamberg, Germany: Universitäts-Verlag Bamberg.
17.
JohnB. E.NewellA. (1989). Cumulating the science of HCI: From S-R compatibility to transcription typing. Proceedings of CHI, 1989 (pp. 109–114). New York: Association for Computing Machinery.
18.
JohnsonT. R.WangH.ZhangJ. (2003). An ACT-R model of human object-location memory. Proceedings of the 25th Annual Meeting of the Cognitive Science Society (p. 1361). Mahwah, NJ: Erlbaum.
19.
KaberD. B.WangX.KimS.-H. (2006). Computational cognitive modeling of operator behavior in telerover navigation. Proceedings of the 2006 IEEE International Conference on Systems, Man, and Cybernetics (CD-ROM). Taipei, Taiwan: IEEE.
20.
KierasD. E. (1985). The why, when, and how of cognitive simulation. Behavior Research Methods Instrumentation and Computers, 17, 279–285.
21.
KierasD. E.WoodS. D.AbotelK.HornofA. (1995). GLEAN: A computer-based tool for rapid GOMS model usability evaluation of user interface designs. Proceedings of the ACM Symposium on User Interface Software and Technology (pp. 91–100). New York: Association for Computing Machinery.
22.
KierasD. E.WoodS. D.MeyerD. E. (1995). Predictive engineering models using the EPIC architecture for a high-performance task. Proceedings of the CHI ′95 Conference on Human Factors in Computing Systems (pp. 11–18). New York: Association for Computing Machinery.
23.
KierasD. E.WoodS. D.MeyerD. E. (1997). Predictive engineering models based on the EPIC architecture for a multimodal high-performance human-computer interaction task. Transactions on Computer-Human Interaction, 4, 230–275.
24.
KukrejaU. (2004). Towards model-based evaluations of human-robot interfaces. Unpublished MS thesis, Department of Computer Science and Engineering, Pennsylvania State University.
25.
KukrejaU.StevensonW. E.RitterF. E. (2006). RUI — Recording user input from interfaces under Windows and Mac OS X. Behavior Research Methods, 38, 656–659.
26.
LewisM.WangJ.HughesS. (2007). USARSim: Simulation for the study of human-robot interaction. Journal of Cognitive Engineering and Decision Making, 1(1), 98–120.
27.
MarbleJ. L.BruemmerD. J.FewD. A.DudenhoefferD. D. (2004). Evaluation of supervisory vs. peer-peer interaction with human-robot teams. HICSS 2004 (p. 50–130). Piscataway, NJ: IEEE Computer Society.
28.
MarrD. (1982). Vision. San Francisco: W. H. Freeman.
29.
MeyerD. E.KierasD. (1997). A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. Psychological Review, 104(1), 3–65.
30.
MurphyR. R.BlitchJ.CasperJ. (2002). AAAI/RoboCup-2001 urban search and rescue events: Reality and competition. AI Magazine, 23(1), 37–42.
31.
NewellA. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
32.
NewellA.SimonH. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
33.
NielsenJ. (1993). Usability engineering. Chestnut Hill, MA: AP Professional Press.
RitterF. E. (2004). Choosing and getting started with a cognitive architecture to test and use human-machine interfaces. MMI-Interaktiv-Journal, 7, 17–37.
36.
RitterF. E.BaxterG. D.JonesG.YoungR. M. (2000). Supporting cognitive models as users. ACM Transactions on Computer-Human Interaction, 7, 141–173.
37.
RitterF. E.LarkinJ. H. (1994). Using process models to summarize sequences of human actions. Human-Computer Interaction, 9, 345–383.
38.
RitterF. E.MajorN. P. (1995, Spring). Useful mechanisms for developing simulations for cognitive models. AISB Quarterly, 91, 7–18.
39.
RitterF. E.Van RooyD.AmantR. (2002). A user modeling design tool based on a cognitive architecture for comparing interfaces. Computer-Aided Design of User Interfaces III, Proceedings of the 4th International Conference on Computer-Aided Design of User Interfaces CADUI′2002 (pp. 111–118). Dordrecht, Netherlands: Kluwer.
40.
RitterF. E.Van RooyD.AmantR.SimpsonK. (2006). Providing user models direct access to interfaces: An exploratory study of a simple interface with implications for HRI and HCI. IEEE Transactions on Systems Man and Cybernetics Part A: Systems and Humans, 36, 592–601.
41.
RoelfsemaP. R. (2005). Elemental operations in vision. Trends in Cognitive Sciences, 9(5), 226–233.
42.
RussellS.NorvigP. (1995). AI: A modern approach. Englewood Cliffs, NJ: Prentice-Hall.
43.
SalvucciD. (2001). Predicting the effects of in-car interface use on driver performance: An integrated model approach. International Journal of Human-Computer Studies, 55(1), 85–107.
44.
SchoellesM. J.GrayW. D. (2001). Argus: A suite of tools for research in complex cognition. Behavior Research Methods Instruments & Computers, 33(2), 130–140.
ScholtzJ. (2003). Theory and evaluation of human robot interactions. System Sciences, 2003. Proceedings of the 36th Annual Hawaii International Conference on System Sciences (p. 125a), Piscataway, NJ: IEEE Computer Society.
ScholtzJ.AntonishekB.YoungJ. (2004). Evaluation of a human-robot interface: Development of a situational awareness methodology. Proceedings of the 37th Annual Hawaii International Conference on System Sciences Track5, 50–130c). IEEE Computer Society, Piscataway, NJ: IEEE.
49.
AmantR.HortonT. E.RitterF. E. (2004). Model-based evaluation of cell phone menu interaction. Proceedings of the CHI′04 Conference on Human Factors in Computer Systems (pp. 343–350). New York: Association for Computing Machinery.
50.
AmantR.HortonT. E.RitterF. E. (2007, May). Model-based evaluation of expert cell phone menu interaction. ACM Transactions on Computer-Human Interaction.14(1).
51.
AmantR.RiedelM. O.RitterF. E.ReifersA. (2005). Image processing in cognitive models with SegMan. Proceedings of HCI International ′05, Vol. 4: Theories Models and Processes in HCI Paper # 1869. Mahwah, NJ: Erlbaum.
52.
AmantR.RiedlM. O. (2001). A perception/action substrate for cognitive modeling in HCI. International Journal of Human-Computer Studies, 55(1), 15–39.
53.
ThimblebyH. (2002). Reflections on symmetry. Proceedings of Advanced Visual Interfaces (pp. 28–33), New York: ACM Press.
54.
TraftonJ. G.SchultzA. C.PerznowskiD.BugajskaM. D.AdamsW.CassimatisN. L.BrockD. P. (2006). Children and robots learning to play hide and seek. Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (pp. 242–249). New York: Association for Computing Machinery.
55.
TreismanA. M.GeladeG. (1980). A feature integration theory of attention. Cognitive Psychology, 12, 97–136.
56.
UllmanS. (1989). Aligning pictorial descriptions: An approach to object recognition. Cognition, 32, 193–254.
57.
UllmanS. (1996). High-level vision. Cambridge, MA: MIT Press.
58.
UrbasL.LeuchterS. (2005). Model based analysis and design of human-machine dialogues through displays. KI - Zeitschrift für künstliche Intelligenz [AI-Journal for AI], 45–51.
59.
YancoH. A.DruryJ. L.ScholtzJ. (2004). Beyond usability evaluation: Analysis of human-robot interaction at a major robotics competition. Human-Computer Interaction, 19(12), 117–149.
