Sage Journals: Discover world-class research

Abstract

Exploring user interaction in specific contexts is often based on simulated environments and semi-functional prototypes of interactive systems. In this paper, we address a combination of context simulation with the Wizard of Oz method, where a human “wizard” simulates missing functionalities or system intelligence. The goal of our work is to provide a software framework for fast and flexible prototyping and concurrent evaluation during user studies embedded in an iterative interaction design processes. Contextual interaction research is particularly challenging in high-dynamic interaction contexts like ambient environments and includes the simulation of various context parameters to elaborate interaction designs in the target context. For this purpose, we have developed a prototyping software framework that allows to setup and handle different contextual situations during user studies. The framework is highly flexible, modular and adjustable at runtime. The modularity allows the application in a big variety of study contexts in in-situ and in-vitro settings. A detailed description of the framework’s requirements and architecture is presented in this paper. The framework is evaluated from two perspectives. We show the applicability of the framework by using it in two different settings including a simulated factory and a pedestrian navigation context. We evaluate the framework with different user groups, namely developers, researchers, and wizards, who conduct contextual user studies. Results are discussed and the insights are presented in this paper. The challenges posed by settings in different contextual environments provides us with in-depth insights about the usage of the ConWIZ – Wizard of Oz framework. The identified improvements and potentials experienced during the usage of the framework have been analyzed and provide valuable findings for further iterations.

Keywords

Wizard of Oz context simulation prototyping human computer interaction study support

Get full access to this article

View all access options for this article.

References

[1]

Alce,

Hermodsson and

Wallergård, WozARd: A Wizard of Oz tool for mobile AR, in: Proc. of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, ACM, 2013, pp. 600–605.

[2]

Angermann, MATLAB-Simulink-Stateflow: Grundlagen, Toolboxen, Beispiele, Oldenbourg Wissenschaftsverlag, 2007.

[3]

Ardito,

Buono,

M.F.

Costabile,

Lanzilotti and

Piccinno, A tool for Wizard of Oz studies of multimodal mobile systems, in: Proc. of the 2nd Conference on Human System Interactions (HSI’09), IEEE Press, Piscataway, NJ, USA, 2009, pp. 341–344.

[4]

J.J.

Barton and

V.V.

UBIWISE, A ubiquitous wireless infrastructure simulation environment, Technical report, 2002.

[5]

Bellucci,

Bottoni and

S.L.

WOEB, Rapid setting of Wizard of Oz experiments and reuse for deployed applications, Technical report, Dipartimento di Informatica, Universita Sapienza di Roma, Italy, 2009.

[6]

Benzmüller,

Horacek,

Kruijff-Korbayová,

Lesourd,

Schiller and

Wolska, DiaWOz-II – A tool for Wizard-of-Oz experiments in mathematics, in: KI 2006: Advances in Artificial Intelligence,

Freksa,

Kohlhase and

Schill, eds, Lecture Notes in Computer Science, Vol. 4314, Springer, Berlin, Heidelberg, 2007, pp. 159–173.

[7]

Cavalluzzi,

Clarizio,

B.N.

De Carolis and

de Rosis, A Persona is not a Person: Designing dialogs with ECAs after Wizard of Oz studies, Technical report, HUMAINE WP6, Workshop on Interaction and Communication, 2005.

[8]

Coutaz,

Salber,

Carraux and

Portolan, NEIMO, a multiworkstation usability lab for observing and analyzing multimodal interaction, in: Conference Companion on Human Factors in Computing Systems (CHI’96), ACM, New York, NY, USA, 1996, pp. 402–403.

[9]

R.C.

Davis,

T.S.

Saponas,

Shilman and

J.A.

Landay, SketchWizard: Wizard of Oz prototyping of pen-based user interfaces, in: Proc. of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST’07), ACM, New York, NY, USA, 2007, pp. 119–128.

10.

[10]

Dey and

Abowd, The context toolkit: Aiding the development of context-aware applications, in: Workshop on Software Engineering for Wearable and Pervasive Computing, Limerick, Ireland, 2000.

11.

[11]

Dey,

Sohn,

Streng and

Kodama, ICAP: Interactive prototyping of context-aware applications, in: Pervasive Computing, Lecture Notes in Computer Science, Vol. 3968, Springer, 2006, pp. 254–271.

12.

[12]

Diaper, The wizard’s apprentice: A program to help analyse natural language dialogues, in: Proc. of the Fifth Conference of the British Computer Society, Human-Computer Interaction Specialist Group on People and Computers V, Cambridge University Press, New York, NY, USA, 1989, pp. 231–243.

13.

[13]

Dow,

MacIntyre,

Lee,

Oezbek,

Bolter and

Gandy, Wizard of Oz support throughout an iterative design process, IEEE Pervasive Computing 4(4) (2005), 18–26.

14.

[14]

Dow,

MacIntyre,

Lee,

Oezbek,

J.D.

Bolter and

Gandy, Wizard of Oz support throughout an iterative design process, IEEE Pervasive Computing 4 (2005), 18–26.

15.

[15]

Drugge,

Hallberg,

Parnes and

Synnes, Wearable systems in nursing home care: Prototyping experience, IEEE Pervasive Computing 5(1) (2006), 86–91.

16.

[16]

Green,

Huttenrauch and

Eklundh, Applying the Wizard-of-Oz framework to cooperative service discovery and configuration, in: 13th IEEE International Workshop on Robot and Human Interactive Communication (ROMAN 2004), 2004, pp. 575–580.

17.

[17]

Grill,

Osswald and

Tscheligi, Task complexity and user model attributes, in: Computers Helping People with Special Needs,

Miesenberger,

Karshmer,

Penaz and

Zagler, eds, Lecture Notes in Computer Science, Vol. 7382, Springer, Berlin, Heidelberg, 2012, pp. 642–649.

18.

[18]

Grill,

Polacek and

Tscheligi, ConWIZ: A tool supporting contextual Wizard of Oz simulation, in: Proc. of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM’12), ACM, New York, NY, USA, 2012, pp. 21:1–21:8.

19.

[19]

Grill,

Polacek and

Tscheligi, Methods towards API usability: A structural analysis of usability problem categories, in: Proc. of HCSE 2012, 2012.

20.

[20]

Grill,

Reitberger,

Obrist,

Meschtscherjakov and

Tscheligi, The Christian Doppler laboratory on contextual interfaces, in: Proc. of the First International Joint Conference on Ambient Intelligence (AmI’10), Springer-Verlag, Berlin, Heidelberg, 2010, pp. 325–332.

21.

[21]

Grill and

Tscheligi, The ConWIZ protocol: A generic protocol for Wizard of Oz simulations, in: EuroCast, Springer, 2013.

22.

[22]

Hanselman and

B.C.

Littlefield, Mastering MATLAB 5: A Comprehensive Tutorial and Reference, Prentice Hall PTR, 1997.

23.

[23]

Hosseini-Khayat,

Ghanam,

Park and

Maurer, ActiveStory enhanced: Low-fidelity prototyping and Wizard of Oz usability testing tool, in: Agile Processes in Software Engineering and Extreme Programming,

Abrahamsson,

Marchesi and

Maurer, eds, Lecture Notes in Business Information Processing, Vol. 31, Springer, Berlin, Heidelberg, 2009, pp. 257–258.

24.

[24]

J.F.

Kelley, An empirical methodology for writing user-friendly natural language computer applications, in: Proc. of CHI’83, ACM, NY, USA, 1983, pp. 193–196.

25.

[25]

S.R.

Klemmer,

A.K.

Sinha,

Chen,

J.A.

Landay,

Aboobaker and

Wang, Suede: A Wizard of Oz prototyping tool for speech user interfaces, in: User Interface Software and Technology, 2000, pp. 1–10.

26.

[26]

Kurz and

Ferscha, Sensor abstractions for opportunistic activity and context recognition systems, in: Proc. of EuroSSC 2010,

Lukowicz,

Kunze and

Kortuem, eds, Lecture Notes in Computer Science, Springer, 2010, pp. 135–149.

27.

[27]

Lee and

Billinghurst, A Wizard of Oz study for an AR multimodal interface, in: Proc. of the 10th International Conference on Multimodal Interfaces (ICMI’08), ACM, New York, NY, USA, 2008, pp. 249–256.

28.

[28]

Li,

J.I.

Hong and

J.A.

Landay, Topiary: A tool for prototyping location-enhanced applications, in: Proc. of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST’04), ACM, New York, NY, USA, 2004, pp. 217–226.

29.

[29]

Li,

J.I.

Hong and

J.A.

Landay, Design challenges and principles for wizard of oz testing of location-enhanced applications, IEEE Pervasive Computing 6(2) (2007), 70–75.

30.

[30]

M.-W.

Lin,

Y.-M.

Cheng and

Yu, Using tactons to provide navigation cues in pedestrian situations, in: Proc. of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges (NordiCHI’08), ACM, New York, NY, USA, 2008, pp. 507–510.

31.

[31]

Liu and

Khooshabeh, Paper or interactive? A study of prototyping techniques for ubiquitous computing environments, in: CHI’03 Extended Abstracts on Human Factors in Computing Systems (CHI EA’03), ACM, New York, NY, USA, 2003, pp. 1030–1031.

32.

[32]

D.V.

Lu and

W.D.

Smart, Polonius: A Wizard of Oz interface for HRI experiments, in: Proc. of the 6th International Conference on Human-Robot Interaction (HRI’11), ACM, New York, NY, USA, 2011, pp. 197–198.

33.

[33]

Lyons,

Skeels and

Starner, Providing support for mobile calendaring conversations: A Wizard of Oz evaluation of dual-purpose speech, in: Proc. of MobileHCI 2005 (MobileHCI’05), ACM, New York, NY, USA, 2005, pp. 243–246.

34.

[34]

Macintyre,

Gandy,

Dow and

J.D.

Bolter, Dart: A toolkit for rapid design exploration of augmented reality experiences, in: Proc. of UIST’04, 2004, pp. 197–206.

35.

[35]

Magnusson,

Pielot,

Anastassova,

Rassmus-Gröhn,

Tollmar and

Roselier, The mobile Oracle: A tool for early user involvement, in: Proc. of MobileHCI 2009, ACM, 2009, p. 84.

36.

[36]

Mäkelä,

E.-P.

Salonen,

Turunen,

Hakulinen and

Raisamo, Conducting a Wizard of Oz experiment on a ubiquitous computing system doorman, in: Proc. of IPNMD 2001, 2001.

37.

[37]

Maulsby,

Greenberg and

Mander, Prototyping an intelligent agent through Wizard of Oz, in: Computer Human Interaction, 1993, pp. 277–284.

38.

[38]

Maulsby,

Greenberg and

Mander, Prototyping an intelligent agent through Wizard of Oz, in: Proc. of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems (CHI’93), ACM, New York, NY, USA, 1993, pp. 277–284.

39.

[39]

M.W.

Newman,

M.S.

Ackerman,

Kim,

Prakash,

Hong,

Mandel and

Dong, Bringing the field into the lab: Supporting capture and replay of contextual data for the design of context-aware applications, in: Proc. of UIST’10, ACM, New York, NY, USA, 2010, pp. 105–108.

40.

[40]

Obrenovic and

J.-B.

Martens, Sketching interactive systems with sketchify, ACM Transactions on Computer-Human Interaction 18 (2011), 4.

41.

[41]

P.P.

Piernot,

R.M.

Felciano,

Stancel,

Marsh and

Yvon, Designing the PenPal: Blending hardware and software in a user-interface for children, in: Proc. of CHI’95, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 1995, pp. 511–518.

42.

[42]

Polacek,

Grill and

Tscheligi, Towards a navigation system for blind people: A Wizard of Oz study, SIGACCESS Accessibility and Computing 104 (2012), 12–29.

43.

[43]

Rogers, Interaction design gone wild: Striving for wild theory, Interactions 18(4) (2011), 58–62.

44.

[44]

Salber and

Coutaz, Applying the Wizard of Oz technique to the study of multimodal systems, in: Human-Computer Interaction, Lecture Notes in Computer Science, Vol. 753, Springer, Berlin, Heidelberg, 1993, pp. 219–230.

45.

[45]

Schlögl,

Doherty,

Karamanis and

Luz, WebWOZ: A Wizard of Oz prototyping framework, in: Proc. of EICS 2010, ACM, New York, NY, USA, 2010, pp. 109–114.

46.

[46]

Serrano and

Nigay, A Wizard of Oz component-based approach for rapidly prototyping and testing input multimodal interfaces, Journal on Multimodal User Interfaces 3 (2010), 215–225.

47.

[47]

Serrano,

Nigay,

Lawson,

Ramsay,

Murray-Smith and

Denef, The openinterface framework: A tool for multimodal interaction, in: CHI’08 Extended Abstracts on Human Factors in Computing Systems, ACM, 2008, pp. 3501–3506.

48.

[48]

Strasser, Alert in the cleanroom: Testing alerting modalities for a task guiding interface, in: CHI’13 Extended Abstracts on Human Factors in Computing Systems (CHI’13), 2013.

49.

[49]

Strasser,

Weiss,

Grill,

Osswald and

Tscheligi, Combining implicit and explicit methods for the evaluation of an ambient persuasive factory display, in: Proc. of AMI 2012, 2012.

50.

[50]

L.A.

Suchman, Plans and Situated Actions: The Problem of Human-Machine Communication, Cambridge University Press, New York, NY, USA, 1987.

51.

[51]

Sutton and

Cole, The CSLU toolkit: Rapid prototyping of spoken language systems, in: Proc. of the 10th Annual ACM Symposium on User Interface Software and Technology (UIST’97), ACM, New York, NY, USA, 1997, pp. 85–86.

52.

[52]

Zachhuber,

Grill and

Tscheligi, Contextual Wizard of Oz – A framework combining contextual rapid prototyping and the Wizard of Oz method, in: Proc. of AMI 2012, 2012.

ConWIZ: The contextual Wizard of Oz

Abstract

Keywords

Get full access to this article

References