Wizard of Oz in a Structured Expert Review (WiSER): An Evaulative Study

Abstract

Time and money constraints are often an issue when selecting a method for testing user interface designs. The Wizard of Oz technique allows usability practitioners to re-create speech user interfaces without building a hi-fidelity prototype. Wizard of Oz in a Structured Expert Review (WiSER) leverages the advantages of Wizard of Oz by combining it with aspects of the Cognitive Walkthrough method. The result is a method which can be efficiently implemented to test speech user interfaces by using usability practitioners as test participants and the designer as the wizard. The participant is presented with a set of tasks to complete using the speech user interface. After each task, problem areas are discussed and design alternatives, explored; modified versions of the two questions identified by Spencer [2000] are addressed for each problem area. This paper covers a direct comparison of WiSER to an online usability test in which both tests are executed on a voice dialer application.

Get full access to this article

View all access options for this article.

References

Dahlback

Jonsson

Ahrenberg

, (1993) Wizard of Oz studies — Why and How, Proceedings of Intelligent User Interfaces 1993 pp. 193–200.

Peissner

Heidmann

Ziegler

, (2001) Simulating recognition errors in speech user interface prototyping, Usability evaluation and interface design: Proceedings of HCI International 2001, Vol. 1 Smith Michael

(Hrsg). Mahwah, N.J.: Lawrence Erlbaum, 2001 pp. 233–237.

Polson

P.G.

Lewis

Rieman

Wharton

, (1992), Cognitive Walkthroughs: A method for theory-based evaluation of user interfaces, International Journal of Man-Machine studies (1992) 36, pp. 741–773.

Rowley

Rhoades

D.G.

, (1992) The cognitive jog through: A fast paced user interface evaluation procedure, Proceedings of ACM CHI 1992. Conference in Human Factors in Computing Systems, pp. 389–395.

Spencer

, (2000) The streamlined cognitive walkthrough method, Working around social constraints encountered in a software development company, Proceedings of ACM CHI 2000. Conference in Human Factors in Computing Systems (New York 2000), pp.119–124.

Wharton

C. W.

Reiman

Lewis

Polson

(1994) The cognitive walkthrough method: A practitioner's guide. In Nielson

J.K.

Mack

R. L.

(Eds.) Usability Inspection Methods. Wiley, New York, 1994. pp. 105–140.

Wixon

, (2003 July+August) Evaluating Usability methods, Why the current literature fails the practitioner, Interactions pp. 29–34.