Too Good to be Bad: The Effect of Favorable Expectations on Usability Perceptions

Abstract

The most common measurements in the task-based usability evaluation paradigm include behavioral (e.g., completion times or errors) and subjective measures (e.g., ratings). Previous work has shown that success and performance in the test tasks do not dictate subjective usability perceptions, which instead are affected by appraisals of the system such as those of its aesthetic appeal. While these appraisals are an outcome of the exposure to and the interaction with the system, less is known about the effect of predispositions (expectations) formed before any exposure. To understand how expectations influence usability perceptions, we devised an experiment wherein 36 subjects read a positive or a negative product review for a novel mobile device (while a control group read nothing) before a usability test. The results demonstrate a surprisingly strong amplifying effect of the positive expectation on the post-experiment ratings, which, interestingly, held even in a condition where the users failed in all of the tasks. We briefly discuss implications of this finding.

Get full access to this article

View all access options for this article.

References

Bangor

Kortum

P.T.

Miller

J.T.

(2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594.

Brooke

(1996). SUS – A quick and dirty usability scale. In Jordan

P. W.

Thomas

Weerdmeester

B. A.

McClelland

I.L.

(Eds.), Usability evaluation in industry. Taylor and Francis, London, UK, 189–194.

Bhattacherjee

(2001). Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly, 25(3), 351–370.

De Angeli

Sutcliffe

Hartman

(2006). Interaction, usability and aesthetics: What influences users' preferences? In Proceedings of the 6th conference on Designing Interactive systems, PA, USA.

Hart

S.G.

Staveland

L.E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock

P.A.

Meshkati

(Eds.), Human Mental Workload, North Holland Press, Amsterdam, Holland, 239–250.

Hassenzahl

Burmester

Koller

(2003). AttrakDiff: Ein fragebogen zur messung wahrgenommene hedonischer und pragmatischer qualität. In Ziegler

Szwillus

(Eds.), Mensch and Computer 2003. Interaktion in Bewegung. B.G. Teubner, Stuttgart, Deutschland, 187–196.

Hornbaek

Lai Chong-Law

(2007). Meta-analysis of Correlations among Usability Measures. In Proceedings of the SIGCHI conference on Human factors in computing systems, California, USA.

Keppel

Wickens

T. D.

(1991). Design and Analysis. A Researcher's Handbook (4th Edition). Prentice Hall New Jersey, USA.

Ofir

Simonson

(2005). The effect of stating expectations on costumer satisfaction and shopping experience. Journal of Marketing Research, XLIV, 164–174.

10.

Szajna

Scamell

R.W.

(1993). The effects of information system user expectations on their performance and perceptions. MIS Quarterly, 17(4), 493–516.

11.

Tranctinsky

Katz

A.S.

Ikar

(2000). What is beautiful is usable. Interacting with Computers, 13(12), 127–145.

12.

Venkatesh

Morris

M.G.

Davis

G.B.

Davis

F.D.

(2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–278.

13.

Watson

Clark

L.A.

Tellegen

(1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070.

14.

Wilkins

W.E.

(1976). The concept of a self-fulfilling prophecy. Sociology of Education, 49, 175–183.