What Kinds of Usability-Problem Description are Useful to Developers?

Abstract

Descriptions of problems found in usability evaluations aim to help developers improve an interactive system. However, little is know about what makes a problem description useful to developers. The present paper describes how four developers assessed the utility of 619 usability problems and relates their assessments to characteristics of how problems were described. Developers find problem descriptions that are clear, propose solutions, and elaborate on why something is a problem of significantly higher utility than problems without such information. Problems coded as persistent for expert users are assessed of higher utility than descriptions of novice users' problems. Reference to observable user actions made no difference to developers' assessment of utility. While developers did not see problems produced with an empirical and an inspection method as differing in utility, methods differed in how problems were described. We conclude by discussing recommendations for how to report the results of usability evaluations.

Get full access to this article

View all access options for this article.

References

Capra

Smith-Jackson

(2005). Developing guidelines for describing usability problems Rep. No. ACE/HCI-2005–002. Blacksburg, VA: Virginia Tech.

Connell

Blandford

Green

(2004). CASSM and cognitive walkthrough: Usability issues with ticket wending machines. Behaviour & Information Technology, 23, 5, 307–320.

Dumas

Molich

Jefferies

(2004). Describing Usability Problems: Are we sending the right message? Interactions, 4, 24–29.

Dumas

Redish

(1999). A practical guide to usability testing. Exeter, England: intellect.

Hornbæk

Frøkjær

(2002). Evaluating User Interfaces with Metaphors of Human Thinking. In Carbonell

Stephanidis

Proceedings of Proceedings of 7th ERCIM Workshop “User Interfaces for All”, Lecture Notes in Computer Science 2615 (pp. 486–507). Berlin: Springer-Verlag.

Hornbæk

Frøkjær

(2004). Usability Inspection by Metaphors of Human Thinking Compared to Heuristic Evaluation, International Journal of Human-Computer Interaction, 17 (3), 357–374.

Hornbæk

Frøkjær

(2005). Comparing usability problems and redesign proposals as input to practical systems development. In Proceedings of ACM Conference on Human Factors in Computing Systems (pp. 391–400. New York, NY: ACM Press.

Jeffries

(1994). Usability Problem Reports: Helping Evaluators Communicate Effectively with Developers. In Nielsen

Mack

R. L.

Usability Inspection Methods (pp. 273–294. New York, NY: John Wiley.

Lavery

Cockton

Atkinson

M. P.

(1997). Comparison of evaluation methods using structured usability problem reports. Behaviour & Information Technology, 16, 5, 246–266.

10.

Landis

Koch

(1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

11.

Molich

(2003). User testing, Discount user testing. www.dialogdesign.dk .

12.

Nielsen

Molich

(1990). Heuristic evaluation of user interfaces. In Proceedings of ACM Conference on Human Factors in Computing (pp. 249–256. New York, NY: ACM Press.