The Relationship Between Levels of User Experience with a Product and Perceived System Usability

Abstract

The System Usability Scale (SUS) is a ten-point assessment tool developed as a reliable low-cost subjective usability scale that can be applied to systems in any number of contexts. Research has demonstrated higher usability ratings from users who claim greater experience with an interface than from those who rate themselves as having less experience. This paper describes research to extend this work by experimentally controlling the experience levels of the users over the course of the study, rather than relying on users’ self-report. Two studies were conducted. In the first, Microsoft Publisher was used over three one hour sessions, with usability being measured with the SUS at the completion of each session. In the second study, MathWorks MATLAB was used over the course of 14 weeks, and SUS usability was measured near the beginning, the middle and end of this time frame. Results from the MS publisher study showed an increase in reported usability with increased experience consistent with the literature, but the data from the MATLAB study did not show this trend. Reasons for this discrepancy are discussed, as are future research directions that could shed further light on these unexpected findings.