Sage Journals: Discover world-class research

Abstract

We describe a comprehensive work-centered evaluation framework for assessing the value of new technology intended to support human performance. A key feature of the framework is that it spans three types of evaluation: (1) usability — How easy is the aid to use? (2) usefulness — How flexible and effective is it in aiding work? and (3) impact — How (much) does it contribute value within the work organization? We argue that all three of these dimensions are important for advancing the product development and facilitating rapid transition of the product to the customer for deployment. A recent evaluation of a work-centered support system prototype is used to illustrate the multi-level work-centered evaluation approach, the types of insights that can be drawn from it, and a way to employ it in a cost effective manner.

Get full access to this article

View all access options for this article.

References

Eggleston

R. G.

Whitaker

R. D.

(2002). Work-Centered Support Systems Design: Using Organizing Frames to Reduce Work Complexity. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 265–269). Santa Monica, CA: Human Factors and Ergonomics Society.

Eggleston

R. G.

Young

M. J.

Whitaker

R. D.

(2000) Work-Centered Support System Technology: A New Interface Client Technology for the Battlespace Infosphere. In Proceedings of NEACON 2000, Dayton OH, 10–12 October, 2000. IEEE 00CH37093.

Grudin

(1992). Utility and usability: Research issues and development contexts. Interacting with Computers 4, 2 (August), 209–217.

Nielsen

(1993). Usability Engineering. San Diego: Morgan Kaufman.

Patterson

E. S.

Nguyen

A. D.

Bozzette

S. A.

Gomes

J. O.

Asch

S. M.

(2003). Comparing Findings from Cognitive Engineering Evaluations. In Proceedings of the Human Factors and Ergonomics Society 47^th Annual Meeting..

Rossi

Freeman

H. E.

Lipsey

M. W

(1999). Evaluation: A Systematic Approach, (6th Edition) Thousand Oaks, CA: Sage Publications.

Roth

E. M.

Gualtieri

J. W.

Elm

W. C.

Potter

S. S.

(2002). Scenario Development for Decision Support System Evaluation. In Proceedings of the Human Factors and Ergonomics Society 46^th Annual Meeting, (pp. 357–361) Santa Monica, CA: Human Factors and Ergonomics Society.

Scott

Roth

E. M.

Deutsch

S. E.

Malchiodi

Kazmierczak

Eggleston

R. G.

Kuper

S. M.

Whitaker

(2002). Using Software Agents in a Work Centered Support System for Weather Forecasting and Monitoring. In Proceedings of the Human Factors and Ergonomics Society 46^th Annual Meeting, (pp.433–437) Santa Monica, CA: Human Factors and Ergonomics Society.

Scott

Roth

E. M.

Deutsch

S. E.

Malchiodi

Kazmierczak

Eggleston

Kuper

S. R.

Whitaker

(2003). Work-Centered Support Systems: A Human-Centered Approach to Intelligent System Design. IEEE Intelligent Systems.

10.

Young

M. J.

Eggleston

R. G.

Whitaker

(2000). Direct Manipulation Interface Technique for Interacting with Software Agents. In Proceedings of the NATO/RTO Symposium on Usability of Information in Battle Management Operations, Oslo, Norway, 10–13 April, 2000, pp 19-1—19-10. RTO-MP-57 (AC/323/(HFM)TP/29), Nov 2000.

11.

Woods

Dekker

(2002). Anticipating the effects of technological change: A new era of dynamics for human factors. Theor. Issues in Ergon. Sci. 1–11.

A Framework for Work-Centered Product Evaluation

Abstract

Get full access to this article

References