The Use of Cognitive Feedback by Experts and Novices in a Judgment Task

Abstract

Cognitive feedback (CFB) has been shown to increase performance in Multiple Cue Probability Learning (MCPL) tasks in both real life and experimental situations. Our experiment focuses on investigating how experts and novices behave differently in applying CFB, which is presented in forms of relative importance weights of each of the cues and the display of function forms (Weaver and Steward, 2000). We believe this study provided some interesting insight into how much domain knowledge is necessary to understand and apply CFB. The study examined how well novices and experts are able to process and use different types of feedback (i.e. CFB and Outcome Feedback) and hence determined which type is more useful for novices or for those with prior knowledge of the task. A total of 20 participants, 10 experts and 10 novices, were given a series of baseball statistics and asked to predict how many wins the team would have in a season based on these statistics. Performance measures (e.g. relative weighting accuracy, Lens Model measures), mental workload and judgment strategy were measured. Domain Knowledge was statistically significant for all weights except one less important cue, “stolen bases.” In other words, domain knowledge impacts weighting accuracy greatly. Domain knowledge was also found to have a significant effect on judgment performance (i.e. Lens Model measures), such as accuracy and knowledge. Mental workload was significantly affected by CFB, but none of the other dependent variables were.

Get full access to this article

View all access options for this article.

References

Adelman

(1981). The Influence of Formal, Substantive, and Contextual Task Properties on the Relative Effectiveness of Different Forms of Feedback in Multiple-cue Probability Learning Tasks. Organizational Behavior and Human Decision Processes. 27, pp. 423–442.

Balzer

W. K.

Hammer

L. B.

Sumner

K. E.

Birchenough

T. R.

Parham

S. E.

Raymark

P. H.

(1994). Effects of Cognitive Feedback Components, Display Format, and Elaboration on Performance. Organizational Behavior and Human Decision Processes. 58, pp. 369–385.

Dennis

R. A.

(1998). The Effect of Feedback Cues, Relevant Information, and Scoring Procedures on Concept Mapping Performance. Unpublished Doctorate Dissertation. University of California, Los Angeles.

Engel

J. D.

Wigton

R. S.

LaDuca

Blacklow

R. S.

(1990). A Social Judgment Theory Perspective on Clinical Problem Solving. Evaluation and the Health Professions. 13, pp. 63–78.

Enis

C. R.

(1995). Expert-novice Judgments and New Cue Sets: Process versus Outcome. Journal of Economic Psychology. 16, pp. 641–662.

Harmon

Rohrbaugh

(1990). Social Judgment Analysis and Small Group Decision Making: Cognitive Feedback Effects on Individual and Collective Performance. Organizational Behavior and Human Decision Processes. 46, pp. 34–54.

Harries

Evans

J. St. B. T.

Dennis

Dean

(1996). A Clinical Judgment Analysis of Prescribing Decisions in General Practice. Le Travail Humain. Vol.59, Issue 1, pp.87–111.

Te'eni

(1990). Direct Manipulation as a Source of Cognitive Feedback: A Human-Computer Experiment with a Judgment Task. International Journal of Man-Machine Studies. 33, pp.453–466.

Weaver

E. A.

Stewart

T. R.

(2000). Giving Feedback: Description of a Method for Fitting Weights and Displaying Function Forms for Feedback to Judges in Judgment Analysis. Online resource: http://cpol.albany.edu.