A growing body of literature suggests that experts are little if at all better than novices in terms of the quality of decision outputs. To explain this counter-intuitive finding, the authors propose a conceptual framework that focuses on initial problem structure as a key moderator of the effect of expertise on performance. Specifically, they argue that the expert–novice performance differential should be greatest at moderate levels of problem structure and weakest at both extremes. To examine this central hypothesis, the authors conduct a controlled experiment that compares experts with novices when solving a complex problem that had characteristics of a moderately ill-structured problem. Relative to novices, the authors find that experts select fewer, but more diagnostic, information inputs and are more consistent when evaluating nonquantified inputs. As a result, they make more accurate and tightly clustered judgments than do novices, and also are more confident in their decisions. To examine the moderating influence of problem characteristics, certain task variables are manipulated to increase or decrease initial problem structure. As hypothesized, the benefits of expertise are less pronounced when solving a problem with increased initial structure.
Get full access to this article
View all access options for this article.
References
1.
AlbaJoseph W., and HasherLynn (1983), “Is Memory Schematic?”Psychological Bulletin, 93(March), 203–31.
2.
AlbaJoseph W., and Wesley HutchinsonJ. (1987), “Dimensions of Consumer Expertise,”Journal of Consumer Research, 13(March), 411–54.
3.
ArmstrongJ. Scott (1985), Long Range Forecasting: From Crystal Ball to Computer, 2d ed.New York: John Wiley & Sons.
4.
BlattbergRobert C., and HochStephen J. (1990), “Database Models and Managerial Intuition: 50% Model and 50% Manager,”Management Science, 36(8), 887–99.
5.
BrehmerBrendt, and JoyceC.R.B. (1988), Human Judgment: The SJT View.New York: North Holland.
6.
BrucksMerrie (1985), “The Effects of Product Class Knowledge on Information Search Behavior,”Journal of Consumer Research, 12(June), 1–16.
7.
BrucksMerrie (1988), “Search Monitor: An Approach for Computer-Controlled Experiments Involving Consumer Information Search,”Journal of Consumer Research, 15(June), 117–21.
8.
CamererColin F., and JohnsonEric J. (1991), “The Process-Performance Paradox in Expert Judgment: How Can Experts Know So Much and Predict So Badly?” in Toward a General Theory of Expertise: Prospects and Limits, Anders EricssonK., and SmithJacqui, eds. New York: Cambridge University Press,195–217.
9.
ChaseWilliam G., and SimonHerbert (1973), “Perception in Chess,”Cognitive Psychology, 4(January), 55–81.
10.
ChiMichelene T.H., FeltovichPaul J., and GlaserRobert (1981), “Categorization and Representation of Physics Problems by Experts and Novices,”Cognitive Science, 5(April/June), 121–52.
11.
ChiMichelene T.H., and GlaserRobert (1980), “The Measurement of Expertise,” in Educational Testing and Evaluation, BakerE.L., and QuellmalzE.S., eds. Newbury Park, CA: Sage Publications.
12.
ChiMichelene T.H., GlaserRobert, and FarrM.J. (1988), The Nature of Expertise.London: Lawrence Erlbaum Associates.
13.
ChiMichelene T.H., GlaserRobert, and ReesErnest (1982), “Expertise in Problem Solving,” in Advances in the Psychology of Human Intelligence, Vol. 1, SternbergRobert J., eds. Hillsdale, NJ: Lawrence Erlbaum and Associates.
14.
Christensen-SzalanskiJay JJ, and BushyheadJames B. (1981), “Physician's Use of Probabilistic Information in a Real Clinical Setting,”Journal of Experimental Psychology: Human Perception and Performance,1, 928–35.
15.
DayGeorge S., and NedungadiPrakesh (1994), “Managerial Representations of Competitive Advantage,”Journal of Marketing, 58(April), 31–44.
16.
EinhornHillel J. (1972), “Expert Measurement and Mechanical Combination,”Organizational Behavior and Human Performance,1, 86–106.
17.
EinhornHillel J. (1986), “Expert Judgment: Some Necessary Conditions and An Example,” in Judgment and Decision Making: An Interdisciplinary Reader, ArkesHal R., and HammondKenneth R., eds. New York: Cambridge University Press.
18.
EricssonK. Anders, and SimonHerbert A. (1984), Protocol Analysis: Verbal Reports as Data.Cambridge, MA: MIT Press.
19.
GoldbergLewis R. (1959), “The Effectiveness of Clinicians’ Judgements: The Diagnosis of Organic Brain Damage from Bender-Gestalt Test,”Journal of Consulting Psychology, 23, 25–33.
20.
GorryG. Anthony, and Scott MortonMichael S. (1971), “A Framework for Management Information Systems,”Sloan Management Review, 13(Fall), 55–70.
21.
HammondKenneth R., HurschC., and ToddF.J. (1964), “Analyzing the Components of Clinical Inference,”Psychological Review, 71, 438–546.
22.
HammondKenneth R., StewartThomas R., BrehmerBrendt, and SteinmannDerick O. (1986), “Social Judgment Theory,” in Judgment and Decision Making: An Interdisciplinary Reader, ArkesHal R., and HammondKenneth R., eds. New York: Cambridge University Press.
23.
JohnsonEric J. (1980), “Expertise in Admissions Judgment,” unpublished dissertation, Department of Psychology, Carnegie-Mellon University.
24.
JohnsonEric J. (1988), “Expertise and Decision Under Uncertainty: Performance and Process,” in The Nature of Expertise, ChiMichelene T.H., GlaserRobert, and FarrM.J., eds. London: Lawrence Erlbaum and Associates.
25.
JohnsonEric J., and Edward RussoJ. (1984), “Product Familiarity and Learning New Information,”Journal of Consumer Research, 11(June), 542–50.
26.
KlaymanJoshua (1988), “On the How and Why (Not) of Learning From Outcomes,” in Human Judgment: The SJT View, BrehmerBerndt, and JoyceC.R.B., ed. New York: Elsevier Science Publishing Company.
27.
LarkinJill H., McDermottJohn, SimonDorothea P., and SimonHerbert A. (1980), “Expert and Novice Performance in Solving Physics Problems,”Science, 208, 1335–42.
28.
MahajanJayashree (1992), “The Overconfidence Effect in Marketing Management Predictions,”Journal of Marketing Research, 29(August), 329–42.
29.
MartinMichael M. (1993), “The Appraiser as an Artist,”The Appraisal Journal, (July), 316–22.
30.
MurphyA.H., and WinklerP. (1977), “Can Weather Forecasters Formulate Reliable Probability Forecasts of Precipitation and Temperature?”National Weather Digest,2, 2–9.
31.
NewellAllen, and RosenbloomPaul S. (1981), “Mechanisms of Skill Acquisition and the Law of Practice,” in Cognitive Skills and Their Acquisition, AndersonJohn R., ed. Hillsdale, NJ: Lawrence Erlbaum and Associates,1–56.
32.
NewellAllen, and SimonHerbert A. (1972), Human Problem Solving.Englewood Cliffs, NJ: Prentice-Hall,.
33.
NisbettRichard E., ZukierHenry, and LemleyRonald E. (1981), “The Dilution Effect: Nondiagnostic Information Weakens the Implications of Diagnostic Information,”Cognitive Psychology, 13, 248–77.
34.
PayneJohn W. (1976), “Task Complexity and Contingent Processing in Decision Making: An Information Search and Protocol Analysis,”Organizational Behavior and Human Performance, 16, 366–87.
35.
PayneJohn W., BettmanJames R., and JohnsonEric J. (1988), “Adaptive Strategy Selection in Decision Making,”Journal of Experimental Psychology: Learning, Memory and Cognition, 14(3), 534–52.
36.
PhelpsR.M., and ShanteauJames (1978), “Livestock Judges: How Much Information Can an Expert Use?”Organizational Behavior and Human Performance, 21, 209–19.
37.
PunjGirish N., and StewartDavid W. (1983), “An Interaction Framework of Consumer Decision Making,”Journal of Consumer Research, 10(September), 181–96.
38.
ReitmanWalter R. (1965), Cognition and Thought: An Information Processing Approach.New York: John Wiley & Sons.
39.
RockhartJohn F. (1979), “Chief Executives Define Their Own Data Needs,”Harvard Business Review, 57(March/April), 81–93.
40.
SawyerJack (1966), “Measurement and Prediction: Clinical and Statistical,”Psychological Bulletin, 66(3), 178–200.
41.
SchroderHarold M., DriverMichael J., and StreufertSiegfried (1967), Human Information Processing: Individuals and Groups Functioning in Complex Situations.New York: Holt, Rinehart and Winston.
42.
ShanteauJames (1988), “Psychological Characteristics and Strategies of Expert Decision Makers,”Acta Psychologica, 68, 203–15.
43.
ShanteauJames (1992a), “How Much Information Does an Expert Use? Is It Relevant?”Acta Psychologica, 81, 75–86.
44.
ShanteauJames (1992b), “Competence in Experts: The Role of Task Characteristics,”Organizational Behavior and Human Decision Processes, 53, 252–66.
45.
ShanteauJames, and StewartThomas R. (1992), “Why Study Expert Decision Making? Some Historical Perspectives and Comments,”Organizational Behavior and Human Decision Processes, 53, 95–106.
46.
SimonHerbert A. (1973), “The Structure of Ill-Structured Problems,”Artificial Intelligence, 4, 181–201.
47.
SimonHerbert A. (1978), “Information Processing Theory of Human Problem Solving,” in Handbook of Learning and Cognitive Processes, Vol. 5, EstesW.K., ed. Hillsdale, NJ: Lawrence Erlbaum Associates,271–96.
48.
SmithGerald F. (1988), “Towards a Heuristic Theory of Problem Structuring,”Management Science, 34(12), 1489–506.
49.
SwellerJohn, MawerRobert F., and WardMark R. (1983), “Development of Expertise in Mathematical Problem Solving,”Journal of Experimental Psychology: General, 112(December), 639–61.
50.
TaylorRonald N. (1974), “Nature of Problem Ill-Structuredness: Implications for Problem Formulation and Solution,”Decision Sciences, 5, 632–43.
51.
VossJames F., and PostTimothy A. (1988), “On the Solving of Ill-Structured Problems,” in The Nature of Expertise, ChiMichelene T.H., GlaserRobert, and FarrM.J., eds. London: Lawrence Erlbaum Associates.
52.
WilsonTimothy D., and SchoolerJonathan W. (1991), “Thinking Too Much: Introspection Can Reduce the Quality of Preferences and Decisions,”Journal of Personality and Social Psychology, 60(2), 181–92.
53.
WinklerRobert L., and PosesRoy M. (1993), “Evaluating and Combining Physicians’ Probabilities of Survival in an Intensive Care Unit,”Management Science, 39(12), 1526–43.
