A discussion of the use of expert performance as the basis for inferring assessment scoring criteria is presented. Characteristics of experts are discussed, leading to a consideration of the appropriateness of using adult experts' performance to measure children's performance. The use of a priori classifications of children is explored, and the suggestioh for examining the use of gifted students' performances as benchmarks is presented. Both positive and negative consequences are described.
Get full access to this article
View all access options for this article.
References
1.
Abedi, J., Baker, E.L., & Herl, H. (1993). Comparing reliabilities obtained by different approaches for performance assessment. Paper presented at the annual meeting of the American Educational Research Association, Atlanta.
2.
Baker, E.L. (1994a). Human benchmarking of natural language systems . In H. F. O'Neil & E. L. Baker (Ed.) Technology assessment
3.
in software applications (pp. 85-97). Hillsdale, NJ: Lawrence Erlbaum Associates.
4.
Baker, E.L. (1994b). Learning-based assessments of history understanding . Educational Psychologist, 29(2), 97-106.
5.
Baker, E.L., Aschbacher, P., Niemi, D., & Sato, E. (1992). CRESST performance assessment models: Assessing content area explanations. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
6.
Baker, E.L., Aschbacher, P., Niemi, D., Chang, S., Weinstock, M. & Herl, H. (1991). Validating measures of deep understanding of history . Paper presented at the annual meeting of the American Educational Research Association, Chicago,
7.
Baker, E.L., Freeman, M., & Clayton, S. (1991). Cognitive assessment of history for large-scale testing. In M. C. Wittrock & E. L. Baker (Eds.), Testing and cognition (pp. 131-153). Englewood, NJ: Prentice Hall.
8.
Baxter, G.P., Elder, A.D., Glaser, R. (1995). Cognitive Analysis of a Science Performance Assessment (CSE Technical Report 398). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing.
9.
Cheng, P.W. (1993). Metacognition and giftedness: The state of the relationship. Gifted Child Quarterly, 37, 105-112.
10.
Chi, M.T.H. (1978). Knowledge structures and memory development. In R. Siegler (Ed.), Children's thinking: What develops? (pp. 73-96). Hillsdale, NJ: Lawrence Erlbaum Associates.
11.
Chi, M.T., Glaser, R., & Farr, M.J. (1988). The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum Associates.
12.
Ericsson, K.A., Krampe, R.T., & Heizmann, S. (1993). Can we create gifted people? In Ciba Foundation, The origins and development of high ability (pp. 222-249). John Wiley & Sons.
13.
Ericsson, K.A., Krampe, R.T., & Tesch-Romer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychology Review, 100, 363-406.
14.
Gentner, D., & Gentner, D.R. (1983). Flowing waters or teeming crowds: Mental models of electricity. In D. Gentner & A. L. Stevens (Eds.) Mental models (pp. 99-129). Hillsdale, NJ: Lawrence Erlbaum Associates.
15.
Gentner. D. R. (1988). Expertise in typewriting. In M. T. H. Chi, R. Glaser , & M. J. Farr (Ed.), The nature of expertise (pp, 1-22). Hillsdale, NJ: Lawrence Erlbaum Associates.
16.
Glaser, R., & Chi, M.T.H. (1988). Overview. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.). The nature of expertise (pp. xv-xxviii). Hillsdale, NJ: Lawrence Erlbaum Associates.
17.
Herl, H.E. (1995). Construct validation of an approach to modeling cognitive structure of experts' and novices' U.S. history knowledge. Unpublished doctoral dissertation, University of Califomia, Los Angeles.
18.
Herl, H., Niemi, D., & Baker, E.L. (1995). Construct validation of an approach to modeling cognitive structure of experts' and novices'
19.
U.S. history, knowledge. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
20.
Johnson, E.J. (1988). Expertise and decision under uncertainty: Performance and process. InM. T. H. Chi, R. Glaser, & M. J. Farr
21.
(Eds.), The nature of expertise (pp. 209-228). Hillsdale, NJ: Lawrence Erlbaum Associates,
22.
Kieras, D.E., (1988). What mental model should be taught: Choosing instructional content for complex engineered systems. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent tutoring
23.
systems: Lessons learned (pp. 85-111). Hillsdale, NJ: Lawrence Erlbaum Associates.
24.
Larkin, J.H. (1983). The role of problem representation in physics . In D. Gentner & A. L. Stevens (Eds.), Mental models (pp. 75-100). Hillsdale, NJ: Lawrence Erlbaum Associates.
25.
Lawrence, J.A. (1988). Expertise on the bench: Modeling magistrates' judicial decision-making, In M. T. H. Chi, R. Glaser, & M. J.
26.
Farr (Eds.), The nature of expertise (pp. 229-260). Hillsdale, NJ: Lawrence Erlbaum Associates.
27.
Lesgold, A., Rubinson. H., Feltovich, P., Glaser, R.Klopfer, D., & Wang, Y. (1988). Expertise in a complex skill: Diagnosing X-ray pictures. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 311-342). Hillsdale, NJ: Lawrence Erlbaum Associates.
28.
Lomask, M., Baron, J., Greig, J., & Harrison, C. (1992). ConnMap: Connecticut's use of concept mapping to assess the structure of students' knowledge of science. Symposium presented at the annual meeting of the National Center for Research in Science Teaching and Learning, Cambridge. MA.
29.
Mayer, R.E. (1989). Models for understanding. Review of Educational Research, 59(1), 43-64,
30.
Means, B., & Gott, S.P. (1988). Cognitive task analysis as a basis for tutor development: Articulating abstract knowledge representations. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent
Miyake, N., & Norman. D.A. (1979). To ask a question one must know enough to know what is not known. Journal of Verbal Learning and Verbal Behavior , 18, 357-364.
33.
O'Neil, H.F., Jr., Allred, K., & Dennis, R.A. (in press). Assessment issues in the validation of a computer simulation of negotiation skills. In H. F. O'Neil , Jr., (Ed.), Workforce readiness: Competencies and assessment . Mahwah, NJ: Lawrence Erlbaum Associates .
34.
O'Neil, H.E., Jr., Baker, E.L., Ni, Y., Jacoby, A., & Swigger, K.M. (1994). Human benchmarking for the evaluation of expert systems. In H. F. O'Neil, Jr, & E. L. Baker (Eds.) Technology assessment in software applications (pp. 13-45). Hillsdale, NJ : Lawrence Erlbaum Associates.
35.
O'Neil, II. E., Jr., Chung, G., & Brown, R. (in press). Use of networked simulations as a context to measure team competencies. In H. F. O'Neil, Jr., (Ed.), Workforce readiness: Competencies and assessment. Mahwah, NJ: Lawrence Erlbaum Associates.
36.
Rogers, K. (1986). Do the gifted think and learn differently? A review of recent research and its implications for instruction. Journal for the Education of the Gifted, 10, 17-39.
37.
Shank, R., & Kass, A. (1988). Knowledge representation in people and machines , In U. Eco, M. Santambrogio, & P. Violi (Eds.), Meaning
38.
and mental representations (pp. 181-200). Bloomington: Indiana University Press.
39.
Shavelson, R.J., Lang, H., & Lewin, B. (1994). On concept maps as potential "authentic" assessments in science (CSE Tech. Rep. No. 388). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
40.
Simon, D.P., & Simon, H.A. (1978). Individual differences in solving physics problems . In R. Siegler (Ed.), Children's thinking: What develops? (pp. 325-348). Hillsdale, NJ: Lawrence Erlbaum Associates.
41.
Staggers, N., & Norcio, A.F. (1993). Mental models: Concepts for human-computer interaction research. International Journal Man-MachineStudies, 38, 587-605.
42.
Sternberg, R.J. (1977). Intelligence, information processing, and analogical reasoning: The componential analysis of human abilities. Hillsdale, NJ: Lawrence Erlbaum Associates.
43.
Sternberg, R.J. (1981). Intelligence and nonentrenchment. Journal of Educational Psychology, 73, 1-16.
44.
Sternberg, R.J. (1988). A unified theory of intellectual exceptionality . In J. D. Day & J. G. Borkowski (Eds.), Intelligence and exceptionality: New directions for theory, assessment, and instructional practices (pp. 135-172). Norwood, NJ: Ablex
45.
Sternberg, R.J., & Clinkenbeard, P.R. (1995). The triarchic model applied to identifying, teaching and assessing gifted children. Roepe Review17(4), 255-260.
46.
Sternberg, R.J., & Davidson, J.E. (1986). Conceptions of giftedness . New York: Cambridge University Press.
47.
Stevens, R., & Lopo, A. (1994). Artificial neural network comparison of expert and novice problem-solving strategies. Proceedings of the Eighteenth Annual Symposium on Computer Applications in Medical Care (pp. 64-68).
48.
Stevens, R., Wang, P., & Lopo, A. (in press). Artificial neural networks can distinguish novice and expert strategies during complex problem-solving. Journal of tke American Medical Informatics Association.
49.
Voss, J.E., & Post, T.A. (1988). On the solving of ill-structured problems. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 261-285). Hillsdale, NJ: Lawrence Erlbaum Associates.
