High interest in performance-based assessment of educational achievement is indicated by recent conferences, special journal issues, and national television coverage. The president's and governors' educational goals reflect the central role of assessment in reform. A prominent view is that changes in assessment, using more complex, meaningful, and integrative performance tasks, will improve education. New assessments present challenges to the curriculum, teaching practices, and presentation of student achievement information to policymakers and to the public. Sufficient high-quality assessments must be available before their impact on educational reform can be assessed. Although interest in performance-based assessment is high, our knowledge about its quality is low. Moreover, few psychometric templates exist to guide the technical practices of assessment developers.
Get full access to this article
View all access options for this article.
References
1.
Advisory Committee on Testing in Chapter 1. (1993, May). Reinforcing the promise, reforming the paradigm. Washington, DC: U.S. Department of Education.
2.
American Educational Research Association. American Psychological Association, and the National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
3.
BakerE. L. (1990a, October). Assessment and public policy: Does validity matter Paper presented at the annual meeting of the American Evaluation Association, Washington, DC.
4.
BakerE. L. (1990b, October). What “probably works” in alternative assessment. Presentation at the conference The Promise and Peril of Alternative Assessment, Office of Educational Research and Instruction, Washington, DC.
5.
BakerE. L. (1991, April). Expectations and evidence for alternative assessment. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
6.
BakerE. L. (in press). Learning-based assessments of history understanding [Special issue]. Educational Psychologist.
7.
BakerE. L.FreemanM.ClaytonS. (1991). Cognitive assessment of history for large-scale testing. In WittrockM. C.BakerE. L. (Eds.), Testing and cognition (pp. 131–153). Englewood Cliffs, NJ: Prentice-Hall.
8.
BakerE. L.LinnR. L. (1990). Advancing educational quality through learning-based assessment, evaluation, and testing (Institutional Grant Proposal for OERI Center on Assessment, Evaluation, and Testing). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing.
9.
BakerE. L.NiemiD.NovakJ.HerlH. (1992). Hypertext as a strategy for teaching and assessing knowledge representation. In DijkstraS.DrammerH. P. M.van MerriënboerJ. J. G. (Eds.), Instructional models in computer-based learning environments (pp. 365–384). Heidelberg, Germany: Springer-Verlag.
10.
BakerE. L.O'NeilH. F.Jr.LinnR. (1990). Performance assessment framework. In AndrioleS.J. (Ed.), Advanced technology for command and control systems engineering (pp. 192–214). Fairfax, VA: AFCEA International.
11.
BaronJ. B. (1991, April). Performance-based assessment at the state level: Developing and implementing high school math and science assessments. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
12.
BennettR. E.RockD. A.WangM. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28, 77–92.
13.
BernheimerL. P.GallimoreR.WeisnerT. S. (1990). Eco-cultural theory as a context for the individual family service plan. Journal of Early Intervention, 14, 219–233.
14.
BraunH. (1991, April). Assessing technology in assessment. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
15.
BraunH. (in press). Assessing technology in assessment. In BakerE.L.O'NeilH. F.Jr. (Eds.), Technology assessment in education and training. Hillsdale, NJ: Erlbaum.
16.
BrayD. W.GrantD. L. (1966). The assessment center in the measurement of potential for business management. Psychological Monographs, 80(17).
17.
BrelandH. M.DanosD. O.KahnH. D.KubotaM. Y.SudlowM. W. (1991). A study of gender and performance on advanced placement history examinations (College Board Report No. 91–4; ETS RR No. 91–61). New York: College Entrance Examination Board.
18.
BrewerR. (1991, April). Panel member for session: Authentic assessment: The rhetoric and the reality. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
19.
BrittonB. (1991, April). Pushing around mental representations by revising texts. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
20.
BursteinL. (1993, April). Validating national curriculum indicators: A conceptual overview of the RAND/CRESST NSF project. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
21.
ByhamW. C.ThorntonG. C.III (1986). Assessment centers. In BerkR.A. (Ed.), Performance assessment: Methods and applications (pp. 143–166). Baltimore, MD: Johns Hopkins University Press.
22.
California Assessment Program. (1991). A new student assessment system for California schools. Sacramento, CA: Author.
23.
California State Department of Education. (1989). A question of thinking: A first look at students' performance on open-ended questions in mathematics. Sacramento, CA: Author.
24.
CampbellC. H.FordP.RumseyM. G.PulakosE. D.BormanW. C.FelkerD. B.DeVeraM. V.RiegelhauptB. J. (1990). Development of multiple job performance measures in a representative sample of jobs. Personnel Psychology, 43, 277–300.
25.
CampbellJ. P.McHenryJ. J.WiseL. L. (1990). Modeling job performance in a population of jobs. Personnel Psychology, 43, 313–333.
26.
CarlsonD. (1991). A proposal for a multi-stage, portfolio-based national examining system. Unpublished report to the Assessment Task Force of the National Council on Education Standards and Testing, Sacramento, CA.
27.
ChiM. T. H.GlaserR.FarrM. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: Erlbaum.
28.
CollisK.RombergT. A. (1991). Assessment of mathematical performance: An analysis of open-ended test items. In WittrockM. C.BakerE. L. (Eds.), Testing and cognition (pp. 82–130). Englewood Cliffs, NJ: Prentice Hall.
29.
DunbarS. B.KoretzD.HooverH. D. (1991). Quality control in the development and use of performance assessments. Applied Measurement in Education, 4, 289–302.
30.
FrankenJ. E. (in press). The effects of stress induced anxiety on individuals and teams in a simulator environment. In O'NeilH. F.Jr.DillingsM. (Eds.), Motivation: Theory and research. Hillsdale, NJ: Erlbaum.
31.
FrederiksenJ. R.CollinsA. (1989). A system approach to educational testing. Educational Researcher, 28(9), 27–32.
32.
GauglerB. B.RosenthalD. B.ThorntonG. C.IIIBentsonC. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493–511.
33.
GearhartM.SaxeG. B.StipekD. J.HakanssonS. W. (1992, June). Portfolios in practice: Integrating assessment and instruction in elementary mathematics. Presentation at the meeting of the Education Commission of the States, Boulder, CO.
34.
GlaserR.LesgoldA.GottS. (1991). Implications of cognitive psychology for measuring job performance. In WigdorA. K.GreenB. F.Jr. (Eds.), Performance assessment for the workplace (Vol. 2, pp. 1–26). Washington, DC: National Academy Press.
35.
GlaserR.LesgoldA.LajoieS. (1987). Toward a cognitive theory for the measurement of achievement. In RonningR. R.GloverJ.ConoleyJ. C.WittJ. C. (Eds.), The influence of cognitive psychology on testing and measurement (pp. 41–85). Hillsdale, NJ: Erlbaum.
36.
GlaserR.RaghavanK.BaxterG. (1992). Cognitive theory as the basis for design of innovative assessment: Design characteristics of science assessments. (CSE Tech. Rep. No. 349). Los Angeles: University of California, National Center for Research on Evaluation Standards, and Student Testing.
37.
GlennanT. K.Jr. (1967). Issues in the choice of development policies. In MarschakT.GlennanT. K.Jr.SummersR. (Eds.), Strategy for research and development (pp. 13–48). New York: Springer-Verlag.
38.
GoldmanS. R.PellegrinoJ.W.BransfordJ. (in press). Assessing programs that invite thinking. In BakerE. L.O'NeilH. F.Jr. (Eds.), Technology assessment in education and training. Hillsdale, NJ: Erlbaum.
39.
GordonE.Speaker (1991, November). Alternatives for measuring performance. Schools that work (Video Conference No. 4) Chicago: North Central Regional Educational Laboratory.
40.
GottS. P. (1988). Apprenticeship instruction for real-world tasks. In RothkopfE. Z. (Ed.), Review of research in education (Vol. XV, pp. 97–169). Washington, DC: American Educational Research Association.
41.
HaertelE. H.WileyD. (1989, October). Poset and lattice representations of ability structures: Implications for test theory. Presentation at the meeting of the National Academy of Education, New York.
42.
HawkinsJ.CollinsA.FrederiksenJ. (1990, September). Interactive technologies and the assessment of learning. Paper presented at the UCLA Center for Technology Assessment conference “Technology Assessment: Estimating the Future,” Los Angeles.
43.
HowardA.BrayD. W. (1988). Managerial lives in transition: Advancing age and changing times. New York: Guilford Press.
JulianE. R.SchumacherC. F. (1988, March). CBT pilot examination: Results and characteristics of CBX. Paper presented at the National Board of Medical Examiners Conference on Computer-Based Testing in Medical Education and Evaluation, Philadelphia.
46.
KintschW. (1992). A cognitive architecture for comprehension. In PickH. L.Jr.van den BroekP.KnillD. C. (Eds.), Cognition: Conceptual and methodological issues (pp. 143–163). Washington, DC: American Psychological Association.
47.
KleinS. P. (1992, April). The effect of content area and test type on bar exam scores. Paper presented at the National Conference of Bar Examiners, Chicago.
48.
KleinS. P.BolusR. E. (1982). An analysis of the relationship between clinical legal skills and bar examination results. Unpublished report prepared for the Committee of Bar Examiners of the State Bar of California and the National Conference of Bar Examiners, Los Angeles.
49.
KoretzD.LewisE.Skewes-CoxT.BursteinL. (1992). Omitted and not-reached items in mathematics in the 1990 National Assessment of Educational Progress (CSE Tech. Rep. No. 357). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
50.
KoretzD.McCaffreyD.KleinS.BellR.StecherB. (1993). The reliability of scores from the 1992 Vermont Portfolio Assessment Program (CSE Tech. Rep. No. 355). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
51.
KoretzD.StecherB.Si DeibertE. (1992). The Veimont Portfolio Assessment Pwgiam: Interim report on implementation and impact, 1991–92 school year (CSE Tech. Rep. No. 350). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
52.
LesgoldA. (in press). Assessment of intelligent training technology. In BakerE. L.O'NeilH. F.Jr. (Eds.), Technology assessment in education and training. Hillsdale, NJ: Erlbaum.
53.
LinnR. L. (1993). Linking the results of distinct assessments. Applied Measurement in Education, 6, 83–102.
54.
LinnR. L.BakerE. L.DunbarS. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 15–21.
55.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan.
56.
MislevyR. J. (1992). Linking educational assessments: Concepts, issues, methods, and prospects. Princeton, NJ: Educational Testing Service, Policy Information Center.
57.
MitchellR. (1992). Testing and learning: How new approaches to evaluation can improve American schools. New York: Free Press.
58.
National Council on Education Standards and Testing. (1992). Raising standards for American education. Washington, DC: Author.
59.
National Education Goals Panel. (1991). Measuring progress toward the national education goals: Potential indicators and measurement strategies (Compendium of interim resource group reports). Washington, DC: Author.
60.
NewmannF. M. (1991). What is a “restructured” school! A framework to clarify means and ends. Madison: University of Wisconsin-Madison, Center on Organization and Restructuring of Schools.
61.
O'NeilH. F.Jr.SugrueB.AbediJ.BakerE.GolanS. (1992). NAEP TRP Task 3a: Experimental motivation study (Final report of experimental studies on motivation and NAEP test performance, deliverable to NCES, Contract No. RS90159001). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
62.
PandeyT. (1990). Power items and the alignment of curriculum and assessment. In KulmG. (Ed.), Assessing higher order thinking in mathematics (pp. 39–51). Washington, DC: American Association for the Advancement of Science.
63.
ReidyE.Discussant (1992, December). Critical issues and future directions in creating a national system of assessments. Symposium conducted at the meeting of the National Education Goals Panel, Washington, DC.
64.
ResnickL. B.ResnickD. P. (1992). Assessing the thinking curriculum: New tools for educational reform. In GiffordB. R.O'ConnorM. C. (Eds.), Changing assessments: Alternative views of aptitude, achievement, and instruction (pp. 37–75). Boston: Kluwer.
65.
SaxeG. B. (1991). Culture and cognitive development: Studies in mathematical understanding. Hillsdale, NJ: Erlbaum.
66.
ScrivenM. (1967). The methodology of evaluation. In TylerR. W.GagnéR. M.ScrivenM. (Eds.) Perspectives on curriculum evaluation (AERA Monograph Series on Curriculum Evaluation, No. 1, pp. 39–83). Chicago: Rand McNally.
67.
The Secretary's Commission on Achieving Necessary Skills. (1991). What work requires of schools: A SCANS report for America 2000. Washington, DC: U.S. Department of Labor.
68.
ShavelsonR. J.BaxterG. P.PineJ.YureJ.GoldmanS. R.SmithB. (1989, October). Performance indicators for large-scale science assessment. Presentation at UCLA Center for Research on Evaluation, Standards, and Student Testing conference Taking Stock of Educational Quality, Los Angeles.
69.
ShavelsonR. J.BaxterG. P.PineJ.YureJ.GoldmanS. R.SmithB. (1991). Alternative technologies for large-scale science assessment: Instrument of education reform. School Effectiveness and School Improvement, 2, 1–18.
70.
ShepardL. (1991). Psychometricians' beliefs about learning. Educational Researcher, 20, 2–16.
71.
SiegelA. I. (1986). Performance tests. In BerkR. A. (Ed.), Performance assessment: Methods and applications (pp. 121–142). Baltimore, MD: Johns Hopkins University Press.
72.
SlavinR. E. (1987). Best-evidence synthesis: An alternative to meta-analytic and traditional reviews. In ShadishW. R.Jr.ReichardtC. S. (Eds.), Evaluation studies: Review annual (Vol. 12, pp. 667–673). Newbury Park, CA: Sage.
73.
SwezeyR.SalasE. (1992). Teams: Their training and performance. Norwood, NJ: Ablex.
74.
TierneyR. J.CarterM. A.DesaiL. E. (1991). Portfolio assessment in the reading-writing classroom. Norwood, MA: Christopher-Gordon.
75.
TuckerM.ResnickL. B. (1990). Setting a new standard: Toward an examination system for the United States (Proposal for the New Standards Project from the Learning Research and Development Center, University of Pittsburgh, and the National Center on Education and the Economy). Pittsburgh, PA: University of Pittsburgh.
76.
U.S. Department of Education. (1991). America 2000: An education strategy. Washington, DC: Author.
77.
van der VleutenC. P. M.SwansonD. B. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2, 58–76.
78.
WebbN. M. (1990). Translation between symbolic representations: Relationship between instruction and performance (CSE Tech. Rep. No. 329). Los Angeles: University of California, Center for the Study of Evaluation.
79.
WebbN. M. (1993). Collaborative group versus individual assessment in mathematics: Group processes and outcomes (CSE Tech. Rep. No. 352). Los Angeles: University of California, Center for the Study of Evaluation.
80.
WebbN. M.YanK.QiS.BusheyB. (1991). Assessment of problem solving: Structured alternatives to traditional word problems. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
81.
WebbN. M.YasuiE. (1992). The influence of problem context on mathematics performance (CSE Tech. Rep. No. 346). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing.
82.
WigdorA. K.GreenB. F.Jr. (1991). Performance assessment for the workplace (Vol. 1). Washington, DC: National Academy Press.
83.
WigginsG. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 70, 703–713.
