In this paper, we object to Michael Scriven's claim that the basic logic of evaluation is criterial and standards-based. We note that valuing is an integral part of perception and that valuing within perception, repeatedly refined, is an even more basic logic of evaluation. We find unpersuasive his claim that making the final synthesis “governed” will diminish bias, noting that bias will find its way into the required statements of need, function, standards and weighting. We offer our alternative for disciplining the synthesis process, by urging more systematic and demanding critiques of emerging interpretations and values, and by more deliberately using competing conceptual organizers (e.g., goals, issues, decisions and elements of the rules Scriven advocates) as temporary and dialectical grounds for reconsidering the evolving meanings of the program, including its merit and shortcoming.
Get full access to this article
View all access options for this article.
References
1.
AngelesP. A. (1981). Dictionary of philosophy. New York: Barnes and Noble.
2.
BolmanL. G.DealT. D. (1991). Reframing organizations: Artistry, choice, and leadership. San Francisco: Jossey-Bass.
3.
ChandlerD. G. (1973). The campaigns of Napoleon. New York: MacMillan.
4.
DenzinN. (1984). The research act. Englewood Cliffs, NJ: Prentice Hall.
5.
DenzinN. K.LincolnY. S. (1994). Handbook of qualitative research. Newbury Park, CA: Sage.
6.
DiederichP. B. (1957). The improvement of essay examinations. Research Memorandum RM-57-3. Princetion, NJ: Educational Testing Service.
7.
Educational Testing Service. (1996, April). Writing skill assessment: Problems and prospects (Policy Issues Perspective series). Princeton, NJ: Breland, H. M..
8.
FlewA. (1984). A dictionary of philosophy; Revised second edition. New York: St. Martin's Press.
9.
FlickU. (1992). Triangulation revisited: Strategy of validation or alternative? Journal for the Theory of Social Behaviour, 22 (2), 175–198.
10.
FournierD. M. (1995). Establishing evaluative conclusions: A distinction between general and working logic. In FournierD. M. (Ed.), Reasoning in evaluation: Inferential links and leaps. New Directions for Evaluation, 68. (pp. 15–32). San Francisco: Jossey-Bass.
11.
GubaE. G.LincolnY. S. (1994) Competing paradigms in qualitative research. In DenzinN. K.LincolnY. S. (Eds.), Handbook of qualitative research. (pp. 105–117). Newbury Park, CA: Sage.
12.
HabermasJ. (1979). Communication and the evolution of society. Boston: Beacon Press.
13.
HouseE. R. (1980). The logic of evaluative argument. In HouseE. R. (Ed.), Evaluating with validity. (pp. 67–96). Beverly Hills, CA: Sage.
14.
HouseE. R. (1993). Professional evaluation. Newbury Park, CA: Sage.
15.
Lévi—StraussC. (1966). The savage mind (2nd ed.). Chicago: University of Chicago Press.
16.
MacDonaldB. (1977). A political classification of evaluation studies. In HamiltonD. (Ed.), Beyond the numbers game. (pp. 224–227). London: Macmillan.
17.
MessickS. (1995). Validity of psychological assessment: Validation of inferences from person's responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
18.
MintzbergH. (1994). The rise and fall of strategic planning: Reconceiving roles for planning, plans, planners. New York: The Free Press.
19.
ParlettM.HamiltonD. (1976). Evaluation as illumination: A new approach to the study of innovatory programmes. In GlassG. V. (Ed.), Evaluation Studies Review Annual, 1, (pp. 140–157). Beverly Hills, CA: Sage.
20.
PirsigR. M. (1974). Zen and the art of motorcycle maintenance. New York: William Morrow.
21.
PetrieH. (1995). Purpose, context, and synthesis: Can we avoid relativism? In FournierD. M. (Ed.), Reasoning in evaluation: Inferential links and leaps. New Directions for Evaluation, 68. (pp. 81–91). San Francisco: Jossey-Bass.
22.
ProppéÓ. (1979). Dialectical evaluation. Urbana, IL: Center for Instructional Research and Curriculum Evaluation (mimeo).
23.
RettigS. (1990). The discursive social psychology of evidence: Symbolic construction of reality. New York: Plenum Press.
24.
SadlerD. R. (1981). Intuitive data processing as a potential source of bias in evaluation. Educational Evaluation and Policy Analysis, 3 (4), 25–31.
25.
SchwandtT. A. (1995). Indexing the practice of reasoning in evaluation. Unpublished manuscript, College of Education, Indiana University.
26.
SchwandtT. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2 (1), 58–72.
27.
ScrivenM. (1966). Primary philosophy. New York: McGraw-Hill.
28.
ScrivenM. (1967). The methodology of evaluation. AERA Monograph Series on Curriculum Evaluation, 1. Chicago: Rand, McNally.
29.
ScrivenM. (1975). Evaluation bias and its control (Occasional paper #4). Kalamazoo, MI: Evaluation Center, Western Michigan University.
30.
ScrivenM. (1980). The logic of evaluation. Inverness, CA: Edgepress.
31.
ScrivenM. (1981). Product evaluation. In SmithN. L. (Ed.), New Techniques for Evaluation (pp. 121–166). Beverly Hills, CA: Sage.
32.
ScrivenM. (1990). The evaluation of hardware and software. Studies in Educational Evaluation, 16, 3–40.
33.
ScrivenM. (1991). Evaluation thesaurus. (4th ed.). Newbury Park, CA: Sage.
34.
ScrivenM. (1992). Evaluation and critical reasoning: Logic's last frontier? In TalaskaR. A. (Ed.), Critical reasoning in contemporary culture (pp. 353–406). Albany: SUNY Press.
35.
ScrivenM. (1994). The final synthesis. Evaluation Practice, 15, 367–382.
36.
ScrivenM. (1995). The logic of evaluation and evaluation practice. In FournierD. M. (Ed.), Reasoning in evaluation: Inferential links and leaps. New Directions for Evaluation, 68. (pp. 49–70). San Francisco: Jossey-Bass.
37.
StakeR. E. (1995). The invalidity of standardized testing for measuring mathematics achievement. In RombergT. A. (Ed.). Reform in school mathematics and authentic assessment. Albany, NY: State University of New York Press
38.
TolstoyL. (1957). War and peace. EdmondsR. (Trans.) Harmondsworth, England: Penguin. (Original work published 1869).
39.
ToulminS. (1983). The construal of reality: Criticism in modern and postmodern science. In MitchellW. J. T. (Ed.), The politics of interpretation (pp. 99–117). Chicago: University of Chicago Press.
