The traditional procedure for obtaining judged rat ings, to ascertain if treatment-related change has oc curred, involves the randomization of the materials to be rated. An alternative approach (linked judgments) is investigated as a potential solution to certain instru mentation-related threats to statistical conclusion valid ity of the incumbent rating procedure. Data from a weight reduction study are presented which suggest that linked raters' judgments provide both a more powerful and a more valid index of treatment effec tiveness than the traditional procedure.
Get full access to this article
View all access options for this article.
References
1.
Campbell, D.T., & Stanley, J.C.Experimental and quasi-experimental designs for research on teaching . In N. L. Gage (Ed.), Handbook of research on teaching. Chicago: Rand McNally, 1963.
2.
Cook, T.D., & Campbell, D. T. Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally, 1979.
3.
Cronbach, L.J., & Furby, L. "How we should measure 'change'—or should we?" Psychological Bulletin,1970, 74, 68-80.
4.
DiLoreto, A.O.Comparative psychotherapy: An experimental analysis. Chicago : Aldine-Atherton, 1971.
5.
Gottman, J.M., & McFall, R. M. Self-monitoring effects in a program for potential H. S. Dropouts. Journal of Consulting and Clinical Psychology , 1972, 39, 273-281.
6.
Howard, G.S., & Maxwell, S.E.Linked raters' judgments: A more sensitive measure of change. Evaluation Review, 1981, 6, 140-146.
7.
Huck, S., & McLean, R. Using a repeated measure ANOVA to analyze the data from a pretest-posttest design: A potentially confusing task. Psychological Bulletin,1975, 82, 511-518.
8.
Johnson, S.M., & White, G.Self-observation as an agent of behavior change. Behavior Therapy, 1971, 2, 488-497.
Meichenbaum, D.H., Gilmore, J.B., & Fedoravicious, A.Group insight versus group desensitization in treating speech anxiety. Journal of Consulting and Clinical Psychology, 1971, 36, 410-421.
11.
Miller, G.A.The magical number seven, plus or minus two: Some limits on our capacity for processing information. In D. C. Beardslee & M. Wertheimer (Eds.), Readings in perception. Princeton NJ: Van Nostrand, 1958.
12.
Myers, J.L.Fundamentals of experimental design. Boston: Allyn & Bacon, 1979.
13.
O'Leary, K.D. & Borkovec, T.D.Conceptual, methodological, and ethical problems of placebo groups in psychotherapy research. American Psychologist , 1978, 78, 821-830.
14.
Parducci, A.Category ratings: Still more contextual effects. In B. Wegener (Ed.), Social attitudes and psychological measurement . Hillsdale NJ: Erlbaum , 1982.
15.
Santa, J.L., Miller, J.J., & Shaw, M.L.Using quasi-F to prevent alpha inflation due to stimulus variation . Psychological Bulletin, 1979, 86, 37-46.
16.
Stuart, R.B., & Davis, B.Slim chance in a fat world: Behavior control of obesity. Champaign IL: Research Press, 1972 .
