The validity of true experiments is threatened by a class of self-report biases that affect all
respondents at pretest but which are diminished by treatment, yielding noncomparable treated
and control subjects at posttest. These biases include inaccurate self-evaluations due to (a) lack
of understanding of dimensions of self-rating, (b) unconscious needs to rationalize sources of
severe emotional distress, (c) distressed or altered states (e.g., drug states) that lower the ability
to report accurately, and (d) deliberate impression management in service of accessing desired
treatment. They are detectable via external criteria, special conditions of measurement, and
retrospective pretests, and may be lessened in several ways. Unchecked, they produce treatment
by self-report bias interactions that undermine the validity of even true experiments.
Get full access to this article
View all access options for this article.
References
1.
Achenbach, T.M., S.H. McConaughy, and C.T. Howell.1987. Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity . Psychological Bulletin101:213-32.
2.
Aiken, L.S.1986. Retrospective self-reports by clients differ from original reports: Implications for the evaluation of drug treatment programs. International Journal of the Addictions21:767-88.
3.
Aronson, S., and L.S. Aiken.1980. Compliance of child care programs with health and safety standards. Impact of program evaluation and advocate training. Pediatrics65.318-25.
4.
Ball, J.C., J.W. Shaffer, and D.N. Nurco.1983. The day-to-day criminality of heroin addicts in Baltimore—A study in the continuity of offence rates. Drug and Alcohol Dependence12:119-42.
5.
Baumeister, R.F.1982. A self-presentational view of social phenomena. Psychological Bulletin91:3-26.
6.
Beck, A.T.1967. Depression: Clinical, experimental, and theoretical aspects . New York: Harper & Row.
7.
—. 1976. Cognitive therapy and the emotional disorders. New York: International Universities Press.
8.
Bedeian, A.G., A.A. Armenakis, and R.W. Gibson.1980. The measurement and control of beta change. Academy of Management Review5:561-66.
9.
Ben-Yehuda, N.1980. Are addicts' self-reports to be trusted?International Journal of the Addictions15:1265-70.
10.
Braginsky, B.M., D.D. Braginsky, and K. Ring.1969. Methods of madness : The mental hospital as a last resort. New York: Holt, Rinehart & Winston.
11.
Bray, J.H., S.E. Maxwell, and G.S. Howard.1984. Methods of analysis with response-shift bias. Educational and Psychological Measurement44:781-804.
12.
Brown, G.W., and T. Harris.1978. Social origins of depression. New York: Free Press.
13.
Chassin, L., C. Presson, S. Sherman, E. Corty, and R. Olshavsky.1985. Predicting the onset of cigarette smoking in adolescents: A longitudinal study.Journal of Applied Social Psychology14:224-43.
14.
Cook, T.D.1985. Postpositivist critical multiplism. In Social Science and Social Policy, edited by R. L. Shotland and M. M. Mark, 21-62. Beverly Hills, CA: Sage.
15.
Cook, T.D., and D.T. Campbell.1979. Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.
16.
Evans, R.I., W.B. Hansen, and M.B. Mittlemark .1977. Increasing the validity of self-reports of smoking behavior in children. Journal of Applied Psychology62:521-23.
17.
Golembiewski, R.T., K. Billingsley, and S. Yeager.1976. Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science12:133-57.
18.
Hochstim, J.R., and K.S. Renne.1971. Reliability of response in a sociomedical population study. Public Opinion Quarterly35:69-79.
19.
Howard, G.S.1980. Response-shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review4:93-106.
20.
Howard, G.S., and P.R. Dailey.1979. Response-shift bias: A source of contamination of self-report measures. Journal of Applied Psychology64:144-50.
21.
Howard, G.S., P.R. Dailey, and N.A. Gulanick.1979. The feasibility of informed pretests in attenuating response-shift bias. Applied Psychological Measurement3:481-94.
22.
Howard, G.S., J. Millham, L. Slaten, and L. O'Donnell.1981. Influence of subject response style effects on retrospective measurements. Applied Psychological Measurement5:89-100.
23.
Howard, G.S., K.M. Ralph, N.A. Gulanick, S.E. Maxwell, D.W. Nance, and S.K. Gerber.1979. Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychological Measurement3:1-23.
24.
Howard, G.S., R.R. Schmeck, and J.H. Bray.1979. Internal validity in studies employing self-report instruments: A suggested remedy. Journal of Educational Measurement16: 129-35.
25.
Hser, Y., M.D. Anglin, and O.P. Chou.1988. Evaluation of drug abuse treatment: A repeated measures design assessing methadone maintenance. Evaluation Review12:547-70.
26.
Jones, E.E., and H. Sigall.1971. The bogus pipeline: A new paradigm for measuring affect and attitude. Psychological Bulletin76:349-64.
27.
Kenrick, D.T., and D.O. Stringfield .1980. Personality traits and the eye of the beholder: Crossing some traditional philosophical boundaries in the search for consistency of all people. Psychological Review87:88-104.
28.
Mabe, P.A., III, and S.G. West.1982. Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology67:280-96.
29.
Messick, S.M.1989. Validity. In Educational Measurement. 3d ed., edited by R. L. Linn, 13-104. New York: Macmillan.
30.
Murray, D.M., and C.L. Perry.1987. The measurement of substance use among adolescents: Why is the "bogus pipeline" method needed?Addictive Behaviors12:225-33.
31.
Neugebauer, R.1981. The reliability of life event reports. In Stressful Life Events and Their Contexts, edited by B. S. Dohrenwend and B. P. Dohrenwend, 85-107. New York: Prodist.
32.
Ross, M.1989. Relation of implicit theories to construction of personal histories. Psychological Review96:341-57.
33.
Ross, M., and M. Comway.1986. Remembering one's own past: The construction of personal histories. In Handbook of Motivation and Cognition . Vol. 1, Foundations of Social Behavior, edited by R. M. Sorrentino and E. T. Higgins, 122-49. New York: Guilford.
34.
Rounsaville, B.J., and H.D. Kleber.1981. Comparison of opiate addicts' reports of psychiatric history with reports of significant-other informants." American Journal of Drug and Alcohol Abuse8:51-69.
35.
Sandler, I., and R. Guenther.1985. Assessment of life stress events. In Measurement Strategies in Health Psychology, edited by P. Karoly, 555-600. New York: Wiley.
36.
Scriven, M.1976. Maximizing the power of causal investigation: The modus operandi method. In Evaluation Studies Review Annual. Vol. 1, edited by G. V Glass, 101-18. Beverly Hills, CA: Sage.
37.
Shaffer, G.S., V. Saunders, and W.A. Owens.1986. Additional evidence for the accuracy of biographical data: Long-term retest and observer ratings. Personnel Psychology39:791-809.
38.
Skinner, H.A.1984. Assessing alcohol use by patients in treatment. In Research Advances in Alcohol and Drug Problems. Vol. 8, edited by R. G. Smart, H. D. Cappell, F. B. Glaser , Y. Israel, H. Kalant, R. E. Popman, W. Schmidt, and E. M. Sellers , 183-208. New York: Plenum.
39.
Sobell, L.C., and M.B. Sobell.1981. Effects of three interview factors on the validity of alcohol abusers' self-reports. American Journal of Drug and Alcohol Abuse8:225-37.
40.
Webb, E. J., D. T. Campbell , R. D. Schwartz, L. Sechrest, and J. B. Grove. (Eds.). 1981. Nonreactive measures in the social sciences. 2d ed. Boston : Houghton Mifflin.
41.
West, S.G., and J.F. Finch. Forthcoming. Measurement and analysis issues in the investigation of personality structure. In Handbook of Personality Psychology , edited by S. Briggs, R. Hogan, and W. Jones.New York: Academic Press.
