This study sought to examine problems in assessing field experiments with conventional survey-based data gathering methods. Two phenomena are examined: (a) increases in reactiveness, and (b) the recalibration of survey scales. The theoretical dimensions of such changes are discussed, including such processes as consistency, saliency, priming, and raised expectations. Methods of identifying these sources of contamination within two-group experimental designs are presented and illustrated with data from a recent field experiment involving 78 female sewing machine operators. In this instance, survey-data gathering processes increased "reactiveness," suggesting a general increase in the salience of experimental variables to subjects; and, scale recalibration appears to have occurred with respect to a primanV intervention variable. Methods of reducing or accommodating these possible sources of contamination are discussed, together with their implications for the protection of human subjects.
Get full access to this article
View all access options for this article.
References
1.
Armenakis, A., & Smith, L.A practical alternative to comparison group designs in OD evaluations: The abbreviated time series design. Academy of Management Journal, 1978,21,499-507.
2.
Armenakis, A. A., Feild, H. S., & Holley, W. H.Guidelines for overcoming empirically identified evaluation problems of organizational development change agents. Human Relations, 1976, 29,1147-1161.
3.
Bowers, D. G.OD techniques and their results in 23 organizations: The Michigan ICL study. Journal of Applied Behavioral Science, 1973, 9, 21-43.
4.
Brightman, H.The need for repeated measurement designs in organizational research. Academy of Management Journal, 1971, 14, 398-402.
5.
Campbell, D. T.Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 1957, 54, 297-312.
6.
Campbell, D. T., & Stanley, J. C.Experimental and quasi-experimental designs for research. Chicago: Rand McNally, 1963.
7.
Child, T. L., & Whiting, J. W.Determinants of level of aspiration and evidence from everyday life. Journal of Abnormal and Social Psychology, 1949, 44, 303-314.
8.
Clegg, C. W., Jackson, P. R., & Wall, T. D.The potential of cross-lagged correlation analysis in field research. Journal of Occupational Psychology, 1977, 50 (3), 177-196.
9.
Cook, T. D., & Campbell, D. T.The design and conduct of quasi-experiments and true experiments in field settings. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally, 1976, Pp. 223-326.
10.
Cummings, T. C., & Molloy, E. S.Improving productivity and the quality of work life. New York: Praeger, 1977.
11.
Ewen, R. B.Weighing components of job satisfaction. Journal of Applied Psychology, 1967,51, 68-73.
12.
Gardner, J. W.The use of the term "level of aspiration."Psychological Review, 1940,47,59-68.
13.
Golembiewski, R. T., Billingsley, K., & Yeager, S.Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 1976, 12, 133-157.
14.
Hackman, J. R., & Lawler, E. E.Employee reactions to job characteristics. Journal of Applied Psychology, 1971, 55, 259-296.
15.
Hackman, J. R., & Oldham, C. R.Development of job diagnostic survey. Journal of Applied Psychology, 1975, 60, 159-170.
16.
Iris, B., & Barrett, G. V.Some relations between job and life satisfaction and job importance. Journal of Applied Psychology, 1972, 56, 301-304.
17.
Katzell, R. A., Bienstock, P., & Faerstein, P. H.A guide to work productivity experiments in the United States 1971-75. New York: New York University Press, 1977.
18.
Kenny, D. A.Cross-lagged panel correlations: A test for spurioustiess. Psychological Bulletin, 1975, 82, 887-903.
19.
Kerlinger, F. N.Foundations of behavioral research. San Fransicso: Holt, Rinehart & Winston, 1973.
20.
King, A. S.Expectation effects in organizational change. Administrative Science Quarterly, 1974, 19, 221-230.
21.
Koch, J. L.Effects of goal specificity and performance feedback to work groups upon peer leadership, performance and attitudes. Human Relations, in press.
22.
Kornhauser, A. W.Mental health of the industrial worker. New York: Wiley, 1965.
23.
Locke, E. A.The nature and causes of job satisfaction. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally, 1976, Pp. 1297-1350.
24.
Locke, E. A.What is job satisfaction? Organizational Behavior and Human Performance, 1969, 4,309-336.
25.
Mobley, W. H., & Locke, E. A.The relationship of value importance to satisfaction. Organizational Behavior and Human Performance, 1970, 5, 463-483.
26.
Porras, J. I., & Berg, P. O.Evaluation methodology in organizational development: An analysis and critique. Journal of Applied Behavioral Science, 1978, 14, 151-173.
27.
Salancik, G. R., & Pfeffer, J.An examination of need-satisfaction models of job attitudes. Administrative Science Quarterly, 1977, 22, 427-456.
28.
Schaffer, R. H.Job satisfaction as related to need satisfaction in work. Psychological Monographs, 1953, 67, (14, Whole No. 364).
29.
Smith, P. C., Kendall, L. M., & Hulin, C. L.The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Chicago: Rand McNally, 1969.
30.
Solomon, R.An extension of control group design. Psychological Bulletin, 1949, 46, 137-150.
31.
Taylor, J. C.The quality of working life: An annotated bibliography, 1957-1972, NTIS PB 218380, January 1973.
32.
Taylor, J. C., & Bowers, D. G.Survey of organizations. Ann Arbor: University of Michigan Institute for Social Research, 1972.
33.
Turner, A. N., & Lawrence, P. R.Industrial jobs and the worker. Boston: Harvard University Graduate School of Business Administration, 1965.
34.
Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L.Unobtrusive measures: Nonreactive research in the social sciences. Chicago: Rand McNally, 1966.
35.
Wemimont, P. F.A systems view of job satisfaction. Journal of Applied Psychology, 1972, 56, 173-176.
36.
Zmud, R. W., & Armenakis, A. A.Understanding the measurement of change. Academy of Management Review, 1978, 3, 661-669.
