Participant Observation and the Pursuit of Truth: Methodological and Ethical Considerations

ALBRECHT G L (1985). Videotape safaris: entering the field with a camera. Qualitative Sociology 8, Winter, 325–43.

AMA CODE OF ETHICS (1991). American Marketing Association.

ANDERSON D R ; LORCH E P ; FIELD D E ; COLLINS P A , & NATHAN JG (1986). Television viewing at home: age trends in visual attention and time with TV. Child Development 57, 1024–33.

BABBIE E (1986). The Practice of Social Research, 4th ed. Belmont CA : Wadsworth Publishing Co.

BELK R W ; SHERRY JF Jr , & WALLENDORF W (1988). A naturalistic inquiry into buyer and seller behavior at a swap meet. Journal of Consumer Research 14, March, 449–70.

BOUCHARD T J Jr (1976). Field research methods: interviewing, questionnaires, participant observation, systematic observation, unobtrusive measures: in Handbook of Industrial and Organizational Psychology. Chicago: Rand McNally.

BULMER M (1982). The merits and demerits of covert participant observation, in Social Research Ethics. New York: Holmes and Meier Publishers, Inc.

CALDER B (1977). Focus groups and the nature of qualitative marketing research. Journal of Marketing Research 14, August, 353–364.

CHURCHILL G A Jr. (1979). A paradigm for developing better measures of marketing/constructs. Journal of Marketing Research 16, February, 64–73.

COTE J A , & FOXMAN ER (1986). A positivist's reactions to a naturalistic inquiry experience: in Advances in Consumer Research 14, Association for Consumer Research.

DENZIN N K , & ERIKSON K (1982). On the ethics of disguised observation: an exchange. In Social Research Ethics. New York: Holmes and Meier Publishers, Inc.

DESHPANDE R (1983). ‘Paradigms lost’: On theory and method in research in marketing. Journal of Marketing 47, Fall, 101–111.

DOUGLAS J D (1976). Investigative Social Research. Beverly Hills, Ca: Sage Publications

FRANCOIS W E (1990). Mass Media Law and Regulation, 5th ed, Ames, Iowa: Iowa State University Press.

GABRIEL C (1990). The validity of qualitative market research. Journal of the Market Research Society 32, 4, 507–518.

GRIFFIN R W (1987). Management, 2nd. ed. Boston, MA: Houghton Mifflin Co.

HILL R P (1992). Homeless women, special possessions, and the meaning of ‘home’: an ethnographic case study. Journal of Consumer Research 18, December, 298–310.

HIRSCHMAN E (1986). Humanistic inquiry in marketing research: philosophy, method and criteria. Journal of Marketing Research 23, August, 237–249.

HODGES L W (1988). Undercover, masquerading, surreptitious taping. Journal of Mass Media Ethics 3, Fall, 26–36.

HOMAN R , & BULMER M (1982). On the merits of covert methods: a dialogue. In Social Research Ethics. New York: Holmes and Meier Publishers, Inc.

HUNT S D (1984). Should marketing adopt relativism? 1984 AMA Winter Educator's Conference: Scientific Method in Marketing. Chicago: American Marketing Association.

JORGENSEN D L (1989). Participant Observation, A Methodology for Human Studies. Newbury Park, CA: Sage Publications.

KAZDIN A E (1982). Observer effects: reactivity of direct observation. In Choosing Observers to Study Behavior. San Francisco: Jossey-Bass Inc.

KIDDER L H , JUDD C M , & SMITH ER (1986). Research Methods in Social Relations, 5th ed. New York: Holt, Rhinehart and Winston.

KIRK J , & MILLER M L (1986). Reliability and Validity in Qualitative Research. Beverly Hills, CA: Sage Publications.

LINCOLN Y S , & GUBA EG (1985). Naturalistic Inquiry. Beverly Hills, CA: Sage Publications.

MRS CODE OF CONDUCT (1992). The Market Research Society.

NEALE J M , & LIEBERT RM (1980). Science and Behaviour, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall.

O'GUINN T C , & FABER RJ (1989). Compulsive buying: a phenomenological exploration. Journal of Consumer Research 16, September, 147–57.

OLSHAVSKY R W (1973). Customer-salesman interaction in appliance retailing. Journal of Marketing Research 10, May, 208–12.

PENNINGTON A L (1968). Customer-salesman bargaining behavior in retail transactions. Journal of Marketing Research 5, August, 255–62.

PETER J P (1981). Construct validity: a review of psychometric basics and recent marketing practices. Journal of Marketing Research 18, May, 133–45.

PETER J P , & OLSON JC (1983). Is science marketing? Journal of Marketing 47, Fall, 111–123.

ROBSON 5 (1991). Ethics: informed consent or misinformed compliance? Journal of the Market Research Society 33, 1, 19–28.

ROBSON S , & WARDLE J (1988). Who's watching whom? A study of the effects of observers on group discussions. Journal of the Market Research Society 30, 3, 333–359.

SCHULER H (1982) Ethical Problems in Psychological Research. New York: Academic Press.

STONEMAN Z , & BRODY GH (1983). Family interactions during three programs. Journal of Family Issues 4, June, 349–65.

SYKES W (1990). Validity and reliability in qualitative market research: a review of the literature. Journal of the Market Research Society 32, 3, 289–328.

The Hawthorne Effect

Graham Chant

Chant Link & Associates Pty Ltd, Australia

One of the major frustrations I have continually encountered when dealing with marketing personnel, and indeed marketing researchers, is their willingness to accept claims arising from old (and I guess, new) studies without examining the original research on which the claimed outcomes are based. In effect it is often the case that marketers accept the conventional wisdom that seems to be inherent in some of these studies, when even a cursory examination of their own motives would suggest the original study generated incorrect conclusions. This applies to the so-called ‘Hawthorne Effect’ in which employees supposedly increase their performance as a result of being observed. I wonder how many partners in research companies would be motivated by the ‘Hawthorne Effect’, and not by monetary incentive?

The ‘Hawthorne Effect’ was mentioned in the Stafford paper, ‘Participant observation and pursuit of truth’ (JMRS 35,1, p 66). While the overall paper makes a significant contribution to marketing research, the conclusions concerning the effects of observations on subjects should not be justified on the basis of the Hawthorne Studies because an examination of the original evidence on which the ‘Hawthorne Effect’ was based shows the studies to be very seriously flawed (for example see Alex Carey, ‘The Hawthorne Studies: A Radical Criticism’ in American Sociological Review, 32, 3, June 1967). The latter paper illustrates that the existence of the ‘Hawthorne Effect’ had some serious (to say the least) defects including:

º The study was based on a sample of five people.

º Two of the people were sacked and replaced after it became apparent that the intervention was not working.

º The replaced employees had a major biasing effect on the outputs of the research. Indeed employee output only showed a significant increase after the two employees who had the lowest output were dismissed and were replaced by employees who were output leaders.

º The so-called friendly supervision that produced greater employee output included paying employees for time not worked (and as might be expected, the piece-rate increased).

º Levels of friendly and relaxed supervision varied throughout the study, however the researchers only provided evidence associated with increasing levels of friendliness and not decreasing levels.

No wonder the researchers discovered an ‘Effect’!

Thus, marketers who see the ‘Hawthorne Effect’ as a means of explaining marketing or marketing research phenomena are making a major mistake.

But I suspect many people who refer to the ‘Hawthorne Effect’ do not see it as applying to their own activities. For example no market researchers that I am aware of have made the mistake of attempting to motivate their staff by using the ‘Hawthorne Effect’ rather than attractive remuneration levels, and we still have to pay incentives to group discussion respondents rather than simply attending to them.

The real issue here is that marketers and market researchers need to be extremely careful about the uncritical use of psychology studies, and especially those where the original data have not been sighted.

Thomas F Stafford Cameron University and Maria Royne Stafford, University of North Texas

We appreciate the comments by Graham Chant regarding the Hawthorne Studies. His words seem to imply (correctly) that the widely-accepted metaphor for subject reaction effects, ‘the Hawthorne Effect’, is often used more for the shared meaning we as researchers ascribe to it, rather than for the veracity of the research study from which it originates. Indeed, as Mr Chant points out, there appear to be some fundamental issues to be dealt with in terms of the research design employed by Western Electric in its study many years ago at its Hawthorne plant.

On the other hand, the wide diffusion and subsequent recognition of the study and its conclusions among the marketing fraternity have ensured that the phenomenon investigated in the Hawthorne study has entered into the methodological paradigm, not as an example of how participation should be carried out, but as part of the mythology of scientific method in the social sciences. Certainly, our use of the term ‘Hawthorne Effect’ is more in line with its paradigmatic interpretation rather than in the context of the study from which the term originated.

Based on Chant's comments, we feel that there are two issues that need to be addressed in this response: (1) the defects of the Hawthorne Studies; and (2) the relative importance of the Hawthorne Studies to our original arguments. We will deal with each of these issues separately.

Mr Chant bases his commentary on Alex Carey's (1967) article concerning methodological biases of the Hawthorne Studies. Carey's article certainly provides insight into some of the problems associated with the experimental design of field research in industrial settings. Clearly, there are methodological problems associated with the Hawthorne Studies, and one should be cautious when generalising from the results. On this point we are in agreement with Mr Chant. The Hawthorne sample is small, and objective generalisation of results is threatened – just as pointed out by Carey and reiterated by Mr Chant.

However, the purpose of our original paper as well as this response is not to generalise from the Hawthorne Studies nor to defend the results. It goes without saying that when one looks hard enough, generally, fault can be found with almost any scientific study – even the most methodologically rigorous. Chant's comments are, therefore, welcome as a means to stimulate discussion regarding some of the methodological issues we face in market research.

Our original paper (Stafford & Stafford 1993) concerned the validity of research findings in participant observation. One of the factors which can affect the validity and thus the ‘truth value’ of findings in participant observation is subjects’ reactions to the presence of the researcher. As indicated above, this reactive phenomenon has for some time commonly been known by market researchers under the rubric of ‘the Hawthorne Effect.’

To use this term as a semantic signal of a concept we are all familiar with and which is central to the theme of our paper is in no way meant to qualify the veracity or the quality of the original Hawthorne studies. One might argue, though, that any research which serves the purpose of bringing such an important potential bias effect to our attention has inherent value outside of an aside from the pragmatic interpretation of the individual study's experimental design, results or conclusions. The Hawthorne studies provide arguably the first widely-recognised caution to researchers against the effects of subject knowledge of experimental observation. Whether one chooses to label this phenomenon ‘subject reaction effects in response to the knowledge of scientific observation’, or employs the more telegraphic ‘Hawthorne Effect’ label, the message and its meaning are the same. We feel sure that certain physicists might also take issue with employment of the term ‘the Uncertainty Principle’ used to describe quantum effects of observation on materials observed at the molecular level. All the same, regardless of opinions over the name given to the effect, physicists around the world would share a common meaning of what the term meant.

This discussion over the propriety of the semantics employed to convey a meaning which we all share should not obscure the importance of the phenomenon underlying the descriptive terminology. That we disagree about what it should be called is not so important as the fact that we all agree that it exists and has significance.

CAREY A (1967). The Hawthorne Studies: a radical criticism. American Sociological Review, 32, 3, 403–416.

RAY M L , and SHERILL P N (1973). Unobtrusive marketing research techniques. In The Dartnell Marketing Manager's Handbook. Chicago: The Dartnell Corporation.

SELLTIZ C ; JAHODA M ; DEUTSCH M , and COOK S W (1959). Research Methods in Social Relations. New York: Holt, Rinehart & Winston.

STAFFORD M R , and STAFFORD T F (1993). Participant observation and the pursuit of truth: methodological and ethical considerations. Journal of the Market Research Society, 35, 1, 63–76.