The Quality of the Quantity: Information Technology and the Evaluation of Data

Abstract

One of the problems with survey data is the need to impute an interpretive frame for the data. Generally this frame is developed ad hoc (in the design of the instrument) and modified post hoc (in explaining findings, apparent contractions, and variances from ad hoc expectations). While statistical methods provide measures of data consistency and validity, they cannot interpret the understandings of respondents. This paper offers an outline for using information technologies to aid researchers in the interpretation of respondent data. One method of re-capturing the interpretive context of subjects is to analyze the natural “talk” that necessarily occurs when subjects respond to survey questions. The new information technologies provide mechanisms for doing this. For example. with surveys that use Computer Assisted Telephone Interview systems, with modifications to the underlying computer technologies and the background database, voice and text records of respondents can be captured and stored. These records can then be used in conjunction with the statistical record. While there are physical limits (i.e. disk space; time, etc.) to such exercises, the basic information derived from text and audio sources can provide important contextual material for the evaluation of numeric data. Simple technological processes can be used to help gain a proxy understanding of the interpretive context (especially in conjunction with statistical evaluation). The data can also be used to further refine the survey instrument and evaluate the performance of the interviewers. A two stage development trajectory is offered. The first consists of a relatively simple modifications to the existing technology using text records, and the second looks to more expensive and complicated development of the core technologies. Other possible applications of these technologies are then discussed, and the significance of the Impact of these efforts are reviewed. Textual, Conversational & Numerical Data, Information Technologies. Qualitative & Quantitative Analysis.

Get full access to this article

View all access options for this article.

References

Andrich, David (1992) “On the Function of Fundamental Measurement in the Social Sciences: The Objective Measurement of Subjective Meaning,” mimeograph paper, International Conference on Social Science Methodology, ISA RC33 Logic and Methodology, Trento, Italy, June 22–26.

Barry, John A, (1991) Technobabble, MIT Press.

Bashe, CJ, LR Johnson, JH Palmer, EW Pugh (1986) IBM's Early Computers, MIT Press.

Briggs, Charles L. (1986) Learning How to Ask: A sociolinguistic appraisal of the role of the interview in social science research, Studies in the Social and Cultural Foundations of Language Series, volume 1, Cambridge University Press.

Burger. Jeff (1993) The Desktop Multimedia Bible, Reading, MA: Addison-Wesley Publishing Co.

Bursetin. L, HE Freeman, KA Sirotnik, G Delandshere, M Hollis (1985) “Data Collection: The Achilles Heel of Evaluation Research,” Sociological Methods and Research, volume 14(1), pp. 65–80.

Calhoun, Craig J. (1981) “The Microcomputer Revolution? Technical Possibilities and Social Choices,” Sociological Methods and Research. volume 9(4), pp. 397–437.

Collins, T. W. (1981) “Social Science Research and the Microcomputer,” Sociological Methods and Research, volume 9(4), pp. 438–460.

Converse, Jean M. and Stanley Presser (1986) Survey Questions: handcrafting the standardized questionnaire. Sage series in Quantitative Applications in the Social Sciences, volume 63, Beverly Hills, CA: Sage Publications.

10.

Davisson, William I. (197) Information Processing: Applications in the Social and Behavioral Sciences. NY: Appleton-Century Crofts.

11.

Fink, James C. (1983) “CATI's First Decade: The Chilton Experience,” Sociological Methods and Research, volume 12 (2), November, pp. 153–168.

12.

Freeman. Howard E. and J. Merrill Shanks (1983) “Special Issue on The Emergence of Computer-Assisted Survey Research,” Sociological Methods and Research, volume 12 (2), November.

13.

Friedheim, Elizabeth (1984) “Field Research and Word Processor Files: A Technical Note,” Qualitative Sociology, vol. 7 (1/2), pp. 90–97.

14.

Fujimura, Joan (1987) “Constructing ‘Doable’ Problems in Cancer Research: articulating alignments,” Social Studies of Science, volume 17. pp. 257–93.

15.

Gassaway, BM , WL Elders , J Cambell (1984) “Word Processors for Qualitative Sociologists: A Review Essay,” Qualitative Sociology, vol. 7 (1/2), pp. 157–168.

16.

Gigerenzer, G , Z Swijtink , T Porter , L Daston , J Beatty , and L Krueger (1989) The Empire of Chance: How Probability Changed Science and Everyday Life, Cambridge University Press.

17.

Goldberg, Adele (1988) A History of Personal Workstations. Reading, MA: Addison-Wesley Publishing Company.

18.

Goldstine, Herman H. (1972) The Computer: from Pascal to von Neumann. Princeton University Press.

19.

Grimshaw, Allen D. (1982) “Foreword: Special issue on Sound-Image Records in Social Interaction Research,” Sociological Methods and Research, volume 11 (2), pp. 121–144.

20.

Grimshaw, Allen D. (1982) “Sound-Image Data Records for Research on Social Interaction,” Sociological Methods and Research, volume 11 (2), pp. 115–119.

21.

Hacking, Ian (1990) The Taming of Chance, Cambridge University Press.

22.

Heise, David R. (1981) “Special Issue on Microcomputers and Social Research,” Sociological Methods and Research, volume 9(4).

23.

Hodges, Matthew E. and Russell M. Sasnett (1993) Multimedia Computing: Case Studies from MIT Project Athena, Reading, MA: Addison- Wesley Publishing Company.

24.

Hong, Lawrence K. (1984) “List Processing Free Responses: Analysis of Open-Ended Questions with Word Processor,” Qualitative Sociology, vol. 7 (1/2), pp. 98–109.

25.

Jones, Susan (1991) Text and Context: Document Processing and Storage, London: Springer-Verlag.

26.

Judge, Peter (1988) Open Systems: The Guide to OSI and Its Implementation, Wellseley, MA: QED Information Sciences, Inc.

27.

Kirk, Rodney C. (1981) “Microcomputers in Anthropological Research,” Sociological Methods and Research, volume 9(4), pp. 473–492.

28.

Krueger, L, G Gigerenzer, and MS Morgan (1987) The Probabilistic Revolution, volumes 1–2, MIT Press.

29.

Lazarsfeld, Paul F. (1961) “Notes on the History of Quantification in Sociology—Trends, Sources and Problems,” Quantification: A History of the Meaning of Measurement in the Natural and Social Sciences, Indianapolis: Bobbs-Merril Company, pp. 147–203.

30.

Lieberson, Stanley (1985) Making It Count: The Improvement of Social Research and Theory, Berkeley: University of California Press.

31.

Lieberson, Stanley (1992) “Einstein, Renoir, and Greeley: Some Thoughts About Evidence in Sociology” American Sociological Review, vol. 57 (2), pp. 1–15.

32.

Lundstrom, David E. (1987) A Few Good Men from Univac, MIT Press.

33.

Lynch. Daniel C. and Marshall T. Rose (1993) Internet System Handbook, Reading, MA: Addison-Wesley Publishing Company.

34.

Mayo-Smith (1910) Statistics and Sociology, Columbia University Press.

35.

Moreau, R. (1984) The Computer Comes of Age: The People, the Hardware and the Software, MIT Press.

36.

Nie. Norman H., CH Hull, JG Jenkins, K Steinbrenner, and DH Bent (1975) SPSS: Statistical Package for the Social Sciences, 2nd edition, NY: McGraw-Hill Book Company.

37.

Podolefsky, Aaron and Christopher McCary (1983) “Topical Sorting: A Technique for Computer Assisted Qualitative Data Analysis,” American Anthropologist, v. 85(4).

38.

Porter, Theodore M. (1986) The Rise of Statistical Thinking, 1850–1900, Princeton University Press.

39.

SAMS Publishing (1994) The Internet Unleashed, Indianapolis, IN: SAMS Publishing.

40.

SAS Institute (1982) SAS User's Guide: Basics, SAS Institute, Inc.

41.

Shanks, Merrill (1983) “The Current Status of Computer-Assisted Telephone Interviewing: recent progress and future prospects.” Sociological Methods and Research, volume 12 (2), November, pp. 119–142.

42.

Sproull, Lee S . & Robert F. Sproull (1982) “Managing and Analyzing Behavioral Records: Explorations in Nonnumeric Data Analysis,” Human Organization, vol. 41 (4).

43.

Stempel, Laurence E. (1982) “Eenie, meenie, minie, mo … What do the data really show?,” American Journal of Obstetrics and Gynecology, volume 144(7), pages 745–752.

44.

Stigler, Stephen M. (1986) The History of Statistics: The Measurement of Uncertainty before 1900, Belknap Press, Harvard University Press.

45.

Sudman, Seymour (1983) “Survey Research and Technological Change,” Sociological Methods and Research, volume 12(2), pp. 217–230.

46.

Sudman, Seymour and Norman N. Bradburn (1982) Asking Questions: A Practical Guide to Questionnaire Design, SF: Jossey-Bass.

47.

Wallace, James and Jim Erickson (1992) Hard Drive: Bill Gates and the Making of the Microsofi Empire, NY: John Wiley & Sons.

48.

Woolf, Harry, ed. (1961) Quantification: A History of the Meaning of Measurement in the Natural and Social Sciences, Indianapolis: Bobbs-Merril Company.