This study reviews considerations for integrating closed-ended items and open-ended questions in a single survey instrument, focusing on contextual effects as a potential pitfall in organizational diagnosis. A randomized posttest-only control group experiment was conducted in a field setting with a small (92 employees) for-profit firm wherein the experimental group received a mixed questionnaire (closed-ended, followed by open-ended, questions) and the control group received only open-ended questions. Individuals receiving the mixed survey responded with a lower response rate and fewer comments than those receiving only the open-ended questions. A thematic content analysis of responses revealed a practically significant difference between groups in respondents' perceptions of the organization's strengths. However, the reported weaknesses of the organization did not differ between groups.
Alderfer, C.P., & Simon, A.F. (2002). Nonresponse rates to organic questionnaire items as evidence of parallel processes during organizational diagnosis. Journal of Applied Behavioral Science, 38, 416.
2.
Armenakis, A.A., & Bedeian, A.G. (1999). Organizational change: A review of theory and research in the 1990s. Journal of Management, 25, 293-315.
3.
Armenakis, A.A., Mossholder, K.W., & Harris, S.G. (1990). Diagnostic bias in organizational consultation . International Journal of Management Science, 18, 563-572.
4.
Babbie, E. (2004). The practice of social research (10th ed.). Belmont, CA: Wadsworth.
5.
Beer, M., & Spector, B. (1993). Organizational diagnosis: Its role in organizational learning. Journal of Counseling & Development, 71, 642-650.
6.
Borkan, J.M. (2004). Mixed method studies: A foundation for primary care research. Annals ofFamily Medicine, 2, 110-122.
7.
Bradburn, N.M., Wansink, B., & Sudman, S. (2004). Asking questions. San Francisco : Jossey-Bass.
8.
Burke, W.W., & Litwin, G.H. (1992). A causal model of organizational performance and change. Journal of Management, 18, 532-545.
9.
Campbell, D.T., & Stanley, J.C. (1966). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin .
10.
Coghlan, D., & Brannick, T. (2001). Doing action research in your own organization . London: Sage.
11.
Creswell, J.W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Thousand Oaks, CA: Sage.
12.
Creswell, J.W., & Plano Clark, V.L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
13.
Creswell, J.W., Shope, R., Plano Clark, V.L., & Green, D.O. (2006). How interpretive qualitative research extends mixed methods research. Research in the Schools, 13, 1-11.
14.
Cummings, T.G., & Worley, C.G. (2005). Organization development and change (8th ed.). Minneapolis/ St. Paul, MN: South-Western Educational Publishing .
15.
Denzin, N.K., & Lincoln, Y.S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (pp. 1-32). Thousand Oaks, CA: Sage.
16.
Dillman, D.A. (2007). Mail and Internet surveys: The tailored design method (2nd ed.). New York: John Wiley.
17.
Di Pofi, J.A. (2001). Organizational diagnostics: Integrating qualitative and quantitative methodology. Journal of Organizational Change Management, 15, 156-172.
18.
Egan, T.M., & Lancaster, C. (2005). Comparing appreciative inquiry to action research: OD practitioner perspectives. Organization Development Journal , 23, 29-35.
19.
Emory, W., & Cooper, D.R. (1991). Business research methods (4th ed.). Homewood, IL: Irwin.
20.
Erickson, P.I., & Kaplan, C.P. (2000). Maximizing qualitative responses about smoking in structured interviews. Qualitative Health Research, 10, 829-841.
21.
Falletta, S.V. (2005). Organizational diagnostic models: A review and synthesis. Retrieved March 12, 2006, from Leadersphere Inc. Web site: http://www.leadersphere.com/img/Orgmodels.pdf
22.
Fielding, N.G., & Fielding, J.L. (1986). Linking data: The articulation of qualitative and quantitative methods in social research (Vol. 4). Beverly Hills, CA: Sage.
23.
Gibson, J.M., & Bahrey, R. (2005). Modality-specificity effects in priming of visual and auditory word-fragment completion. Journal of General Psychology, 132, 110-117.
24.
Golden-Biddle, K., Wiebe, E., & Locke, K. (2006). Using qualitative methodology to study the dynamics of organizational change. In R. Lines , I. G. Stensaker, & A. Langley (Eds.), New perspectives on organizational change and learning (pp. 413-425). Bergen, Norway: Fagbokforlaget.
25.
Greene, J.C., Caracelli, V.J., & Graham, W.F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis , 11, 255-274.
26.
Guilford, J.P., & Fruchter, B. (1973). Fundamental statistics in psychology and education . New York: McGraw-Hill.
27.
Harrison, M.I., & Shirom, A. (1999). Organizational diagnosis and assessment: Bridging theory and practice. Thousand Oaks, CA: Sage.
28.
Harvey, D.F., & Brown, D.R. (1996). An experiential approach to organization development (5th ed.). Upper Saddle River, NJ: Prentice Hall.
29.
Head, T.C., Griffin, R.W., Bateman, T.S., Lohman, L., & Yates, V.L. (1988). The priming effect in task design research. Journal of Management, 14, 31-38.
30.
Houtkoop-Steenstra, H. (2000). Interaction and the standardized survey interview. Cambridge, UK: Cambridge University Press.
31.
Jackson, K.M., & Trochim, W.M.K. (2002). Concept mapping as an alternative approach for the analysis of open-ended survey responses. Organizational Research Methods, 5, 307-332.
32.
Johnson, B., & Turner, L.A. (2003). Data collection strategies in mixed methods research . In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 297-320). Thousand Oaks, CA: Sage.
33.
Koch, J.L., & Rhodes, S.R. (1979). Problems with reactive instruments in field research . Journal of Applied Behavioral Science, 15, 485-501.
34.
Kramer, R.M. (1996). Divergent realities and convergent disappointments in the hierarchical relation: Trust and the intuitive auditor at work. In R. M. Kramer & T. R. Tyler (Eds.), Trust in organizations: Frontiers of theory and research (2nd ed., pp. 212-275). London: Sage.
35.
Krippendorf, K. (2004). Content analysis: An introduction to its methodology . Thousand Oaks, CA: Sage.
36.
Lasorsa, D.L. (2003). Question-order effects in surveys: The case of political interest, news attention, and knowledge. Journalism and MassCommunications Quarterly, 80, 499-522.
37.
Miles, M.B., & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
38.
Mingers, J. (2006). A critique of statistical modeling in management science from a critical realist perspective: Its role within multimethodology . Journal of the Operational Research Society, 57, 202-219.
39.
Moss, S.E., & Lawrence, K.G. (1997). The effects of priming on the self-reporting of perceived stressors and strains. Journal of Organizational Behavior, 18, 393-412.
40.
Onwuegbuzie, A.J., & Johnson, R.B. (2006). The validity issue in mixed research. Research in the Schools, 13, 48-63.
41.
Paul, J. (1996). Between-method triangulation in organizational diagnosis. International Journal of Organizational Analysis , 4, 135-156.
42.
Peterson, R.A. (2005). Response construction in consumer behavior research . Journal of Business Research, 58, 348-353.
43.
Rossman, G.B., & Wilson, B.L. (1991). Numbers and words revisited: Being ``shamelessly eclectic.'' Evaluation Review, 9, 627-643.
44.
Salancik, G.R., & Pfeffer, J. (1977). An examination of need-satisfaction models of job attitudes. Administrative Science Quarterly,22, 407-427.
45.
Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys: Experiments on question form, wording, and context. New York: Academic Press.
46.
Schwarz, N. (1999). How the questions shape the answers. American Psychologist, 54, 93-105.
47.
Schwarz, N., & Hippler, H.J. (1995). Subsequent questions may influence answers to preceding questions in mail surveys. Public Opinion Quarterly , 59, 93-115.
48.
Schwarz, N., Hippler, H.J., Deutsch, B., & Strack, F. (1985). Response scales effects of category of range on reported behavior and comparative judgments. Public Opinion Quarterly, 45, 368-395.
49.
Schwarz, N., & Sudman, S. (Eds.). (1996). Answering questions. San Francisco: Jossey-Bass.
50.
Smith, C.P. (1992). Introduction: Inferences from verbal material . In C. P. Smith (Ed.), Motivation and personality: Handbook of thematic content analysis (pp. 1-17). Cambridge, UK: Cambridge University Press.
51.
Sproull, N.L. (1988). Handbook of research methods: A guide for practitioners and students in the social sciences. Metuchen, NJ: Scarecrow Press.
52.
Styhre, A., & Sundgren, M. (2005). Action research as experimentation. Systemic Practice and Action Research, 18, 53-72.
53.
Sudman, S., Bradburn, N.M., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco: Jossey-Bass.
54.
Teddlie, C., & Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral siences. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 3-50). Thousand Oaks, CA: Sage.
55.
Todorov, A., (2000). Context effects in national health surveys: Effects of preceding questions on reporting serious difficulty seeing and legal blindness . Public Opinion Quarterly, 64, 65-78.
56.
Tulving, E., & Schacter, D.L. (1990). Priming and human memory systems. Science, 247, 272-301.
57.
Weisburg, H.F., Krosnick, J.A., & Bowen, D.D. (1989). An introduction to survey research and data analysis (2nd ed.). Glenview, IL: Scott Foresman.
58.
Wyer, R.S., & Hartwick, J. (1978). The role of information retrieval and conditional inference process in belief formation and change. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 13). New York: Academic Press.
