The researcher, who pretests a questionnaire conventionally, is confident that a small number of interviews reveals numerous problems afflict the questionnaire. The standardized interviewer is asked to observe respondent’s verbal behaviors in order to realize whether respondent misunderstands question. But this strategy fails when respondent does not show any misunderstanding signal. The study, reported in this article, illustrates how to exceed this limit, by means of the analysis of respondent’s comments released in the flexible interview, which researcher uses to unveil incongruities between question and answer; the questions, when many respondents’ misunderstandings occur, should be reworded. This study shows that the number of problems and the number of the problematic questions identified by this unconventional procedure are much higher than the number of problems and the number of problematic questions determined by the conventional pretesting. The strength of respondents’ comments analysis is to discover problems the questionnaire poses for respondents, even when the respondents do not give any misunderstanding signal.
AkmajianA.DemersR.FarmerA.HarnishR.. 1990. Linguistics: Introduction to Language and Communication. Cambridge, MA: MIT Press.
2.
BichiR.. 2002. L’intervista biografica: una proposta metodologica. Milano, Italy: Vita e Pensiero.
3.
BiemerP.GrovesR.LybergL.MathiowetzN.SudmanS.. 2004. Measurement Errors in Surveys. New York, NY: Wiley.
4.
BrinkmannS.. 2014. Interview. New York, NY: Springer.
5.
BradburnN.SudmanS.. 1979. Improving Interview Method and Questionnaire Design. San Francisco, CA: Jossey Bass.
6.
CannellC. F.KahnR. L.. 1957. The Dynamics of Interviewing: Theory, Technique, and Cases. New York, NY: Wiley.
7.
ConradF.SchoberM.. 2008. “New Frontiers in Standardized Survey Interviewing.” Pp 173–188 in Handbook of Emergent Methods, edited by Hesse-BiberS. N.LeaveyP.. New York, NY: Guilford Press.
8.
ConverseJ.PresserS.. 1986. Survey Questions: Hand Crafting the Standardized Questionnaire. Newbury Park, CA: Sage.
9.
DijkstraW.. 2002. “Transcribing, Coding, and Analyzing Verbal Interactions in Survey Interviews.” Pp 401–426 in Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, edited by MaynardD.Houtkoop-SteenstraH.SchaefferN.Van der ZouweD.. New York, NY: Wiley.
10.
DykemaJ.LepkowskyJ. M.BlixtS.. 1997. “The Effect of Interviewer and Respondent Behavior on Data Quality: Analysis of Interaction Coding in a Validation Study.” Pp 287–310 in Survey Measurement and Process Quality, edited by LybergL. E.BiemerP. P.CollinsM.de LeeuwE.DippoC.SchwarzN.TrewinD.. New York, NY: Wiley.
11.
EcoU.. 2008. Trattato di semiotica generale. Milano, Italy: Bompiani.
12.
FowlerF.MangioneT.. 1990. Standardized Survey Interviewing: Minimizing Interviewer Related Error. Newbury Park, CA: Sage.
GrovesR.. 1989. Survey Errors and Survey Costs. New York, NY: Wiley.
15.
Houtkoop-SteenstraH.. 2000. Interaction in the Standardized Survey Interview: The Living Questionnaire. Cambridge, MA: Harvard University Press.
16.
KamberelisG.DimitriadisG.. 2013. Focus Groups: From Structured Interviews to Collective Conversations. New York, NY: Routledge.
17.
MarradiA.. 2007. Metodologia delle scienze sociali. Bologna, Italy: Il Mulino.
18.
MauceriS.. 2003. Per la qualità del dato nella ricerca sociale. Strategie di progettazione e conduzione dell’intervista con questionario. Milano, Italy: Franco Angeli.
19.
MaynardD.. 2006. “Cognition on the Ground.” Discourse Studies8:105–15.
20.
MaynardD.Houtkoop-SteenstraH.SchaefferN.Van der ZouwenD.. 2002. Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York, NY: Wiley.
MontesperelliP.. 2014. Comunicare e interpretare. Introduzione all’ermeneutica per la ricerca sociale. Milano, Italy: Egea.
23.
Morton-WilliamsJ.. 1979. “The Use of Verbal Interaction Coding’ for Evaluating a Questionnaire.” Quality & Quantity13:59–75.
24.
OksenbergL.CannellC.KaltonG.. 1991. “New Strategies for Pretesting Survey Questions”. Journal of Official Statistics7:349–65.
25.
OngenaY. P.DijkstraW.. 2007. “A Model of Cognitive Processes and Conversational Principles in Survey Interview Interaction.” Applied Cognitive Psychology21:145–63.
26.
PeirceCharles Sanders. 1931. Collected Papers. Cambridge, MA: Harvard University Press.
27.
PitroneM. C.. 2010. Il sondaggio. Milano, Italy: Franco Angeli.
28.
PresserS.CouperM.LesslerJ.MartinE.RothgebJ.SingerS.. 2004. Methods for Testing and Evaluating Survey Questions. New York, NY: Wiley.
29.
SartoriG., ed. 1984. Social Science Concepts: A Systematic Analysis. London, UK: Sage.
30.
SchaefferN.. 1991. “Conversation with a Purpose or Conversation? Interaction in Standardized Interview.” Pp 367–391 in Introduction to Survey Quality, edited by P. Paul. NY: Wiley.
31.
SchaefferN.MaynardD.. 1996. “From Paradigm to Prototype and Back Again: Interactive Aspects of Cognitive Processing in Standardized Interview.” Pp 65–88 in Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, edited by SchwarzN.SudmanS.. San Francisco: Jossey-Bass.
32.
SchaefferN.MaynardD.. 2002. “Occasions for Intervention: Interactional Resources for Comprehension in Standardized Survey Interviews.” Pp 261–280 in Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, edited by MaynardD.Houtkoop-SteenstraH.SchaefferN.Van der ZouwenD.. New York, NY: Wiley.
33.
SchoberM.. 1999. “Making Sense of Questions: An Interactional Approach.” Pp 77–93 in Cognition and Survey Research, edited by SirkenM.R.HermanJ.SchechterS.SchwarzN.TanurJ. M.TourangeauR.. New York, NY: Wiley.
SchoberM.ConradF.. 1998. “Response Accuracy When Interviewers Stray from Standardization.” Pp 940-945 in Proceedings of the Joint Statistical Meeting of the American Statistical Association, Section on Survey Methods Research. Alexandria, Egypt.
36.
SchoberM.ConradF.. 2002. “A Collaborative View of Standardized Survey Interviews.” Pp 67–94 in Standardization nad Tacit Knowledge: Interaction and Practice in the Survey Interview, edited by MaynardD.Houtkoop-SteenstraH.SchaefferN.Van der Zouwen. New York, NY: Wiley.
37.
SchoberM.ConradF.FrickerS.. 2004. “Misunderstanding Standardized Language in Research Interviews.” Applied Cognitive Psychology18:169–88.
38.
SchutzA.. 1932. Der Sinnhafte Aufban der Sozialen Welt. Wien (Oesterreich): Springer.
39.
SykesW.CollinsM.. 1992. “Anatomy of the Survey Interview.” Journal of Official Statistics8:277–91.
40.
SykesW.Morton-WilliamsJ.. 1987. “Evaluating Survey Questions.” Journal of Official Statistics2:191–207.
41.
SuchmanH.JordanB.. 1990. “Interactional Troubles in Face-to-Face Survey Interviews.” Journal of the American Statistical Association85:232–53.
42.
SudmanS.BradburnN.. 1982. Asking Questions: A Practical Guide to Questionnaire Design. San Francisco, CA: Jossey-Bass.
43.
TylerS.. 1969. Cognitive Anthropology. New York, NY: Holt.
44.
VioliP.. 1997. Significato ed esperienza. Milano, Italy: Bompiani.
45.
WillisG.. 2005. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage.
