Measuring scientific interest: the effect of knowledge questions on interest ratings

Abstract

Research into the publics' interest in, knowledge of and attitudes towards science has captured the attention of politicians and educators. Sample survey research has been employed to assess the diverse aspects of the public understanding of science. However, surveys are subject to various biases that may affect the findings, calling into question both the reliability and the validity of the measures concerned. In this study we look at one common bias—that of context effects. Context effects occur when a question influences responses to later questions. The effects of answering one of four different sets of science questions (physical or life science, and easy or difficult questions) on what people report as their interest in science and what they think science is, were investigated using a split ballot format (n = 2099). Two approaches from social psychology, framing and consistency, are used to predict the effects of these knowledge questions on subsequent responses. Context effects were found and were more in line with the framing explanation. The results signal the need for caution in interpreting findings from surveys of the public understanding of science.

Get full access to this article

View all access options for this article.

References

Evans, G. A. , and Durant, J. R. , 1989, Understanding of science in Britain and the USA. British Social Attitudes: Special International Report, edited by R. R. Jowell , S. Witherspoon , and L. Brooks (Aldershot: Gower) p. 105.

and Jobe, J. B. , and Mingay, D. J. , 1991, Cognition and survey measurement: history and overview. Applied Cognitive Psychology, 5, 175-192.

and Evans, G. A. , and Durant, J. R. , 1989, Understanding of science in Britain and the USA. British Social Attitudes: Special International Report, edited by R. R. Jowell , S. Witherspoon , and L. Brooks (Aldershot: Gower).

and Grice, H. P. , 1975, Logic and conversation. Syntax and Semantics: Speech Acts, Vol. 3, edited by P. Cole , and J. L. Morgan (New York: Academic Press) pp. 41-58.

Jonsdottir, G. , O'Muircheartaigh, C. A. , and Gaskell, G. , 1990, Context effects in social surveys. Presented at the International Conference on Measurement Errors in surveys, 11-14 November 1990.

Bem, D. J. , 1972, Self-perception theory. Advances in Experimental Psychology, Vol. 6, edited by L. Berkowitz (New York: Academic Press).

Schwarz, N., and Bless, N., 1990, An inclusion/exclusion model of assimilation and contrast effects in social judgement. Unpublished manuscript, ZUMA, Mannheim. It is worth mentioning that respondents might believe that their guesses were correct. The only people whom we can be certain are aware of their low scores are those who answer `don't know'. This is dealt with later in the paper.

Greenwald, A. G. , 1975, On the inconclusiveness of crucial cognitive tests of dissonance versus self-perception theories. Journal of Experimental Social Psychology, 11, 490-499.

Bishop, G. F. , Oldendick, R. W. , and Tuchfarber, A. J. , 1984, Effects of questions wording and format on political attitude consistency. Public Opinion Quarterly, 42, 81-92.

10.

They are difficult questions, but the way they are phrased respondents might think they are not that difficult. The first question asked if they `remember anything special that your US Representative has done for your district', and the second `Is there any legislative bill that has come up in the House of Representatives, on which you remember how your congressman has voted' (p.512; emphasis added). On the face of it, these seem like questions that should be answered by people who follow politics. The effects in Bishop et al.'s study could be particularly large because they appear easy, but are in fact difficult. The shifts may also be accentuated because the question used to gauge political interest in the States asks people how informed they are. Bishop, G. F. , Oldendick, R. W. , and Tuchfarber, A. J. , 1984, Effects of questions wording and format on political attitude consistency. Public Opinion Quarterly, 42.

11.

Bishop, G. F. , Oldendick, R. W. , and Tuchfarber, A. J. , 1984, Effects of questions wording and format on political attitude consistency. Public Opinion Quarterly, 42, 516 (italics in original).

12.

Bishop, G. F. , Oldendick, R. W. , and Tuchfarber, A. J. , 1984, Effects of questions wording and format on political attitude consistency. Public Opinion Quarterly, 42, 515.

13.

Durant, J. R. , Evans, G. A. , and Thomas, G. P. , 1992, Public understanding of science in Britain: the role of medicine in the popular representation of science. Public Understanding of Science, 1, 173.

14.

15.

Searle, J. R. , 1990, Consciousness, explanatory inversion, and cognitive science. The Behavioral and Brain Sciences, 13, 585-642.

16.