Three surveys compared the accuracy of predictions based on forced-choice and subjective probability scales. The latter produced significantly more accurate election predictions and significantly reduced the percentage of undecided, or “Don't Know” responses, compared to forced-choice scales in all three surveys. Analysis indicates subjective probability scales decrease sampling error and confirms there is an inherent source of error in traditional forced-choice questions about voting intentions not attributable to sampling error. The results are discussed with respect to (1) sampling and measurement errors in forced-choice and subjective probability scales measuring behavioral intentions, (2) their practical application, and (3) cognitive theory, especially support theory.
Get full access to this article
View all access options for this article.
References
1.
BradburnN. M.SudmanS. (1991) The current status of questionnaire research. In BiemerP. P.GrovesR. M.LybergL. E.MathiowetzN. A.SudmanS. (Eds.), Measurement errors in surveys. New York: Wiley. Pp. 29–40.
2.
BudescuD. V.WeinbergS.WallstenT. S. (1988) Decisions based on numerically and verbally expressed uncertainties. Journal of Experimental Psychology: Human Perception and Performance, 14, 281–294.
3.
CorlettT. (1996) Sampling errors in practice. Journal of the Market Research Society, 38, 308–318.
4.
CrespiI. (1988) Pre-election polling: sources of accuracy and error. New York: Sage.
5.
CurticeJ. (1997) Are the opinion polls ready for 1997?Journal of the Market Research Society, 39, 317–330.
EdwardsA. L. (1985) Experimental design in psychological research. (5th ed.) New York: Harper & Row.
8.
EisenhowerD.MathiowetzN. A.MorgansteinD. (1991) Recall error: sources and bias reduction techniques. In BiemerP. P.GrovesR. M.LybergL. E.MathiowetzN. A.SudmanS. (Eds.), Measurement errors in surveys. New York: Wiley. Pp. 127–144.
9.
ErevI.WallstenT. S.BudescuD. V. (1994) Simultaneous over- and underconfidence: the role of error in judgment processes. Psychological Review, 101, 519–527.
10.
FischhoffB.SlovicP.LichtensteinS. (1977) Knowing with certainty: the appropriateness of extreme confidence. Journal of Experimental Psychology: Human Perception and Performance, 3, 552–564.
11.
FlannellyK. J.FlannellyL. T.McLeodM. S.Jr. (1998) Comparison of election predictions, voter certainty and candidate choice on political polls. Journal of the Market Research Society, 40, 337–346.
12.
FlannellyK. J.McLeodM. S.Jr. (1991) Predicting consumer demand for alternative transportation services among suburban commuters. Transportation Research Record, 1280, 73–81.
13.
FlannellyL. T. (in press) Using feedback to reduce students' judgment bias on test questions. Journal of Nursing Education.
14.
FlannellyL. T.FlannellyK. J. (in press) Reducing people's judgment bias about their level of knowledge. Psychological Record.
15.
GelmanA.KingG. (1993) Why are American presidential election campaign polls so variable when votes are so predictable?British Journal of Political Science, 24, 409–451.
16.
GendallP. J.EsslemontD.DayD. (1991) A comparison of two versions of the Juster Scale using self-completion questionnaires. Journal of the Market Research Society, 33, 257–263.
17.
Gonzalez-VallejoC.WallstenT. S. (1992) Effects of probability mode on preference reversal. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 855–864.
18.
GwyerP.CliffordB. R. (1997) The effects of the cognitive interview on recall, identification, confidence and the confidence/accuracy relationship. Applied Cognitive Psychology, 11, 121–145.
19.
HawkinsS. A. (1994) Information processing strategies in riskless preference reversals: the prominence effect. Organizational Behavior and Human Decision Processes, 59, 1–26.
20.
HochS. J. (1985) Counterfactual reasoning and accuracy in predicting personal events. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 719–731.
21.
HoekJ. A.GendallP. J. (1993) A new method of predicting voting behaviour. Journal of the Market Research Society, 35, 361–373.
22.
JaegerR. M. (1984) Sampling in education and the social sciences. New York: Longman.
23.
KrosnickJ. A. (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213–236.
24.
LauR. R. (1994) An analysis of the accuracy of “trial heat” polls during the 1992 presidential elections. Public Opinion Quarterly, 59, 589–605.
25.
LevinJ.FoxJ. A. (1997) Elementary statistics in social research. (7th ed.) New York: Longman.
26.
McGuiganF. J. (1978) Experimental psychology: a methodological approach. (3rd ed.) Englewood Cliffs, NJ: Prentice Hall.
27.
McLeodM. S.Jr.FlannellyK. J.HendersonB. H. (1988) Commuting behavior of Hawaii state workers in Honolulu: implications for transportation system management strategiesTransportation Research Record, 1170, 53–59.
28.
MooreD. S. (1998) Sampling is not enumerating…. AMSTAT News, February, 2–3.
29.
PickeringJ. F.IsherwoodB. C. (1974) Purchase probabilities and consumer durable buying behaviour. Journal of the Market Research Society, 16, 203–226.
30.
PulfordB. D.ColmanA. M. (1997) Overconfidence: feedback and item difficulty effects. Personality and Individual Differences, 23, 125–133.
31.
RottenstreichY.TverskyA. (1997) Unpacking, repacking, and anchoring: advances in support theory. Psychological Review, 104, 406–415.
32.
SchwarzN.HipplerH-J. (1991) Response alternatives: the impact of their choice and presentation order. In BiemerP. P.GrovesR. M.LybergL. E.MathiowetzN. A.SudmanS. (Eds.), Measurement errors in surveys. New York: Wiley. Pp. 41–56.
33.
SchwarzN.StrackF.HipplerH-J.BishopG. (1991) The impact of administration mode on response effects in survey measurement. Applied Cognitive Psychology, 5, 193–212.
34.
SiegelS. (1956) Nonparametric statistics for the behavioral sciences. New York: McGraw Hill.
35.
SollJ. B. (1996) Determinants of overconfidence and miscalibration: the roles of random error and ecological structure. Organizational Behavior and Human Decision Processes, 65, 117–137.
36.
TverskyA.KoehlerD. J. (1994) Support theory: a nonextensional representation of subjective probability. Psychological Review, 101, 547–567.
37.
TverskyA.SattathS.SlovicP. (1988) Contingent weighting in judgment and choice. Psychological Review, 95, 371–384.
38.
ValloneR. P.GriffinD. W.LinS.RossL. (1990) Overconfident prediction of future actions and outcomes by self and others. Journal of Personality and Social Psychology, 58, 582–592.
39.
WallstenT. S.BudescuD. V.ZwickR. (1993) Comparing the calibration and coherence of numerical and verbal probabilistic judgments. Management Science, 39, 176–190.
