Measuring attitude toward the ad: A test of using arbitrary scales and “ p

Abstract

Attitude toward the ad is an important construct frequently measured in advertising and marketing research. However, it is somewhat ambiguous in regard to how to quantify attitude toward the ad with a numerical scale. This study examines the practice and effect of using arbitrary scales when measuring attitude toward the ad (i.e., 1–5, 1–7, 1–9, 0–10, and 0–100). A longitudinal experiment with thousands of adult participants (Time 1: n = 2,366, Time 2: n = 1,165) was conducted. The experimental results revealed that different scales, in general, will lead to consistent findings, but if the conventional p < .05 criterion is used, the study conclusion may differ. Based on these findings, theoretical discussions and practical recommendations are provided.

Keywords

attitude attitude measure attitude toward the ad replicability crisis scale length scale point

Get full access to this article

View all access options for this article.

References

Ajzen

(2001). Nature and operation of attitudes. Annual Review of Psychology, 52(1), 27–58. https://doi.org/10.1146/annurev.psych.52.1.27

Alwin

D. F.

(1992). Information transmission in the survey interview: Number of response categories and the reliability of attitude measurement. Sociological Methodology, 22, 83–118. https://doi.org/10.2307/270993

Amrhein

Greenland

McShane

(2019). Scientists rise up against statistical significance. Nature, 567, 305–307. https://doi.org/10.1038/d41586-019-00857-9

Bearden

W. O.

Netemeyer

R. G.

(Eds.). (1999). Handbook of marketing scales: Multi-item measures for marketing and consumer behavior research. SAGE.

Beltramini

R. F.

(1982). Rating scale selection in advertising research: Does it matter which one is used? Current Issues and Research in Advertising, 5(1), 39–58.

Bergkvist

Langner

(2017). Construct measurement in advertising research. Journal of Advertising, 46(1), 129–140. https://doi.org/10.1080/00913367.2017.1281778

Birkett

N. J.

(1986). Selecting the number of response categories for a Likert-type scale. In Proceedings of the American Statistical Association (pp. 488–492).

Bohner

Dickel

(2011). Attitudes and attitude change. Annual Review of Psychology, 62, 391–417. https://doi.org/10.1146/annurev.psych.121208.131609

Carroll

A. E.

(2017). Science needs a solution for the temptation of positive results. New York Times. https://www.nytimes.com/2017/05/29/upshot/science-needs-a-solution-for-the-temptation-of-positive-results.html

10.

Cicchetti

D. V.

Shoinralter

Tyrer

P. J.

(1985). The effect of number of rating scale categories on levels of interrater reliability: A Monte Carlo investigation. Applied Psychological Measurement, 9(1), 31–36. https://doi.org/10.1177/014662168500900103

11.

Cox

E. P.

III . (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 17(4), 407–422. https://doi.org/10.1177/002224378001700401

12.

Dawes

(2008). Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 50(1), 61–77. https://doi.org/10.1177/147078530805000106

13.

Goggin

S. N.

Stoker

(2014, August). Optimal scale length and single-item attitude measure: Evidence from simulations and a two-wave experiment. Paper Presented at the 2014 American Political Science Association Annual Conference, Washington, DC, United States.

14.

Jacoby

Matell

M. S.

(1971). Three-point Likert scales are good enough. Journal of Marketing Research, 8(4), 495–500. https://doi.org/10.2307/3150242

15.

Krosnick

J. A.

Judd

C. M.

Wittenbrink

(2005). The measurement of attitudes. In Albarracín

Johnson

B. T.

Zanna

M. P.

(Eds.), The handbook of attitudes (pp. 21–76). Lawrence Erlbaum.

16.

Leung

S. O.

(2011). A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point Likert scales. Journal of Social Service Research, 37(4), 412–421. https://doi.org/10.1080/01488376.2011.580697

17.

Levin

A. M.

Levin

I. P.

Cook

M. P.

(2010). Measuring and accounting for cross-country response biases in marketing food and drink products. Journal of International Consumer Marketing, 22(2), 213–222. https://doi.org/10.1080/08961530903476295

18.

Lissitz

R. W.

Green

S. B.

(1975). Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 60(1), 10–13. https://doi.org/10.1037/h0076268

19.

Lozano

L. M.

García-Cueto

Muñiz

(2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73–79. https://doi.org/10.1027/1614-2241.4.2.73

20.

Park

S. Y.

Morton

C. R.

(2015). The role of regulatory focus, social distance, and involvement in anti-high-risk drinking advertising: A construal-level theory perspective. Journal of Advertising, 44(4), 338–348. https://doi.org/10.1080/00913367.2014.1001503

21.

Peter

J. P.

(1979). Reliability: A review of psychometric basics and recent marketing practices. Journal of Marketing Research, 16(1), 6–17. https://doi.org/10.2307/3150868

22.

Preston

C. C.

Colman

A. M.

(2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104(1), 1–15. https://doi.org/10.1016/s0001-6918(99)00050-5

23.

Rand

D. G.

(2012). The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments. Journal of Theoretical Biology, 299, 172–179. https://doi.org/10.1016/j.jtbi.2011.03.004

24.

Reich

B. J.

Soule

C. A. A.

(2016). Green demarketing in advertisements: Comparing “buy green” and “buy less” appeals in product and institutional advertising contexts. Journal of Advertising, 45(4), 441–458. https://doi.org/10.1080/00913367.2016.1214649

25.

Remington

Tyrer

P. J.

Newson-Smith

Cicchetti

D. V.

(1979). Comparative reliability of categorical and analogue rating scales in the assessment of psychiatric symptomatology. Psychological Medicine, 9(4), 765–770. https://doi.org/10.1017/s0033291700034097

26.

Revilla

M. A.

Saris

W. E.

Krosnick

J. A.

(2014). Choosing the number of categories in agree–disagree scales. Sociological Methods & Research, 43(1), 73–97. https://doi.org/10.1177/0049124113509605

27.

Schutz

H. G.

Rucker

M. H.

(1975). A comparison of variable configurations across scale lengths: An empirical study. Educational and Psychological Measurement, 35(2), 319–324. https://doi.org/10.1177/001316447503500210

28.

Shaw

M. E.

Wright

J. M.

(1967). Scales for the measurement of attitudes. McGraw-Hill.

29.

Sung

Choi

S. M.

(2011). Increasing power and preventing pain. Journal of Advertising, 40(1), 71–86. https://doi.org/10.2753/joa0091-3367400105

30.

Viswanathan

Sudman

Johnson

(2004). Maximum versus meaningful discrimination in scale response: Implications for validity of measurement of consumer perceptions about products. Journal of Business Research, 57(2), 108–124. https://doi.org/10.1016/s0148-2963(01)00296-x

31.

Wasserstein

R. L.

Lazar

N. A.

(2016). The ASA’s statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108

32.

Wasserstein

R. L.

Schirm

A. L.

Lazar

N. A.

(2019). Moving to a world beyond “p< 0.05.” The American Statistician, 73(S1), 1–19. https://doi.org/10.1080/00031305.2019.1583913

33.

Weng

L. J.

(2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64(6), 956–972. https://doi.org/10.1177/0013164404268674

34.

Wood

Randall

(2018). How bad is the government’s science? Wall Street Journal. https://www.wsj.com/articles/how-bad-is-the-governments-science-1523915765

35.

M. L.

Leung

S. O.

(2018). Effects of varying numbers of Likert scale points on factor structure of the Rosenberg Self-Esteem Scale. Asian Journal of Social Psychology, 21(3), 119–128. https://doi.org/10.1111/ajsp.12214

36.

Xue

(2008). The moderating effects of product involvement on situational brand choice. Journal of Consumer Marketing, 25(2), 85–94. https://doi.org/10.1108/07363760810858828

37.

Zaichkowsky

J. L.

(1985). Measuring the involvement construct. Journal of Consumer Research, 12(3), 341–352. https://doi.org/10.1086/208520

Measuring attitude toward the ad: A test of using arbitrary scales and “ p < .05” criterion

Abstract

Keywords

Get full access to this article

References