Statistical power and effect size are not considered sufficiently by marketing researchers. The authors discuss how better attention to these two factors can improve the planning, execution, and reporting of marketing and consumer research. Suggestions are offered about how to increase effect size and improve statistical power.
Get full access to this article
View all access options for this article.
References
1.
AckoffRussell L. and EmshoffJames R. (1975), “Advertising Research at Anheuser-Busch, Inc. (1963–68),”Sloan Management Review (Winter), 1–15.
2.
ArmstrongJ. Scott (1979), “Advocacy and Objectivity in Science,”Management Science, 25 (May), 423–38.
3.
BagozziRichard P. (1977), “Structural Equation Models in Experimental Research,”Journal of Marketing Research, 14 (May), 209–26.
4.
BakanDavid (1966), “The Test of Significance in Psychological Research,”Psychological Bulletin, 66 (December), 423–37.
5.
BanksSeymour (1965), Experimentation in Marketing. New York: McGraw-Hill Book Company.
6.
BelkRussell W. (1974), “An Exploratory Assessment of Situational Effects in Buyer Behavior,”Journal of Marketing Research, 11 (May), 156–63.
7.
BlattbergRobert C. (1979), “The Design of Advertising Experiments Using Statistical Decision Theory,”Journal of Marketing Research, 16 (May), 191–202.
8.
BlattbergRobert C. and SenSubrata K. (1973), “An Evaluation of the Application of Minimum Chi-Square Procedures to Stochastic Models of Brand Choice,”Journal of Marketing Research, 10 (November), 421–7.
9.
BoruchRobert E. and GomezHernando (1977), “Sensitivity, Bias, and Theory in Impact Evaluations,”Professional Psychology (November), 411–34.
10.
BrewerJames K. (1972), “On the Power of Statistical Tests in the” American Educational Research Journal, American Educational Research Journal, 9 (Summer), 391–401.
11.
BuschPaul and BushRonald F. (1978), “Women Contrasted to Men in the Industrial Salesforce: Job Satisfaction, Values, Role Clarity, Performance, and Propensity to Leave,”Journal of Marketing Research, 15 (August), 438–48.
12.
CarlsmithJ. Merrill, EllsworthPhoebe C., and AronsonElliot (1976), Methods of Research in Social Psychology. Reading, Massachusetts: Addison-Wesley.
13.
ChaseLawrence J. and BaranStanley J. (1976), “An Assessment of Quantiatitive Research in Mass Communication,”Journalism Quarterly, 53 (Summer), 308–11.
14.
ChaseLawrence J. and ChaseRichard B. (1976), “A Statistical Power Analysis of Applied Psychological Research,”Journal of Applied Psychology, 61 (April), 234–37.
15.
ChaseLawrence J. and TuckerRaymond K. (1975), “A Power-Analytic Examination of Contemporary Communication Research,”Speech Monographs, 42 (March), 29–41.
16.
ChevalierMichel (1975), “Increase in Sales Due to In-Store Display,”Journal of Marketing, 12 (November), 426–31.
17.
ChurchillGilbert A. (1979), “A Paradigm for Developing Better Measures of Marketing Constructs,”Journal of Marketing Research, 16 (February), 64–73.
18.
ClearyT. Anne, LinnnRobert, and WalsterG. William (1970), “Effect of Reliability and Validity on Power of Statistical Tests,” in Sociological Methodology, BorgattaEdgar F. and BohrnstedtGeorge W., eds. San Francisco: Jossey-Bass, 130–38.
19.
CohenJacob (1962), “The Statistical Power of Abnormal-Social Psychological Research: A Review,”Journal of Abnormal and Social Psychology, 65, 145–53.
20.
CohenJacob (1965), “Some Statistical Issues in” Psychological Research, in Handbook of Clinical Psychology, WolmanB. B., ed. New York: McGraw-Hill Book Company, 95–121.
21.
CohenJacob (1973), “Statistical Power Analysis and Research Results,”American Educational Research Journal, 10 (Summer), 225–9.
22.
CohenJacob (1977), Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press.
23.
CookThomas D. and CampbellDonald T. (1979), Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand-McNally.
24.
CookThomas D., GraderCharles R., HenniganKaren M., and FlayBrian R. (1978), “History of the Sleeper Effect: Some Logical Pitfalls in Accepting the Null Hypothesis,”Psychological Bulletin, 86, 662–79.
25.
CoxKeith K. (1970), “The Effect of Shelf Space Upon Sales of Branded Products,”Journal of Marketing Research, 7 (February), 55–8.
26.
DalaiSiddhartha R. and SrinivasanV. (1977), “Determining Sample Size for Pretesting Comparative Effectiveness of Advertising Copies,”Management Science, 23 (August), 1284–94.
27.
DayGeorge S. and HeelerRoger M. (1971), “Using Cluster Analysis to Improve Marketing Experiments,”Journal of Marketing Research, 8 (August), 340–7.
28.
DillonWilliam R., GoldsteinMathew, and LementLucy (1981), “Analyzing Qualitative Predictors with Too Few Data: An Alternate Approach to Handling Sparse-Cell Values,”Journal of Marketing Research, 18 (February), 63–72.
29.
DoylePeter and FenwickIan (1975), “Planning and Estimation in Advertising,”Journal of Marketing Research, 12 (February), 1–6.
30.
DudewiczEdward J. (1976), Introduction to Statistics and Probability. New York: Holt, Rinehart, and Winston.
31.
EskinGerald J. and BaronPenny H. (1977), “Effects of Price and Advertising in Test-Market Experiments,”Journal of Marketing Research, 14 (November), 499–508.
32.
FisherRonald A. (1946), Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd.
33.
FornellClaes and LarckerDavid F. (1981), “Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,”Journal of Marketing Research, 18 (February), 39–50.
34.
FriedmanHerbert (1968), “Magnitude of Experimental Effect and a Table for Its Rapid Estimation,”Psychological Bulletin, 70 (October), 245–51.
35.
GibbonsJean Dickinson, OlkinIngram, and SobelMilton (1977), Selecting and Ordering Populations: A New Statistical Methodology. New York: John Wiley & Sons, Inc.
36.
GinterJames, CooperMartha, ObermillerCarl, and PageThomas (1981), “The Design of Advertising Experiments Using Statistical Decision Theory: An Extension,”Journal of Marketing Research, 18 (February), 120–3.
37.
GivonMoshe M. (1980), “Determination of Optimal Sample Sizes in the Beta-Binomial Brand Choice Model,”Journal of Marketing Research, 17 (May), 258–67.
38.
GivonMoshe M. and HorskyDan (1979), “Application of a Composite Stochastic Model of Brand Choice,”Journal of Marketing Research, 16 (May), 258–67.
39.
GlassGene V. and Ralph HakstianA. (1969), “Measures of Association in Comparative Experiments: Their Development and Interpretation,”American Educational Research Journal, 6 (May), 403–14.
40.
GoldenLinda L. (1979), “Consumer Reactions to Explicit Brand Comparisons of Advertisements,”Journal of Marketing Research, 16 (November), 517–32.
41.
GreenPaul E. (1973), “On the Analysis of Interactions in Marketing Research Data,”Journal of Marketing Research, 10 (November), 410–20.
42.
GreenPaul E., Douglas CarrollJ., and DeSarboWayne S. (1978), “A New Measure of Predictor Variable Importance in Multiple Regression,”Journal of Marketing Research, 15 (August), 356–60.
43.
GreenPaul E., Douglas CarrollJ., and DeSarboWayne S. (1980), “Reply to A Comment on a New Measure of Predictor Variable Importance in Multiple Regression,”Journal of Marketing Research, 17 (February), 116–8.
44.
GreenwaldAnthony G. (1975a), “Consequences of Prejudice Against the Null Hypothesis,”Psychological Bulletin, 82 (January), 1–19.
45.
GreenwaldAnthony G. (1975b), “Does the Good Samaritan Parable Increase Helping? A Comment on Darley and Batson's No-Effect Conclusion,”Journal of Personality and Social Psychology, 32 (October), 578–83.
46.
GreenwaldAnthony G. (1975c), “Significance, Non-significance, and Interpretation of an ESP Experiment,”Journal of Experimental Social Psychology, 11 (March), 180–191.
47.
GreenwaldAnthony G. (1976), “Within-Subject Designs: To Use or Not to Use?” Psychological Bulletin, 83 (March), 314–20.
48.
GraderCharles L., CookThomas, KarenHennigan, BrianFlay, CynthiaAessis, and JeromeHalamaj (1978), “Empirical Tests of the Absolute Sleeper Effect Predicted from the Discounting Cue Hypothesis,”Journal of Personality and Social Psychology, 36 (October), 1061–74.
49.
DelHawkins (1970), “The Effects of Subliminal Stimulation on Drive Level and Brand Preference,”Journal of Marketing Research, 7 (August), 322–6.
50.
HaysWilliam L. (1963), Statistics for Psychologists. New York: Holt, Rinehart, and Winston.
51.
JacksonBarbara Bund (1980), “Comment on ‘A New Measure of Predictor Variable Importance in Multiple Regression’,”Journal of Marketing Research, 17 (February), 113–5.
52.
JacobyJacob (1977), “Information Load and Decision Quality: Some Contested Issues,”Journal of Marketing Research, 14 (November), 568–73.
53.
JacobyJacob, SpellerDonald E., and BerningCarol A. K. (1975), “Constructive Criticism and Programmatic Research: Reply to Russo,”Journal of Consumer Research, 2 (September), 154–6.
54.
JacobyJacob, SpellerDonald E., and KohnCarol A. (1974a), “Brand Choice Behavior as a Function of Information Load,”Journal of Marketing Research, 11 (February), 63–9.
55.
JacobyJacob, SpellerDonald E., and KohnCarol A. (1974b), “Brand Choice Behavior as a Function of Information Load: Replication and Extension,”Journal of Consumer Research, 1 (June) 33–42.
56.
JohanssonJohnny K. and McAlisterLeigh (1980), “Using Variable Parameter Regression to Capture Individual Variations in Choice Models,”paper presented at National Meeting of TIMS/ORSA, Washington, D.C., May 7.
57.
JöreskogKarl G. (1974), “Analyzing Psychological Data by Structural Analysis of Covariance Matrices,” in Measurement, Psychographics, and Neural Information Processing, KrantzD. H., eds. San Francisco: Freeman, 1–56.
58.
KalwaniManohar U. and MorrisonDonald G. (1977), “Estimating the Proportion of ‘Always Buy’ and ‘Never Buy’ Consumers: A Likelihood Ratio Test with Sample Size Implications,”Journal of Marketing Research, 14 (November), 601–6.
59.
KalwaniManohar U. and MorrisonDonald G. (1980), “Sample Size Requirements for Zero-Order Models,”Journal of Marketing Research, 17 (May), 721–27
60.
KatzerJeffrey and SodtJames (1973), “An Analysis of the Use of Statistical Theory in Communication Research,”The Journal of Communications, 23 (September), 251–65.
61.
KeppelGeoffrey (1973), Design and Analysis: A Researcher's Handbook. Englewood Cliffs, New Jersey: Prentice-Hall, Inc.
62.
KeselmanH. J., GamesPaul A., and RoganJoanne C. (1980), “Type I and Type II Errors in Simultaneous and Two-Stage Multiple Comparison Procedures,”Psychological Bulletin, 88 (September), 356–8.
63.
KrollRichard M. and ChaseLawrence J. (1975), “Communication Disorders: A Power Analytic Assessment of Recent Research,”Journal of Communication Disorders, 8, 237–47.
64.
LaTourStephen A. (1981a), “Effect Size Estimation: A Commentary on Wolf and Bassler,”Decision Sciences, 12 (January), 136–41.
65.
LaTourStephen A. (1981b), “Variance Explained: It Measures Neither Importance or Effect Size,”Decision Sciences, 12 (January), 150–60.
66.
LodishLeonard and PekelmanDov (1978), “Increasing Precision of Marketing Experiments by Matching Sales Areas,”Journal of Marketing Research, 15 (August), 449–55.
67.
LutzRichard J. and KakkarPradeep (1975), “The Psychological Situation as a Determinant of Consumer Behavior,” in Advances in Consumer Research, Vol. 2, SchlingerMary J., ed. Chicago: Association for Consumer Research, 439–53.
68.
MarascuiloLeonard A. and McSweeneyMaryellen (1977), Nonparametric and Distribution-Free Methods for the Social Sciences. Belmont, California: Wadsworth.
69.
McAlisterLeigh (1979), “Choosing Multiple Items from a Product Class,”Journal of Consumer Research, 6 (December), 213–24.
70.
MeehlPaul E. (1967), “Theory Testing in Psychology and Physics: A Methodological Paradox,”Philosophy of Science, 16 (June), 103–15.
71.
MilliganGlenn W. (1979), “A Computer Program for Calculating Power of the Chi-Square Test,”Educational and Psychological Measurement, 39 (October), 681–4.
72.
MilliganGlenn W. (1980), “Factors That Affect Type I and Type II Error Rates in the Analysis of Multidimensional Contingency Tables,”Psychological Bulletin, 87 (March), 238–44.
73.
MontgomeryDavid B. (1975), “New Product Distribution—An Analysis of Supermarket Buyer Decisions,”Journal of Marketing Research, 12 (August), 255–64.
74.
MorrisonDenton E. and RamonE. Henkel, eds. (1970), The Significance Test Controversy. Chicago: Aldine.
75.
MorrisonDonald G. (1966), “Testing Brand-Switching Models,”Journal of Marketing Research, 3 (November), 401–9.
76.
MyersJerome L. (1980), Fundamentals of Experimental Design. Boston: Allyn and Bacon.
77.
OverallJohn E. and DalaiSudhin N. (1965), “Design of Experiments to Maximize Power Relative to Cost,”Psychological Bulletin, 64 (November), 339–50.
78.
PermutSteven E., MichelAllen J., and JosephMonica (1976), “The Researcher's Sample: A Review of the Choice of Respondents in Marketing Research,”Journal of Marketing Research, 13 (August), 278–83.
79.
PeterJ. Paul (1981), “Construct Validity: A Review of Basic Issues and Marketing Research,”Journal of Marketing Research, 18 (May).
80.
PetersonRobert A. and MahajanVijay (1976), “Practical Significance and Partitioning Variance in Discriminant Analysis,”Decision Sciences, 7 (October), 649–58.
81.
PhillipsLawrence D. (1973), Bayesian Statistics for Social Sciences. London: Thomas Nelson.
82.
PlattJohn R. (1964), “Strong Inference,”Science, 146 (October 16), 347–53.
83.
RosekransFrank M. (1969), “Statistical Significance and Reporting Test Results,”Journal of Marketing Research, 6 (November), 451–55.
RozeboomWilliam W. (1960), “The Fallacy of the Null Hypothesis Significance Test,”Psychological Bulletin, 57 (September), 416–28.
86.
RussoJ. Edward (1974), “More Information is Better: A Reevaluation of Jacoby, Speller, and Kohn,”Journal of Consumer Research, 1 (December), 68–72.
87.
RussoJ. Edward, MetcalfBarbara L., and StephensDebra (in press), “Toward an Empirical Technology for Identifying Misleading Advertising,” in Information Processing Research in Advertising, HarrisRichard J., ed. Hillsdale; New Jersey: Lawrence Earlbaum.
88.
SawyerAlan G. (1975), “Demand Artifacts in Laboratory Experiments in Consumer Research,”Journal of Consumer Research, 1 (March), 20–30.
89.
SawyerAlan G. and Dwayne BallA. (1980), “Statistical Power in Market Measurement and Analysis Research,” in Market Measurement and Analysis, Vol. II, LeoneRobert, ed. Austin, Texas: The Institute for Management Sciences, 147–67.
90.
SawyerAlan G., WorthingParker, and SendakPaul (1979), “The Role of Laboratory Experiments to Test Marketing Strategies,”Journal of Marketing, 43 (Summer), 60–7.
91.
SechrestLee and YeatonWilliam H. (1981a), “Empirical Bases for Estimating Effect Size,” in Reanalyzing Program Evaluation: Policies and Practices for Secondary Analysis of Social and Educational Programs, BoruchR. F., WortmanP. M., and CordrayD. S., eds. San Francisco: Jossey-Bass.
92.
SechrestLee and YeatonWilliam H. (1981b), “Estimating Magnitudes of Experimental Effects.” Ann Arbor: Institute of Social Research, University of Michigan.
93.
ShieldsWilliam S. and HeelerRoger M. (1979), “Analysis of Contingency Tables with Sparse Values,”Journal of Marketing Research, 16 (August), 382–6.
94.
ShoemakerRobert and StaelinRichard (1976), “The Effects of Sampling Variation on Sales Forecasts for New Consumer Product,”Journal of Marketing Research, 13 (May), 138–43.
95.
SmithMary Lee and GlassGene V. (1977), “Meta-Analysis of Psychotherapy Outcome Studies,”American Psychologist, 32 (September), 752–60.
96.
SrinivasanV. (1977), “A Theoretical Comparison of the Predictive Power of the Multiple Regression and Equal Weighting Procedures,”Research Paper No. 347, Stanford University (February).
97.
SrinivasanV. and KesavanR. (1976), “An Alternate Interpretation of the Linear Learning Model of Brand Choice,”Journal of Consumer Research, 13 (September), 76–83.
98.
SternthalBrian, DholakiaRuby, and LeavittClark (1978), “The Persuasive Effect of Source Credibility: Tests of Cognitive Response,”Journal of Consumer Research, 4 (March), 252–60).
99.
SternthalBrian, PhillipsLynn W., and DholakiaRuby (1978), “The Persuasive Effect of Source Credibility: A Situational Analysis,”Public Opinion Quarterly, 42 (Fall), 285–314.
100.
StevensJames P. (1980), “Power of the Multivariate Analysis of Variance Tests,”Psychological Bulletin, 88 (November), 728–37.
101.
SummersJohn O. (1974), “Less Information is Better?” Journal of Marketing Research, 11 (November), 467–8.
102.
SutcliffeJ. P. (1980), “On the Relationship of Reliability to Statistical Power,”Psychological Bulletin, 88 (September), 509–15.
103.
TuckerLedyard R. and LewisCharles (1975), “A Reliability Coefficient for Maximum Likelihood Factor Analysis,”Psychometrika, 38 (March), 1–10.
104.
TverskyAmos and KahnemanDaniel (1971), “Belief in the Law of Small Numbers,”Psychological Bulletin, 76 (August), 105–10.
105.
WalsterG. William and Anne ClearyT. (1970), “Statistical Significance as a Decision Rule,” in Sociological Methodology, BorgattaEdgar F. and BohrnstedtGeorge W., eds. San Francisco: Jossey-Bass, 246–54.
106.
WheatleyJohn J. and OshikawaSadaomi (1970), “The Relationship Between Anxiety and Positive and Negative Advertising Appeals,”Journal of Marketing Research, 7 (February), 85–9.
107.
WilkieWilliam L. (1974), “Analysis of Effects of Information Load,”Journal of Marketing Research, 11 (November), 462–6.
108.
WinerB. J. (1971), Statistical Principles in Experimental Design. New York: McGraw-Hill Book Company.
109.
WinerRussell S. and RyanMichael J. (1979), “Analyzing Cross-Classification Data: An Improved Method for Predicting Events,”Journal of Marketing Research, 16 (November), 539–44.
110.
WolfGerrit and BasslerJohn (1980), “Importance of Hypotheses: Group Comparison Analysis,”Decision Sciences, 11 (January), 27–41.
111.
WoodwardJ. A. and OverallJohn E. (1976). “A Computer Program for Calculating Power of the” F-Test, Educational and Psychological Measurement, 36 (Spring), 165–8.
