For the past half-century there has been a growing use of and interest in quantitative market research and the concomitant use of advanced statistical techniques within it. A review of scaling methods in market research and their analyses, validity and reliability is therefore, arguably, overdue. Marketing scholars and marketing research practitioners every time are in doubt about the validity of results in their research conduct, even over presentation and colorful reports, which can sometimes obfuscate the actual validity of a market research survey. Developing a valid and reliable scale in marketing research is very important for data collection and thereby deriving appropriate conclusions from the collected data. A reliable and valid scale is genuinely the foundation of any survey based research approach. This note explores fundamental theories and concepts behind these two issues of scale evaluation and the various approaches of evaluating the reliability and validity of a scale. An illustrative example has also been considered to discuss important issues regarding reliability and validity specifically how one can practically deal with reliability and validity.
Get full access to this article
View all access options for this article.
References
1.
BeardenW. O.NetemeyerR. G. (1999) Handbook of Marketing Scales: Multi-Item Measures for Marketing and Consumer Behavior Research, SAGE Publications.
2.
BormanW. C. (2001) “An Examination of the Comparative Reliability, Validity, and Accuracy of Performance Ratings Made Using Computerized Adaptive Rating Scales”Journal of Applied Psychology, 86(5), 965.
3.
CampbellD. T.FiskeD. W. (1959) “Convergent and Discriminant Validity by the Multitrait-Multimethod Matrix”, Psychological Bulletin, 56, 81–105.
4.
CampbellJohn P. (1976) “Psychometric Theory”, in DunetteMarvin D. (Ed.), Handbook of Industrial and Organizational PsychologyChicago Rand McNally, Inc.
5.
CardinetJ.TourneurY.AllalL. (1981). “Extension of Generalizability Theory and Its Applications in Educational Measurement”, Jour. Educ. Meas., 18, 183204.
6.
CarminesE. G.ZellerR. A. (1980) Reliability and Validity Assessment, London: Sage.
7.
ChurchillG. (1979), “A Paradigm for Developing Better Measures of Marketing Constructs”, Journal of Marketing Research, 6A-73.
8.
CooperD.SchindlerP. (2003) Business Research Methods, Tata Mcgraw-Hill Publishing Company Ltd., New Delhi.
9.
CronbachL. J. (1951) “Coefficient Alpha and the Internal Structure of Tests”, Psychometrika, 31, 93–96.
10.
CronbachL. J. (1971) “Test validation”, in ThorndikeR. L. (Ed.), Educational Measurement, 2nd ed., American Council on Education, Washington, D.C.
11.
CronbachL. J.GleserG. C.NandaH.RajaratnamN. (1972). The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. Wiley, New York. 410 p.
12.
DabholkarP. A.ThorpeD. I.RentzJ. O. (1996) “A Measure of Service Quality For Retail Stores: Scale Development and Validation”, Journal of the Academy of Marketing Science, 24(1), 3–16.
13.
De GruitjerP. N. M.Van Der KampL. J. T. (Eds.). (1976). Advances in Psychologica Land Educational Measurement.New York: Wiley.
14.
GreenleafE. A. (1992) “Improving Rating Scale Measures by Detecting and Correcting Bias Components in Some Response Styles, Journal of Marketing Research, 29, 176–188.
15.
HairJ. F.AndersonR. E.TathamR. L.BlackW. C. (2003) ‘Multivariate Data Analysis’5th ed, Pearson Education, New Delhi.
16.
HinkinT. (1995) “A Review of Scale Development Practices in the Study of Organizations”, Journal of Management, 21(5), 967–988.
17.
HunterJ. E.SchmidtF. L. (1990) Methods of Meta-analysis: Correcting error and Bias in Research Findings, Newsbury Park: SAGE Publications.
18.
KlineP. (1979). Psychometrics and Psychology, Academic Press, London.
19.
KlineP. (1986). A Handbook of Test Construction. Methuen, New York.
20.
LandisJ. R.KochG. G. (1977). “The Measurement of Observer Agreement for Categorical Data”, Biometrics, 33, 159–174.
21.
McKelvieS.J. (1992). “Dose Memory Test-retest reliability?”Journal of Gen. Psychology, 119(1), 59–72.
22.
MalhotraN. K. (2004) Marketing Research: An Applied Orientation, 4thed, Pearson Education.
23.
NunallyJ. C. (1970). Introduction to Psychological Measurement. McGraw-Hill., New York.
24.
NunnallyJ. (1967) Psychometric Methods, McGraw-Hill Book Co., New York.
25.
NunnallyJ. (1979) Psychometric Theory, 2nd ed., McGraw-Hill, New York.
26.
ParikhD. (2006) “Measuring Retail Service Quality: An Empirical Assessment of the Instrument”, to appear in Vikalpa: The Journal of Decision Makers.
27.
PeterJ. (1979) “Reliability: A Review of Psychometric Basics and Recent Marketing Practices”, Journal of Marketing Research, 16 (Feb), 6–17.
28.
PeterJ. (1981) “Construct Validity: A Review of Basic Issues and Marketing Practices”, Journal of Marketing Research, 18(May), 133–145.
29.
RobinsonJ. P.ShaverP. R.WrightsmanL. S. (1991) “Criteria for Scale Selection and Evaluation” in Measures of Personality and Social Psychological Attitudes, RobinsonJPShanverPRWrightmanLS (Eds.), Academic Press, San Diego, California.
30.
RobinsonJ. P.ShaverP. R. (1973) Measures of Psychological Attitudes, Ann Arbor, MI: Survey Research Center Institute for Social Research, University of Michigan.
31.
RuaneJ. M. (2005) Essentials of Research Methods, Blackwell Publishing Ltd., Oxford.
32.
SellitzClaireWrightsmanLawrence S.CookStuart W. (1976) Research Methods in Social Relations, 3rded, New YorkHolt, Rinehart, and Winston.
33.
ShavelsonR. J.WebbN. M. (1991). Generalizability Theory: A Primer. Sage Publ., Newbury Park, CA. 137 p.
34.
ShroutP.E.FleissJ. L. (1979). “Intra-class Correlations: Uses in assessing rater reliability”, Psychological Bulletin, 86, 420–428.
35.
SinghJ. (2001) Methodology and Techniques of Social Research, Kanishka Publishers, New Delhi.
36.
StrubeM. J. (2000). Reliability and Generalizability Theory. p. 2366 in Reading and Understanding More Multivariate Statistics.
37.
GrimmL. G.YarnoldP. R. (Eds.). Am. Psych. Assoc., Washington D.C.
38.
ThorndikeR. L.HagenE. P. (1977). Measurement and Evaluation in Psychology and Education. New York: Wiley.
