Different purposes of measurement reflect corresponding differences in marketing problems underlying the need for measurement. Reliability assessment in marketing, which is based on classical reliability theory, continues to focus on scaling of individual respondents and ignores other purposes of measurement. Besides the shortcomings in reliability assessment, the academic literature does not make any recommendations for efficiencies in scale usage by marketing firms in applied settings. Generalizability theory has long been identified as an alternative psychometric approach that can be applied to design efficient measurement and can explicitly take into account differences in the purpose of measurement. The authors take a generalizability theory perspective to (1) reiterate the rationale for taking into account the purpose of measurement when assessing reliability, (2) propose a general method to optimize the design of marketing measurement in applied studies, and (3) conduct an empirical study on service quality measurement that illustrates the dual advantages of optimizing measurement and the generalizability approach to marketing measurement.
Get full access to this article
View all access options for this article.
References
1.
AakerDavid A. (1996), Building Strong Brands.New York: The Free Press
2.
AndersonEugene W. (1994), “Cross-Category Variation in Customer Satisfaction and Retention,”Marketing Letters, 5(January), 19–30.
3.
AndersonEugene W., FornellClaes, and LehmannDonald R. (1994), “Customer Satisfaction, Market Share, and Profitability: Findings From Sweden,”Journal of Marketing, 58(July), 53–66.
4.
BabakusEmin, and BoilerGregory W. (1992), “An Empirical Assessment of the SERVQUAL Scale,”Journal of Business Research, 24, 253–68.
5.
BeardenWilliam O., NetemeyerRichard G., and MobleyMary F. (1993), Handbook of Marketing Scales: Multi-Item Measures for Marketing and Consumer Behavior Research.Newbury Park, CA: Sage Publications.
6.
BitnerMary Jo, and HubbertAmy R. (1994), “Encounter Satisfaction Versus Overall Satisfaction Versus Quality: The Customers Voice,” in Service Quality: New Directions in Theory and Practice, RustRoland T., and OliverRichard L., eds. Thousand Oaks, CA: Sage Publications.
7.
BoltonRuth N., and DrewJames H. (1991a), “A Longitudinal Analysis of the Impact of Service Changes on Customer Attitudes,”Journal of Marketing, 55(January), 1–9.
8.
BoltonRuth N., and DrewJames H. (1991b), “A Multistage Model of Customers Assessments of Service Quality and Value,”Journal of Consumer Research, 17(March), 375–84.
9.
BouldingWilliam, KalraAjay, StaelinRichard, and ZeithamlValarie A. (1993), “A Dynamic Process Model of Service Quality: From Expectations to Behavioral Intentions,”Journal of Marketing Research, 30(February), 7–27.
10.
BrennanRobert (1983), Elements of Generalizability Theory.Iowa City, IA: ACT Publications.
11.
BrownStephen W., and SwartzTeresa A. (1989), “A Gap Analysis of Professional Service Quality,”Journal of Marketing, 53(April), 92–98.
12.
BrownTom J., ChurchillGilbert A.Jr., and Paul PeterJ. (1993), “Research Note: Improving the Measurement of Service Quality,”Journal of Retailing, 69(Spring), 127–39.
13.
BrunerGordon C.II, and HenselPaul J. (1993a), “Multi-item Scale Usage in Marketing Journals: 1980 to 1989,”Journal of the Academy of Marketing Science, 21(Fall), 339–44.
14.
BrunerGordon C.II, and HenselPaul J. (1993b), Marketing Scales Handbook: A Compilation of Multi-item Measures.Chicago: American Marketing Association.
15.
BurdickRichard K., and GraybillFranklin A. (1992), Confidence Intervals on Variance Components.New York: Marcel Dekker.
16.
CarmanJames M. (1990), “Consumer Perceptions of Service Quality: An Assessment of the SERVQUAL Dimensions,”Journal of Retailing, 66(Spring), 33–55.
17.
ChurchillGilbert A.Jr. (1979), “A Paradigm for Developing Better Measures of Marketing Constructs,”Journal of Marketing Research, 16(February), 64–73.
18.
ChurchillGilbert A.Jr., and Paul PeterJ. (1984), “Research Design Effects on the Reliability of Rating Scales: A Meta-Analysis,”Journal of Marketing Research, 21(November), 360–75.
19.
CooilBruce, and RustRoland T. (1994), “Reliability and Expected Loss: A Unifying Principle,”Psychometrika, 59(June), 203–16.
20.
CooilBruce, and RustRoland T. (1995), “General Estimators for the Reliability of Qualitative Data,”Psychometrika, 60(June), 199–220.
21.
CronbachLee J., GleserGoldine C., NandaHarinder, and RajaratnamNageswari (1972), The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles.New York: John Wiley & Sons.
22.
CronbachLee J., RajaratnamNageswari, and GleserGoldine C. (1963), “Theory of Generalizability: A Liberalization of Reliability Theory,”British Journal of Statistical Psychology, 16(November), 137–63.
23.
CroninJ. Joseph, and TaylorSteven A. (1992), “Measuring Service Quality: A Reexamination and Extension,”Journal of Marketing, 56(July), 55–68.
24.
CroninJ. Joseph, and TaylorSteven A. (1994), “SERVPERF Versus SERVQUAL: Reconciling Performance-Based and Perceptions-Minus-Expectations Measurement of Service Quality,”Journal of Marketing, 58(January), 125–31.
25.
DevlinSusan J., and DongH. K. (1994), “Service Quality from the Customers Perspective,”Marketing Research, 6(1), 5–13.
26.
DevlinSusan J., DongH. K., and BrownMarbue (1993), “Selecting a Scale for Measuring Quality,”Marketing Research, 5(Summer), 12–17.
27.
FarleyJohn U., and LehmannDonald R. (1986), Meta-Analysis in Marketing: Generalization of Response Models.Lexington, MA: Lexington Books.
28.
FornellClaes (1992), “A National Customer Satisfaction Barometer,”Journal of Marketing, 56(January), 6–21.
29.
GerbingDavid W., and AndersonJames C. (1988), “An Updated Paradigm for Scale Development Incorporating Unidimensionality and Its Assessment,”Journal of Marketing Research, 25(May), 186–92.
30.
HartleyH. O., and RaoJ. N. K. (1967), “Maximum Likelihood Estimation for Mixed Analysis of Variance,”Biometrika, 54, 93–108.
31.
HartleyH. O., RaoJ. N. K., and LamotteL. (1978), “A Simple Synthesis-Based Method of Variance Component Estimation,”Biometrics, 34, 233–44.
32.
HauserJohn R., SimesterDuncan I., and WernerfeltBirger (1994), “Customer Satisfaction Incentives,”Marketing Science, 13(Fall), 327–50.
33.
HughesMarie Adele, and GarrettDennis E. (1988), “Inter-Coder Reliability Estimation Approaches in Marketing: A Generalizability Theory Framework for Quantitative Data,”Journal of Marketing Research, 27(May), 185–95.
34.
IacobucciDawn, GraysonKent A., and OstromAmy L. (1994), “The Calculus of Service Quality and Customer Satisfaction: Theoretical and Empirical Differentiation and Integration,”Advances in Services Marketing and Management, 3, 1–67.
35.
JacobyJacob (1978), “Consumer Research: A State of the Art Review,”Journal of Marketing, 42(April), 87–96.
36.
KhuriA. I., and SahaiHardeo (1985), “Variance Components Analysis: A Selective Literature Survey,”International Statistical Review, 53(3), 279–300.
37.
LastovickaJohn L. (1983), “Convergent and Discriminant Validity of Television Commercial Rating Scales,”Journal of Advertising, 12(2), 14–23.
38.
MarcoulidesGeorge A. (1995), “Designing Measurement Studies Under Budget Constraints: Controlling Error of Measurement and Power,”Educational and Psychological Measurement, 55(3), 423–28.
39.
NarverJohn C., and SlaterStanley F. (1990), “The Effect of a Market Orientation on Business Profitability,”Journal of Marketing, 54(4), 20–35.
40.
NunnallyJum C. (1978), Psychometric Theory, 2d ed.New York: McGraw-Hill.
41.
ParasuramanA., BerryLeonard L., and ZeithamlValarie A. (1991), “Refinement and Reassessment of the SERVQUAL Scale,”Journal of Retailing, 67(Winter), 420–50.
42.
ParasuramanA., BerryLeonard L., and ZeithamlValarie A. (1993), “Research Note: More on Improving Service Quality Measurement,”Journal of Retailing, 69(Spring), 140–47.
43.
ParasuramanA., ZeithamlValarie A., and BerryLeonard L. (1985), “A Conceptual Model of Service Quality and Its Implications for Future Research,”Journal of Marketing, 49(Fall), 41–50.
44.
ParasuramanA., ZeithamlValarie A., and BerryLeonard L. (1988), “SERVQUAL: A Multiple-Item Scale for Measuring Consumer Perceptions of Service Quality,”Journal of Retailing, 64(Spring), 12–40.
45.
ParasuramanA., ZeithamlValarie A., and BerryLeonard L. (1994a), “Reassessment of Expectations as a Comparison Standard in Measuring Service Quality: Implications for Further Research,”Journal of Marketing, 58(January), 111–24.
46.
ParasuramanA., ZeithamlValarie A., and BerryLeonard L. (1994b), “Alternative Scales for Measuring Service Quality: A Comparative Assessment Based on Psychometric and Diagnostic Criteria,”Journal of Retailing, 70(January), 201–30.
47.
PeterJ. Paul (1979), “Reliability: A Review of Psychometric Basics and Recent Marketing Practices,”Journal of Marketing Research, 16(February), 6–17.
48.
PeterJ. Paul, and RayMichael L. (1984), “Introduction,” in Measurement Readings for Marketing Research, Paul PeterJ., and RayMichael L., eds. Chicago: American Marketing Association, vii.
49.
PetersonRobert A. (1994), “A Meta-Analysis of Cronbach's Coefficient Alpha,”Journal of Consumer Research, 21(September), 381–91.
50.
RaoC. R. (1971), “Estimation of Variance and Covariance Components—MINQUE Theory,”Journal of Multivariate Analysis, 1, 257–75.
51.
ReibsteinDavid, BatesonJohn E., and BouldingWilliam (1988), “Conjoint Analysis Reliability: Empirical Findings,”Marketing Science,1(Summer), 271–86.
52.
RentzJoseph O. (1987), “Generalizability Theory: A Comprehensive Method for Assessing and Improving the Dependability of Marketing Measures,”Journal of Marketing Research, 24(February), 19–28.
53.
RentzJoseph O. (1988), “An Exploratory Study of the Generalizability of Selected Marketing Measures,”Journal of the Academy of Marketing Science, 16(Spring), 141–50.
54.
RustRoland T., and CooilBruce (1994), “Reliability Measures for Qualitative Data: Theory and Implications,”Journal of Marketing Research, 31(February), 1–14.
55.
RustRoland T., ZahorikAnthony J., and KeininghamTimothy L. (1995), “Return on Quality (ROQ): Making Service Quality Financially Accountable,”Journal of Marketing, 59(April), 58–70.
56.
SalkinH. M. (1975), Integer Programming.Reading, MA: Addison-Wesley.
57.
SandersP. F., TheunissenT. J. J. M., and BaasS. M. (1989), “Minimizing the Number of Observations: A Generalization of the Spearman-Brown Formula,”Psychometrika, 54, 587–98.
58.
SandersP. F., TheunissenT. J. J. M., and BaasS. M. (1991), “Maximizing the Coefficient of Generalizability Under the Constraint of Limited Resources,”Psychometrika, 56, 87–96.
59.
SatterthwaiteF. E. (1946), “An Approximate Distribution of Estimates of Variance Components,”Biometrics Bulletin, 2, 110–14.
60.
SchmalenseeDiane H. (1994), “Finding the Perfect Scale,”Marketing Research, 6(Fall), 24–27.
61.
SearleShayle R., CasellaGeorge, and McCullochCharles E. (1992), Variance Components.New York: John Wiley & Sons.
62.
ShavelsonRichard J., and WebbNoreen M. (1991), Generalizability Theory: A Primer.Newbury Park, CA: Sage Publications.
63.
SpechlerJay W. (1991), When America Does It Right: Case Studies in Service Quality.Norcross, GA: Institute of Industrial Engineers.
64.
SteenkampJan-Benedict E. M., and van TrijpHans C. M. (1991), “The Use of LISREL in Validating Marketing Constructs,”International Journal of Research in Marketing, 8(November), 283–99.
65.
TeasKenneth R. (1994), “Expectations as a Comparison Standard in Measuring Service Quality: An Assessment of a Reassessment,”Journal of Marketing, 58(January), 132–39.
66.
ZahorikAnthony J., and RustRoland T. (1992), “Modeling the Impact of Service Quality on Profitability: A Review,”Advances in Services Marketing and Management, 1, 247–76.
