This study compares the performance of four commonly used approaches to measure consumers’ willingness to pay with real purchase data (REAL): the open-ended (OE) question format; choice-based conjoint (CBC) analysis; Becker, DeGroot, and Marschak's (BDM) incentive-compatible mechanism; and incentive-aligned choice-based conjoint (ICBC) analysis. With this five-in-one approach, the authors test the relative strengths of the four measurement methods, using REAL as the benchmark, on the basis of statistical criteria and decision-relevant metrics. The results indicate that the BDM and ICBC approaches can pass statistical and decision-oriented tests. The authors find that respondents are more price sensitive in incentive-aligned settings than in non-incentive-aligned settings and the REAL setting. Furthermore, they find a large number of “none” choices under ICBC than under hypothetical conjoint analysis. This study uncovers an intriguing possibility: Even when the OE format and CBC analysis generate hypothetical bias, they may still lead to the right demand curves and right pricing decisions.
AbramsJack, (1964), “A New Method for Testing Pricing Decisions,”Journal of Marketing, 28(January), 6–9.
2.
AdamDaniel, (1958), Les Réactions du Consommateur Devant le Prix.Paris: Sedes.
3.
AndersonJames C., JainDipak, and ChintaguntaPradeep K., (1993), “Understanding Customer Value in Business Markets: Methods of Customer Value Assessment,”Journal of Business-to-Business Marketing, 1(1), 3–30.
4.
AroraNeeraj, and HuberJoel, (2001), “Improving Parameter Estimates and Model Prediction by Aggregate Customization in Choice Experiments,”Journal of Consumer Research, 28(September), 273–83.
5.
ArrowKenneth, SolowRobert, PortnoyPaul R., LeamerEdward E., RadnerRoy, and SchumanHoward, (1993), “Report from the NOAA Panel on Contingent Valuation,”Federal Register, 58(10), 4601–4614.
6.
BackhausKlaus, WilkenRobert, VoethMarkus, and SichtmannChristina, (2005), “An Empirical Comparison of Methods to Measure Willingness to Pay by Examining the Hypothetical Bias,”International Journal of Market Research, 47(5), 543–62.
7.
BalistreriEdward, McClellandGary, PoeGregory, and SchulzeWilliam, (2001), “Can Hypothetical Questions Reveal True Values? A Laboratory Comparison of Dichotomous Choice and Open-Ended Contingent Values with Auction Values,”Environmental and Resource Economics, 18(3), 275–92.
8.
BeckerGordon M., DeGrootMorris H., and MarschakJacob, (1964), “Measuring Utility by a Single-Response Sequential Method,”Behavioral Science, 9(3), 226–32.
9.
BelloniAlexandre, FreundRobert, SeloveMatthew, and SimesterDuncan, (2008), “Optimizing Product Line Designs: Efficient Methods and Comparisons,”Management Science, 54(9), 1544–52.
10.
BishopRichard C., WelshMichael P., and HeberleinThomas A., (1992), “Some Experimental Evidence on the Validity of Contingent Valuation,” working paper, Department of Agricultural Economics, University of Wisconsin.
11.
BohmPeter, LindénJohan, and SonnegårdJoakim, (1997), “Eliciting Reservation Prices: Becker-DeGroot-Marschak Mechanisms vs. Markets,”The Economic Journal, 107(443), 1079–1089.
12.
BrazellJeff D., DienerChristopher G., KarniouchinaEkaterina, MooreWilliam L., SeverinValerie, and UldryPierre-Francois, (2006), “The No-Choice Option and Dual Response Choice Designs,”Marketing Letters, 17(4), 255–68.
13.
BrownThomas C., ChampPatricia A., BishopRichard C., and McCollumDaniel W., (1996), “Which Response Format Reveals the Truth about Donations to a Public Good,”Land Economics, 72(2), 152–66.
14.
CameronTrudy A., and JamesMichelle D., (1987), “Estimating Willingness to Pay from Survey Data: An Alternative to Pretest-Market Evaluation Procedure,”Journal of Marketing Research, 26(November), 389–95.
15.
ChernevAlexander, (2003), “Reverse Pricing and Online Price Elicitation Strategies in Consumer Choice,”Journal of Consumer Psychology, 13(1–2), 51–62.
16.
DingMin, (2007), “An Incentive-Aligned Mechanism for Conjoint Analysis,”Journal of Marketing Research, 44(May), 214–23.
17.
DingMin, GrewalRajdeep, and LiechtyJohn, (2005), “Incentive-Aligned Conjoint Analysis,”Journal of Marketing Research, 42(February), 67–82.
18.
DongSongting, DingMin, and HuberJoel, (2010), “A Simple Mechanism to Incentive-Align Conjoint Experiments,”Internationl Journal of Research in Marketing, 24(4), 312–23.
19.
EfronBradley, and TibshiraniRobert J., (1993), An Introduction to the Bootstrap.Boca Raton, FL: Chapman & Hall/CRC.
20.
FrykblomPeter, (1997), “Hypothetical Question Modes and Real Willingness to Pay,”Journal of Environmental Economics and Management, 34(3), 275–87.
21.
GaborAndré, and GrangerClive W.J., (1966), “Price as an Indicator of Quality: Report on an Enquiry,”Economica, 33(February), 43–70.
22.
GautschiDavid A., and RaoVithala R., (1990), “A Methodology for Specification and Aggregation in Product Concept Testing,” in Telecommunications Demand Modelling: An Integrated View, Alain de FontenayMary, ShugardH., and SibleyDavid S., eds. Amsterdam: North-Holland, 37–63.
23.
GhiselliEdwin E., CampbellJohn P., and ZedeckSheldon, (1981), Measurement Theory for the Behavioral Sciences.San Francisco: W.H. Freeman.
24.
HanemannW. Michael, (1984), “Welfare Evaluations in Contingent Valuation Experiments with Discrete Responses,”American Journal of Agricultural Economics, 66(3), 332–41.
25.
HarrisonGlenn W., and Elisabet RutströmE., (2008), “Experimental Evidence on the Existence of Hypothetical Bias in Value Elicitation Methods,” in Handbook of Experimental Economics Results, PlottCharles R., and SmithVernon L., eds. Amsterdam: North-Holland, 727–67.
26.
HarstadRonald, M., (2000), “Dominant Strategy Adoption and Bidders’ Experience with Pricing Rules,”Experimental Economics, 3(3), 261–80.
27.
HuberJoel, (1997), “What We Have Learned from 20 Years of Conjoint Research: When to Use Self-Explicated, Graded Pairs, Full Profiles, or Choice Experiments,” in Sawtooth Software Conference Proceedings.Sequim, WA: Sawtooth Software, [available at http://www.sawtoothsoftware.com/download/techpap/1997Proceedings.pdf#page=257].
28.
HuberJoel, and ZwerinaKlaus, (1996), “The Importance of Utility Balance in Efficient Choice Designs,”Journal of Marketing Research, 33(August), 307–317.
29.
JedidiKamel, and JagpalSharan, (2009), “Willingness to Pay: Measurement and Managerial Implications,” in Handbook of Pricing Research in Marketing, RaoVithala R., ed. Cheltenham, UK: Edward Elgar Publishing, 37–60.
30.
JedidiKamel, JagpalSharan, and ManchandaPuneet, (2003), “Measuring Heterogeneous Reservation Prices for Product Bundles,”Marketing Science, 22(2), 107–130.
31.
JedidiKamel, and John ZhangZ., (2002), “Augmenting Conjoint Analysis to Estimate Consumer Reservation Price,”Management Science, 48(10), 1350–1368.
32.
KaasKlaus P., and RuprechtHeidrun, (2006), “Are the Vickrey Auction and the BDM-Mechanism Really Incentive Compatible? Empirical Results and Optimal Bidding Strategies in the Case of Uncertain Willingness-to-Pay,”Schmalenbach Business Review, 55(1), 37–55.
33.
KagelJohn H., HarstadRonald M., and LevinDan, (1987), “Information Impact and Allocation Rules in Auctions with Affiliated Private Values: A Laboratory Study,”Econometrica, 55(6), 1275–1304.
34.
KagelJohn H., and LevinDan, (1993), “Independent Private Value Auctions: Bidder Behaviour in First-, Second- and Third-Price Auctions with Varying Numbers of Bidders,”Economic Journal, 103(419), 868–79.
35.
KohliRajeev, and MahajanVijay, (1991), “A Reservation-Price Model for Optimal Pricing of Multiattribute Products in Conjoint Analysis,”Journal of Marketing Research, 28(August), 347–54.
36.
KrinskyItzhak, and Leslie RobbA., (1986), “On Approximating the Statistical Properties of Elasticities,”Review of Economics and Statistics, 68(November), 715–19.
37.
LouviereJordan J., and WoodworthGeorge, (1983), “Design and Analysis of Simulated Consumer Choice or Allocation Experiments: An Approach Based on Aggregate Data,”Journal of Marketing Research, 20(November), 350–67.
38.
LuskJayson L., and SchroederTed C., (2004), “Are Choice Experiments Incentive Compatible? A Test with Quality Differentiated Beef Steaks,”American Journal of Agricultural Economics, 86(2), 467–82.
39.
MassartPascal, (1990), “The Tight Constant in the Dvoretzky-Kiefer-Wolfowitz Inequality,”Annals of Probability, 18(3), 1269–83.
40.
MitchellRobert C., and CarsonRichard T., (1989), Using Surveys to Value Public Goods: The Contingent Valuation Method.Washington, DC: Resources for the Future.
41.
OrmeBryan, K., (2003), “Which Conjoint Method Should I Use?” Research Paper Series, Sawtooth Software.
42.
OrmeBryan, K., (2006), Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research.Madison: Research Publishers.
43.
ParkTimothy, LoomisJohn B., and CreelMichael, (1991), “Confidence Intervals for Evaluating Benefits Estimates from Dichotomous Choice Contingent Valuation Studies,”Land Economics, 67(1), 64–73.
44.
ParkYoung-Hoon, DingMin, and RaoVithala R., (2008), “Eliciting Preference for Complex Products: A Web-Based Eliciting Upgrading Method,”Journal of Marketing Research, 45(October), 562–74.
45.
PoeGregory L., Severance-LossinEric K., and WelshMike P., (1994), “Measuring the Difference (X-Y) of Simulated Distributions: A Convolutions Approach,”American Journal of Agricultural Economics, 76(November), 904–915.
46.
RaoVithala R., and SattlerHenrik, (2003), “Measurement of Price Effects with Conjoint Analysis: Separating Informational and Allocative Effects of Price,” in Conjoint Measurement. Methods and Applications, GustaffssonAnders, HerrmannAndreas, and HuberFrank, eds. Berlin: Springer, 47–66.
47.
ShafferGreg, and John ZhangZ., (1995), “Competitive Coupon Targeting,”Marketing Science, 14(4), 395–416.
48.
ShafferGreg, and John ZhangZ., (2000), “Competitive Pay to Switch or Pay Not to Switch: Third Degree Price Discrimination in Markets with Switching Costs,”Journal of Economics & Management Strategy, 9(3), 397–424.
49.
SilvaAndres, NaygaRodolfo M.Jr., CampbellBen L., and ParkJohn, (2007), “On the Use of Valuation Mechanisms to Measure Consumers’ Willingness to Pay for Novel Products: A Comparison of Hypothetical and Non-Hypothetical Values,”International Food and Agribusiness Management Review, 10(2), 165–80.
SonnierGarrett, AinslieAndrew, and OtterThomas, (2007), “Heterogeneity Distributions of Willingness-to-Pay in Choice Models,”Quantitative Marketing Economics, 5(3), 313–31.
52.
SteenkampJan-Benedict E.M., and WittinkDick R., (1994), “The Metric Quality of Full-Profile Judgments and the Number-of-Attribute Levels Effect in Conjoint Analysis,”International Journal of Research in Marketing, 11(3), 275–86.
53.
SwaitJoffre, and LouviereJordan J., (1993), “The Role of the Scale Parameter in the Estimation and Comparison of Multinomial Logit Models,”Journal of Marketing Research, 30(August), 305–314.
54.
ThalerRichard, (1985), “Mental Accounting and Consumer Choice,”Marketing Science, 4(3), 199–214.
55.
ToubiaOlivier, HauserJohn, and GarciaRosanna, (2007), “Probabilistic Polyhedral Methods for Adaptive Choice-Based Conjoint Analysis: Theory and Application,”Marketing Science, 26(5), 596–610.
56.
TrainKenneth, (2003), Discrete Choice Methods with Simulation.Cambridge, UK: Cambridge University Press.
57.
TrainKenneth, and WeeksMelvyn, (2005), “Discrete Choice Models in Preference Space and Willingness-to-Pay Space,” in Applications of Simulation Methods in Environmental and Resource Economics, ScarpaRiccardo, and AlberiniAnna A., eds. Dordrecht, the Netherlands: Springer, 1–16.
58.
VarianHal, R., (1992), Microeconomic Analysis, Vol. 3. New York: W.W. Norton.
59.
VerleghPeeter W.J., SchiffersteinHendrik N.J., and WittinkDick R., (2002), “Range and Number-of-Levels Effects in Derived and Stated Measures of Attribute Importance,”Marketing Letters, 13(1), 41–52.
60.
VoelcknerFranziska, (2006), “An Empirical Comparison of Methods for Measuring Consumers’ Willingness to Pay,”Marketing Letters, 17(2), 137–49.
61.
VoelcknerFranziska, (2008), “The Dual Role of Price: Decomposing Consumers’ Reactions to Price,”Journal of the Academy of Marketing Science, 36(3), 359–77.
62.
WangTuo, VenkateshR., and ChatterjeeRabikar, (2007), “Reservation Price as a Range: An Incentive-Compatible Measurement Approach,”Journal of Marketing Research, 44(May), 200–213.
63.
WertenbrochKlaus, and SkieraBernd, (2002), “Measuring Consumers’ Willingness to Pay at the Point of Purchase,”Journal of Marketing Research, 39(May), 228–41.
64.
WittinkDick A., KrishnamurthiLakshman, and NutterJulia B., (1989), “The Effect of Differences in the Number of Attribute Levels on Conjoint Results,”Marketing Letters, 1(2), 113–23.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
