Making Recommendations More Effective Through Framings: Impacts of User- Versus Item-Based Framings on Recommendation Click-Throughs

Abstract

Companies frequently offer product recommendations to customers, according to various algorithms. This research explores how companies should frame the methods they use to derive their recommendations, in an attempt to maximize click-through rates. Two common framings—user-based and item-based—might describe the same recommendation. User-based framing emphasizes the similarity between customers (e.g., “People who like this also like…”); item-based framing instead emphasizes similarities between products (e.g., “Similar to this item”). Six experiments, including two field experiments within a mobile app, show that framing the same recommendation as user-based (vs. item-based) can increase recommendation click-through rates. The findings suggest that user-based (vs. item-based) framing informs customers that the recommendation is based on not just product matching but also taste matching with other customers. Three theoretically derived and practically relevant boundary conditions related to the recommendation recipient, the products, and other users also offer practical guidance for managers regarding how to leverage recommendation framings to increase recommendation click-throughs.

Keywords

advice taking algorithms explanations framing recommender systems

Many companies provide customers with product recommendations that have been generated by algorithmic recommender systems: Spotify and Netflix recommend songs or movies for their subscribers, and TripAdvisor and Yelp provide recommendations for hotels or restaurants. Amazon suggests which products consumers might want to buy, and the New York Times recommends different news articles. These personalized recommendations help customers find offerings they likely are interested in and also increase their loyalty (Gupta et al. 2006; Kamakura et al. 2005). According to a survey by Spotify, 65% of customers find a new favorite song in the personalized playlists they receive (Johnson 2015), and Netflix asserts that its recommender system effectively reduces customer churn and saves the company more than $1 billion annually (Gomez-Uribe and Hunt 2015).

To improve the accuracy of these algorithmic recommendations, recommender systems frequently adopt a hybrid approach that accounts for both common preferences across customers and common attributes across products (Amatriain and Basilico 2016). Each recommendation thus is based on both user and product input; it is not straightforward to explain the basis of the recommendation descriptively. In our interviews with members of a major European e-commerce company, the data scientists expressed different opinions about whether user or product input best described the basis for their recommender system, which actually uses various inputs. In turn, this company, and others alike, could choose which component to emphasize when explaining how it derives recommendations for customers. Some companies already highlight that their recommendations are user-based by focusing on overlaps in customer preferences, such as “Customers who viewed this item also viewed…” by Amazon and “Customers also watched…” by Netflix. In contrast, other companies emphasize that recommendations are item-based, such as “Similar to [what you have listened to]” by Spotify and “More in Health” by the New York Times.

The question that then arises is which framing, user-based or item-based, is more effective in triggering clicks on a recommendation. We aim to provide an answer to this question by framing the same recommendation as user-based or item-based and comparing customers’ decisions to click on it. Such clicks can increase conversion rates by stimulating customers to explore other product offerings (Xu, Duan, and Whinston 2014). Prior research on customers’ responses to recommendations has focused primarily on the underlying recommendation algorithms (Ariely, Lynch, and Aparicio 2004; Hennig-Thurau, Marchand, and Marx 2012; Ying, Feinberg, and Wedel 2006) or characteristics of recommended products (Cooke et al. 2002; Pathak et al. 2010), with limited attention to the framing provided to describe the recommendations. This gap is surprising for two main reasons. First, many recommendations rely on input from both users and items, so companies can choose to highlight different elements. Second, altering recommendation framing is a nearly zero-cost effort. To address this managerially relevant gap, we manipulate recommendation framing (user-based vs. item-based) but keep the underlying algorithms and recommended products constant.

Our central proposition is that, compared with item-based framing, user-based framing informs customers that the recommendation is generated through product matching (i.e., the recommended product is similar to the focal product) but also indicates taste matching between users (i.e., the focal product liked by oneself is also liked by other users). Customers extract information from similar others’ tastes to predict their own liking of unfamiliar products (Yaniv, Choshen-Hillel, and Milyavsky 2011), so this information should provide an additional guarantee to customers that the product will match their tastes. Consequently, we predict that recommendations framed as user-based (vs. item-based) attract more click-throughs. This prediction rests on the assumption that consumers perceive that a recommendation accurately matches their taste. We thus consider three important boundary conditions (related to the product, the customer, and other users) that might lead to perceptions of unsuccessful taste matching, such that the advantage of user-based framing over item-based framing shrinks or even reverses.

Our investigation spans a variety of data sources and consumption domains. Two field experiments involve article recommendations within WeChat, the top mobile app in China (Studies 1a and 1b). Study 2 is a behavioral experiment with painting recommendations. Then two scenario experiments focus on book recommendations (Studies 3 and 4), followed by a behavioral experiment in the same domain (Study 5). Across these various methods and product contexts, we consistently find that user-based framing attracts more click-throughs on recommendations than item-based framing when customers perceive that others’ preferences match their own. We further propose three boundary conditions that potentially cause the recommendation recipient to perceive the taste matching as unsuccessful, so the advantage of user-based framing over item-based framing decreases. These boundary conditions in turn offer substantive guidance for companies on how to adapt the framing of their recommendations to maximize recommendation click-throughs.

Theoretical Background

Recommendation Systems and Explanations to Customers

Recommendation systems are an automated, data-driven tool that companies frequently adopt to fulfill their customers’ personalization needs (Hinz and Eckert 2010; Ricci, Rokach, and Shapira 2015). Depending on what customers have viewed, liked, or purchased, these systems predict what other products they could be interested in and deliver instant suggestions. Research in marketing and information systems highlights such recommender systems as important determinants of sales (Bodapati 2008; Fleder and Hosanagar 2009; Pathak et al. 2010). Two typical methods inform these recommendations. First, collaborative filtering identifies customers who are similar in their product rating history and recommends items that one customer likes to similar other customers. The product ratings might be explicitly provided by customers or inferred from their online behavior. Second, content-based filtering identifies the product attributes that a customer likes and recommends products with similar attributes (Ansari, Essegaier, and Kohli 2000). Because each method has shortcomings, companies often combine them to improve the performance of their hybrid recommender systems. Examples include Amazon’s “item-to-item collaborative filtering” (Linden, Smith, and York 2017), and the New York Times’ collaborative topic modeling (Spangher 2015). Extensive research suggests ways to improve the prediction accuracy of recommendation algorithms using hybrid frameworks (Zhang et al. 2018).

The computationally complex algorithms pose challenges for explaining recommendations to customers. A clear, concise, accurate explanation is crucial, because it promotes customers’ trust in the recommender systems (Wang and Benbasat 2007) and acceptance of recommendations (Cramer et al. 2008; Kramer 2007). To the best of our knowledge, no research in marketing has suggested the optimal methods for explaining recommendations. In information systems literature, Tintarev and Masthoff (2015) identify five recommendation explanation types. Two explanations are particularly relevant to our research: collaborative-based and content-based. As their names imply, collaborative-based explanations such as “Customers who bought this item also bought…” rely on recommender systems that adopt collaborative filtering, whereas content-based explanations, such as “Recommended because you said you owned…,” involve recommender systems that use content-based filtering. The other explanation types either overlap with the content-based explanation (e.g., cased-based that specifies the items compared by the underlying algorithm) or assume unique inputs (e.g., demographic-based; Tintarev and Masthoff 2015).

Rather than explanation styles yoked to distinct, specific recommendation algorithms (Tintarev and Masthoff 2015), we define recommendation framing according to the various explanations that might be provided, even with the same recommender system. Because most recommender systems take a hybrid approach that combines the input from users (i.e., interuser similarity in preferences) and the input from items (i.e., interitem similarity in attributes), we compare framings that highlight one input over the other, user-based framing versus item-based framing. Accordingly, the goal of the current research is to establish the causal impact of alternating between the user-based and item-based framings on click-throughs of recommendations, rather than to provide an exhaustive categorization of explanation styles (Tintarev and Masthoff 2015).

Comparisons of Item-Based and User-Based Framing

As we detail in Figure 1, with user-based framing, the provided explanations draw attention to the shared tastes of consumers of a focal item. This framing describes how the target user (u) is similar to other users (u′), due to their shared interest in the focal item (i), and it indicates that the focal item (i) and recommended item (i′) are related because they attract the same users (u′). Item-based framing instead highlights the match between the focal and the recommended products (i and i′), either with or without specifying their shared properties. For example, “More in Health” suggests that recommended articles will be similar to the focal item, because they fall in the same news category; “Similar to this item” also emphasizes the relationship between the items but does not cite specific product attributes.

Figure 1.

Illustration of the definitions of user-based and item-based framing.

As these definitions make clear, both user-based and item-based framings suggest product matching between items i and i′ as the basis for the recommendation. User-based framing matches products by consumers; item-based framing suggests that products are matched on their attributes. Notably, user-based framing also suggests taste matching (users’ shared taste in the focal product) as the basis for recommendation, such that it offers informational value beyond that provided by item-based framing. According to advice-taking research, consumers extract information from others’ tastes to predict their own satisfaction with unfamiliar products (Morvinski, Amir, and Muller 2017; Yaniv, Choshen-Hillel, and Milyavsky 2011) and tend to adopt others’ preferences if they believe those others’ tastes match their own (Hilmert, Kulik, and Christenfeld 2006; Naylor, Lamberton, and Norton 2011). Therefore, we reason that user-based framing offers additional information (i.e., about others’ tastes) that can reduce customers’ uncertainty about whether they will like or dislike the recommended item. By offering additional information about taste matching beyond product matching, user-based framing can serve as a sort of double-guarantee that customers will enjoy the recommended item and thus should be more effective in triggering click-throughs. Formally,

H₁ : User-based framing increases recommendation click-throughs relative to item-based framing.

This predicted advantage of user-based framing is premised on customers’ perception that the taste matching is successful. Taste matching provides valid information for customers to infer their liking of the recommended item only if they believe others’ preferences reflect their personal tastes. With automated recommendations, many factors could influence the extent to which customers perceive taste matching as successful and potentially reduce or even reverse the framing effect, such that user-based framing actually becomes disadvantageous compared with item-based framing. We consider three such factors that might provide important boundary conditions to the framing effect. We purposefully select a range of factors related to the customer segment (i.e., more or less consumption experience), the products (i.e., more or less attractive focal products), and other users (i.e., more or less similar to the recommendation recipient).

Consumption Experience, Focal Attractiveness, and Dissimilarity Cues

User-based framing differs from item-based framing in the implication that the recommender system attempts to match users on the basis of their tastes in the focal product. Customers who have accumulated more experience in a consumption domain may be less likely to perceive this taste matching as successful, for two reasons. First, customers develop more refined and sophisticated tastes as they acquire more experience within a consumption category (Bettman, Luce, and Payne 1998). With more experience, customers are better able to differentiate products and develop a more complex understanding of the category (Alba and Hutchinson 2002). Second, more experienced customers have accrued more observations of individual differences in tastes and therefore likely regard their own taste as idiosyncratic (Packard and Berger 2016). Accordingly, they might deem a shared interest in a single or a limited set of products (i.e., focal products) as insufficient for taste matching, leaving them reluctant to converge with or rely on other users’ preferences. In contrast, inexperienced customers whose tastes are still coarse (Hoeffler and Ariely 1999) may be less skeptical of a match between their own and others’ tastes (Becker 1991), leading to the advantage of user-based over item-based framing. We predict,

H₂ : The advantage of user-based framing relative to item-based framing decreases for customers with more consumption experience in the focal domain.

Customers’ perceptions of taste-matching success also likely depend on the products themselves. We propose that taste matching may appear less accurate if the focal product is less attractive, because customers constantly learn about their own preferences through their reactions to different products (Ariely and Hoeffler 1999; West, Brown, and Hoch 2002). More attractive focal products would serve as salient and diagnostic signals of personal preferences (Zunick, Teeny, and Fazio 2017), which in turn should promote perceived success in taste matching with other users who presumably also like the attractive focal product. In contrast, people tend to view less attractive products as less indicative of their taste or even a negative signal of preferences, lowering the perceived accuracy of taste matching and resulting in a smaller advantage or even a disadvantage of user-based framing relative to item-based framing. Specifically,

H₃ : The advantage of user-based framing over item-based framing diminishes for unattractive focal products.

Finally, in ambiguous situations, in which the identities of other customers are not revealed, people tend to assume self–other similarity (Naylor, Lamberton, and West 2012). However, some companies provide information about the users who are the basis for the recommendation, explicitly (e.g., location of other users on booking.com) or implicitly (e.g., books of “teens’ choice” on Amazon). When this information points to a dissimilarity between users, it may undermine the value of taste matching. As existing research shows, dissimilarity on certain dimensions (e.g., gender) activates thoughts of self–other dissimilarity in other domains (e.g., product attitudes; Tuk et al. 2019). Customers thus might categorize a recommendation as reflecting “nonself” tastes if it is associated with dissimilar others and deem taste-matching efforts unsuccessful. In this case, we no longer expect an advantage of user-based framing over item-based framing but rather predict that it becomes disadvantageous, because consumers tend to avoid dissimilar others’ tastes (Berger and Heath 2008). Formally:

H₄ : User-based framing decreases recommendation click-throughs relative to item-based framing in the presence of cues suggesting self–other dissimilarity.

To summarize, we posit that, compared with item-based framing, user-based framing provides additional information about the preferences of other users that customers can use to reduce their uncertainty about the recommendation; as a result, it encourages them to click on it. The informational value of user-based framing and whether it benefits or harms recommendation click-throughs depends on the perceived success of taste matching. We conducted six studies to test these predictions and our conceptual framework (see the Web Appendix for the full results of all the studies). Studies 1a and 1b test H₁ (main effect) in field experiments with article recommendations. The results affirm the advantage of user-based framing over item-based framing in a managerially relevant setting. Study 2 tests H₂ that consumption experience functions as a moderator, in a setting that provides painting recommendations. For Studies 3 and 4, we created book-shopping scenarios to test H₃. We find consistent support for our hypotheses, whether the attractiveness of the focal product is rated by a separate batch of customers (Study 3; analogous to data gathered by companies from prior customers) or by the same customers (Study 4). In Study 4, we also leverage information about the ages of other customers to establish a dissimilarity cue that leaves user-based framing disadvantageous relative to item-based framing, as predicted in H₄. Study 5 strengthens the support for H₄ by using gender as a different cue of dissimilarity. In all these studies, the recommendations involve products with which customers are unfamiliar, a design element that establishes insights into how to market novel products. Because customers are unlikely to hold prior beliefs about these products, managerial strategies likely make a big difference (Cooke et al. 2002). Our findings thus add unique theoretical insights and suggest managerial strategies for companies.

Studies 1a and 1b

We conducted Studies 1a and 1b in collaboration with a media company that regularly pushes articles to its subscribers on WeChat, the top mobile app in China (Novet 2017). These two field studies differ primarily in the item-based framing, which we varied to ensure that the user-based framing is responsible for the increased click-throughs. This company also offers an ideal context to test the predicted main effect (H₁) for two reasons. First, it primarily publishes articles about social science research, and its subscribers represent a highly homogeneous community. In this context, readers likely view taste matching as successful in general. Second, this company had not used recommendations before we ran Study 1a, so we could observe the unique effects of framing, unaffected by prior practice.

Study 1b, conducted 14 months after Study 1a, then offers a conceptual replication with completely new stimuli. During the 14-month interval, the company did not adopt any other article recommendation and witnessed a 52% increase in the number of subscribers (from 70,488 to 107,338) on WeChat. These changes should minimize carry-over effects from Study 1a to Study 1b. Because we had no access to individual users’ data, we conducted both experiments at the article level (for a similar design, see Gong et al. 2017).

Study 1a

Article selection

Before the experiment started, we carefully selected 71 original articles that had been previously pushed to all subscribers, according to four criteria. First, the number of times people had read each article could not exceed 400, which is low compared with the overall average 3,071 (as of August 2017, immediately before Study 1a). Second, the article had been pushed to subscribers at least three months ago, to ensure that it was likely to be unfamiliar to most readers. Third, it reported on research on human beings, which is the main content the company disseminates, to avoid the risk that the article topic would seem odd to readers. Fourth, the article could not contain time- or event-specific content (e.g., “Top research of 2016”), because timeliness might interfere with the framing effects.

Study design

We assigned these preselected articles to three conditions: no recommendation (N = 9), user-based framing (N = 31), or item-based framing (N = 31). The assignment used stratified randomization; each condition includes approximately the same percentage of articles published in different years (12% published in 2014, 20% in 2015, 48% in 2016, and 20% in 2017). This approach helped exclude bias due to publication timing. With the control condition (no recommendation), we test whether a recommendation per se is effective, regardless of its specific framing. We limit the sample size for this control condition, because it is not our focal interest and to maximize the statistical power of the contrast between user-based and item-based framings. The recommended articles, with user-based or item-based framing, attracted more reads than nonrecommended articles (p < .001; see Figure W1 in the Web Appendix). We do not discuss the nonrecommended articles further.

Procedure

We randomly paired one article in the user-based framing with another in the item-based framing. The 31 pairs of recommendations then were distributed randomly across 31 days. Every weekday, the company pushed one set of articles to all subscribers. Each set had a headline article that was most salient to readers, which served as the focal article. Each pair of recommended articles was inserted toward the end of the focal article. Therefore, the readers would only see the recommendations if they were really interested in the focal article and finished reading it. The recommendation consisted of the recommendation framing and the title of the recommended article (a hyperlink to click on and read). The user-based framing read, “People who like this article also like…,” and the item-based framing specified the category that both the focal and the recommended articles fell in “More analyses of scientific research” (all focal articles were in this category). The order in which the two framings appeared (one preceded the other) was counterbalanced across days.

To measure click-throughs on the recommendations, we calculated the click-through rate (CTR) for each recommended article:

CTR = \frac{CurrentRead - InitialRead}{Focal Read} \times 100.

InitialRead is the number of reads of the recommended article before the experiment started. It does not differ by the framing condition (p = .543). CurrentRead is the number of reads after the experiment started, recorded at four time points of 24 hours, 48 hours, 72 hours, and two weeks after the recommendation, which enables us to determine whether the framing effect varies over time. The number of reads of the focal article (FocalRead) also was recorded at these four time points. Table 1 presents the descriptive statistics.

Table 1.

Means of Number of Article Reads in Study 1a.

	Focal Articles	User- Based Framing	Item- Based Framing	Non-recommended
Before experiment	0 (0)	316 (93)	294 (93)	274 (103)
24 hours	2,595 (2,280)	343 (125)	306 (96)	—
48 hours	2,610 (2,404)	344 (126)	307 (97)	—
72 hours	2,793 (2,564)	345 (126)	308 (97)	—
2 weeks	3,071 (2,693)	349 (132)	310 (98)	275 (103)

Notes: Standard deviations are in parentheses.

Results

There were 17 missing cases because we could not observe the reads of some articles at some time points. Furthermore, we excluded one outlier article in the user-based condition from the analysis, because its CTR (M = 19.25% across the time points) was disproportionately higher than the average of all the other articles (.61%). The final data set contains 228 observations: 112 in the user-based condition and 116 in the item-based condition. Due to the nested structure of the data (articles nested within days), we constructed a multilevel model with CTR as the outcome variable and random intercepts at the day level. The recommendation faming served as the predictor (0 = item-based, 1 = user-based). Because time did not moderate the framing effect (p = .919), we focus on the overall effect. Table W1 in the Web Appendix summarizes the regression results. Consistent with H₁, CTR is significantly higher in the user-based condition than in the item-based condition (M = .72% vs. M = .51%; b = .22, SE = .06, t(196) = 3.79, p < .001). Including the outlier article added to the error of estimation but also magnified the framing effect (M = 1.26% vs. M = .56%; b = .70, SE = .22, t(199) = 3.11, p = .002).

Follow-up survey

These results provide initial evidence that user-based framing outperforms item-based framing. Recall that we propose this effect arises because, unlike item-based framing, user-based framing offers additional informational value by suggesting taste matching as part of the recommendation strategy. To determine whether readers interpret the two framings in this way, we distributed a follow-up survey to the subscribers (N = 780; 67% female; M_age = 24.4 years, SD_age = 5.7). Note that we do not know whether the survey participants also participated in our experiment, because the experiment was conducted on the article level. The survey participants were randomly assigned to read the user-based framing (N = 409) or item-based framing (N = 371) that we used in the field experiment; then, they rated the extent to which they agreed with eight statements (1 = “strongly disagree,” and 6 = “strongly agree”). Half of the statements referred to product matching as the basis for the recommendation (e.g., “The recommendation is based on articles that are similar to what I have read,” “The recommendation is based on the categorization of articles”; Cronbach’s α = .68), whereas the other half referred to taste matching (e.g., “The recommendation is based on readers who have similar preferences with me,” “The recommendation is based on the categorization of readers”; Cronbach’s α = .70).

To test whether both user-based and item-based framings imply product matching to customers but only user-based framing suggests taste matching as a recommendation strategy, we submitted the perceived product-matching and perceived taste-matching scores to a 2 (two dependent measurements) × 2 (recommendations framing: user-based vs. item-based) mixed analysis of variance. A main effect of the measurement arises; participants more readily recognize product matching than taste matching as the basis for recommendations (F(1, 778) = 226.04, p < .001), suggesting that product matching is the default perceived recommendation strategy. In addition, we find a significant interaction between measurement and framing (F(1, 778) = 9.10, p = .003). In support of our reasoning, participants recognize product matching as the basis for the recommendation equally in both user-based and item-based conditions (M_user = 4.83, M_item = 4.82; t(778) = −.08, p = .941). However, participants in the user-based framing condition agree that taste matching is a basis for the recommendation to a greater extent than participants in the item-based framing condition (M_user = 4.38, M_item = 4.18, t(778) = 3.44, p = .001). That is, user-based framing (vs. item-based framing) offers information about taste matching, in addition to product-matching information.

Discussion

Consistent with H₁, Study 1a demonstrates that framing recommendations as user-based rather than item-based attracts more click-throughs in a field setting. It also provides support for the notion that perceived taste matching differentiates user- from item-based framing. It remains unclear, however, whether the framing effect really is due to the additional informational value of user-based framing or if readers instead avoid reading more articles in the same category, a response potentially evoked by the item-based framing that read “More analyses of scientific research.” In Study 1b, we thus use a different item-based framing operationalization but keep the user-based framing constant. If the framing effect in Study 1a is due to the informational value of user-based framing, it should emerge regardless of whether the item-based framing specifies the article category.

Study 1b

Article selection and study procedure

Study 1b contains a new set of articles and a more generic item-based framing (“Similar to this article”). We selected the recommended articles using criteria similar to those we applied in Study 1a, except we also required that they had not been recommended in Study 1a. We increased the constraint on the number of reads before recommendation, from 400 to 480 reads, to ensure a decent sample size and account for the substantial increase in the number of subscribers to the company. With these criteria, we identified 66 articles, half randomly assigned to the user-based and the other half to the item-based framing condition. The procedure is the same as in Study 1a, and the experiment lasted for 33 days.

Results

Similar to Study 1a, we excluded an outlier article in the item-based condition that had a peculiarly high CTR (M = 24.35%) relative to the average of all the other articles (M = .95%). Thus we retain 258 observations in the final data set. Unlike Study 1a, we did not balance the year of publication across conditions; more articles published in 2018 were assigned to the user-based condition than to the item-based condition (49% vs. 37%; p = .073). Therefore, we control for publication bias (0 = published before 2018, 1 = published in 2018) in the analysis. Using the same multilevel modeling approach as in Study 1a (time did not moderate the framing effect; p = .945), we find a lower CTR for articles published in 2018 than for those published before 2018 (M = .58% vs. M = 1.06%; b = −.49, SE = .13; t(223) = −3.80, p < .001). More importantly, controlling for the publication year, CTR is higher in the user-based condition than in the item-based condition (M = 1.25% vs. M = 1.06%; b = .19, SE = .08; t(223) = 2.44, p = .015). Table W1 in the Web Appendix summarizes the regression results. The advantage of user-based framing persists but shrinks in magnitude without the covariate (M = 1.01% vs. M = .88%; b = .14, SE = .08; t(224) = 1.79, p = .075). Including the outlier article made the framing effect insignificant (b = −.32, SE = .28; t(227) = −1.12, p = .262).

Discussion

Study 1b strengthens the support for H₁ by showing that the advantage of user-based framing over item-based framing persists when the item-based framing does not specify the category of the articles. Taken together, the framing effects observed in Studies 1a and 1b cannot be accounted for by avoidance of same-category items. We next seek to provide evidence for our conceptualization by testing boundary conditions on the framing effect.

Study 2

With Study 2 we examine our prediction that user-based framing outperforms item-based framing in terms of recommendation CTR for inexperienced customers but not for experienced customers within a consumption domain (H₂). We displayed the article recommendations in Studies 1a and 1b at the end of the focal article, which guaranteed that readers liked the focal article, because they would only see the recommendation if they read it to the end. In line with this element, in Study 2 we provide product recommendations only to participants who indicated that they liked the focal product.

Participants and Design

We recruited 403 participants located in the United States from Amazon Mechanical Turk (MTurk; 186 female participants; M_age = 37.3 years, SD_age = 11.6). Data from MTurk offer reliability comparable to those gathered from offline laboratories (Horton, Rand, and Zeckhauser 2011; Paolacci, Chandler, and Ipeirotis 2010). After giving their informed consent, participants entered an “Online Museum” study and viewed 50 paintings created between the seventeenth and twentieth centuries. Each painting was paired with a hidden recommended painting with the same theme (e.g., seascape). The recommendations feature either a user-based (N = 195) or item-based (N = 208) framing. All paintings were obtained from Google Art Project. We operationalized our proposed moderator, consumption experience, as the frequency of visiting art museums, measured on a continuous scale.

Procedure

Participants viewed the 50 focal paintings in a random sequence. Next to each focal painting, there was a “like” button in the shape of a heart. We told participants to mark their favorite paintings by pressing the button. To ensure that they provided their honest opinions, we told them that they would enter into a lottery for postcards of their favorite paintings. After participants clicked on a “like” button, another button appeared, indicating either “People who like this painting also like…” (user-based framing) or “Similar painting to this” (item-based framing), depending on the randomly assigned condition. They could click on this button to view the recommended painting in a pop-up window, as well as exit the pop-up window any time to continue viewing the focal paintings. The CTR for each recommendation was calculated as

CTR = \frac{N who viewed the recommended}{N who liked the focal} \times 100.

The denominator is the number of participants who liked the focal painting, and the numerator indicates how many of them chose to view the recommended painting. The CTR varied between 0% and 100%. After participants finished viewing all the focal paintings, they saw both the user-based and item-based framing and indicated which one they encountered in the “Online Museum” (94% answered correctly). To measure participants’ consumption experience, we asked them to indicate how often they visited art museums in their life (1 = “never,” 2 = “seldom,” 3 = “sometimes,” 4 = “often,” and 5 = “very often”). We deliberately chose this single-item measurement to maximize the number of observations per level of consumption experience and thus to obtain reliable CTRs to estimate the effects of framing. We ended the study with a few demographic questions.

Results

Similar to Study 1, we conducted the analyses at the level of each recommended painting. We calculated the CTR for each recommendation per framing and per level of consumption experience, resulting in a data set with 499 observations. One observation was missing because one focal painting in the item-based condition was not liked by any participants who never went to art museums. We regressed CTR on framing (0 = item-based, 1 = user-based), consumption experience (continuous from 1 to 5), and their interaction. The results, as plotted in Figure 2, indicate a significant interaction between framing and consumption experience (b = −2.23, SE = .97; t(495) = −2.30, p = .022). In support of H₂, the CTR is higher in the user-based condition than in the item-based condition among people who never (b = 7.83, SE = 2.38; t(495) = 3.29, p = .001), seldom (b = 5.60, SE = 1.68; t(495) = 3.33, p < .001), or sometimes (b = 3.36, SE = 1.37; t(495) = 2.45, p = .015) visited art museums. We find no significant difference across framings for those who often (b = 1.13, SE = 1.68; t(495) = .67, p = .501) or very often (b = −1.10, SE = 2.37; t(495) = −.47, p = .642) visited art museums. On average, the CTR is slightly but significantly higher in the user-based condition than in the item-based condition (M = 46.63% vs. 43.23%; F(1, 495) = 6.15, p = .014), probably because most participants have rather limited experience with arts (median = 2 of 5). Moreover, the CTR decreases with more consumption experience (b = −6.70, SE = .68; t(495) = −9.78, p < .001) in the user-based condition, but this trend is attenuated in the item-based condition (b = −4.46, SE = .69; t(495) = −6.49, p < .001). Table W2 in the Web Appendix summarizes the regression results.

Figure 2.

Regression results of Study 2.

Discussion

Study 2 provides support for H₂; the advantage of user-based framing over item-based framing diminishes for people with more consumption experience. Also consistent with our theorizing that more experienced people are less likely to perceive taste matching as accurate, we find that greater consumption experience induces a greater decrease in the recommendation CTR when it is framed as user-based as opposed to item-based.

The paradigms we use in Studies 1a, 1b, and 2 guarantee that participants like the focal product; they see the recommendation only if they finish reading the article or like the focal painting. In Study 3, we relax this criterion so that all participants receive a product recommendation regardless of whether they expressed interest in the focal product; this allows us to test for a moderating role of the attractiveness of the focal product (H₃).

Study 3

According to our theorizing, liking the focal product is a necessary prerequisite for taste matching to be perceived as successful and thus for the advantage of user-based framing over item-based framing to arise. To test this assumption explicitly, we vary the attractiveness of the focal products and inquire into people’s intentions to click on the recommendation. We expect the advantage of user-based framing to diminish or even reverse for less attractive focal products (H₃). Moreover, if focal attractiveness affects perceived success in taste matching, it should relate more positively to CTR when the recommendations are framed as user-based as opposed to item-based.

Participants and Study Design

Fifty participants located in the United States were recruited from MTurk to participate in a study about shopping for novels on Amazon (18 female participants; M_age = 37.57 years, SD_age = 12.32). The majority (56%) had never purchased a novel on Amazon. We manipulated the framing within participants. Unlike the previous studies, the user-based framing emphasized users’ actions (“Customers who viewed this book also viewed…”) rather than likes. This variation is purposeful; the decision to view a book’s webpage might be driven merely by the appearance of the cover, but liking a book requires understanding it. The item-based framing followed Study 2 (“Similar to this item”). This setup might be less engaging than previous studies. However, it taps into an important situation in which customers are merely browsing products without concrete goals.

Book Selection

We took a convenience sample of 50 novels from the “Literature and Fiction” category on Amazon that had garnered fewer than 200 reviews before the experiment started, such that they were presumably unfamiliar to most of our participants. A pretest with a separate batch of 50 participants from MTurk (23 female participants; M_age = 33.6 years, SD_age = 8.4) confirms that these books are unfamiliar to MTurk workers (maximum mean familiarity is 2.14 of 10, where higher values indicate more familiarity). From the 50 books, we randomly selected 25 candidates as focal books and the other 25 candidates as recommended books. Then we randomly paired a candidate from the focal set with another from the recommended set. We aimed for an equal number of focal–recommended pairs per framing condition for the study.

The pretest demonstrates that the distribution of mean attractiveness scores across the 25 focal books centered around the scale midpoint (M = 5.21 of 10, SD = .64). We selected six focal books (three per framing condition) that represent this distribution for extrapolation (M = 5.04, SD = .64). The attractiveness scores of the selected books do not differ by framing condition (p = .941).

Procedure

In the main study, participants viewed the preselected focal books in random sequences, each accompanied by a preassigned recommended book. For each recommendation, participants indicated whether they would click on the recommended book, on a ten-point scale (1 = “Definitely not,” and 10 = “Definitely yes”). After they finished viewing all the recommendations, they selected the reasons that they had seen for recommendation: (1) user-based framing, (2) item-based framing, (3) both, or (4) neither. The study ended with demographic questions.

Results

We excluded eight participants who recalled neither the user-based nor the item-based framing. This exclusion is important, because it rules out the possibility that the framing effect shrinks due to participants’ lack of attention to the recommendations associated with less attractive focal books. The final data set includes 252 observations (6 books nested within 42 participants). We regressed participants’ intention to click on the recommended book on three predictors: recommendation framing, the score of focal attractiveness as obtained from the pretest, and their interaction. The regression model allowed for a random intercept for each participant. Table W2 in the web appendix summarizes the regression results.

We find a significant interaction effect between framing and focal attractiveness (b = 1.10, SE = .53; t(207) = 2.08, p = .039). Consistent with H₃, the advantage of user-based framing decreases for less attractive focal books. To illustrate, when focal attractiveness is one standard deviation above the mean, user-based framing increases people’s intention to click on the recommendation relative to item-based framing (b = .87, SE = .42; t(207) = 2.07, p = .039). No framing effect emerges at the mean level of focal book attractiveness (b = .22, SE = .29; t(207) = .77, p = .441) or at one standard deviation below the mean (b = −.43, SE = .43; t(207) = −1.00, p = .317). For very unattractive books (1 out of 10), the model even predicts that user-based framing lowers click-through intentions compared with item-based framing (b = −4.22, SE = 2.16; t(207) = −1.96, p = .052). Furthermore, in support of our theorizing, focal book attractiveness predicts intentions to click for the user-based framing (b = 1.09, SE = .44; t(207) = 2.48, p = .014), but this trend is absent for item-based framing (b = −.01, SE = .29; t(207) = −.02, p = .981).

When all cases are included, the moderation by focal attractiveness is in the same direction and marginally significant (t(247) = 1.76, p = .080). For the similar patterns with and without data exclusion, see Figure W2 in the Web Appendix.

Discussion

In support of H₃, Study 3 establishes focal product attractiveness as a boundary condition for the advantage of user-based framing over item-based framing. It renders insights into the framing effect in a setting where customers are merely browsing products without explicit signals of their interest in the focal product. In Study 4, we aim to replicate this finding using a different procedure; we also test whether presenting a salient cue of self–other dissimilarity makes user-based framing disadvantageous relative to item-based framing (H₄).

Study 4

Cue of Self–Other Dissimilarity

The majority of MTurk workers are at least 25 years of age (Ipeirotis 2010), so we use the age group “18–24 years” as a dissimilarity cue. For this study, a bar graph indicates other customers’ ages, under the title “Age of interested customers,” with three bars: “18–24,” “25–55,” and “above 55.” We highlighted the “18–24” bar and informed participants that it represented the age of customers who also viewed the recommended book. A pretest (N = 101; 62 female participants; M_age = 36.7 years, SD_age = 12.4) confirmed that most MTurk workers (89%) are older than 24 years and perceive themselves as more similar to other customers in their age group than to people in the 18–24 year group (p < .001).

Participants and Study Design

We recruited 360 participants from MTurk, who are at least 25 years old (169 female participants; M_age = 37.6 years, SD_age = 1.19), and randomly assigned them to three conditions: user-based framing (“Customers who viewed this item also viewed…”), item-based framing (“Similar to this item”), and user-based framing with the age group dissimilarity cue.

Procedure

The procedure is similar to that in Study 3, with two differences. First, instead of presenting participants with preselected books, we allowed them to self-select three focal books to view from nine books, thereby simulating browsing behavior in online stores. Second, the attractiveness of focal books was rated by the participants rather than based on the score from the pretest, which captures the heterogeneity of ratings across individuals. At the end of the study, participants evaluated how attractive they found each focal book using a ten-point scale (1 = “Not at all,” and 10 = “Very attractive”; M = 6.94, SD = 1.96). The attractiveness was not influenced by the assigned conditions (p = .353; overall M = 6.94, SD = 1.96).

Results

As in Study 3, we excluded participants (N = 133) who could not recall the framing they saw, leaving a data set with 680 observations (3 books nested within 227 participants). We took the same analysis approach as in Study 3, with two dummy predictors: user-based condition and dissimilarity cue condition, each of which could interact with the rating of the focal book’s attractiveness. Because the dissimilarity (age group) cue did not interact with focal attractiveness (p = .845) and including this interaction term did not increase model fit (p = .364), we dropped it from the analysis to focus on the main effect of dissimilarity. Figure 3 plots the results.

Figure 3.

Regression results of Study 4.

In line with Study 3 results, we find a significant interaction of focal book attractiveness and recommendation framing when the dissimilarity cue is absent (b = .24, SE = .10; t(451) = 2.32, p = .021). Specifically, user-based framing (vs. item-based framing) increased participants’ intention to click on the recommended book when focal attractiveness scored one standard deviation above the mean (b = .70, SE = .36; t(224) = 1.93, p = .055) but not when it scored at the mean (b = .24, SE = .30; t(224) = .78, p = .434) or one standard deviation below the mean (b = −.23, SE = .36; t(224) = −.64, p = .526). For very unattractive books (1 out of 10), user-based framing even lowered click-through intentions relative to item-based framing (b = −1.18, SE = .68; t(224) = −1.74, p = .084). In addition, focal attractiveness relates more positively to click-through intentions in the user-based condition (b = .50, SE = .08; t(451) = 6.22, p < .001) than in the item-based condition (b = .26, SE = .06; t(451) = 4.01, p < .001). However, when the dissimilar cue is present, user-based framing (vs. item-based framing) decreases intentions to click on recommended books (b = −.89, SE = .32; t(224) = −2.84, p = .005).

When all cases are included, we replicate the reversal of the framing effect (t(357) = −3.11, p = .002). The moderation by focal attractiveness is in the same direction but not significant (t(717) = 1.58, p = .113). For the similar patterns with and without data exclusion, see Figure W3 in the Web Appendix.

Discussion

Study 4 replicates the findings of Study 3 with a different procedure, strengthening the support for H₂. Furthermore, consistent with H₃, we find that the presence of a cue suggesting dissimilarity with other users makes user-based framing disadvantageous compared with item-based framing, regardless of the attractiveness of the focal books. To provide additional support for H₄ and in line with prior research (Naylor, Lamberton, and West 2012), in Study 5 we used gender composition as a different cue of self–other dissimilarity. Moreover, we include both dissimilar (most other users are a different gender) and similar (most other users are the same gender) cue. In line with our theorizing and prior research (Naylor, Lamberton, and West 2010), we anticipate that cueing customers with their similarity to other users will have an effect similar to user-based framing that lacks information about the identity of other users.

Study 5

Study Design

Study 5 follows the design of Study 2 (painting) but in the domain of books. We selected 57 books of various genres (e.g., comics, thrillers, philosophy) that were not available on the market when the study was conducted (i.e., “coming soon” category), so participants were unlikely to be familiar with them. We selected another 57 coming-soon books as recommendations and paired them with the focal books. Participants viewed the book covers, titles, author names, and genres and then marked books they would like to read by clicking on a heart button. Next, the recommendation button popped up, indicating either “Customers who like this also like…” in the user-based condition or “Similar book to this” in the item-based condition. In both conditions, we told participants that the recommendation came from readers on Amazon. Moreover, participants had the chance to win a book that they marked as “would like to read.”

In the user-based condition, next to the recommendation button, participants also saw the gender composition of people who liked the focal book. Of the 57 focal books, 21 were predominantly liked by male participants (95%–100%), and 21 were mainly liked by female participants (95%–100%), so 42 books offered a cue of self–other similarity, and 42 provided a cue of self–other dissimilarity (see Table 2). In addition, 15 neutral books were liked about equally by participants of both genders (45%–55% male). These neutral books serve two purposes. First, their presence creates a more realistic book-shopping scenario, in which customers encounter books that attract either gender and those that appeal to both genders. Second, the neutral books, combined with similar-cue and dissimilar-cue books, increase the power of the contrasts relative to the item-based condition (i.e., same 57 books compared across conditions). For the similar-cue books, we expect to replicate the moderating role of consumption experience from Study 2. For the dissimilar-cue books, in line with Study 4, we anticipate that user-based framing will decrease CTR.

Table 2.

Design of Study 5, User-Based Framing.

	Similar Cue	Dissimilar Cue
Male participants	21 books liked by 95% to 100% men	21 books liked by 95% to 100% women
Female participants	21 books liked by 95% to 100% women	21 books liked by 95% to 100% men
Total	57 books (42 + 15 neutral books)	57 books (42 + 15 neutral books)

Participants and Procedure

Three hundred sixteen MTurk workers participated in the study (159 male participants; M_age = 35.51 years, SD_age = 1.61). After viewing the focal books, participants indicated their experience with book shopping on the item, “How often do you visit bookstores (online or offline) in general?” with the same scale from Study 2. Compared with participants in Study 2 (paintings), participants in Study 5 had more experience with books (median = 3 vs. 2; significantly higher mean, p < .001). The study ended with a few demographic questions.

Results

We calculated separate CTRs for similar-cue books, dissimilar-cue books, and the item-based frame books. We then regressed the CTRs on two dummy predictors: similar-cue books and dissimilar-cue books, each of which could interact with consumption experience. Similar to Study 4, the interaction between the dissimilar cue and consumption experience is insignificant (p = .975), and including the interaction term does not increase model fit (p = .975). We thus drop the interaction (for the full regression results, see Table W3 in the Web Appendix). The results, as plotted in Figure 4, show that for similar-cue books, as in Study 2, there is a significant interaction between framing and experience (b = −4.88, SE = 1.63; t(623) = −3.00, p = .003). Specifically, user-based framing is more advantageous for participants who never visit bookstores (b = 8.91, SE = 4.35; t(623) = 2.05, p = .041), but this advantage decreases and even reverses as they gain more experience (seldom b = 4.03, SE = 2.98; t(623) = 1.35, p = .176; sometimes b = −.85, SE = 2.03; t(623) = −.42, p = .676; often b = −5.73, SE = 2.15; t(623) = −2.66, p = .008; very often b = −1.61, SE = 3.24; t(623) = −3.28, p = .001). In support of H₄, user-based framing becomes disadvantageous, relative to item-based framing, for dissimilar-cue books (b = −6.55, SE = 1.94; t(623) = −3.38, p < .001).

Figure 4.

Regression results of Study 5.

Discussion

Using the paradigm from Study 2, Study 5 conceptually strengthens support for H₄. Cueing customers to recognize self–other dissimilarity leads to a disadvantage of user-based framing relative to item-based framing. This study also generalizes the role of consumption experience to the domain of books.

General Discussion

Customers frequently receive product recommendations from recommender systems, and companies often frame them as user-based (e.g., “People who like this also like…”) or item-based (e.g., “Similar to this item”). We compare these two framings while keeping the actual recommendation constant (or randomized, as in the field studies) and thereby demonstrate the advantage of user-based framing over item-based framing in terms of recommendation CTR. In two field experiments with the mobile app WeChat (Study 1a and 1b), we establish that recommending articles with user-based (vs. item-based) framing increases recommendation CTR. Study 2 identifies consumption experience as an important boundary condition for the framing effect; Studies 3 and 4 show that the effect shrinks and even reverses for unattractive focal products. Finally, Studies 4 and 5 reveal that cueing customers to their dissimilarity with other users makes user-based framing less effective than item-based framing. Table 3 summarizes the studies and the hypotheses they support. We took care to test our predictions using various product categories (articles, books, and paintings) and different paradigms mimicking real recommendation practices to establish the generalizability and robustness of the effects. The results in turn offer several contributions to literature, practical suggestions for companies that use product recommendations in their marketing strategy, and directions for further research.

Table 3.

Overview of Studies.

Study	Data Source	Outcome Variable	Supported Hypotheses
Studies 1a and 1b	Field experiment	CTR of recommended books (0%–100%)	H₁
Study 2	Behavioral experiment	CTR of recommended paintings (0%–100%)	H₂
Study 3	Scenario experiment	Intention to click on recommended books (1–10)	H₃
Study 4	Scenario experiment	Intention to click on recommended books (1–10)	H₃ and H₄
Study 5	Behavioral experiment	CTR of recommended books (0%–100%)	H₂ and H₄

Theoretical Implications

Prior investigations of recommender systems have primarily focused on technical designs (e.g., Ansari, Essegaier, and Kohli 2000; Ariely, Lynch, and Aparicio 2004; Hennig-Thurau, Marchand, and Marx, 2012) or the consequences of their use (e.g., Bodapati 2008; Fleder and Hosanagar 2009; Pathak et al. 2010). Little research has explored the ideal ways for companies to communicate the basis of recommendations to their customers. Our research represents an initial attempt to fill this gap by comparing the effects of user-based and item-based framings on recommendation CTR. Simply changing the framing of recommendations can have an impact on this metric. We thus emphasize the importance of studying the effect of framing in addition to the technical aspects of the underlying algorithms.

Our findings also advance understanding of customers’ interpretations of recommendations. As our follow-up survey in Study 1a shows, customers recognize product matching more readily than taste matching, regardless of the recommendation framing. In two pilot studies (see the Web Appendix), we also find that product matching is perceived as a more dominant recommendation strategy than taste matching. This primacy of product matching might result from the visual salience of products, relative to the latency of customers: on a typical product webpage, customers see products, not other customers, and can directly compare the products but not themselves with others. The results of our survey show that the difference between the two framings is due to taste matching. By signaling that taste matching is part of the recommendation strategy, beyond product matching, user-based framing offers additional informational value for customers that, presumably, mitigates their uncertainty about their satisfaction with the recommendation.

More broadly, our work contributes to advice-taking research (Aral and Walker 2012; Iyengar, Van den Bulte, and Lee 2015; Müller-Trede et al. 2018). Prior studies have focused on how customers take advice from other users; we investigate customers’ tendency to follow recommendations generated by algorithms. Consistent with findings that indicate that customers adopt others’ choices (Morvinski, Amir, and Muller 2017) and opinions (e.g., online reviews; Chen and Xie 2008; Zhu and Zhang 2010), we demonstrate that mentioning others’ preferences can encourage customers to click on recommended products. However, a fundamental difference between following recommendations and adopting others’ preferences is that the former depends on customers’ understanding of the “black box” of recommender systems, whereas the latter pertains to how customers navigate the social world. Recommendations framed as user-based (vs. item-based) might exert more influence on customers by adding a social component to the recommender system.

Managerial Implications

Companies heavily invest in recommender systems; global spending is estimated at $5.9 billion in 2019 (International Data Corporation 2019). Our research suggests that it is not only the technical aspects of recommender systems that matter; the framing of recommendations exerts a notable influence as well. Companies might fail to maximize recommendation click-throughs if they rely only on item-based framing. Managers must not only develop effective recommender systems but also devote attention to how to frame the recommendations for customers. Adapting the framing, while keeping the underlying algorithm and the recommended product constant, comes with nearly zero cost, unlike developing and improving technical aspects of recommender systems.

Our field studies suggest a general advantage of user-based framing over item-based framing in a setting where customers’ tastes are homogeneous and they show deep interest in the focal item (e.g., they read the entire article). Studies 2 through 5 document situations in which this advantage can diminish or even reverse. These boundary conditions are particularly important for companies to consider when deciding on the framing that they want to utilize. First, customers with less consumption experience are particularly susceptible to the impact of recommendation framing. Managers can identify these customers by analyzing their past behavior and infer the degree to which they possess consumption experience in a specific domain. Customers who seldom listen to classical music probably know little about this genre, for example, so they likely follow the lead of other classical music fans and exhibit high responsiveness to user-based framing.

Second, in situations in which customers are merely browsing on a website and do not necessarily express interest in focal products (as was the case in the paradigms of Studies 3 and 4), utilizing a user-based framing is unlikely to be advantageous compared with an item-based framing. Conversely, a user-based framing is more advantageous than item-based framing for attractive products; it can trigger customers to click the recommendation when they already have expressed some interest in the focal product, such as by reading an article or watching a video to the end. Managers can infer the attractiveness of focal products by tracking customers’ real-time behavior and thereby decide whether to prioritize user-based framing. Moreover, considering that user-based framing appears particularly beneficial for products that receive high ratings from prior customers, if managers cannot easily infer a particular target customer’s attitude toward the focal product, they still can decide whether to prioritize user-based framing, depending on prior customers’ reactions to it.

Third, user-based framing is less effective than item-based framing when it is coupled with a cue suggesting that others (on whom the recommendation is based) are dissimilar to the recommendation recipient. This insight is critical for companies that present prior customers’ information to target customers (e.g., “teens’ choices”). If these selected others differ from the target customer in salient ways, the target customer might avoid a recommendation framed as user-based. To maximize the value of user-based framing, managers either should not display any cues suggestive of differences or else should selectively emphasize other customers who are similar to the target customer in some important aspect. If these displays of information cannot be adjusted, managers might compare the backgrounds of the target customer and others, then choose a user-based framing only if a match exists and item-based framing if not.

Fourth, customers more readily recognize product matching than taste matching (as shown in the follow-up survey for Study 1a). However, the advantage of user-based framing stem from customers’ awareness of the taste-matching effort and their recognition of successful taste matching. Therefore, it is important for companies to make user-based framings salient, such as by increasing the font size or underscoring the framing, if they intend to leverage its value to the fullest.

Caveats and Calls for Further Research

We purposefully compare generic user-based and item-based framings, which are common in the marketplace, to generate externally valid and practically relevant insights. However, both framings can vary in their specificity. For example, user-based framing can refer to a specific group of users, such as friends (e.g., Spotify’s “what friends are listening to”), which may alter how likely customers are to perceive taste matching as successful. A generic user-based framing is unlikely to prompt customers to question their similarity with ambiguous other users, but referring to specific friends could more easily trigger perceptions of dissimilarity. Typically, customers know their friends’ tastes and therefore recognize fine-grained differences in them. In that sense, referring to friends’ preferences might backfire for user-based framing, making it less effective than item-based framing. We encourage continued research into this practically relevant issue.

Similarly, companies might specify standards for item categorization. Instead of merely mentioning that the recommended item is similar to a focal item or that the two fall in a rather broad category (e.g., romantic novels), companies might emphasize books by the same author or movies by the same director. Noting the primacy of product matching as the perceived recommendation strategy, we speculate that the width of the category exerts little influence on the difference between user-based and item-based framing. However, it is possible that item categorization variations could affect certain customers; for example, those with greater consumption experience within a product category might find item-based framing more attractive if the item categorization is narrower, because they are motivated to deepen their knowledge of specific categories (Clarkson, Janiszewski, and Cinelli 2013).

Alternatively, recommendation framing might be analyzed along dimensions other than an emphasis on different inputs (i.e., users or items), such as whether it refers to the target customer’s own past behavior as a basis for recommendation. Spotify uses “Because you have listened to X” in parallel with a more generic “Similar to X” to explain its recommendations. Does explicitly referring to customers’ own tastes make a difference? On the one hand, personalized explanations (“you” and “your” behavior) might cause customers to perceive greater effort by the recommender system and the recommendation as more self-relevant. On the other hand, personalization could raise customers’ awareness that their private information has been collected and prompt reactance to the recommendations. Additional research could compare different recommendation framings along multiple dimensions to achieve a fuller understanding of their roles.

Although we explore three theoretically derived, practically relevant moderators, a variety of factors could shift the perceived success of taste matching and thus moderate the framing effect. According to social influence literature, for example, customers tend to perceive more self–other dissimilarity as their distance grows (Meyners et al. 2017). Their perceptions of taste-matching success thus might depend on their geographical distance. Another pertinent factor is customers’ perception of the size of the group of other users (Argo, Dahl, and Manchanda 2005), as defined by the type of product. Customers interested in a niche product may infer a small group of interested other users; those considering a mainstream product likely presume a large group. Larger groups can be more influential but also appear more heterogeneous in their tastes (Latane 1981). Studies of such influences could deepen understanding of framing effects across communities.

We suggest that user-based framing is advantageous compared with item-based framing because it signals taste matching. In our work, we provide support for this theorizing in product domains in which taste is an important decision criterion (articles, paintings, and books). We speculate that for products primarily differentiated by quality (e.g., utilitarian products such as laptops), customers’ reaction to recommendations could be less sensitive to their perception of taste matching; in such instances, the informational value of taste matching is likely to diminish. Future research could examine if the advantage of user-based framing relative to item-based framing depends on whether taste or quality is the more salient decision criterion for a particular product.

Importantly, the more customers are familiar with the digital world, the more experienced they are with recommender systems and might develop their own understanding of how these systems work. For instance, ethnographic work on recommendations shows that experienced customers tend to game with the recommender system to generate desired recommendations (Devendorf and Goodman 2014). This suggests that experienced customers might interact with recommender systems more rationally and deliberately choose to click or not to click on recommendations with the purpose of improving the quality of future recommendations. For instance, customers might resist a recommendation related to the opposite gender’s taste not only because they perceive a mismatch with their own taste, but also to avoid misidentification by the recommender system and to prevent any future recommendations associated with the other gender. The implication is that customers who are more experienced with recommender systems could be more likely to scrutinize taste matching efforts. We see this as a fruitful avenue for future research.

As a concluding remark, in a blog post, Netflix has acknowledged that it provides explanations for why it has recommended a movie or show to gain customers’ trust (Amatriain and Basilico 2012). Our research advances this notion by revealing that when companies explain a recommendation to their customers, the decision of which framing to use, user-based or item-based, is crucial in terms of its impact on recommendation click-throughs.

Supplemental Material

Supplemental Material, DS_10.1177_0022242919873901 - Making Recommendations More Effective Through Framings: Impacts of User- Versus Item-Based Framings on Recommendation Click-Throughs

Supplemental Material, DS_10.1177_0022242919873901 for Making Recommendations More Effective Through Framings: Impacts of User- Versus Item-Based Framings on Recommendation Click-Throughs by Phyliss Jia Gai and Anne-Kathrin Klesse in Journal of Marketing

Footnotes

Acknowledgments

The authors thank Jijitang, a Chinese research social network platform that kindly collaborated on the field experiment. They thank Jennifer Argo, Darren Dahl, Christoph Fuchs, Michael Haenlein, Ajay Kohli, Anne ter Braak, and Raji Srinivasan for insightful comments.

Associate Editor

Praveen Kopalle

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed the following financial support for the research, authorship, and/or publication of this article: Financial support from the Netherlands Organization for Scientific Research (VENI grant 451-15-023 awarded to Anne-Kathrin Klesse) is gratefully acknowledged.

Online supplement:

References

Alba

Joseph W.

Wesley Hutchinson

(2002), “Dimensions of Consumer Expertise,” Journal of Consumer Research, 13 (4), 411.

Amatriain

Xavier

Basilico

Justin

(2012), “Netflix Recommendations: Beyond the 5 Stars (Part 1),” The Netflix Tech Blog (April 6), https://medium.com/netflix-techblog/netflix-recommendations-beyond-the-5-stars-part-1-55838468f429.

Amatriain

Xavier

Basilico

Justin

(2016), “Past, Present, and Future of Recommender Systems: An Industry Perspective,” in Proceedings of the 10th ACM Conference on Recommender Systems. New York: Association for Computing Machinery, 211–14.

Ansari

Asim

Essegaier

Skander

Kohli

Rajeev

(2000), “Internet Recommendation Systems,” Journal of Marketing Research, 37 (3), 363–75.

Aral

Sinan

Walker

Dylan

(2012), “Identifying influential and suscepttible members of social networks,” Science, 337 (6092), 337–41.

Argo

Jennifer J.

Dahl

Darren W.

Manchanda

Rajesh V.

(2005), “The Influence of a Mere Social Presence in a Retail Context,” Journal of Consumer Research, 32 (2), 207–12.

Ariely

Dan

Hoeffler

Steve

(1999), “Constructing Stable Preferences: A Look Into Dimensions of Experience and Their Impact on Preference Stability,” Journal of Consumer Psychology, 8 (2), 113–39.

Ariely

Dan

Lynch

John G.

Aparicio

Manuel

IV (2004), “Learning by Collaborative and Individual-Based Recommendation Agents,” Journal of Consumer Psychology, 14 (1/2), 81–95.

Becker

Gary S.

(1991), “A Note on Restaurant Pricing and Other Examples of Social Influences on Price,” Journal of Political Economy, 99 (5), 1109–16.

10.

Berger

Jonah

Heath

Chip

(2008), “Who Drives Divergence? Identity Signaling, Outgroup Dissimilarity, and the Abandonment of Cultural Tastes,” Journal of Personality and Social Psychology, 95 (3), 593–607.

11.

Bettman

James R.

Luce

Mary Frances

Payne

John W.

(1998), “Constructive Consumer Choice Processes,” Journal of Consumer Research, 25 (3), 187–217.

12.

Bodapati

Anand V.

(2008), “Recommendation Systems with Purchase Data,” Journal of Marketing Research, 45 (1), 77–93.

13.

Chen

Yubo

Xie

Jinhong

(2008), “Online Consumer Review: Word-of-Mouth as a New Element of Marketing Communication Mix,” Management Science, 54 (3), 477–91.

14.

Clarkson

Joshua

Janiszewski

Cinelli

Melissa D.

(2013), “The Desire for Consumption Knowledge,” Journal of Consumer Research, 39 (6), 1313–29.

15.

Cooke

Alan D.J.

Sujan

Harish

Sujan

Mita

Weitz

Barton A.

(2002), “Marketing the Unfamiliar: The Role of Context and Item-Specific Information in Electronic Agent Recommendations,” Journal of Marketing Research, 39 (4), 488–97.

16.

Cramer

Henriette

Evers

Vanessa

Ramlal

Satyan

Someren

Maarten Van

Rutledge

Lloyd

Stash

Natalia

Aroyo

Lora

Wielinga

Bob

(2008), “The Effects of Transparency on Trust in and Acceptance of a Content-Based Art Recommender,” User Modeling and User-Adapted Interaction, 18 (5), 455–96.

17.

Devendorf

Laura

Goodman

Elizabeth

(2014), “The Algorithm Multiple, the Algorithm Material: Reconstructing Creative Practice,” presentation from Contours of Algorithmic Life, University of California, Davis (May 15).

18.

Fleder

Daniel

Hosanagar

Kartik

(2009), “Blockbuster Culture’s Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity,” Management Science, 55 (5), 697–712.

19.

Gomez-Uribe

Carlos A.

Hunt

Neil

(2015), “The Netflix Recommender System,” ACM Transactions on Management Information Systems, 6 (4), 1–19.

20.

Gong

Shiyang

Zhang

Juanjuan

Zhao

Ping

Jiang

Xuping

(2017), “Tweeting as a Marketing Tool: A Field Experiment in the TV Industry,” Journal of Marketing Research, 54 (6), 833–35.

21.

Gupta

Sunil

Hanssens

Dominique

Hardie

Bruce

Kahn

William

Kumar

Lin

Nathaniel

, et al. (2006), “Modeling Customer Lifetime Value,” Journal of Service Research, 9 (2), 139–55.

22.

Hennig-Thurau

Thorsten

Marchand

André

Marx

Paul

(2012), “Can Automated Group Recommender Systems Help Consumers Make Better Choices?” Journal of Marketing, 76 (5), 89–109.

23.

Hilmert

Clayton J.

Kulik

James A.

Christenfeld

Nicholas J.S.

(2006), “Positive and Negative Opinion Modeling: The Influence of Another’s Similarity and Dissimilarity,” Journal of Personality and Social Psychology, 90 (3), 440–52.

24.

Hinz

Oliver

Eckert

Jochen

(2010), “The Impact of Search and Recommendation Systems on Sales in Electronic Commerce,” Business & Information Systems Engineering, 2 (2), 67–77.

25.

Hoeffler

Steve

Ariely

Dan

(1999), “Constructing Stable Preferences: A Look into Dimensions of Experience and Their Impact on Preference Stability,” Journal of Consumer Psychology, 8 (2), 113–39.

26.

Horton

John J.

Rand

David G.

Zeckhauser

Richard J.

(2011), “The Online Laboratory: Conducting Experiments in a Real Labor Market,” Experimental Economics, 14 (3), 399–425.

27.

International Data Corporation (2019), “Worldwide Spending on Artificial Intelligence Systems Will Grow to Nearly $35.8 Billion in 2019, According to New IDC Spending Guide” (March 11), https://www.idc.com/getdoc.jsp?containerId=prUS44911419.

28.

Ipeirotis

Panagiotis G

. (2010), “Demographics of Mechanical Turk,” CeDER-10–01 working paper, New York University.

29.

Iyengar

Raghuram

Van den Bulte

Christophe

Lee

Jae Young

(2015), “Social Contagion in New Product Trial and Repeat,” Marketing Science, 34 (3), 408–29.

30.

Johnson

Chris

(2015), “From Idea to Execution: Spotify’s Discover Weekly,” (November 16), https://www.slideshare.net/MrChrisJohnson/from-idea-to-execution-spotifys-discover-weekly/26-Google_Form_1_Results.

31.

Kamakura

Wagner

Mela

Carl F.

Bodapati

Anand

Fader

Pete

Naik

Prasad

Neslin

Scott

, et al. (2005), “Choice Models and Customer Relationship Management,” Marketing Letters, 16 (3/4), 279–91.

32.

Kramer

Thomas

(2007), “The Effect of Measurement Task Transparency on Preference Construction and Evaluations of Personalized Recommendations,” Journal of Marketing Research, 44 (2), 224–33.

33.

Latane

Bibb

(1981), “The Psychology of Social Impact,” American Psychologist, 36 (4), 343–56.

34.

Linden

Greg

Smith

Brent

York

Jeremy

(2017), “Two Decades of Recommender Systems at Amazon.com,” IEEE Internet Computing, 7 (1), 12–18.

35.

Meyners

Jannik

Barrot

Christian

Becker

Jan U.

Goldenberg

Jacob

(2017), “The Role of Mere Closeness: How Geographic Proximity Affects Social Influence,” Journal of Marketing, 81 (5), 49–66.

36.

Morvinski

Coby

Amir

Muller

Eitan

(2017), “‘Ten Million Readers Can’t Be Wrong!,’ or Can They? On the Role of Information about Adoption Stock in New Product Trial,” Marketing Science, 36 (2), 290–330.

37.

Müller-Trede

Johannes

Choshen-Hillel

Shoham

Barneron

Meir

Yaniv

Ilan

(2018), “The Wisdom of Crowds in Matters of Taste,” Management Science, 64 (4), 1779–1803.

38.

Naylor

Rebecca Walker

Lamberton

Cait Poynor

Norton

David A.

(2011), “Seeing Ourselves in Others: Reviewer Ambiguity, Egocentric Anchoring, and Persuasion,” Journal of Marketing Research, 48 (3), 617–31.

39.

Naylor

Rebecca Walker

Lamberton

Cait Poynor

West

Patricia M.

(2012), “Beyond the ‘Like’ Button: The Impact of Mere Virtual Presence on Brand Evaluations and Purchase Intentions in Social Media Settings,” Journal of Marketing, 76 (6), 105–12.

40.

Novet

Jordan

(2017), “China’s WeChat Captures Almost 30% of the Country’s Mobile App Usage: Meeker Report,” CNBC, (May 31), https://www.cnbc.com/2017/05/31/wechat-captures-about-30-percent-of-chinas-mobile-app-usage-meeker-report.html.

41.

Packard

Grant

Berger

Jonah

(2016), “How Language Shapes Word of Mouth’s Impact,” Journal of Marketing Research, 54 (4), 572–88.

42.

Paolacci

Gabriele

Chandler

Jesse

Ipeirotis

Panagiotis G.

(2010), “Running Experiments on Amazon Mechanical Turk,” Judgment and Decision Making, 5 (5), 411–19.

43.

Pathak

Bhavik

Garfinkel

Robert

Gopal

Ram D.

Venkatesan

Rajkumar

Yin

Fang

(2010), “Empirical Analysis of the Impact of Recommender Systems on Sales,” Journal of Management Information Systems, 27 (2), 159–88.

44.

Ricci

Francesco

Rokach

Lior

Shapira

Bracha

, eds. (2015), Recommender Systems Handbook. New York: Springer.

45.

Spangher

Alexander

(2015), “Building the Next New York Times Recommendation Engine,” The New York Times (August 11), https://open.blogs.nytimes.com/2015/08/11/building-the-next-new-york-times-recommendation-engine/.

46.

Tintarev

Nava

Masthoff

Judith

(2015), “Designing and Evaluating Explanations for Recommender Systems,” in Recommender Systems Handbook, Ricci

Francesco

Shapira

Bracha

Rokach

Lior

, eds. New York: Springer, 353–82.

47.

Tuk

Mirjam A.

Verlegh

Peeter W.J.

Smidts

Ale

Wigboldus

Daniël H.J.

(2019), “You and I Have Nothing in Common: The Role of Dissimilarity in Interpersonal Influence,” Organizational Behavior and Human Decision Processes, 151 (March), 49–56.

48.

Wang

Weiquan

Benbasat

Izak

(2007), “Recommendation Agents for Electronic Commerce: Effects of Explanation Facilities on Trusting Beliefs,” Journal of Management Information Systems, 23 (4), 217–46.

49.

West

Patricia M.

Brown

Christina L.

Hoch

Stephen J.

(2002), “Consumption Vocabulary and Preference Formation,” Journal of Consumer Research, 23 (2), 120–35.

50.

Lizhen

Duan

Jason A.

Whinston

Andrew

(2014), “Path to Purchase: A Mutually Exciting Point Process Model for Online Advertising and Conversion,” Management Science, 60 (6), 1392–412.

51.

Yaniv

Ilan

Choshen-Hillel

Shoham

Milyavsky

Maxim

(2011), “Receiving Advice on Matters of Taste: Similarity, Majority Influence, and Taste Discrimination,” Organizational Behavior and Human Decision Processes, 115 (1), 111–12.

52.

Ying

Yuanping

Feinberg

Fred

Wedel

Michel

(2006), “Leveraging Missing Ratings to Improve Online Recommendation Systems,” Journal of Marketing Research, 43 (3), 355–65.

53.

Zhang

Shuai

Yao

Lina

Sun

Aixin

Tay

(2018), “Deep Learning Based Recommender System: A Survey and New Perspectives,” ACM Computing Surveys, 1 (1), 1–35.

54.

Zhu

Feng

Zhang

Xiaoquan (Michael)

(2010), “Impact of Online Consumer Reviews on Sales: The Moderating Role of Product and Consumer Characteristics,” Journal of Marketing, 74 (2), 133–48.

55.

Zunick

Peter V.

Teeny

Jacob D.

Fazio

Russell H.

(2017), “Are Some Attitudes More Self-Defining Than Others? Assessing Self-Related Attitude Functions and Their Consequences,” Personality and Social Psychology Bulletin, 43 (8), 1136–49.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.23 MB