Abstract
Measuring the impact of social media communication is a prominent and pertinent challenge; the commercialization of social media influencers (SMIs) in the form of so-called influencer marketing makes this effort even more complicated. Companies that embrace influencer marketing have limited control over content and context, so they must evaluate both the SMIs and the content they post, prior to and during their collaborations. Although quantitative success metrics (e.g., number of followers, number of likes) are readily available, it remains unclear whether such metrics offer appropriate proxies for evaluating an SMIs or the outcomes of an influencer marketing campaign. By combining secondary data on influencer marketing campaigns from Instagram with an online survey among marketers, this study finds that professionals generally rely on an SMI’s reach and number of interactions as success metrics. When they must trade off across multiple metrics, these professionals predominantly rely on comment sentiment, indicating their implicit awareness that the commonly used metrics are inadequate. A regression analysis affirms that only the sentiment measure correlates positively with professional content evaluations, so this study both challenges the use of common quantitative metrics to evaluate SMI content and emphasizes the relevance of content-based metrics.
Introduction
Social media influencers (SMIs) are increasingly prevalent in social media domains. These content providers have acquired large audiences of thousands or even millions of followers on social media platforms such as Facebook, YouTube, or Instagram, such that they can communicate to and influence a vast number of people with their messages (Uzunoğlu & Misci Kip, 2014). These large audiences also attract the attention of brand marketers, which enter into paid collaborations with SMI for advertising or product placement purposes, in a tactic called influencer marketing (de Veirman, Cauberghe, & Hudders, 2017).
Selecting SMIs to collaborate with and measuring the outcomes of the resulting campaigns represent two main challenges to influencer marketing. The social media environment is fragmented across different platforms, so reviewing and evaluating SMIs manually would be extremely laborious. Then each influencer marketing campaign could include dozens or hundreds of sponsored posts by the SMIs, making it impossible for managers to assess them individually. Brand marketers and agencies therefore use various metrics as key performance indicators (KPIs), both for the selection of SMIs and for measuring the outcomes (Fay & Larkin, 2017; Peters, Chen, Kaplan, Ognibeni, & Pauwels, 2013). For example, to measure the reach and impact of SMIs, they might use the number of followers or number of interactions with each post (Peters et al., 2013). Such metrics are publicly available, easy to interpret, and comparable across social media platforms. Yet they might not be optimal or even suitable for selecting SMIs or predicting and then evaluating a campaign’s advertising effectiveness.
Consider the number of followers metric as an example. Many followers might indicate the substantial experience of SMIs, who likely adopt a professional communication approach that enables them to design appealing, appropriate posts that evoke positive responses by the audience. Yet the number of followers marks the maximum potential reach, and an algorithmic selection of posts (as used by Facebook and Instagram) could lead to a significantly smaller audience (Isaac, 2016). Similarly, a large number of interactions may indicate engaging, interesting content that is likely to provoke the audience’s positive attitude toward the influencer marketing campaign. However, a high number of interactions (e.g., comments) do not reveal the content or sentiment of the interactions. Because SMIs also are well aware of the use and importance of such metrics, they might try to optimize their results, using acceptable methods (e.g., raffles to increase engagement) or disputable practices (e.g., buying followers) (Smith, 2017). To help clarify the value of metrics for influencer marketing, this article establishes three main research questions:
To address these questions, the next section outlines a conceptual background of social media metrics and influencer marketing. The “Research design and data collection” section describes the methodology, followed by the “Data analysis and results” section. Finally, the “Discussion” section provides a discussion of the findings.
Conceptual Background
SMIs and Influencer Marketing
With the rise of social media, adopted by consumers worldwide, SMIs have become increasingly broad phenomena. These influencers are “opinion leader[s] in digital social media who communicate[s] to an unknown mass audience” (Gräve, 2017, p. 1; Uzunoğlu & Misci Kip, 2014). Digital social media encompass various platforms, including major networks such as Facebook, YouTube, Instagram, and Twitter, and also special interest platforms like Twitch. In terms of their audience, SMIs range from individuals with a few thousand followers to celebrities with several million fans. The former are sometimes referred to as micro-influencers (Wissman, 2018) or micro-celebrities (Abidin, 2016; Marwick, 2015); the latter often attract vast audiences and can be considered international celebrities. For example, Felix Arvid Ulf Kjellberg (also known as “PewDiePie”), who publishes entertaining videos, has more than 66 million subscribers on YouTube (Socialblade, 2018b). Huda Kattan (“hudabeauty”) shares posts about makeup and cosmetics with more than 27 million followers on Instagram (Socialblade, 2018c).
On these platforms, SMIs cover various topics and target groups, including fashion, gaming, and entertainment, by (allegedly) sharing their personal lives. As a result, SMIs are perceived as accessible, friendly, people next door, who offer unbiased opinions and therefore possess high credibility and trustworthiness (Abidin, 2015; de Veirman et al., 2017). In turn, they represent efficient sources of electronic word of mouth and can be effective marketing tools (King, Racherla, & Bush, 2014). Brand marketers thus increasingly engage in influencer marketing, entering into paid collaborations with SMIs to promote brands and products (Carter, 2016; Lee & Watkins, 2016; Schwemmer & Ziewiecki, 2018). Industry reports suggest that most brands and companies plan on increasing their investments in influencer marketing too, so this advertising channel appears likely to remain relevant for marketing communication (Linqia, 2017). Collaborations with SMIs might range from single posts endorsing a product to long-term ambassadorships to launches of comprehensive product ranges dedicated to and branded by a single SMIs (Hobbs, 2015; Lee & Watkins, 2016; Strugatz, 2016). The compensation for SMIs varies widely too, from a free product sample to hundreds of thousands of dollars (Carter, 2016).
As noted in the “Introduction” section, though, for brand marketers, influencer marketing strategies are complicated by the identification and selection of relevant, suitable collaboration partners (Carter, 2016), as well as the difficulties of measuring the outcomes of campaigns. The social media environment is opaque in this sense, in that it is dynamic and fast-paced, as well as fragmented across different platforms. Beyond a few notoriously popular SMIs, marketers would have a hard time finding effective SMIs that are relevant for their brand communication, in terms of reach and target market. Some influencer marketing campaigns include dozens of different SMIs, which makes it even more effortful to make a selection and then assess their impacts. Evaluations of influencer marketing campaigns might seek to quantify the outcome of a campaign in general or determine whether the collaboration should continue. For these various tasks, marketers often turn to metrics to quantify the (potential) impact of SMI communications.
Social Media Metrics
Technological advances and the introduction of new platforms and participatory features have profoundly changed the Internet, transforming individual users from mere content consumers into active participants and content creators (Gerlitz & Helmond, 2013). Social media networks offer many ways to interact with content, including subscribing, (dis)liking, sharing, and commenting. Such activities produce data and metrics that together constitute meaningful information, and much of it is publicly available on social media networks. Depending on the platform, the number of followers (e.g., Instagram), likes and comments (e.g., Facebook), or views of a post (e.g., YouTube) are observable to everyone. Some of these metrics are meaningful per se (e.g., number of subscribers), but others can have more abstract meanings (e.g., clicking a Like button; Sumner, Ruge-Jones, & Alcorn, 2018). Platforms such as influencerdb.net, socialblade.com, or socialbakers.com systematically collect such metrics in databases, compute additional metrics (e.g., influencerdb.net, 2018, offers “Estimated Post Values” in U.S. dollars and “Like-Follower-Ratios”), and offer to help marketing professionals find SMIs and measure their impact. These metrics actually serve as proxies; for example, the actual number of attentive views of a post would be more meaningful than the number of followers of an account. However, private companies own the most popular social media networks and decide what information to make publicly available, which information is restricted to the account owner (i.e., business accounts on Facebook and Instagram receive details about the number of views of their posts), and what information is not available at all.
In research on social media metrics, most studies adopt a brand communication perspective, such as a stream of research that investigates how companies and brands can manage their social media presence efficiently and communicate effectively, e.g. by eliciting many interactions (de Vries, Gensler, & Leeflang, 2012; Peters et al., 2013). Other authors examine the outcomes of communication on social media on consumers’ offline behavior, sales, or returns on investment (Fay & Larkin, 2017; Fisher, 2009; Kumar, Bezawada, Rishika, Janakiraman, & Kannan, 2016). Such insights might help establish efficient influencer marketing campaigns, but influencer marketing also creates a special challenge in this regard. That is, for other types of paid and owned content on social media, brand marketers typically produce the content themselves and retain control of the context in which it appears. In contrast, influencer marketing leaves brand managers with very limited control of the content that SMIs post and the context in which it appears, because SMIs create the content themselves; brand marketers can only define rough guidelines. Brand marketers accept this limited control, because it allows SMIs to communicate genuinely and retain their authenticity and trustworthiness (de Veirman et al., 2017). Therefore, the posted content does not necessarily adhere to a carefully crafted brand identity but rather is composed according to the intuition of the influencer. A similar limitation exists with regard to the context: A firm can review a channel for general brand fit, but subsequent posts could feature conflicting content, leading to negative perceptions of the posts in this context. Thus, the firm needs some way to evaluate the content quality and to measure the advertising effectiveness of the sponsored content in the context in which it appears. Because doing so is difficult, as previously noted, metrics often serve as KPIs to evaluate SMIs, single posts, and campaigns overall. The empirical study in the next section seeks to determine which metrics practitioners should use.
Research Design and Data Collection
The research design relies on real influencer marketing campaigns focused on a German audience, which then were included as examples in a survey of German marketing professionals. Germany is generally comparable to other countries, though the share of companies that engage in social media activities is slightly below the European average (Eurostat, 2019). Whereas SMIs communicating in English have the potential to reach an international audience, SMIs communicating in German likely engage in national campaigns. This study leverages Instagram, which is especially relevant for influencer marketing, with its more than 800 million users (de Veirman et al., 2017; Linqia, 2017; Systrom, 2017). The well-established practice of using campaign-specific hashtags on this platform also makes it possible to identify posts related to a campaign more easily.
First, I identified influencer marketing campaigns on Instagram directed toward a German audience with a consistent campaign hashtag, by researching online press coverage and Instagram profiles of popular German SMIs. Only campaigns with a distinctive hashtag were included (i.e., a hashtag used only in the context of the campaign). I identified five suitable campaigns in 2017, featuring the brands L’Oréal Paris (cosmetics), Rama (margarine), Rittersport (chocolate), Coral (detergent), and Fitvia (detox products). The data collection relied on Netlytic, a cloud-based tool that performs automated collections of publicly available social media posts (Gruzd, 2017), which gathered all Instagram posts that included the focal campaign hashtag. All posts were screened to eliminate those that were not part of the original campaign by the brand (e.g., posts mimicking the campaign, posts by individuals with small audiences of <1,000).
Second, I expanded this data set with information about the SMI, such as the number of followers they had at the time the post was created, collected from Socialblade (2018a). To compute a sentiment-related metric, “net sentiment” (i.e., number of positive comments minus number of negative comments; Fay & Larkin, 2017), the Linguistic Inquiry and Word Count (LIWC) dictionary tool classified all SMI posts as positive, neutral, or negative (Kahn, Tobin, Massey, & Anderson, 2007; Wolf et al., 2008). Sentiment analysis provides a method to make affect quantifiable, which is especially helpful for very large data sets in social media contexts (Puschmann & Powell, 2018). The analysis was conducted using a German LIWC dictionary (Wolf et al., 2008), so no translations were necessary. Table 1 summarizes the descriptive statistics of the data set.
Descriptive Statistics of Collected Campaign Posts.
Third, I conducted an online survey among social media professionals, including brand managers, public relations managers, and employees of advertising agencies. To collect a convenience sample, I used posts on social media platforms (e.g., Facebook, LinkedIn) and personal contacts in various companies in the consumer goods industry and advertising agencies, whom I encouraged to share the survey within their organizations. Prior to the actual survey, the respondents answered questions regarding their professional experience to verify their fit with the target group. Specifically, respondents indicated whether they worked in the field of marketing and for how long, and whether they had prior work experience with social media marketing. Only respondents with professional experience completed the actual survey questions, offering their professional opinions. In total, 76 respondents (
The first section of the survey featured an open question, to gain insight about the most relevant KPIs used in practice. Respondents had to identify the metrics they considered most relevant for influencer marketing. Each respondent could state up to five KPIs. Next, they had to trade off different KPIs to rank nine fictitious influencer marketing campaigns presented to these respondents, according to their level of success. These unnamed campaigns were characterized by three KPIs: number of interactions (i.e., sum of all likes and comments), interaction rate (i.e., number of followers divided by the sum of all likes and comments; Fisher, 2009), and a sentiment index (i.e., positive minus the negative comments; Fay & Larkin, 2017). To ensure realistic values, I derived three different values for each KPI from the data related to the influencer marketing campaigns: minimum, maximum, and mean. Table 2 lists the KPI values included in the stimuli. These nine values were combined in a Latin square design (Green, 1974) to obtain values for the nine fictitious influencer marketing campaigns. The respondents then had to evaluate the campaigns from a professional perspective and indicate which campaigns they considered most successful by ranking them, 1 to 9, with a drag and drop tool.
Values of KPIs in the Conjoint Analysis.
Finally, in the third section of the survey, each respondent evaluated three randomly chosen SMI posts out of a sample of 30 posts that came from the initial data set, equal to 6 posts per campaign. The posts for each campaign reflect overall variance in the campaign (i.e., scoring low, medium, and high values for each KPI). For every post, the number of likes and comments, as well as the comments themselves, was deleted. Only the SMI’s user name, picture, and caption were visible, as the examples in Figure 1 depict. Before each evaluation, respondents indicated if they were aware of the SMI and the brand. If they were, they also had to complete items indicating their attitudes toward the SMI (three-item scale; Martin, Kwai-Choi Lee, & Yang, 2004) and the brand (three-item scale; Sengupta, Goodstein, & Boninger, 1997).

Exemplary campaign posts for L’Oréal and Rama.
After respondents viewed the post, they offered general attitudes toward the advertisement and their professional evaluations. The professional evaluation consisted of three items on 7-point Likert-type scales: “The post shown is an example of well-implemented influencer marketing,” “The post shown is of high quality,” and “The influencer is proficient in performing professional influencer marketing” (Cronbach’s α = .857; developed according to Singh, Balasubramanian, & Chakraborty, 2000).
All respondents were informed that their participation was anonymous and that their data would be used only for research purposes. All Instagram posts presented in the survey and used in the data analysis were publicly available on the Internet.
Data Analysis and Results
Following the structure of the survey, the data analysis consists of three parts. First, to classify the freely stated KPIs, I grouped identical and similar terms. The grouping reveals the most relevant types of KPI. Second, I used traditional conjoint analysis to assess the rank orders of the fictitious influencer marketing campaigns. The respondents’ task is to sort a given number of choice options, with a given number of characteristics, according to their personal preference. Thus, the respondents must make trade-offs across the different choice characteristics, which reveals the relative importance of each characteristic (Johnson, 1974). Third, the evaluation of the SMIs’ posts and the respective KPIs provided input for a linear regression analysis. Estimating the relation between abstract KPIs and professional evaluation of a specific SMI post should help clarify the extent to which professional evaluations of a post are reflected by the corresponding KPIs used for the post and the SMI.
Stated KPI Classification
The classification of stated KPIs features two dimensions: quantitative versus qualitative and post/influencer related versus sales/brand related. The results show a clear preference for quantitative KPIs (Table 3); qualitative measures are hardly mentioned. Furthermore, KPIs directly related to posts are mentioned more often, especially the number of interactions/interaction rate and total reach (which ultimately relates to the SMI, given its correlation with the number of followers). The ultimate goal of influencer marketing—that is, performance measures related to the firm—appears less often in these free responses.
Classification of Stated KPIs.
Thus, the findings support the notion that managers mostly rely on KPIs that are easily quantifiable and readily available. This choice is reasonable in a way; it minimizes the effort needed to evaluate influencer marketing campaigns. However, it is questionable whether this shortcut also is a proxy for actually desired outcomes, namely, authentic, high-quality content corresponding to the aspired brand image, which increases brand awareness, enhances the brand image, and ultimately leads to increased sales.
Conjoint Analysis
The KPIs in the second section of the survey include the post-related dimensions mentioned most often in the free response: interactions, reach, and sentiment. These KPIs represent measures that practitioners use to evaluate campaigns. The conjoint analysis yields plausible partworth utilities for all three KPIs; the higher the KPI, the higher its assigned utility value (see Table 4). Goodness-of-fit measures also can help evaluate the estimation: Pearson’s
Results of the Conjoint Analysis.
Besides the estimated partworth utilities, I seek to determine the relevance assigned to a KPI if multiple KPIs are available, as reflected in relative importance values (Johnson, 1974). If forced to trade off the three KPIs, respondents assign the highest weight (43.3%) to the net sentiment in their evaluations of an influencer marketing campaign. The number of interactions (30.3%) ranks second, and the interaction rate is least important (26.4%). These results are inconsistent with the findings from the first section of the survey, in which participants mentioned the number of interactions most often. This discrepancy indicates that the importance of the number of interactions and reach may be due to their mere presence and availability. When considered in relation to other measures like net sentiment, managers attach greater importance to this sentiment measure rather than relying solely on the quantity of reactions.
Regression Analysis
A linear regression can reveal the extent to which the three abstract metrics used in the conjoint analysis relate to actual professional attitudes toward an SMI post, as expressed in a direct evaluation. The regression model includes
The results reveal an adjusted
Results of the Regression Analysis.
Dummy variable indicating whether a promotional code is included in the post.
The model estimation for the covariates shows several significant effect sizes, indicating the need to control for individual differences. However, the effect of the dummy variable
Discussion
This article addresses two of the main challenges to influencer marketing: the identification and selection of SMIs and the evaluation of influencer marketing campaigns using metrics as KPIs. To do so, it starts from a foundation of three research questions, related to the metrics brand marketers and agencies use, the ones they consider most important for evaluating an influencer marketing campaign, and the metrics that can proxy for actual perceived quality of the content of an influencer marketing campaign.
Regarding the first two questions, the empirical study shows that marketers mostly rely on quantitative, readily available metrics, like the number of interactions or reach, to evaluate influencer marketing activities. Almost 53% of all mentions fall into this category. The overreliance on these KPIs seems driven largely by their mere availability. In contrast, the conjoint analysis revealed that in the presence of additional KPIs that resemble the sentiment of the audience, managers attach greater importance to such measures than to purely count-based metrics. Managers therefore seem (at least implicitly) aware that the informative value of the metrics they predominantly use is subpar. The preference for the net sentiment metric could arise because positive net sentiment is the desired outcome, such that marketers strive for more positive comments and few negative comments and are willing to pay for it; it also could indicate that net sentiment provides a seemingly effective proxy of general advertising effectiveness, in that this metric contains more information than a simple difference in the number of positive and negative comments. The former explanation might be embraced by marketing professionals, but the regression analysis suggests the latter is the case. For the last research question, only the net sentiment metric relates positively to professional attitudes toward a post, therefore emphasizing the importance of such KPIs for evaluating the success of influencer marketing activities on an abstract level, without requiring assessments of every single campaign post. Managers should assign greater emphasis to sentiment measures drawn from comments to evaluate influencer marketing activities rather than looking at simple heuristics like reach or interaction rate. However, this finding should not be taken to mean that the latter measures are not important: Managers should consider (potential) reach and interaction rate when choosing SMIs for collaborations. Still, it is important to understand that these conditions are necessary, not sufficient, to guarantee high-quality SMI content.
This research also is subject to several limitations that offer potential for further research advances. First, I only used 30 posts from 5 example influencer marketing campaigns on Instagram to gather professional attitudes, and the survey covered a limited number of KPIs. In addition, this study focused on German influencer marketing campaigns and accordingly surveyed German professionals. Continued research could broaden the scope by reviewing a greater variety of influencer marketing activities on different social media platforms in different countries, as well as more and varied KPIs, to ensure that the results generalize to all cases of influencer marketing. Second, the computation of certain KPIs could be subject to further investigation. Recent research indicates that, among automated text classification methods, machine learning algorithms such as random forests outperform lexicon-based approaches like LIWC (Hartmann, Huppertz, Schamp, & Heitmann, 2018). Thus, it would be worthwhile to investigate whether the findings of the current analysis hold when advanced classification methods with higher accuracy compute the sentiment metric.
In the near future, the fields of influencer marketing and SMIs promise to remain dynamic and challenging, due to the development of new features (e.g., Instagram stories), emerging social media platforms, regulatory changes, and increasing professionalization of both SMIs and specialized marketers. Researchers thus must continue to extend knowledge and best practices in this field.
Footnotes
Acknowledgements
Special thanks to Annika Greff for her help in the data collection.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
