Context Information Can Increase Revenue in Online Display Advertising Auctions: Evidence from a Policy Change

Abstract

Ad exchanges, where real-time auctions for display ad impressions take place, have historically emphasized user targeting, and advertisers sometimes did not know on which sites their ads would appear; in other words, they had no ad context. More recently, some ad exchanges have been encouraging publishers to provide context information to ad buyers that would allow them to adjust their bids for ads at specific sites. This article explores the empirical effect of a change in context information provided by a private European ad exchange. Analyzing this as a quasi-experiment using difference in differences, the authors find that average revenue per impression rose when the exchange provided subdomain information to ad buyers. Thus, ad context information is important to ad buyers, and they will act on it. Revenue per impression rises for nearly all sites, which is what auction theory predicts will happen when rational buyers with heterogeneous preferences are given more information. Exceptions are sites with thin markets prior to the policy change; consistent with theory, these sites do not show a rise in prices. This study adds evidence that ad exchanges with reputable publishers, particularly smaller volume, high-quality sites, should provide ad buyers with context information, which can be done at almost no cost.

Digital display advertising has rapidly become popular among advertisers due to new targeting options as well as lower transaction costs enabled by ad tech. A central component of the industry is real-time bidding (RTB) markets for display ad impressions. When a user requests a page from a website, an impression (i.e., an opportunity to advertise to that user) becomes available. The site publisher can sell this impression on an RTB market by submitting a bid request to the ad exchange, which often includes a cookie identifying the user. The ad exchange subsequently broadcasts the bid request to potential ad buyers typically through intermediaries called demand-side platforms (DSPs). In response, ad buyers submit bids for the impression, and the exchange sells the impression in an auction. This entire process occurs within 400 milliseconds so that the ad loads almost instantaneously for the user. The publisher is paid the winning price less a commission to the exchange. An alternative to RTB is programmatic direct advertising, which allows ad buyers to prenegotiate a guaranteed number of impressions at a particular site and are often processed using the same technology infrastructure as RTB. ¹ Together, RTB and programmatic direct advertising are the dominant way that online display ads are sold, with projected spending set to reach $81.58 billion representing 86.5% of total display ad spending in the United States in 2021 (eMarketer 2021b). Besides their economic importance, display advertising markets have been of interest to researchers (Berman 2018; Budak et al. 2014; Choi and Mela 2019; Goldfarb and Tucker 2011a, b; Johnson, Shriver, and Du 2020; Lambrecht and Tucker 2013; Miller and Skiera 2017; Rafieian and Yoganarasimhan 2021; Reiley, Lewis, and Schreiner 2012; Tunuguntla and Hoban 2021), in part because they provide detailed tracking of ad transactions and user, ad buyer, and publisher behavior.

Historically, the display advertising industry has focused on user targeting, and a thriving market for third-party data that describe the demographic characteristics and online behavior of each user cookie has developed. Recently, user targeting has come under question due to privacy concerns, and several recent articles have focused on estimating the increased value of an impression when it is accompanied by a cookie, which can be connected to user-level behavioral data (Johnson, Shriver, and Du 2020; Marotta, Abhishek, and Acquisti 2019; Miller and Skiera 2017). As user information has become limited, the display advertising industry has put an increasing emphasis on context as an alternative way to target ads. Industry groups, like the Interactive Advertising Bureau (IAB), have encouraged publishers to include the exact URL in the bid request (IAB 2016), and publishers are increasingly doing so. In January 2020, 80% of bid requests passing through a large meta-exchange included the exact URL and another 5% included the subdomain. ² However, little empirical research exists on how increasing context information has affected the value and selling prices of impressions in RTB auctions.

This article provides evidence that display advertising buyers value information about the context of the impression even when they have access to rich information about users. Our analysis focuses on a private European ad exchange, where a relatively small number of reputable digital publishers agree to offer impressions to a preapproved group of advertisers through RTB. Private exchanges reduce fraud (Choi and Sayedi 2021) and provide brand safety for advertisers and publishers. This market provides us with a natural experiment, where the exchange moved abruptly to providing subdomain information (e.g., www.nytimes.com/section/business) for each impression. After analyzing data on the sales on this exchange using a quasi-experimental difference-in-differences approach (Goldfarb and Tucker 2014), we find that when buyers had information on the subdomain where the impression would appear, revenue rose for nearly all sites. We argue that this happened because different buyers preferred different sites, and they all increased their bids for their preferred sites. Our study serves as an empirical validation that providing more information about items for sale in an auction affects the auction outcomes and is consistent with theoretical predictions that prices will rise when buyers have heterogeneous preferences (Hummel and McAfee 2016; Rafieian and Yoganarasimhan 2021).

This finding has practical importance for the display advertising industry. We show that providing more context transparency on the private exchange benefits both advertisers (who can place their ads on sites they prefer) and publishers (who get higher prices). The growing number of private exchanges, where the information provided by publishers is well-controlled, should provide context information at least at the subdomain level. Display ad sales on such private ad exchanges exceeded those on open exchanges for the first time in 2020 (eMarketer 2021a), so the implications are not inconsequential.

Our findings also speak to context disclosure policies at the large open exchanges. Managing information on the open exchanges presents greater challenges, and exchanges have moved more gradually toward greater context transparency. Consequently, determining how increased context transparency has affected prices has been difficult. Without direct empirical evidence, some publishers remain dubious about providing detailed context information. As discussed previously, about 15% of impressions listed on a major meta-exchange in January 2020 provided only a high-level domain. Even when a domain is listed, it might not reflect the true domain where the impression will be served (e.g., when nbc.universal.com is listed as the domain in the bid request for any NBCU-owned website but the ad appears on a different domain, such as telemundo.com). We provide empirical evidence that the industry's efforts to increase context transparency were likely beneficial to advertisers and publishers, and we suggest the open exchanges should continue to promote context transparency and eliminate context fraud, including “domain spoofing” (AdExchanger 2020b; DIGIDAY 2020). Even though the Authorized Digital Sellers (ads.txt) protocol from the IAB was touted to increase transparency and reduce domain spoofing, fraud remains an issue (Fou 2020). ³ If reducing fraud increases an advertiser's confidence about where its ads appear, then it will likely increase prices for publishers that are desirable to some advertisers. In light of this, our findings also provide an explanation for why premium publishers have migrated to private exchanges where information is better controlled and buyers have more confidence in what they are buying.

Despite recent efforts to increase context transparency, it was not obvious that context information would affect ad buyers’ valuations for ads. When RTB was introduced to display advertising, the industry touted its capability to allow ad buyers to target specific users no matter what sites they were viewing (e.g., retargeting), and the industry has often dismissed ad context as a poor proxy for the richer user-targeting options available in digital advertising. However, there is large body of research showing that the website where an ad is placed can have an effect on the consumer's response to the ad. Most of these studies are conducted in the lab (e.g., Janssens, Pelsmacker, and Geuens 2012; Moore, Stammerjohan, and Coulter 2005; Shamdasani, Stanaland, and Tan 2001; Yaveroglu and Donthu 2008), but there is growing evidence from the field that context changes the effect of ads on users (Bleier and Eisenbeiss 2015; Goldfarb and Tucker 2011a; Lambrecht and Tucker 2013; Shehu, Abou Nabout, and Clement 2020). If an ad has a greater effect in a particular context, then we would expect ad buyers to value some site placements more than others. However, it was unknown prior to the policy change how much ad buyers would value context information relative to the rich user information already available. In addition, even if they valued it, it was unclear whether they would put that information to use given the complex decision-making environment for ad buyers in RTB. Thus, this setting represents a strong empirical test of whether buyers make complete use of information in ad auctions.

While it was not clear whether ad buyers would respond to context information at all, it was also not clear how this information would affect prices in the RTB market as a whole. Theoretically, providing ad buyers with the subdomain where the ad will appear allows them to put a more precise valuation on each impression that they bid on. If ad buyers all have the same (homogeneous) preferences over sites, then when ad buyers bid their valuations for individual sites rather than the average valuation of the bundle, prices will rise for some sites and fall for others. However, if ad buyers prefer different sites (heterogeneous preferences), then auction prices can rise uniformly across all sites. This somewhat surprising result has been shown theoretically for ad auctions (Hummel and McAfee 2016) and empirically in an auction for used cars (Tadelis and Zettelmeyer 2015). But even under heterogeneous preferences, if too much information about an impression is provided, the market for a particular impression can become too thin (Levin and Milgrom 2010; Hummel and McAfee 2016), leading to a decline in prices. We find prices rose for most sites when more context information was provided, which is consistent with heterogeneous preferences among ad buyers. Our findings suggest that ad buyers value context information, that ad buyers have heterogeneous preferences for sites, and that providing context information moves the market to a higher point on the revenue curve for the platform and for most sites.

To provide further evidence that prices rise because ad buyers have heterogeneous preferences for sites and bid higher on the sites they each prefer, we provide several mechanism checks. We show that when context information became available, (1) ad buyers concentrated their purchases at fewer sites, (2) prices rose for nearly all sites, and (3) most buyers purchased from fewer sites. We also analyze a period when a single buyer was provided with context information and show that the buyer with the context information won more impressions, suggesting that the buyer was bidding higher on preferred sites. In total, these findings are consistent with theoretical predictions for second-price auctions, where buyers have heterogeneous preferences for products and markets remain sufficiently thick.

Finally, we investigate heterogeneous treatment effects and find that prices did not rise as much for sites that had fewer ad buyers prior to the policy change or were rated as providing low-quality advertising impressions (called nonpremium sites) by industry experts. This is consistent with the hypothesized deconflation (Levin and Milgrom 2010) that can occur in ad auctions when too much information is provided about an item sold in an auction. Levin and Milgrom (2010) describe conflation as a situation “in which similar but distinct products are treated as identical in order to make markets thick or reduce cherry-picking”; deconflation is the opposite: products are unbundled and prices fall. We also find that sites providing a very small volume of premium impressions saw ad prices rise the most, suggesting that sites providing ads to a desirable niche audience benefit the most from context disclosure.

These findings are based on the observed outcomes in the exchange we study. This stands in contrast to much of the existing literature on ad auctions, which estimates structural models on auction outcomes based on bid-level data and then reports counterfactual predictions for policy changes. For example, Johnson (2013) estimates a structural model from U.S.-based ad auction data and uses this model to predict that both ad buyers and publishers are worse off when the platform introduces stricter privacy policies reducing user-targeting options. Similarly, Lu and Yang (2020) use a structural model to predict how much an ad platform may improve its revenue by optimizing the level of information provided about users. Rafieian and Yoganarasimhan (2021) fit a structural model and use counterfactuals to predict the effect of limiting both user and context information on auction revenue and find that reducing context information affects revenue, but user information has a greater effect. Our approach is complementary to these studies in that it is based on careful analysis of the outcomes of an actual policy change rather than structural assumptions that ad buyers and publishers behave rationally and consistent with auction theory.

We believe the effects we find may be amplified in the future, as privacy concerns lead to restrictions on the amount of user information that can be collected and shared with ad buyers. For example, Google recently announced efforts to limit the use of third-party cookies in its Chrome browser by making “disable third-party cookies” the default setting (AdExchanger 2020a, c). Similarly, major ad-supported publishers like The New York Times have moved to protect users by decreasing the amount of user information collected on their sites (Berjon 2020) and some publishers have moved to eliminate user tracking all together (Edelman 2020). Thus, user targeting will be less straightforward in the future, and industry experts predict that contextual targeting will become more relevant (AdExchanger 2019; Tan 2019). This study provides direct empirical evidence that context information is valuable to ad buyers.

In the next section, we briefly review findings on information disclosure from the literature on auctions, which shows that prices may go up or down when more information is provided to all bidders depending on market thickness and the heterogeneity in bidders’ valuations. In the following section, we describe the institutional setting and policy change in more detail. We then analyze the empirical effect of information disclosure on ad prices using a difference-in-differences analysis, which shows that prices rose on average across the exchange. Mechanism checks show that advertisers’ budgets were concentrated on fewer sites and competition at each site decreased, but the right tail of the distribution of winning bids increased (i.e., winning bids started to spread out more), and nearly all sites saw an increase in prices. Following that, we compare the behavior of one buyer that received early access to the context information with a synthetic control made of up of buyers without this information and find that this buyer won more auctions. This provides convergent evidence that ad buyers bid higher when they have context information for each impression, and markets thin out a bit but remain sufficiently thick for prices to rise. We then proceed with evidence for heterogeneous treatment effects related to (1) competition and (2) the size and quality of a site and conclude with a summary of findings and a discussion of the study's implications.

Predicted Effect of Information Disclosure on Auctions

Theoretical predictions for how information disclosure affects ad auctions are mixed, with some researchers arguing that ad prices achieved in the auction should go up when more information is available about each impression (Hummel and McAfee 2016) while others argue they should go down (Levin and Milgrom 2010). These predictions depend critically on (1) the valuations of ad buyers for sites and (2) the number of bidders with positive valuations for each impression after the information is disclosed. If ad buyers all prefer the same sites, then prices will rise for the desirable sites and fall for the others. However, prices can rise for all sites if each ad buyer prefers different sites (i.e., buyers have heterogeneous preferences [Hummel and McAfee 2016] and markets are thick enough).

For example, consider an ad exchange selling impressions in a second-price auction ⁴ where there are two sites. Bidder i's valuation for an impression $\mathrm{j}$ at site $\mathrm{k}$ is composed of the value for the site ${\mathrm{s}}_{\mathrm{ik}}$ plus their value for remaining features of the impression ${\mathrm{r}}_{\mathrm{ij}}$ such as the cookie and time of day. When context information is disclosed to all bidders, their bids are based on their unique valuation for each site. When context information is withheld, bidders are forced to base their bids on their average site valuation. For simplicity in this example, we assume that the bidders know that the two sites place equal numbers of impressions in the auction; thus, the valuations are:

$${\mathrm{v}}_{\mathrm{ijk}}=\{\begin{array}{cc}{\mathrm{r}}_{\mathrm{ij}}+{\mathrm{s}}_{\mathrm{ik}}& \mathrm{if}\mathrm{the}\mathrm{site}\mathrm{is}\mathrm{disclosed}\\ {\mathrm{r}}_{\mathrm{ij}}+({\mathrm{s}}_{\mathrm{i}1}+{\mathrm{s}}_{\mathrm{i}2})/2& \mathrm{if}\mathrm{the}\mathrm{site}\mathrm{is}\mathrm{not}\mathrm{disclosed}.\end{array}$$

(1)

For the examples that follow, we assume that ${\mathrm{s}}_{\mathrm{i}1}$ and ${\mathrm{s}}_{\mathrm{i}2}$ are independent, meaning that buyers’ valuations for the two sites are not correlated. We also assume that the valuation for each impression ${\mathrm{r}}_{\mathrm{ij}}$ is independent of ${\mathrm{s}}_{\mathrm{i}1}$ and ${\mathrm{s}}_{\mathrm{i}2}$ . Specifically, we assume they are distributed as follows:

$${\mathrm{r}}_{\mathrm{ij}}\sim \mathrm{N}(0,\omega ){\mathrm{s}}_{\mathrm{i}1}\sim \mathrm{N}(\mu \text{–}\delta /2,\sigma ){\mathrm{s}}_{\mathrm{i}2}\sim \mathrm{N}(\mu +\delta /2,\sigma )$$

Here, $\omega$ represents variation in bidders’ values for individual impressions, $\delta$ is the difference in average valuation between the two sites, and $\sigma$ represents the variation across bidders in valuations for sites.

When $\sigma$ is large and $\delta$ small, we have what Tadelis and Zettelmeyer (2015) refer to as “heterogeneous bidders” or “horizontal differentiation between sites.” When there are a sufficient number ( $\mathrm{N}$ ) of heterogeneous bidders, disclosure will increase auction prices for both sites because bidders bid more for the sites they uniquely prefer (Chen, Li, and Yu 2018; Chakraborty 1999; Hummel and McAfee 2016; Palfrey 1983). Figure 1, Panel A, illustrates this scenario, showing simulated selling prices for 1,000 impressions where there are 50 bidders with heterogeneous preferences for sites ( $\delta =0,\sigma =1$ ). As can be seen from the horizontal lines in Figure 1, Panel A, average prices rise for both sites, although prices may rise more for one site than the other depending on the realizations of ${\mathrm{s}}_{\mathrm{i}1}$ and ${\mathrm{s}}_{\mathrm{i}2}$ . Under context disclosure, the distribution of winning prices also has greater variance and a longer tail. As we show, the auction outcomes in the exchange we study are consistent with this scenario.

Figure 1.

Effect of information disclosure on auction prices.

The potential for prices to rise across the board when ad buyers have more information about impressions has also been shown analytically by Hummel and McAfee (2016). They prove that average prices rise when there are at least four bidders with independently and symmetrically distributed valuations, so long as the distributions are not fat tailed. Intuitively, this happens because the additional information leads to a better match between ad buyers and impressions. While this result has not been tested empirically for ad auctions, it has been observed in other types of auctions. Tadelis and Zettelmeyer (2015) report on a field experiment in which additional information on condition was provided to bidders in a used-car auction, resulting in an increase in prices for both high- and low-quality cars. They argue that the bidders have heterogeneous preferences for condition and that the information produces a better match between buyers and cars.

Even when there are heterogeneous preferences for sites ( $\sigma$ large and $\delta$ small), if there are an insufficient number of bidders, prices may fall. This is illustrated in Figure 1, Panel B, which shows that average prices fall under information disclosure when there are N = 2 bidders. Because there are so few bidders, some impressions have a second-highest bid of zero, as shown by the large number of points with prices at zero for Site 2. (This does not happen for Site 1 because realizations of ${\mathrm{s}}_{11}$ and ${\mathrm{s}}_{21}$ are both high by chance.) Levin and Milgrom (2010) describe this phenomenon in the context of user targeting and refer to it as an “orphaned category.” They argue that ad platforms should conflate markets, which involves providing less information about items in the auction so that such orphaned categories are indistinguishable to buyers, thereby creating thicker markets. Not disclosing site information is one strategy for conflation, when markets are thin.

Thus, Panels A and B of Figure 1 show prices may rise or fall when relevant information is disclosed to bidders in an auction. For a given set of bidders and preferences, the literature on ad auctions concludes that the relationship between auction outcomes and information disclosed is concave, with an intermediate amount of information (or equivalently bundling) producing the highest revenue (Rafieian and Yoganarasimhan 2021).

Context disclosure can lead to winners and losers among sites when ad buyers have homogeneous preferences for sites. We can simulate this scenario by setting $\delta$ higher and $\sigma$ lower. As Figure 1, Panel C, shows, when bidders have homogeneous preferences for one site over the other, prices rise dramatically for the preferred site and fall for the other. This outcome may be revenue neutral for the auction platform (depending on the mix of sites it represents), but it has a substantial effect on revenue for individual sites and may make publishers of less-preferred sites dissatisfied with the platform. The potential for this type of behavior by ad buyers was a serious concern to the platform operator we study. However, the data from before and after the policy change suggest that ad buyers’ preferences for sites are largely heterogeneous (with some limitations, which we discuss subsequently).

Finally, because it is possible that ad buyers do not value context information above and beyond cookie information, in Figure 1, Panel D, we simulate a case in which bidders show more variation in their value for individual impressions (based on the cookie) versus sites (i.e., $\sigma$ is small and $\omega$ is large). As the graph shows, when bidders place more importance on the cookie and other features of the impression than the context, then context disclosure has a modest effect on ad markets.

Collectively, the simulations in Figure 1 show that context disclosure may (1) uniformly raise prices for all sites, (2) uniformly lower prices, or (3) raise prices for some sites but not others depending on the number of bidders and their valuations for sites. It was also possible ex ante that this policy change would have no effect at all because ad buyers may not behave rationally. The theoretical and structural literature on auctions relies on the assumption that bidders will maximize their expected value given available information, as do the simulations in Figure 1. However, research on managerial decision making shows that managers are often risk averse (Amihud and Lev 1981), relying merely on historical performance patterns (Busenitz and Barney 1997; Little 1970) such that they do not change their investment decisions when receiving better information (Lambert, Leuz, and Verrecchia 2007). Given the many potential targeting options available to display ad buyers, they may not have the time or incentive to adjust their bidding strategies for each site. Consequently, it is unclear whether ad buyers will put context information to use at all.

In summary, it is difficult to predict whether ad context information will affect auction outcomes for three reasons: (1) if site placement is not valued by ad buyers then the change will have no effect; (2) even if buyers value the ad context, they may not change their bidding strategy due to the complexity of the advertising environment; and (3) even when ad buyers are behaving optimally, the effect of information disclosure on auction outcomes is a complex function of buyer valuations and market thickness, and prices may fall or rise for particular sites or overall. Thus, it remains an empirical question how context information will affect RTB ad auction outcomes. Next, we describe the institutional setting where we study the effect of a change in context disclosure.

Institutional Setting

We investigate a change in the ad context disclosure policy at a major private ad exchange in Europe. In a private exchange, a relatively small number of digital publishers agree to offer display ad impressions to a pre-approved group of ad buyers through RTB. This makes private exchanges distinct from open RTB display advertising exchanges like Google Ad Manager (formerly DoubleClick), where any ad buyer or publisher may participate and thousands do. Although RTB began with the open exchanges, as concerns about transparency, fraud, and brand safety have grown, premium publishers including Hearst, Technorati, Conde Nast, CBS, NBCUniversal, IDG TechNetwork, The Weather Channel, and Vox have created private exchanges that offer ads at a smaller set of reputable websites. Participating ad buyers and sites are vetted prior to bidding, and the relatively small number of participants increases transparency and brand safety for both ad buyers and publishers. Software platforms for running ad exchanges such as Google Admeld make it easy for small groups of publishers to build the necessary infrastructure to run a private exchange.

Policy Change

This private exchange offers us a unique opportunity to study the effect of a change in context information provided to ad buyers. Prior to April 2016, buyers (including advertisers themselves and intermediaries acting on behalf of multiple advertisers) on the exchange we study purchased banner ads on the publishers’ websites without any knowledge of where the impression would appear. The only form of context targeting available to ad buyers was buying ads on a “channel,” where channels represented broad content categories like “news,” “automotive” or “finance.” Many ad buyers chose to place their ads on “run of network,” meaning their ads may appear on any of the participating publishers’ websites. Ad buyers also received cookie information in the bid request and engaged in user targeting, including retargeting.

In April 2016, a single buyer was given access to context information in the bid request. Specifically, this buyer was given information about the subdomain where the ad would appear (e.g., nytimes.com/section/business). In May 2016, after observing an aggregate rise in revenue when one buyer had site information, the exchange made the site information for each impression available to all bidders. When the policy changed, ad buyers were notified by the ad exchange, often through personal phone calls. Throughout the analysis, we use the term “site” to refer to these subdomains, recognizing that some of these “sites” are subdomains belonging to the same domain. The specifics of how users determined bids specifically for sites varied by DSP. In addition to whitelisting or blacklisting sites, they could also use programmatic strategies to bid differently for different sites or combinations of sites and user profiles. Figure W1 in Web Appendix A shows an example of how buyers were able to restrict their bids to particular subdomains in their bidding criteria.

The private exchange provides us with a well-controlled setting to study the effect of context information on ad buyers’ valuations for ads. The participating publishers contractually agreed to sell all their digital ad inventory exclusively through this exchange. Publishers could choose between RTB and programmatic direct sales, and we observed all sales in both formats. Before the change, ad buyers knew that their ads would appear on one of the participating reputable sites and not “anywhere,” as in open exchanges. Publisher fraud is not an issue in this setting. Finally, the policy change happened all at once. Buyers could easily manage their buys to target specific sites, and all sales were fully observed. This sudden change in policy allows us to take a quasi-experimental approach to estimate the effect of context information on ad prices.

During this entire period, which was prior to the General Data Protection Policy coming into effect, ad buyers had access to the user cookie ID for each impression, and there was an active market for third-party data on past cookie behavior. Therefore, the policy change gives us insight into how ad buyers value context information over and above the rich user behavior data available at the time.

Executives at the exchange hoped that this change toward greater transparency would make the exchange as a whole even more appealing to ad buyers and earn it certification as a brand-safe platform. However, these executives had lengthy internal debates about the change. While some at the company were confident that revenue would rise for most or all sites, others were concerned that cherry-picking of the most desirable sites would lower revenues for less desirable sites and potentially overall revenue for the platform (which gets a 2.5% commission on all sales). As discussed in the previous section, these outcomes depend on the distribution of preferences for sites among the buyers, which was ex ante unobservable to the exchange.

Participating Websites

As discussed in the previous section, the effect of context disclosure in an ad auction depends critically on how the buyers value the websites where ads appear. The participating sites vary substantially in the types of content they provide and include one of the top three news sites in the country (according to SimilarWeb), one of the top three sports sites, and a variety of special interest and community sites similar to Quora.com, WebMD.com, Allrecipes.com, or Zillow.com in the United States. ⁵ However, while some sites might be considered niche content, all of them are reputable, and none would be considered extremist content or clickbait (as you might find in the open exchanges).

The difference-in-differences analysis focuses on the change in average revenue per impression for the 57 sites that participated in the market in both 2015 and 2016. Table W1 in Web Appendix B summarizes the supply of impressions and average revenue per impression for these sites during the week preceding the policy change. Notably, each site put on average 3,370,257 impressions into the auction during this week and sold 3,287,055, leaving an average of 2.5% of impressions unsold. The low unsold inventory reflects the high desirability of impressions sold on this private exchange.

The primary data set we use for the analysis includes the number of impressions sold and the average selling price per impression for each website-buyer pair on each day. For our main analysis, which focuses on the effect of the policy change on the average selling prices for each site, we roll up to the website-week level. The data spans a period of seven months in 2016 that covers 11 weeks before the change (within the months of January–March 2016), four weeks where one ad buyer had access to context information (April 2016), and 12 weeks where all ad buyers received context information (within the months of May–July 2016). Similar data for the same months in the previous year are also available. ⁶

Context Disclosure Increased Revenue per Impression

Our goal is to identify how the policy change affected the average selling prices for impressions—that is, the revenue per impression. As discussed in a previous section, if ad buyers have heterogeneous preferences for the sites and there are a sufficient number of buyers for each site, then prices should rise overall. If there are sites that are undesirable to most buyers, such that the markets thin out, then prices may fall for some sites or overall. It is also possible that the market might not be affected at all if buyers do not value the ad context, the transaction costs of customizing bids to specific sites are too high, or buyers do not behave rationally.

The full disclosure of context information coincided with the spring and summer, when there is a lower supply of impressions and an increase in prices in both years. This observed decline in supply of impressions (and increase in prices) is consistent with seasonal patterns in web traffic, which tend to decline in the summer. Thus, the policy change is confounded by the seasonal decrease in supply. We do a formal difference-in-differences analysis with additional controls to account for this. (Figures W2 and W3 in Web Appendix C plot the supply of impressions and the overall weekly average revenue per impression in years 2015 and 2016.)

Difference-in-differences analysis

To show that the policy change increased revenue per impression for the platform, we estimate a regression model following a difference-in-differences approach, which compares the increase in average revenue per impression during the policy change in 2016 to the increase in average revenue per impression at the same time during the previous year. The regression equation for this analysis is:

$$\begin{array}{rl}{\mathrm{y}}_{\mathrm{it}}=& {\alpha }_0+{\alpha }_1\mathrm{partia}{\mathrm{l}}_{\mathrm{t}}+{\alpha }_2\mathrm{ful}{\mathrm{l}}_{\mathrm{t}}+{\alpha }_3\mathrm{year}{16}_{\mathrm{t}}\\ & +{\beta }_1\mathrm{partia}{\mathrm{l}}_{\mathrm{t}}\times \mathrm{year}{16}_{\mathrm{t}}+{\beta }_2\mathrm{ful}{\mathrm{l}}_{\mathrm{t}}\times \mathrm{year}{16}_{\mathrm{t}}\\ & +{\gamma }_1\mathrm{suppl}{\mathrm{y}}_{\mathrm{t}}+{\gamma }_2\mathrm{adspen}{\mathrm{d}}_{\mathrm{t}}+{\delta }_{\mathrm{i}}+{\epsilon }_{\mathrm{it}},& \end{array}$$

(2)

where ${\mathrm{y}}_{\mathrm{it}}$ is the average revenue per impression on site $\mathrm{i}$ in week $\mathrm{t}$ , $\mathrm{year}{16}_{\mathrm{t}}$ is a 0–1 indicator for the year when the policy change happened, and $\mathrm{partia}{\mathrm{l}}_{\mathrm{t}}$ and $\mathrm{ful}{\mathrm{l}}_{\mathrm{t}}$ are 0–1 indicators for the weeks of the year when the partial and full disclosure policies were in effect. The parameters on the interaction terms ( ${\beta }_1$ and ${\beta }_2$ ) are the key coefficients that estimate the effect of the disclosure policy under the parallel trends assumption. The regression equation also includes an intercept and main effects for $\mathrm{partia}{\mathrm{l}}_{\mathrm{t}}$ , $\mathrm{ful}{\mathrm{l}}_{\mathrm{t}}$ , and $\mathrm{year}{16}_{\mathrm{t}}$ . Thus, our estimate of the effect of the policy change is the change in revenue per impression observed in 2016 above and beyond the change in revenue observed during the same period in 2015.

To further account for fluctuations that might affect average revenue per impression, Equation 2 includes two controls: the total supply of impressions placed in the RTB market in week $\mathrm{t}$ ( $\mathrm{suppl}{\mathrm{y}}_{\mathrm{t}}$ ) and the total national ad spending on digital advertising ( $\mathrm{adspen}{\mathrm{d}}_{\mathrm{t}}$ ). The latter was sourced from a third-party data aggregator. These controls further account for shocks in supply or demand, above the seasonality controlled for by the previous year's revenues per impression. We expect the effect of higher supply to be negative and the effect of more spending to be positive. In addition to these two controls, we factor in site fixed effects ( ${\delta }_{\mathrm{i}}$ ) that further capture heterogeneity between sites.

The difference-in-differences analysis makes several key assumptions. The most critical assumption is that 2015 provides a suitable proxy for the seasonal fluctuation in average revenue per impression; that is, the parallel trends assumption is met. In the regression, the 2015 revenues are further augmented with controls for the supply provided to the exchange ( $\mathrm{suppl}{\mathrm{y}}_{\mathrm{t}}$ ) and the total advertising spending in the market ( $\mathrm{adspen}{\mathrm{d}}_{\mathrm{t}}$ ). Together, this should result in minimal differences between predicted revenues per impression in 2015 and 2016 prior to the policy change, which we confirm with a placebo test (discussed next). The second major assumption is that the two time-varying controls are unaffected by the treatment, that is, that they are exogenous. Given that publishers are contractually obligated to sell all impressions through this platform (via RTB or programmatic direct), the supply of impressions is driven primarily by the number of users who visit each site. While publishers could increase or reduce impressions in the auction by moving them into programmatic direct, we do not observe that they do. This assumption would be violated if publishers were strategically changing the design of sites to increase or decrease impressions in anticipation of the policy change at the exchange, but this seems unlikely. Rather, the total supply of impressions is likely exogenously determined by the number of users visiting the sites. The total spending on digital ads in the country ( $\mathrm{adspen}{\mathrm{d}}_{\mathrm{t}}$ ) includes other platforms like social media and search engines, and is also likely exogenous to prices on this display advertising exchange. These controls are arguably exogenous, but as with any empirical exercise, there remains a risk that they do not capture all potential threats, so we report the difference-in-differences regression without these two controls as well. The analysis also assumes that the linear, separable model is correctly specified.

The regression is estimated using weekly data for each site from January to July for both 2015 and 2016. While the raw data are reported at the daily level, we summarized them at the weekly level to avoid having periods in which a site did not offer any impressions (and thus the auction price would be missing). The dependent variable (average weekly revenue per impression for each website) is computed by dividing the total weekly revenue for each site by the total impressions that the site submitted to the RTB platform in that week, including sold and unsold impressions. As our estimate of the treatment effect relies on a comparison between years, we include only the 57 sites that sold impressions in both 2015 and 2016. This results in 3,058 site-week-year-level observations, which is slightly fewer than 57 sites × 27 weeks × 2 years due to a few sites not selling impressions in the first few weeks of 2015. The regression is weighted by the number of impressions that each site placed in the RTB market in that week, so that the estimated effect of the policy change is an average change in revenue per impression across all impressions in the RTB market. (This avoids overweighting sites that place fewer impressions in the auction, which, as we show later, saw larger increases in revenue per impression.)

Table 1, Column 1 shows estimated coefficients for the regression in Equation 2. We focus on the increase in average revenue per impression for two periods: the partial disclosure period when one buyer had subdomain information and the full disclosure period when all buyers had subdomain information. ⁷ The key coefficients of interest are ${\beta }_1$ and ${\beta }_2$ , which show that average revenue per impression increased by €.108 per thousand impressions during the period when one buyer had access to the subdomain information for each impression and by €.154 when all buyers had access to the subdomain information. This increase of €.154 for full disclosure is above and beyond the increase in average revenue per impression seen for the same months in 2015 (see the coefficients ${\alpha }_1$ and ${\alpha }_2$ ) and the other controls. This substantial increase in average revenue per impression represents the effect of moving this auction from “channel”-level context disclosure to subdomain-level disclosure.

Table 1.

Diff.-in-Diff. Analysis of the Change in Average Revenue per Impression (€ per Thousand) Due to Context Disclosure.

	(1)	(2)	(3)
	Average Revenue per Impression(CPM in €)	Without Controls	Placebo Test
Effect of Policy Change
Partial × Year16 ( ${\beta }_1$ )	.108***	.098***
	(.032)	(.028)
Full × Year16 ( ${\beta }_2$ )	.154***	.111***
	(.039)	(.043)
Placebo × Year16			–.084
			(.060)
Baselines
Constant ( ${\alpha }_0$ )	.904***	.929***	.842***
	(.153)	(.017)	(.151)
Partial ( ${\alpha }_1$ )	–.006	.016
	(.020)	(.022)
Full ( ${\alpha }_2$ )	.182***	.239***
	(.035)	(.038)
Placebo			.096
			(.060)
Year16 ( ${\alpha }_3$ )	.193**	.234***	.249***
	(.027)	(.027)	(.046)
Controls
Supply ( ${\gamma }_1$ )	–.001**		–.001***
	(.0004)		(.0004)
Adspend ( ${\gamma }_2$ )	.0002		–.0003
	(.0005)		(.001)
Site FE ( ${\delta }_{\mathrm{i}}$ )	Yes	Yes	Yes
N	3058	3058	1234
Adjusted R2	.7785	.7716	.7565

*Significant at the 10% level.

**Significant at the 5% level. ***Significant at the 1% level.

Notes: Unit of analysis is the site-week, and regression is weighted by the number of impressions the site offered to the RTB market in that week. Standard errors, in parentheses, are clustered at the week level. Clustering by publisher or two-way clustering by publisher and week also resulted in significant effects of partial and full disclosure. Robustness checks (1) filtering out calendar week 9 in 2015 when supply spiked, and (2) filtering out the largest buyer showed substantively similar results (see Web Appendix D, Table W2).

All control variables show the expected signs: average revenue per impression is lower when there are more impressions available in the market (i.e., supply is higher) and higher when there is higher ad spending in the country where the exchange operates. A robustness check without the controls reported in Table 1, Column 2, shows effects that are attenuated without the controls but still significant and substantial. As another robustness check, we use an alternative approach to accounting for seasonality in supply and demand using monthly fixed effects. For this specification, the treatment effect estimates are €.105 under partial disclosure and €.115 under full disclosure and are significant at the 5% level. (The regression is reported in Table W4 in Web Appendix D.)

Parallel trends assumption

The difference-in-differences strategy for identifying the effect of disclosing context information to buyers relies mainly on the assumption that the seasonal pattern of revenue per impression is similar in 2015 and 2016 (Datta, Knox, and Bronnenberg 2018; Goldfarb and Tucker 2014). While display advertising markets can change quickly, we believe this is a reasonable assumption for this private exchange, which was quite stable. The sites in our data set are all well established with stable traffic and inventory. The contracts between the exchange and the publishers cannot be terminated easily or quickly. No sites dropped out of the private exchange during our observation window. Ad buyers, too, sign long-term contracts with the exchange, and buyer turnover in 2016 is fairly comparable to 2015 (see Table 2). We can also see from Figure W3 in Web Appendix C that the observed supply decline in 2015 was slightly more pronounced than in 2016, suggesting that the change in average revenue per impression in 2016 relative to 2015 is a conservative estimate of the causal effect of the policy change. Taken together, this evidence suggests that the parallel trend assumption is reasonable in this context.

Table 2.

Websites and Ad Buyers Participating in the Exchange Before and After Ad Context Disclosure.

	2015	2016 Prechange(Continuing + New)	2016 Postchange(Continuing + New)	Total
Sites	68	57 + 10 = 67	67 + 0 = 67	78
Buyers	504	322 + 197 = 519	429 + 138 = 567	710

We further assess the parallel trends assumption with a placebo test. We accomplish this by modifying the regression in Equation 2 to estimate the increase in average revenue per impression between 2015 and 2016 for each week beginning with week 7, relative to the baseline of weeks 2–6. Specifically, the regression we estimate is as follows:

$$\begin{array}{c}{\mathrm{y}}_{\mathrm{it}}={\alpha }_0+{\alpha }_1\mathrm{year}{16}_{\mathrm{t}}+\sum _{\mathrm{i}=7}^{28}{\alpha }_{\mathrm{i}}\mathrm{wee}{\mathrm{k}}_{\mathrm{it}}+\sum _{\mathrm{i}=7}^{28}{\beta }_{\mathrm{i}}\mathrm{wee}{\mathrm{k}}_{\mathrm{it}}\\ \times \mathrm{year}{16}_{\mathrm{t}}+{\gamma }_1\mathrm{suppl}{\mathrm{y}}_{\mathrm{t}}+{\gamma }_2\mathrm{adspen}{\mathrm{d}}_{\mathrm{t}}+{\delta }_{\mathrm{i}}+{\epsilon }_{\mathrm{it}},\end{array}$$

(3)

where $\mathrm{wee}{\mathrm{k}}_{\mathrm{it}}$ takes the value 1 when period $\mathrm{t}$ is the ${\mathrm{i}}^{\mathrm{th}}$ week of the year. The weekly year-over-year effects ( $\beta$ s) are plotted in Figure 2. Under the parallel trends assumption, the weekly effects estimates should be nonsignificant for weeks 7–12, which were prior to the policy change. The estimated effects for these pretreatment periods are mostly nonsignificant and below zero, suggesting that the parallel trends assumption holds. To see how the policy change affected average revenue per impression over time, we can examine the weekly effects for weeks 13–28. While we do not see significant weekly effects during the partial disclosure period, after full disclosure, there is a clear rise in the weekly effects, and most are significant. Note that these weekly effects have lower power than the pooled effects reported in Table 1. A stronger placebo test in Table 1, Column 3, compares prices in weeks 2–6 with weeks 7–12 (i.e., assuming ${\beta }_{\mathrm{i}}$ in Equation 3 are all equal) and also shows a nonsignificant placebo effect.

Figure 2.

Placebo test showing estimates of weekly year-over-year effects.

Thus, we find little evidence that the parallel trends assumption does not hold, suggesting that revenue per impression in the RTB auction increased when ad buyers were provided subdomain information for each impression. The policy change moved from disclosing nearly no information about where an ad would appear to disclosing the subdomain, which suggests that ad buyers value knowing the site where their ad will appear even if they do not know precisely what content it will be placed next to. Theory suggests that information disclosure will raise auction prices when (1) ad buyers have heterogeneous preferences for sites and (2) markets remain sufficiently thick for each site. In the next section, we provide additional evidence for this mechanism, but first we rule out an alternative explanation for the increase in average revenue per impression.

Increase in Revenue Is Not Due to Advertisers Shifting Budgets from Programmatic Direct

Prior to the policy change, ad buyers could not buy impressions at specific sites through the RTB market, but they could buy impressions at a specific site by making a programmatic direct deal with that site. These programmatic direct deals are often more expensive than prevailing prices in the RTB market. So, one explanation for why average revenue per impression increased is that ad buyers moved money out of programmatic direct deals and into the RTB market when it became possible to buy impressions at specific subdomains via RTB. However, we can rule this out by examining the proportion of impressions sold via programmatic direct and RTB in 2016. (The publishers were contractually obligated to sell all impressions through this exchange, so we observe how many impressions they sold through each sales channel.) Table 3 shows there is no obvious shift from programmatic direct to RTB associated with the policy change; the proportion of impressions sold via programmatic direct was nearly constant across the three periods. Rather than using the RTB market as a replacement for impressions buyers previously purchased via programmatic direct, it appears that they were adjusting their bids in the RTB market and increasing overall budgets for display advertising.

Table 3.

Average Monthly Impressions (in Millions) Sold Through RTB and Programmatic Direct in 2016 Before and After the Policy Change.

	No Disclosure		Partial Disclosure		Full Disclosure
	Imp.	%	Imp.	%	Imp.	%
Programmatic direct	1,320.1	59.4%	1,117.7	59.4%	1,144.8	59.4%
RTB	901.8	40.6%	763.5	40.6%	782.2	40.6%
Sold	828.8	37.3%	744.3	39.6%	741.7	38.5%
Unsold	73.0	3.3%	19.2	1.0%	40.5	2.1%

As prices rose, the total revenue in the RTB market also increased, which raises the question, “Where did this additional budget come from?” Figure 3 plots the spending for each buyer by month and shows that total spending increased in May, June, and July 2016 after full disclosure (but it did not increase in those same months in 2015). Further, the increased spending is not due to one or two buyers increasing their spend; most buyers increased their budgets in those months. We cannot say where the additional funds came from but can only speculate that buyers reallocated from other media (e.g., TV, search).

Figure 3.

Total spending in each month for each buyer.

Mechanism Checks

As discussed previously, information disclosure increases prices in an auction when a sufficient number of bidders have heterogeneous preferences for the items. In this section, we provide additional evidence that this mechanism is at play when context information was disclosed in this private exchange. Specifically, we show that (1) the market for each site was sufficiently thick after the policy change, (2) buyers appear to have been bidding higher for their preferred sites, and (3) most individual sites experienced an increase in revenue.

Market for Impressions at Each Site Remained Sufficiently Thick

A necessary condition for prices to rise in the auction is that the number of bidders for each impression does not fall too low. We do not observe the individual bids, so we cannot say how many bidders were bidding on each individual impression; however, the average number of daily buyers for each site gives us a proxy for the number of ad buyers who were submitting competitive bids for impressions at a given site. That said, we observed little change in the average number of daily buyers for each site (see Figure W4 and Table W5 in Web Appendix E). Even the site with the fewest average daily buyers had an average of more than eight buyers each day after the policy change. Thus, the market for impressions at each site remained competitive after the policy change.

Information Disclosure Resulted in Better Match Between Buyers and Sites

If buyers have heterogeneous preferences and are bidding more for their preferred sites when they have context information, then we would expect that buyers purchase from a smaller range of sites after full disclosure. To confirm that this happens, we computed the entropy of the buyers’ purchases across sites as

$$\mathrm{entropy}=\text{–}\sum _{\mathrm{i}\in {\mathrm{S}}_{\mathrm{j}}}{\mathrm{p}}_{\mathrm{ij}}\log ({\mathrm{p}}_{\mathrm{ij}}){\mathrm{p}}_{\mathrm{ij}}={\displaystyle \frac{\mathrm{spen}{\mathrm{d}}_{\mathrm{ij}}}{\sum _{\mathrm{i}\in {\mathrm{S}}_{\mathrm{j}}}\mathrm{spen}{\mathrm{d}}_{\mathrm{ij}}},}$$

(4)

where ${\mathrm{S}}_{\mathrm{j}}$ is the set of sites purchased by buyer $\mathrm{j}$ and $\mathrm{spen}{\mathrm{d}}_{\mathrm{ij}}$ is the amount that buyer $\mathrm{j}$ spent on site $\mathrm{i}$ before any disclosure (or after full disclosure). We find that the entropy dropped for the majority of buyers. The average before any disclosure is 1.035 and after is .9416. (As a benchmark, if every buyer purchased in proportion to the impressions offered, then the entropy would be 1.64; the entropy is 0 if the buyer purchases from one site only.) The average number of sites that each buyer purchases at also falls by .63 sites. While a t-test comparing the mean entropy is not significant, the direction of the change is consistent with the proposed mechanism, namely that buyers are bidding more for and concentrating their purchases at their preferred sites.

We also find that the right tail of the distribution of winning bids increases, consistent with the simulation results in Figure 1, Panel A. Figure W5 in Web Appendix F plots the distribution of average prices paid by each buyer for each site and shows a distinct increase in the proportion of impressions selling for €1.50–€2.00 CPM when ad buyers are provided with context information. We do not see a similar shift in the distribution for these same time periods in 2015.

Most Websites Experienced an Increase in Revenue per Impression

If preferences are heterogeneous and markets remain thick, then most sites should see a rise in prices. Figure 4 plots the estimated effect of full context disclosure for individual sites and shows that revenue per impression rose for the majority of sites. These site-specific estimates are based on a regression with the same specification as our main difference-in-differences analysis (Equation 2), except that sites are interacted with the treatment indicators. There are few orphaned sites; only one site shows a significant drop in revenue per impression. The effect of context disclosure for most sites is either neutral or positive, with a few sites that gain substantially. This is consistent with some ad buyers having strong preferences for impressions at a particular site (in addition to the information they already had about the user from the cookie).

Figure 4.

Estimated change in revenue for individual sites.

Taken together, our mechanism checks paint a picture that is consistent with the changes expected for information disclosure in a competitive auction in which buyers have heterogeneous preferences. If each buyer is raising their bids for a different subset of sites, then average daily buyers for each site should fall slightly (as shown in Table W5 in Web Appendix E), buyers should be purchasing at a smaller range of sites, the winning bids should have a longer right tail (Figure W5 in Web Appendix F), and prices should rise for most sites (Figure 4).

Partial Information Disclosure Advantages the Bidder with Information

As a final mechanism check, we investigate the effect of the policy change for the period in which only the buyer who used a particular DSP was provided with context information (partial disclosure). Theoretical research on information disclosure and bundling has focused on the cases like those illustrated in Figure 1, in which all bidders have access to the same information and product offerings (Eaton 2005; Hummel and McAfee 2016; Milgrom and Weber 1982; Tadelis and Zettelmeyer 2015). However, during a one-month period, the auction platform initially provided site information to one buyer only.

Because the theoretical literature on auctions is largely silent about such partial disclosure to a single party, we first review another simulation showing the effect of disclosure to a single bidder. Specifically, we assume that one buyer has the site information and will bid their valuation under disclosure, while the other 49 buyers bid their valuation without disclosure (see Equation 1). The outcome under partial disclosure depends on the site valuations for the buyer with information; we assume this buyer has low value for Site 1 ( ${\mathrm{s}}_{\mathrm{i}1}=\text{–}1.5$ ) and a high value for Site 2 ( ${\mathrm{s}}_{\mathrm{i}2}=1.5$ ). The simulation otherwise follows the same assumptions as the simulation for the full disclosure case. Figure 5 shows simulated winning prices for no disclosure versus disclosure to one bidder; the auctions won by the treated bidder are colored red. In this scenario, when information is disclosed to just one bidder, that buyer's bids for Site 2 go from $(1.5-1.5)/2=0$ to 1.5 on average, resulting in the treated bidder winning more often. In this example, the treated bidder wins 5.6% of impressions without disclosure and 38.8% when they have context information. Partial disclosure also raises the overall average selling price by €.111. Thus, when there is a large number of bidders who are heterogeneous in their valuations, partial disclosure can result in the treated bidder winning more and prices rising overall. We note, however, that the outcome of partial disclosure depends critically on the treated bidder's site valuations; in the extreme, disclosure to a bidder with very low valuations will not affect the market at all.

Figure 5.

Effect of information disclosure to a single bidder on auction prices.

To understand what happened in practice at the private exchange, we analyze the behavior of the bidder who obtained exclusive access to context information in the partial disclosure period relative to other bidders. This buyer was a DSP bidding on behalf of several advertisers. Prior to the policy change, the treated bidder was purchasing across a wide variety of sites and paying higher prices than the average of all other bidders, suggesting that bidder was purchasing impressions served to highly desirable users (cookies).

To obtain a counterfactual for what would have happened if this buyer had not gotten context information, we use a synthetic control analysis (Abadie, Diamond, and Hainmueller 2010; Abadie and Gardeazabal 2003; Tirunillai and Tellis 2017) to construct a counterfactual buyer that resembles the treated buyer during the pre-intervention period. Specifically, the counterfactual synthetic control is a convex combination of untreated buyers that matches the treated buyer as closely as possible on the weekly outcomes prior to treatment. In addition, the synthetic control also matches the treated buyer on several pretreatment covariates known as “predictors” in the synthetic control literature. The weights that define the control buyer are chosen such that the counterfactual buyer's weekly outcomes and predictors approximate the treated buyer's during the period prior to the policy change, week by week. Then, the weighted average of the posttreatment outcomes for the untreated bidders is used to estimate a counterfactual for how the treated buyer would have behaved if not provided with context information.

We use the synthetic control method to estimate the effect of partial disclosure on two outcomes: (1) the average winning price paid by the treated bidder in the partial disclosure period and (2) the number of impressions won. (Note that average winning price is the same as average revenue per impression but at the buyer-week level rather than the site-week level.) Technically, prices paid by buyers who did not have access to the context information may have been affected somewhat since they are participating in the auction with the treated bidder (i.e., the stable unit treatment value assumption required by the synthetic control is violated by the auction). However, this analysis provides some evidence that the treated bidder was changing bidding behavior during the partial disclosure period.

The synthetic control is constructed to match the treated buyer on the following predictors: (1) number of impressions won in each genre, (2) average price paid in each genre, (3) total number of impressions won, and (4) average price paid. We used genres to create the predictors, because ad buyers were able to target channels prior to the policy change. We created the predictors based on February–July 2015 as well as February–March 2016. In constructing the synthetic control, we filtered out daily observations for the control buyers with substantially higher prices (e.g., €10 CPM) or lower volumes (<500 impressions in a day). These unusual observations are likely due to highly targeted ad buys that are not representative of the types of prices paid by the treated buyer, and they represent only .8% of impressions. The core identifying assumption of the synthetic control is a form of unconfoundeness or ignorability; specifically, we assume the weekly pretreatment outcomes plus these pretreatment predictors represent the key ways in which the treated buyer is different than the untreated and fully account for the reasons why the treated buyer was selected for the test. The synthetic control was constructed separately for the average winning-price outcome and the impressions-won outcome.

Figure 6, Panel A, shows that the trajectory of the synthetic buyer for average winning price closely follows the treated buyer's price, which suggests that the synthetic buyer nicely mimics the treated buyer prior to policy change. (Table W6 in Web Appendix G compares the predictors for the synthetic buyer to the synthetic control, and those are also largely similar.) Consistent with the simulation in Figure 5, the additional context information caused a modest increase in the average winning price for the treated bidder, as can be seen by comparing the treated bidder to the synthetic control in Figure 6a after the policy change.

Figure 6.

Comparison of average winning price and number of impressions won for treated buyer versus synthetic control.

We repeated this analysis (including regenerating the synthetic control buyer) to estimate the effect of partial disclosure on the number of impressions won by the treated bidder. To the left of the vertical line, the number of impressions won is similar for the treated and synthetic control bidder, suggesting that the algorithm successfully found a comparable synthetic buyer. After the policy change, we see a large gap between the treated and synthetic buyer in the number of impressions won during partial disclosure, and this seems to revert back after full disclosure. While impressions won by the treated ad buyer increased substantially after the policy change, this number remained flat for the synthetic control. The statistical significance of this result is confirmed by placebo tests reported in Figure W7 in Web Appendix G.

Taken together, the synthetic control analysis shows that the treated bidder purchased substantially more impressions during the partial disclosure period at a slightly higher price. ⁸ This is consistent with the simulation reported in Figure 5 and helps explain the significant market-level price increase we see during the partial disclosure period in Table 1.

Heterogeneous Treatment Effects

Thus far, the analysis has focused on how providing context transparency affected the revenue per impressions for the market as a whole. The analysis suggests that buyers value context information and have heterogeneous preferences for sites, which leads to revenue gains for all sites. Next, we turn to the practical question of whether certain types of sites benefited more from this policy change. First, we show that sites with more buyers prior to the policy change (i.e., those with thicker markets) saw a greater increase in revenue after the policy change. Second, we show that smaller-volume, high-quality sites benefited the most from the policy change.

Sites with Thick Markets Experienced Greater Increases in Revenues

Theoretically, the effect of context disclosure on revenues should be moderated by the competitiveness of the market (see Figure 1, Panels A and B). This motivates an investigation of heterogeneous treatment effects across sites with stronger or weaker competition among buyers prior to the policy change. We create a dummy variable for sites that had fewer buyers (thin markets) prior to the policy change. We set the cutoff point at the first quartile, which is 28 average daily buyers. (Figure W4 in Web Appendix E shows the average daily buyers for each site prior to the policy change.) This variable captures market thinness before the policy change, and we assume that sites with thinner markets before the policy change were likely to have thinner markets after the change. Figure W4 indicates that this assumption is reasonable because market competitiveness was similar before and after treatment. This pretreatment covariate is also not contaminated by the treatment. Our simulations and the literature on auctions lead us to expect that sites with thick markets experience a greater increase in average revenue per impression because it is more likely that several buyers will value those impressions more when provided with context information.

We plot the estimated effects for sites with thin versus thick markets in Figure 7. The estimated increase in revenue per impression after full disclosure is €.154 for sites with thick markets, while it is €.004 for thin markets. ⁹ This increase is also small in relative terms: revenue per impression for sites with thin markets increased by .2% from the pre-disclosure period to the period with full disclosure in 2016. In comparison, the increase in revenue per impression for sites with thick markets was 14.0%. The finding that prices did not increase for sites with thinner markets is consistent with the literature on auctions and the simulations, thus serving as an additional mechanism check. Table W7 in Web Appendix H presents full regression results.

Figure 7.

Differential effect of context disclosure for sites with thin markets.

Small, Premium Sites Saw the Greatest Increase in Revenues

For our final analysis, we examine heterogeneous treatment effects for sites of different quality and size. While slightly less theoretically motivated, this analysis gives us an answer to the practical question, “Which sites benefit the most from context transparency?” Even though we study a private exchange with generally brand-safe sites, not all of them are—what advertisers may consider—the highest-quality advertising outlets. To categorize sites according to their quality, we asked three industry experts (a head of media planning, a media planner, and a trader in RTB auctions) to classify the sites into those that provide premium and nonpremium advertising environments, which is a common industry categorization. All experts were familiar with the sites and had purchased media from the ad exchange in the past. In addition, we categorized sites into small versus large according to the number of impressions they provided to the RTB market, ¹⁰ which leads to four categories of sites: premium large, premium small, nonpremium large, and nonpremium small.

To show the moderating effect of our size/quality measure on average revenue per impression, we estimated a regression interacting the treatment effect with indicators for these four size/quality groups. The estimated effects are plotted in Figure 8. Premium small sites benefit the most from the policy change, with an estimated effect more than three times that of the average (€.469). These sites typically serve a niche audience with a very specific topical interest (e.g., a website that provides content targeted at physicians) and when ad buyers know what they are bidding for, some of them value these sites far more than had they been in a bundle of unknown sites. Thus, the data suggest that small premium sites have the most to gain by increasing context transparency. Table W8 in Web Appendix H provides full regression results.

Figure 8.

Differential effect of context disclosure for sites of varying quality and size.

Discussion and Implications

This study investigates a specific change in context information, in which a private exchange moved from providing context information at the “channel” level to providing the subdomain associated with each ad impression in the bid request. Our analysis of the policy change shows that buyers value context information above and beyond user information and act on it as soon as context information is available. Consequently, average revenue per impression rose after the policy change relative to the previous year. As we illustrate with a simulation reported in Figure 1, Panel A, these effects are consistent with a scenario in which ad buyers prefer different sites. Such heterogeneous preferences lead to an increase in prices with context disclosure (Tadelis and Zettelmeyer 2015), so long as the market does not become thin (Hummel and McAfee 2016; Levin and Milgrom 2010). Under this scenario, ad buyers bid higher for the sites they each prefer after the policy change, and prices rise overall.

Several mechanism checks provide convergent evidence that this is indeed what happened: (1) markets thinned out slightly, suggesting fewer bidders were bidding on each site, but remained sufficiently thick for prices to rise; (2) entropy dropped for the majority of buyers, and the distribution of winning bids shifted to the right, meaning that winning bids were more dispersed when ad buyers were provided with context information; (3) most individual sites saw an increase in average revenue per impression; and (4) consistent with our simulations in Figure 5, partial disclosure of information to a single buyer resulted in this buyer winning more auctions and higher prices overall. We are also able to rule out some alternative explanations. For one, the change in average revenue per impression was not due to ad buyers shifting budgets from programmatic direct to RTB. In addition, ad buyer and site turnover was not responsible for the observed increase in revenue per impression. Our evidence therefore points toward buyers increasing their bids and budgets in response to the policy change.

Finally, to answer the practical question of whether certain types of sites benefited more from the policy change, we also investigate heterogeneous treatments effects: the increase in average revenue per impression was most pronounced for sites with a large number of average daily buyers prior to the policy change (again, consistent with our simulations in Figure 1, Panel A). From a more managerial standpoint, we show that small, premium sites benefited the most from context disclosure. When ad buyers know the ad context for the impression, some of them value these sites far more than had they been in a bundle of unknown sites. Yet, there are almost no losers of the policy change in this market; we mainly see sites that benefit more and sites that benefit less.

These changes in revenues were economically meaningful for this private exchange. The average weekly supply of impressions offered through RTB for a site in our sample is roughly 3.5 million (see Table W1), sold for an average CPM of €.88. According to our analysis, average CPM rises by about €.154 when all ad buyers are provided with context information (see Table 1, Column 1, Full disclosure × Year 16). Therefore, on average, each site generates an additional yearly revenue of (3.5 million/1,000) × €15.8 × 52 weeks = €28,756, and the overall revenue across all sites increases by about €28,756 × 57 = €1,639,092 per year. This ad exchange receives a 2.5% share of revenues, so the additional yearly revenue for the exchange is roughly €1,639,092 × 2.5% = €40,977. Note that this calculation depends on the figures obtained from our sample and is highly dependent on the scale of the ad exchange and the sites in its portfolio. In addition, our data does not allow us to investigate whether the effect might vanish in the future, yet we see no reason why the effect would not continue. Therefore, these rough calculations show that the policy change is associated with a substantial increase in revenue for the ad exchange and its publishers.

Although we demonstrate the impact of information disclosure on average revenue per impression utilizing a data set that consists of winning auction outcomes, we would be able to gain more insight into a buyer's valuations by investigating data on individual ad buyers’ bids for specific impressions. Instead, we only have data on the selling prices for the winning bids, which makes it difficult to determine precisely how the context information affected each bidder's valuation. In addition, if we had data on individual bids, we could better assess market competitiveness by counting the number of ad buyers bidding for each impression. Despite these limitations, our study represents an important step in empirically understanding the effect of context information in advertising auctions. Moreover, our results have important implications for different market players in digital marketing.

Implications

The extensibility of our findings to other markets depends critically on the sites and ad buyers that participate in the market. Some sites participating in the exchange we study provide niche content, but even the smaller websites were reputable brand-safe media outlets that had been vetted by the exchange. This mix of sites is typical of private exchanges, and so our results strongly suggest that private exchanges should provide information at the subdomain level in the bid request. Although we did not observe buyer entry due to the policy change within our observation window (three months postdisclosure), it is possible that buyers would want to join this specific private market if others do not provide the same level of context disclosure. Such buyer entry might increase auction revenue even further.

While our results translate fairly directly to other private exchanges, which represent a growing share of the display advertising market (eMarketer 2021a), it is more difficult to predict how context disclosure might affect open exchanges. Open markets attract a much wider range of sites, including low-traffic sites like blogs and low-quality advertising environments such as clickbait websites. When context is transparent, some publishers may become orphaned by ad buyers, as evidenced by the recent drop in demand for advertising at the alt-right site Breitbart.com in the United States (Bhatturai 2017) when buyers became aware that their programmatic ads were appearing on the site. Overall, we might find a larger share of sites with thin markets at the open exchanges and that prices for impressions at these sites might fall with more context disclosure (as we illustrated with simulations). This could be exacerbated if buyers find it difficult to evaluate low-volume sites and instead bid on a short list of sites they are familiar with.

Despite this, our data suggest that advertisers value context even when they have access to detailed user-targeting information. This confirms that the IAB’s efforts to increase context transparency in the open exchanges have had some impact on these markets. Our case study suggests that the IAB's ongoing efforts to increase context transparency on the open exchanges has benefited publishers that can maintain thick markets even when context information is available. If fraud continues to remain a problem (AdExchanger 2020b), large publishers may continue to move to private markets, which already process the majority of display advertising impressions as of 2020 (eMarketer 2021a).

Our analysis shows that context information provides ad buyers with additional information about the value of an impression above and beyond the rich cookie information available to European ad buyers in 2016. That is, context information is complementary to user information. If context information is also a partial substitute for user information, then context information will become even more important as ad buyers’ access to user information becomes more limited. Regulations like the General Data Protection Policy already limit the amount of user-level targeting that is possible. In addition, Google recently announced efforts to limit the use of third-party cookies in its Chrome browser by making “disable third-party cookies” the standard setting (AdExchanger 2020a, c). Thereby, the market share of browsers (including Mozilla, Safari, and Chrome) that inhibit tracking will grow to more than 80% in many countries in the next two years. Thus, user targeting will be less straightforward in the future, and industry experts predict that contextual targeting will become more relevant (AdExchanger 2019; Tan 2019).

Several empirical studies (Johnson, Shriver, and Du 2020; Marotta, Abhishek, and Acquisti 2019; Miller and Skiera 2017) investigate the effect of reduced access to user information on outcomes of ad auctions. However, in practice the information technology required to track user behavior and the organizational measures that ensure compliance with privacy regulations come at a cost and sometimes a prohibitive one, making it unattractive for publishers to enable cookie tracking. In contrast, providing ad buyers with more context information is nearly costless to publishers; we provide convergent evidence that doing so results in substantial revenue increases. This seems to be borne out in the industry as publishers such as The New York Times have moved to protect users by decreasing the amount of user information collected on their sites (Berjon 2020), while still providing context information. Other publishers in the Netherlands have eliminated user tracking all together (Edelman 2020). Interestingly, these publishers report that digital revenue rose when they eliminated user targeting and concentrated on context transparency.

While it is speculating beyond our data, we expect full URL disclosure to be very attractive to ad buyers who can then target ads against the specific content that an ad is placed next to. In fact, startups such as Grapeshot (acquired by Oracle in 2018), Peer39, and Leiki (acquired by DoubleVerify in 2018) have been building machine learning tools to help ad buyers determine which URLs represent the best advertising opportunities based on the text on the page. However, finer levels of context disclosure may lead to thin markets and deconflation for specific URLs or particular content topics. At the same time, it may encourage content creators and publishers to focus on content that is appealing to consumers and ad buyers rather than simply attracting an audience to generate impressions (Gal-Or, Geylani, and Yildirim 2012). We encourage future theoretical and empirical research that investigates how such fine-grained context disclosure will impact publishers’ incentives to produce content and the welfare of advertisers, publishers, and content consumers.

Supplemental Material

sj-pdf-1-mrj-10.1177_00222437211070219 - Supplemental material for Context Information Can Increase Revenue in Online Display Advertising Auctions: Evidence from a Policy Change

Supplemental material, sj-pdf-1-mrj-10.1177_00222437211070219 for Context Information Can Increase Revenue in Online Display Advertising Auctions: Evidence from a Policy Change by Sıla Ada, Nadia Abou Nabout and Elea McDonnell Feit in Journal of Marketing Research