Sage Journals: Discover world-class research

Abstract

This study combines data analysis with multilevel processing of visual communication to classify the visual frames of state-sponsored social media propaganda. We relied on Twitter’s Election Integrity data to sample five propaganda targets of the Internet Research Agency, including Russian and American partisan groups, and explored how their operations deviated from canonical state propaganda marked by symbols of national identity and heroic masculinity. The results show that the visual frames employed by the Internet Research Agency are designed to embody the vox populi with relatable, familiar, or attractive faces of ordinary people. The results also indicate that Internet Research Agency influence operations displayed cultural acuity and familiarity with the social identity of their targets, and that the visual narrative the agency crafted trafficked primarily in the tropes of regular guys or attractive young women. We discuss these findings and argue that state-sponsored propaganda has attuned to subcultural and visual affordances of social platforms.

Keywords

Disinformation Internet Research Agency propaganda Twitter visual framing

Introduction

Influence operations on social platforms have employed explicit textual and visual narratives that deviate from the grammar of state propaganda. Canonical war propaganda, purposefully tailored to boost morale and provide up-to-date information on the war effort, was tailored to traffic in the symbology of national identity, military prowess, and heroic masculinity—overt tropes embodied in proud wives waving well-equipped soldiers off to an honorable war. The balancing act of war propaganda typically entailed psychological tricks to boost morale while seeking to appear as truthful as possible (Welch, 2016). War visuals would go hand in hand with the war rhetoric, including the strategic objective of capturing territory and defeating the enemy on the ground. The absence of direct military action that marked the Cold War period gave way to propaganda efforts dedicated to sowing distrust, as exemplified by the KGB’s disinformation campaign framing HIV as an American biological weapon (Selvage, 2019).

Pomerantsev (2019) argued that propaganda has always accompanied war, usually as a handmaiden to attrition warfare. But the mediatization of traditional institutions (Strömbäck, 2008), of late by social media platforms, has effectively flipped the equation. Information warfare takes central stage and military operations, when unavoidable, follow large-scale influence operations embracing a range of measures performed by state or non-state actors aimed at influencing public opinion (Arif et al., 2018). This inversion requires a fundamentally different strategy, as influence operations are kept relatively separate from overt belligerent efforts. The target of propaganda efforts is no longer restricted to geography nor populated by military personnel (Steel, 2015), occupying instead communication networks now populated by Twitterbots, false accounts, sockpuppets, trolls, and paid influencers (Bastos and Mercea, 2019; Benkler et al., 2018).

In this article, we consider whether tropes of heroic resistance or military endurance, typical of canonical state propaganda, track in a context of information but not attrition warfare. Visual propaganda research has found that emotional and human-interest frames tend to garner the largest amount of interest on social media (Seo, 2019). To this end, we probe whether social media propaganda seeks to embody the vox populi by employing relatable, familiar, or alternatively attractive faces of ordinary people―that which in the US context is referred to as “the man on the street” (MOTS). We refer to such familiar or attractive faces as “the guy next door” and the “implausibly attractive young woman.” The former is embodied by an unassuming yet average male lacking overt tropes of hegemonic masculinity like strength and dominance commonly embraced by Russian propaganda (Romanets, 2017), while the latter is characterized by tangible exploitation of the female body for maximum erotic impact and objectification (Davis, 2018).

In the following, we analyze the visual frames in profile images of Twitter accounts operated by the Internet Research Agency (IRA), a company specialized in online influence operations based in St. Petersburg, Russia. With profile pictures lending credibility to user’s communications (Morris et al., 2012), as well as seizing attention, eliciting emotions (Rose, 2016), and galvanizing engagement (Seo, 2014), we posit that IRA seized the currency of profile images to attain social embeddedness. Its curated impersonation points to a strategic shift from the visual representation of the enemy through stereotypical images in war propaganda (Welch, 2013) to self-presentation as ordinary citizens.

Previous work

Framing (Entman, 1993) and strategic narrative studies (Miskimmon et al., 2014) have established that the way the press depicts issues otherwise removed from citizens’ own experience has a powerful effect on how reality is perceived. In the absence of contradicting personal experience, individuals are likely to rely on media framings resulting from complex interactions between journalistic norms and economic considerations, with far-reaching social and cultural implications (De Vreese, 2005). By adapting framing contestation to our current information ecosystem, this framework has in some measure offered a testable alternative to the established, but arguably more limited, framework of propaganda studies that traditionally uncovered systemic biases entrenched in modern mass media (Herman and Chomsky, 2010). The contemporary disinformation ecosystem can thus be explored more granularly by repurposing the framing framework to account for instances of digital network framing.

A seminal suggestion to this effect has been offered by Entman and Usher (2018), who sought to revisit the cascade network model of frame activation against the backdrop of a mature social media ecosystem. Influence operations could thus be traced through strategies of persuasion and impression management, typically explored in framing studies and strategic narratives. This conceptual framework defines narratives as dynamic entities circulating in digital network frames that connect to structural grievances. The centralized topology of mainstream media is attached to decentralized social platforms, which may subtly and elusively alter the original frame. Frames employed by the press may thus undergo subtle reframing with accounts that source information to users subsequently redistributing this information to their followers. The resulting topology is that of a multistep flow of information diffusion (Hilbert et al., 2016), which starts with mainstream media reporting and may return to the press after bouncing around different groups of social media users that can skew the frame toward polarization.

This repositioning of media institutions in public communication was advanced by multiple related developments, including the growing multiplication of the sources circulating symbols of social life, mass participation in the creation of online content (Chadwick, 2013), and the rise of citizen journalism as an alternative to dominant news organizations (Papacharissi, 2015). In the context of the 2011 popular uprisings of the Arab Spring in Egypt, Papacharissi (2015) characterizes such reporting as attempts to communicate an accurate and authentic version of events in response to the censorship and manipulation perpetuated by that country’s then authoritarian regime. Such digital storytelling enabled the bottom-up rise of crowd-sourced elites elevated by networked publics to a position of prominence.

One notable logic underpinning recent approaches to influencing public perceptions consists of painting reality in frames familiar to selected groups on social media (Chadwick et al., 2018; Del Vicario et al., 2016; Ferrara, 2017; Siegel and Tucker, 2018). By means of carefully edited visual content, the so-called Islamic State terrorist organization sought to both polarize and to attract young supporters to its theocracy by giving prominence to its extreme violence while promising thrills and individual stardom to new recruits (Andersen and Sandberg, 2020; Siegel and Tucker, 2018). Conversely, in the course of the 2016 US elections, the Russian Internet Research Agency attempted to accentuate socio-political cleavages among citizens and voters through hyperpartisan messaging (Bastos and Farkas, 2019), so as to similarly polarize but also disengage voters from the electoral process. Among the many techniques they employed—discussed below and in greater detail elsewhere (Mueller, 2019)—a key visual prop it utilized consisted of misappropriating profile pictures (Benkler et al., 2018).

IRA activity drew scholarly attention particularly to its expertly pitched right-wing framings and beliefs in the course of the 2016 US elections (Benkler et al., 2018). The framing cultivated by the agency, a clear departure from mainstream propaganda, is best unpacked against the backdrop of countercultural theory. Historically, normative interpretations of countercultures foregrounded their digression from a reference-category embodied by mainstream culture. However, alternative interpretations posit that subcultures exhibit a style, a key to unlocking their meaning (Gelder, 2005). Illustratively, Andersen and Sandberg (2020) highlight that cultural artifacts at the heart of violent Jihadist propaganda are designed to seduce its audience. In their words, “the so-called ‘jihadi cool’ creates a seductive subculture, attracting young people to participate in global Islamist terrorism because it is considered cool and exciting” (Andersen and Sandberg, 2020: 13). Similarly, the IRA sought to cultivate a larger range of distinct sections of the US public, extending from left-wing, to Black, LGBT, or right-wing, anti-immigration and nationalist activists (Freelon et al., 2020). In this article, we explore the use of profile pictures as resonant cultural artifacts harnessed by the IRA to achieve such segmentation.

Research aim and questions

The Twitter profiles operated by the Internet Research Agency combined human-operated and automated accounts to perform as de facto crowd-sourced elites that occupied central positions in retweet networks throughout 2016 (Badawy et al., 2019). IRA operatives engaged in trolling by intervening in an online conversation to spark a reaction among readers at the behest and for the benefit of their patron. Leveraging black, white, and gray propaganda, IRA activity likewise testified to the aim to amplify a resonant discourse among targeted subcultures through the impersonation of its membership, including Black Lives Matter activists (Bastos and Farkas, 2019; Freelon et al., 2020; Stewart et al., 2017). In contrast to traditional public influence propaganda, marked by mass-produced, mass-distributed campaigns, the IRA activity was remarkable in its ability to identify and appropriate the subculture or social identity of relatively narrowly defined targets (Jensen, 2018; Kim et al., 2018; Linvill and Warren, 2020).

Previous single-case research on the IRA intervention in the Black Lives Matter movement (Arif et al., 2018) pointed to a concerted attempt to inject content on both sides of framing contests enveloping the movement. A framing contest ensues as action-oriented interpretations constructed by individuals or groups to mobilize support for and/or demobilize opposition to their efforts are met by neutralizing forces engaging in counter-framing (Benford and Snow, 2000). In the case of #BlackLivesMatter, framing contests on Twitter pitted activist interpretations of protest as a response to the structural injustice perpetuated by law enforcement and government actions victimizing African Americans against, on the other hand, the counter-framing of the movement’s protests as a usurpation of the rule of law attested, for instance, by the shooting of police officers (Stewart et al., 2017). The IRA accounts profiled in the analysis of the #BlackLivesMatter framing contest by Arif et al. (2018) exploited this divide using profile information and the retweet functionality to exhibit associations with either side of the contest.

IRA profiles were analyzed as impersonations of imagined audiences (Arif et al., 2018), which on Twitter are an amalgamation of direct connections with one’s followers and indirect links with the larger Twitter social graph (Marwick and Boyd, 2011) through, for example, @-mentions or retweets. The personas taken on by those profiles were strategic constructions tailored to their partisan clusters, ultimately able to portray an interesting and authentic self (Arif et al., 2018). They enlisted their profile information and pictures in the service of that apparent goal, self-describing themselves as either African American or, conversely, white male or female. They displayed cultural acuity alongside a sensitivity to the social identity of either side as supporters or detractors of the #BlackLivesMatter movement on Twitter. They were thus able to demonstrate an appreciation of the values, symbols, and views attached to their respective group membership (McGarty et al., 2014; Tajfel, 1978). This was achieved through the embracement of relevant subcultural styles articulated linguistically as well as visually through a partisan-political alignment and news consumption. The careful social curation driving these accounts prevented them from crossing the divide, instead embedding themselves firmly on their side of the fault-line (Arif et al., 2018).

This meticulous curation projected the personification of ordinary citizens at the level of individual accounts. An earlier study of one of the most prominent IRA Twitter accounts (Xia et al., 2019) proposed that it engaged strategically in self-presentation, thereby resorting to social interaction as a stage where one performs an impression of oneself that resonates with an imagined audience (Goffman, 1959). Concentrating on its self-presentation, the authors were able to undertake an in-depth scrutiny of the language used by Jenna Abrams—the moniker adopted by the IRA account—specifically to embody an ordinary albeit opinionated Twitter user. Their interest was thus in how Jenna Abrams constructed an authentic and culturally resonant persona. Yet, while that examination was able to tease out the language used in the performance of Abram’s persona of an ordinary yet passionate American girl (Xia et al., 2019), it did not probe the visual elements of her self-presentation.

Visual framing research, on the other hand, is largely focused on how the press curates and selects photos to cover international conflicts or electoral politics. Parry (2010) contrasted how two British newspapers framed the war using press photographs to represent the Israel-Lebanon conflict. Brantner et al. (2011) investigated the effects of visual framing on the emotional response to different versions of news stories covering the Gaza conflict in 2009. A larger body of scholarship has expanded framing theory and second-level agenda setting by inspecting the visual presentation of presidential candidates in electoral campaigns, including markers of nonverbal behavior (Coleman and Banning, 2006; Grabe and Bucy, 2009). Beyond the studies reviewed hitherto, literature exploring the visual framing of propaganda campaigns and influence operations, particularly within an information ecosystem marked by digital and networking technologies, is still forthcoming.

In the following, we take up this task and explore the visual tropes found in IRA Twitter accounts by employing open, manual coding of the visual content (Rose, 2016) and statistical analysis of aggregate compositional, chromatic, as well as emotional or social patterns (Fyfe and Ge, 2018). Building on the above studies, we hypothesize that the IRA profiles embodied real and ordinary citizens (Arif et al., 2018), displaying cultural acuity and a sensitivity to the social identity of their targets, not only in language use (Miller, 2019), but also in their visual self-presentation with profile images (Xia et al., 2019). As such, we seek to scale up the initial assessment of cognate studies by way of a systematic examination of profile images deployed by the Internet Research Agency across multiple fault-line issues it targeted in its operations.

Data and methods

We leverage the data set released by Twitter’s Elections Integrity initiative that reportedly identified and ultimately removed the universe of false accounts, Twitterbots, and sockpuppets operated by the Internet Research Agency (Elections Integrity, 2018). The first release of the data included 2752 accounts the company recognized were operated by the IRA (United States Senate Committee, 2017). This list was expanded in early 2018 to include 3814 IRA-linked accounts (Twitter, 2018). The data set was updated again to include 3836 Twitter accounts, along with the totality of images, videos, and profile images used or created by these accounts. Although some accounts were created as far back as 2009, most were created in 2013 (29%) and 2014 (40%), with a smaller share of accounts created in 2015 (12%) and 2017 (12%). These accounts amassed a combined following of over six million users (N = 6,386,749). This is the canonical IRA data set we analyze in this study.

Informed by previous research that identified IRA propaganda targets (Bastos and Farkas, 2019; Farkas and Bastos, 2018), we applied regular expression to profile descriptions and sampled five distinct groups: Muscovites, Black Lives Matter activists, Conservatives, Trump supporters, and Christians. We identified Russian accounts distributing domestic, white propaganda by querying the database for users that self-reported “Москва” as their location (n = 206). Next, we identified black propaganda accounts targeting Conservatives, Trump, Christian, and Black Lives Matter supporters. We proceeded by sampling profiles with the keyword “Conservative” in the profile description (n = 121) and repeated the process (avoiding selecting users more than once) for the keywords “Trump” (n = 134), “God,” “Christ,” or “Jesus” (n = 154), and finally “Black” or “BLM” (n = 79). After removing false positives and disambiguating profiles with multiple matches, the sampled data returned 75 Black Lives Matter activists, 61 Christians, 104 Conservatives, 201 Muscovites, and 57 Trump supporter profiles. Figure 1 provides an overview of the profiled groups.

Figure 1.

From left to right, Twitter profile pictures of Trump supporters, Christians, Conservatives, Black Lives Matter activists, and self-identified Muscovites.

The multimodal information curated by IRA accounts was unpacked by image-captioning the Twitter profile images (Martinec and Salway, 2005), so that their visual content was translated to text for downstream analysis (see Supplemental material). We did not find any profile with no image or “Twitter eggs” (the default image for profile picture), but we identified stock photos of American actress Kaley Cuoco (Figure 1: first row, third column; third row, fourth column) and pop star Ariana Grande (Figure 1: fourth row, fifth column). A visual inspection of the profile images, vis-à-vis the demographic group they targeted, shows that the semantic relationship in IRA multimodal content is of complementarity as opposed to illustration, when the image supports the text, or anchorage, when the text supports the image (stat = 0). Instead of anchoring the textual information, the profile images are conceivably subordinated to the profile description and the content posted by these accounts. Yet, the overwhelming presence of young individuals (Figure 1) constitutes new information not readily available in the profile description and conceivably outside the scope of their political messaging. As such, the interplay between profile picture and profile description is one where both modalities contribute individually to the message, as each part contains details that the other does not (Otto et al., 2019).

We collated the profile images of each of the five groups to pursue exploratory visual data analysis (Fyfe and Ge, 2018; Rogers, 2013). This pixel-level analysis can uncover patterns in visual media to produce a largescale representation and offer a broad-based “situational awareness” that is otherwise inaccessible (Manovich, 2020). We organized the profile images according to brightness and the color saturation of each image through an ImageMeasure2 plug-in for ImageJ (Rasband, 2012). Two members of the team, with expertise in professional photography, subsequently coded each of the images to identify features in the composition, framing, and photographic techniques employed to produce the profile image. The coding procedure was informed by Rose (2016) and included the categories “number of people in the photo,” “race,” “gender,” “emotion,” “clothing,” and “background scenario” (see Supplemental material: Coding Variables). These observations allowed us to generate the following categories accounting for the social identity of the portrayed individuals: gender (male, female, or group), race (Black, White, Asian, or other), and emotion (happy, friendly, tense, sad, or angry). It also allowed us to generate the following technical categories identifying the framing of individuals: composition (close-up, medium shot, three-quarter shot, or long shot), scenario (indoor vs outdoor), and camera angle (high, neutral, or low angle). These technical categories drawn from visual framing studies and allow for probing whether the profile composition seeks to depict regular individuals instead of canonical tropes of state propaganda, where low angles and three-quarter shots are traditionally employed to portray the subject in a position of symbolic power over the viewer (Mandell and Shaw, 1973).

In addition to explicit categories, our inductive coding rendered a key category devised to identify whether the profile photo explored sensual undertones, a subtle feature and proxy for attractiveness present in many profiles featuring women or sexualizing the female body. Sensuality has been defined as the involvement of multisensory surfaces, curves, and textures that evoke femininity, corporeality, and eroticism (Pritchard and Morgan, 2011) in contrast to cognitive or affective associations (Cho et al., 2015; Holbrook and Hirschman, 1982). We relied upon the inferential definition employed in product management literature, as well as research cautioning against a clear definition of sensuality (Hofmeester et al., 1996), to establish the binary category “sensual” inductively based on prominent features of the profile photos of the accounts. These categories provide latitude for probing whether the profile composition portrays regular individuals instead of propaganda tropes trafficking on symbols of power and heroism.

The final key category devised during the coding process refers to the extent to which professional photographic techniques were employed to produce the image. Professional photographic work generates aesthetic elements that differ substantively from amateur photography, notwithstanding recent advances in smartphone cameras, particularly the expert use of hue, brightness, and light controls (Leaver et al., 2020). We refer to this category as Quality to distinguish between professional-looking photos from those that are discernibly produced by users themselves (though both may have been misappropriated by IRA operatives). We reviewed the coding procedures repeatedly until we found the devised categories to be exhaustive and exclusive. Intercoder reliability was calculated on a random subsample of 90 images, with Krippendorff’s alpha = 1 for race, male, female, and number of persons; alpha = .926 for background scenario; alpha = .828 for camera angle and quality; alpha = .75 for sensual; and alpha = .716 for the category emotion. Table 1 offers an overview of the coded categories.

Table 1.

Coding categories applied to the Twitter profile image of Internet Research Agency accounts impersonating Black Lives Matter, Christians, Conservatives, Muscovites, and Trump supporters.

File	2625497896_profile.png	2625539381_profile.png	2671070290_profile.png
Description	White man swimming. He plays with a dolphin and smiles.	Young white couple hugging outdoors. Man wears glasses. Woman smiles. Four other men in the background.	Young white man smiling, wearing glasses, and a black cap
Race	W	W	W
Composition	Close-up	Close-up	Close-up
Angle	High angle	Neutral	Neutral
Quality	Professional	Amateur	Amateur
In/outdoor (I/O)	O	O	I
People	1	2	1
Gender (male/female/group)	M	G	M
Sensual (true/false)	F	F	F
Emotion	Happy	Happy	Happy

IO: in/outdoor.

We further inspected the images by relying on the bwimage R package (Biagolini and Macedo, 2019) to identify patterns in the groups of profile images. The package analyzes digital images as cells holding information about the intensity of color channels red, green, blue, and alpha (representing transparency). We transform the color intensity data to pure black & white data, rendered into a binary matrix with the threshold_color function set to 50%, which is the function controlling the averaged intensity of red, green, and blue. Computer-supported analysis of digital images is commonly employed in the field of ecology to estimate vertical vegetation complexity and canopy openness (Díaz and Lencinas, 2018). Given the aims of this study, we adapted functions for detecting vegetation complexity to establish metrics of profile image complexity against a white background (Zehm et al., 2003). This method allows for systematically extracting information from images and processing the images in user profile at scale.

We expect the triangulation of these methods to partially overcome the limitation underpinning the visual content analysis, which allows for greater reflexivity and contextualization of the data, but whose renderings of visual representation are difficult to scale due to the polysemic nature of visual information (Bock et al., 2012). We therefore sought not only to code the images by looking and counting visual cues, but also to compile the contextual information with descriptors of the profile photos to account for the context in which the image was produced. Although challenging, the integration of qualitative and quantitative processing of visual content allows for multilevel processing of visual communication (Margolis and Pauwels, 2011) that can leverage the generalizability characteristic of quantitative research with the vantage point inherent to visual research, namely greater reflexivity and higher contextualization of the data.

Results

We proceed by processing the images in a computer program and extracting metadata such as color, brightness, and saturation to order the images on a Cartesian plane. The visualizations confirmed patterns evidenced in the data, with Figure 2 showing the predominance of blonde and professionally shot photos featuring clusters of implausibly attractive young women in the target groups of Christians, Conservatives, and Trump supporters. Indeed, a visual inspection shows that sexual and sensual exploitation of the female body is largely restricted to Christians, Conservatives, and Trump supporters, particularly the latter (x = .23, .22, and .32, compared with x = .07, and .13 for Muscovites and Black Lives Matter activists). The Muscovite images are marked by low color saturation and low brightness, in one quadrant, and high color saturation when the profile picture features subjects outdoors. This group comprises compositions that speak to notions of virility, control, and natural life producing greater brightness latitude and color saturation. Individuals in the Muscovite group often exhibit dark hair and a black background, or dark clothes, thereby shifting most of the images to the lower-left quadrant.

Figure 2.

(a) Plot of 221 profile images combined for Christian, Conservatives, and Trump supporter groups, ordered by brightness (X axis) and color saturation (Y axis). Higher definition available at: https://doi.org/10.6084/m9.figshare.14798571.v1 and (b) plot of 201 profile images in the Muscovite group, ordered by brightness (X axis) and color saturation (Y axis). Higher definition available at: https://doi.org/10.6084/m9.figshare.14798616.v1.

The use of indoor and outdoor images in association with the use of professionally produced instead of homemade photos offers additional insights. Profile images in the Moscow group are more likely to prioritize outdoor areas and often feature blue skies, while US-targeted Twitter profiles are more likely to feature indoor areas with a clear background. Whereas Russian profiles often feature adventurous backgrounds populated by masculine individuals, US-targeted Twitter profiles are more likely to feature implausibly attractive young women shot against a soft, balanced lighting and featuring professional makeup and composition in the image. While the appearance of these women would be plausible when encountered at random, they are not so when uniformly grouped as Trump supporters, a section of the electorate whose demographics deviate from the compositional motifs consistently employed in IRA profiles.

The chromatic variance displayed across groups, along with the topical emphasis on regular individuals, shows IRA’s inclination to pick average-looking men but unordinary attractive young women. The former embody unassuming yet average individuals lacking overt features of masculine tropes like strength and dominance, the typical hegemonic masculinity embraced by Russian propaganda as a trope for the country’s self-assured resurgence under the Putin regime (Romanets, 2017). Conversely, the latter are characterized by tangible exploitation of shapes and textures that evoke femininity and eroticism, so that women are displayed with their appearance coded for maximum erotic impact and objectification (Davis, 2018; Mulvey, 1975).

The profile images in the Russian cohort are marked by unambiguous symbols and aspects associated with virility and adventure, while rightwing Americans are targeted with a composite of average young men and sensually crafted images of young women typical of soft advertisement employed by the cosmetic industry (Xie and Zhang, 2013). The contrast within American targeted audiences is also revealing. The profile images selected for Conservatives, Trump supporters, and Christians present higher brightness compared with the images selected for the Black Lives Matter group, with the latter presenting a wider range of saturation and brightness.

Figure 3(a) shows the Conservative group on a Cartesian plane, thereby highlighting how compositional elements relate within the set. The agglutination of the brightest images and the low color saturation in the lower-right quadrant of the visualization results from a large set of close-ups, especially of fair-haired women. The images are bright and suggest artificial illumination typical of professional photography in image capture or post-production editing. Figure 3(b), in opposition, comprises mostly close-ups with substantively distinct coordinates, from the highest contrast to the background in the images. The photographic elements are not standardized, with higher dispersion and illumination, which results in many images with a greater spectrum of color saturation.

Figure 3.

(a) A total of 104 profile images of Conservatives, ordered by brightness (X axis) and color saturation (Y axis). Higher definition available at: https://doi.org/10.6084/m9.figshare.14798559.v1 and (b) 75 profile images of Black Lives Matter, ordered by brightness (X axis) and color saturation (Y axis). Higher definition available at: https://doi.org/10.6084/m9.figshare.14798499.v1.

The visualization of Trump supporters in Figure 4(a) is interesting because of the chromatic splintering, with profiles in the lower-right quadrant concentrating most profile images of blond women with soft lighting. The lower-left quadrant, on the other hand, shows dark-haired women along a few outliers. Figure 4(b) shows the Christian subgroup, with profile images largely occupying two sections in the center of the plot. The cluster on the right includes profile images with lower contrast and low color saturation, while the left cluster displays profile images with greater brightness and darker elements, such as black hair or dark clothing.

Figure 4.

(a) In all, 57 profile images of Trump supporters, ordered by brightness (X axis) and color saturation (Y axis). Higher definition available at: https://doi.org/10.6084/m9.figshare.14798652.v1 and (b) 61 profile images of Christians, ordered by brightness (X axis) and color saturation (Y axis). Higher definition available at: https://doi.org/10.6084/m9.figshare.14798553.v1.

The visual analysis of the images is consistent with the aggregate results gained from our manual coding of the material, but the quantitative analysis of the coded material provides greater granularity to the assessment of the compositional tropes employed by the IRA. First, the indoor vs outdoor polarity is marked by a cleavage, with Black Lives Matter activists, Trump supporters, Christians, and Conservatives appearing regularly indoors, while Muscovites appear habitually outdoors (x = .84, .84, .73, and .78, respectively, compared with x = .55). Male individuals are more likely to appear in the Black Lives Matter and Moscow groups, but females dominate the profile composition of Christians, Conservatives, and particularly Trump supporters (x = .55 and .52, respectively, compared with x = .30, .16, and .11). Neutral angles are predominant across groups, but there are also important differences. Black Lives Matter activists and Christians are depicted with high angles (.41 and .36, respectively). Low angles are more common among Muscovites and Trump supporters (.15 and .18, respectively, against an average of .6 for the other groups).

Emotions are surprisingly balanced across groups, with expressions of happiness and tenderness being prevalent, but there are also noticeable differences. Black Lives Matter activists are more likely to appear tense (x = .41 compared with $\dot{x}$ = .12). Christians, Muscovites, and Trump supporters are mostly tender ( $\dot{x}$ = .42), while Conservatives are overwhelmingly happy (x = .55). Racial diversity is minimal, with whiteness prevailing across Christians, Muscovites, and Trump supporters and Black Lives Matter activists being largely black ( $\dot{x}$ = .90 for both sets of groups). Selfies and self-portraits prevail in the profiles of Black Lives Matter activists and Conservatives ( $\dot{x}$ = .66), while Christians and Trump supporters are mostly framed with regular close-ups ( $\dot{x}$ = .85). While these groups resorted mostly to close-ups, which account for ¾ of the profile photos, Muscovites are mostly framed in close to medium angles typical of a three-quarter shot ( $\dot{x}$ = .95).

The compositional undertones employed across groups also offer interesting contrasts. Amateur profile photos are predominant across the groups, except for Trump supporters and Christian profiles. For Trump supporters, there is a strong divide between professional and non-professional profile photos. Nearly one-third of Christians also employed professional photos in their profiles. These differences are compounded by the number of individuals featured in the profile photo: Trump supporters, Muscovites, and Black Lives Matter activists are groups that consistently show a single headshot. The profile of Trump supporters is particularly prone to featuring implausibly attractive young women, likely photographed by professionals, while Muscovites and Black Lives Matter activists present a greater ratio of regular individuals. Christians and Conservatives average 1.15 people per profile, compared with 0.99 for Muscovites, as photos featuring groups of people are extensive in the first cohort.

This is consistent with the word frequency extracted from the manual description of the images. Muscovites and Conservatives are more likely to feature men, while Trump supporters and Christians favor women (again speaking to the gender unevenness in profile compositions). White, young, and girl are the three central terms used to describe the profile of Conservatives, compared with man and attire for Black Lives Matter activists and man and smile for Muscovites. Conversely, Trump supporters and Christians were largely coded with the descriptors woman, smile, and endearing. In short, the word frequency extracted from the description summarizes Trump supporters and Christian profiles as “beautiful woman smiling,” while Black Lives Matter profiles are broadly described as “man wearing” and the descriptor of a specific attire (see Supplemental material: Categories Summary).

We triangulated the visual inspection of the data with quantitative processing of the images using the bwimage R package (Biagolini and Macedo, 2019). This approach allowed us to identify the extent to which the average distribution of colors in each of the five groups is less than or equal to the threshold. We applied the function denseness_total to obtain information on color density, a logical test to calculate the threshold below which 75% of the canvas occurs. The automated image processing is consistent with the results of our qualitative coding of the images, as the average variation found in Trump supporters and Christians are similar ( $\dot{x}$ = .6241 and $\dot{x}$ = .6014) and at odds with those found in the Black Lives Matter activists ( $\dot{x}$ = .5652), Conservatives ( $\dot{x}$ = .4815), and Muscovite groups ( $\dot{x}$ = .5276). These differences remain even at the threshold level of 80%, with Trump supporters and Christians again showing similar scores ( $\dot{x}$ = .9229 and $\dot{x}$ = .9132) compared with Black Lives Matter activists ( $\dot{x}$ = .9011), Conservatives ( $\dot{x}$ = .8648), and Muscovite groups ( $\dot{x}$ = .8729).

Finally, we have considered the ethical dilemmas in displaying profile pictures that might have been taken from real people, as the IRA may have conceivably misappropriated profile images of real users, against the also possible scenario that these images may have been taken from stock photos of celebrities and models. In the end, our concerns were offset by the realization that the images explored in this study have been recontextualized to such an extent that they are detached from potentially existing personas, reflecting instead the profiles manufactured by the IRA to portray fabricated Russian, American, or European Twitter identifies.

Conclusion

The results presented in this study highlight common visual tropes employed by state-sponsored propaganda that deviate from the canonical tropes of war propaganda, revolving around ideas of heroism, adventure, and resistance. The IRA propaganda campaign represents a point of departure by resorting instead to ordinary individuals but unordinary attractive young women, often portrayed with sophisticated makeup in professionally shot photos against a soft, balanced lighting. The chromatic and compositional variance identified in the profile photos shows that US conservative groups are targeted with a composite of average young men and sensually crafted images of young women typical of soft advertisements employed by the cosmetic industry. As such, the weaponization of the guy next door is counterintuitive given the lasting reliance of state propaganda on tropes of military prowess.

The profile images in the Russian cohort, on the other hand, are marked by tropes of virility and domination. Emphasis is shifted from ordinary-looking male individuals to compositional undertones emphasizing power and adventure. Black Lives Matter activists are framed with another compositional frame altogether; depicted largely with high angles projecting tense expressions in a single headshot. These compositional choices do not merely depict gender inequalities and gendered stereotypes supporting traditional notions of femininity and masculinity. More than merely reflecting entrenched inequalities, the compositional tropes explored by the IRA appear to celebrate racial and gender discrimination.

The results are also consistent with the notion that state-sponsored propaganda on social media exploits existing social tensions by framing contentious issues in a familiar but partisan light. IRA activity expertly pitched to right-wing framings and beliefs, but the agency also targeted distinct sections of the US electorate, including liberal subgroups, chief among which were Black Lives Matter activists (Freelon et al., 2020). Religious, racial, and cultural groups are targeted with disguised social media accounts exploring biases and prejudices, but also perceived markers of membership of these groups. While the profile pictures weaponized by the IRA may have been misappropriated from real-world users, there is nonetheless an unordinary investment in editing, along with the careful curation sustaining the aggregated content.

In contrast to mass-produced, mass-distributed traditional propaganda, IRA digital disinformation specifically targeted narrowly defined demographic, social, and political groups. Yet, the multitude of targeted social groups, and the broad ideological spectrum of their interests, suggest that the target of the propaganda efforts is not the social groups the campaign chooses to personify, be they Conservatives, Democrats, Black Lives Matter activists, or Russian nationals. The target of this influence operation was social cohesion itself, which was either supported (as in the case of domestic Russian operations) or attacked by deepening social distrust with overheated rhetoric reminiscent of past campaigns of disinformation (Selvage, 2019). The emphasis and perhaps exaggeration placed on existing social divisions are leveraged to amplify threats and ultimately erode social trust and social cohesion. Within this line of thought, shared beliefs among the citizenry can be undermined if the population ceases to believe they are a part of a common community that encourages reciprocal trust.

This may conceivably be achieved by infiltrating cultural subgroups and sowing distrust among targeted communities. The guy next door represents a considerable departure from archetypes of heroic masculinity; indeed, the guy next door is the embodiment of unassuming, dependable, yet average masculinity. He is not just ordinary, but also local, thus submitting a sense of familiarity that effortlessly evolves to notions of reliability and trustworthiness. The visual trope of the affable guy next door may conceivably ease the labor-intensive costs of infiltration. Similarly, the cohort of implausibly attractive young women may follow an analogous tactic of infiltration and subversion that exploits the male gaze and the depiction of women as sexual objects for the pleasure of the male viewer (Mulvey, 1975, 2006).

However implausibly attractive as a group they may be, this visual trope is seemingly an optimal candidate for becoming embedded with Trump supporters. Ultimately, this analysis sheds new light on the visual aesthetics and grammar of self-presentation by IRA profiles (Xia et al., 2019) that linked individual performance to a social identity strategically catering to target subcultures, thereby marking a break with conventional state propaganda. Moreover, it highlights the salience of profile images as a key prop in social media propaganda that can be deployed by state actors to attain social embeddedness among target groups. Future research can continue to develop this line of enquiry through examinations of profile images by other types of actors, before and after potentially polarizing events such as protests or elections; and, to the same end, can expand the visual analysis to comparisons between profile images and other imagery pushed by the same accounts among their followings.

It remains unclear whether the IRA campaign reflects broader shifts in the execution of influence operations. Some observers have argued that it may reflect a pragmatic approach by a private outfit to deliver a policy objective set by its patron, the Russian state (Miller, 2019). Nevertheless, state propaganda seems unlikely to altogether cease trafficking in the symbology of national identity and heroic or hegemonic masculinity (see for example Romanets, 2017). In other quarters, and while the IRA carried out their unorthodox campaign, Chinese influence operations promoted nationalism with a narrative that elevated fighting and dominance using militaristic tropes. The Chinese corps of hawkish “wolf-warrior” diplomats mirror the patriotic blockbuster Rambo-style movie Wolf Warrior 2, where a former Chinese special-operations commando saves compatriots in a war-torn African country by defeating Western-led mercenaries. This campaign, waged at first on Chinese soil and later on Twitter (Zhong et al., 2020), relied on the canonical propaganda symbology of national identity and military prowess to boost China’s influence and control of global narratives (Hille, 2020).

Finally, the methodological approach developed for this study maximizes the benefits of a mixed methods design combining the qualitative coding and analysis of photographic images with statistical and computer processing of the data. The triangulation of various approaches to visual inspection allowed us to reach saturation, and ultimately validity, which are considerable roadblocks in the transferability and generalizability of visual research (Bock et al., 2012). As such, we expect this approach to inform future research in the production of visuals that combine data analysis with the vantage point afforded by the visual research, including greater reflexivity, the high contextualization applied to the data, and hopefully greater insights into the potentials and limitations of the explored data.

Supplemental Material

sj-pdf-1-nms-10.1177_14614448211026580 – Supplemental material for Guy next door and implausibly attractive young women: The visual frames of social media propaganda

Supplemental material, sj-pdf-1-nms-10.1177_14614448211026580 for Guy next door and implausibly attractive young women: The visual frames of social media propaganda by Marco Bastos, Dan Mercea and Fábio Goveia in New Media & Society

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The first author acknowledges financial support from Twitter, Inc. research grant 50069SS.

ORCID iDs

Marco Bastos

Dan Mercea

Supplemental material

Supplemental material for this article is available online.

Author biographies

Marco Bastos is the University College Dublin Ad Astra Fellow at the School of Information and Communication Studies and Senior Lecturer at City, University of London. His research leverages network science and computational methods to explore the intersection of communication and critical data studies.

Dan Mercea is Reader in the Department of Sociology at City, University of London. He holds a PhD in Communication Studies from the Department of Sociology at University of York and is the author of Civic Participation in Contentious Politics, Basingstoke: Palgrave (2016).

Fábio Goveia is an Assistant Professor at the Federal University of Espírito Santo and coordinator of the Laboratory on Studies on Image and Cyberculture (Labic). This research was conducted while Dr Goveia conducted postdoctoral research in the Department of Sociology at City, University of London under the supervision of Dr Marco Bastos.

References

Andersen

Sandberg

(2020) Islamic State propaganda: between social movement framing and subcultural provocation. Terrorism and Political Violence 32: 1506–1526.

Arif

Stewart

Starbird

(2018) Acting the part: examining information operations within #BlackLivesMatter discourse. Proceedings of the ACM on Human-Computer Interaction 2(CSCW): 20.

Badawy

Addawood

Lerman

, et al. (2019) Characterizing the 2016 Russian IRA influence campaign. Social Network Analysis and Mining 9(1): 31.

Bastos

Farkas

(2019) “Donald Trump Is My President!”: the Internet Research Agency propaganda machine. Social Media + Society. Epub ahead of print 6 August. DOI: 10.1177/2056305119865466.

Bastos

Mercea

(2019) The Brexit Botnet and user-generated hyperpartisan news. Social Science Computer Review 37(1): 38–54.

Benford

Snow

(2000) Framing processes and social movements: an overview and assessment. Annual Review of Sociology 26(1): 611–639.

Benkler

Faris

Roberts

(2018) Network Propaganda: Manipulation, Disinformation, and Radicalization in American Politics. Oxford: Oxford University Press.

Biagolini

Jr Macedo

(2019) bwimage: a package to describe image patterns in natural structures. F1000Research 8: 1168.

Bock

Isermann

Knieper

(2012) Quantitative content analysis of the visual. In: Hughes

(ed.) SAGE Visual Methods. London: SAGE, 313–334.

10.

Brantner

Lobinger

Wetzstein

(2011) Effects of visual framing on emotional responses and evaluations of news stories about the Gaza conflict 2009. Journalism & Mass Communication Quarterly 88(3): 523–540.

11.

Chadwick

(2013) The Hybrid Media System: Politics and Power. Oxford: Oxford University Press.

12.

Chadwick

Vaccari

O’Loughlin

(2018) Do tabloids poison the well of social media? Explaining democratically dysfunctional news sharing. New Media & Society 20(11): 4255–4274.

13.

Cho

Fiore

Russell

(2015) Validation of a fashion brand image scale capturing cognitive, sensory, and affective associations: testing its role in an extended brand equity model. Psychology & Marketing 32(1): 28–48.

14.

Coleman

Banning

(2006) Network TV news’ affective framing of the presidential candidates: evidence for a second-level agenda-setting effect through visual framing. Journalism & Mass Communication Quarterly 83(2): 313–328.

15.

Davis

(2018) Objectification, sexualization, and misrepresentation: social media and the college experience. Social Media + Society. Epub ahead of print 13 July. DOI: 10.1177/2056305118786727.

16.

De Vreese

(2005) News framing: theory and typology. Information Design Journal 13(1): 51–62.

17.

Del Vicario

Bessi

Zollo

, et al. (2016) The spreading of misinformation online. Proceedings of the National Academy of Sciences of the United States of America 113(3): 554–559.

18.

Díaz

Lencinas

(2018) Model-based local thresholding for canopy hemispherical photography. Canadian Journal of Forest Research 48(10): 1204–1216.

19.

Elections Integrity (2018) Data Archive (17 October edition). San Francisco, CA: Twitter, Inc.

20.

Entman

(1993) Framing: toward clarification of a fractured paradigm. Journal of Communication 43(4): 51–58.

21.

Entman

Usher

(2018) Framing in a fractured democracy: impacts of digital technology on ideology, power and cascading network activation. Journal of Communication 68(2): 298–308.

22.

Farkas

Bastos

(2018) IRA propaganda on Twitter: stoking antagonism and tweeting local news. In: Proceedings of the 9th international conference on social media and society, Copenhagen, 18–20 July, pp. 281–285. New York: ACM.

23.

Ferrara

(2017) Disinformation and social bot operations in the run up to the 2017 French presidential election. First Monday 22(8).

24.

Freelon

Bossetta

Wells

, et al. (2020) Black trolls matter: racial and ideological asymmetries in social media disinformation. Social Science Computer Review. Epub ahead of print 7 April. DOI: 10.1177/0894439320914853.

25.

Fyfe

(2018) Image analytics and the nineteenth-century illustrated newspaper. Journal of Cultural Analytics. Epub ahead of print 26 October. DOI: 10.22148/16.026.

26.

Gelder

(2005) The Subcultures Reader. London: Routledge.

27.

Goffman

(1959) The Presentation of Self in Everyday Life. New York: Anchor.

28.

Grabe

Bucy

(2009) Image Bite Politics: News and the Visual Framing of Elections. Oxford: Oxford University Press.

29.

Herman

Chomsky

(2010) Manufacturing Consent: The Political Economy of the Mass Media. New York: Random House.

30.

Hilbert

Vásquez

Halpern

, et al. (2016) One step, two step, network step? Complementary perspectives on communication flows in Twittered citizen protests. Social Science Computer Review 35: 444–461.

31.

Hille

(2020) “Wolf warrior” diplomats reveal China’s ambitions. Financial Times, 12 May. Available at: https://www.ft.com/content/7d500105-4349-4721-b4f5-179de6a58f08

32.

Hofmeester

Kemp

JAM

Blankendaal

ACM

(1996) Sensuality in product design: a structured approach. In: Proceedings of the SIGCHI conference on human factors in computing systems, Vancouver, BC, Canada, 13–18 April, pp. 428–435. New York: Association for Computing Machinery (ACM).

33.

Holbrook

Hirschman

(1982) The experiential aspects of consumption: consumer fantasies, feelings, and fun. Journal of Consumer Research 9(2): 132–140.

34.

Jensen

(2018) Russian trolls and fake news: information or identity logics? Journal of International Affairs 71(15): 115–124.

35.

Kim

Hsu

Neiman

, et al. (2018) The stealth media? Groups and targets behind divisive issue campaigns on Facebook. Political Communication 35(4): 515–541.

36.

Leaver

Highfield

Abidin

(2020) Instagram: Visual Social Media Cultures. Hoboken, NJ: John Wiley & Sons.

37.

Linvill

Warren

(2020) Troll factories: manufacturing specialized disinformation on Twitter. Political Communication 37: 447–467.

38.

McGarty

Thomas

Lala

, et al. (2014) New technologies, new identities, and the growth of mass opposition in the Arab Spring. Political Psychology 35(6): 725–740.

39.

Mandell

Shaw

(1973) Judging people in the news—unconsciously: effect of camera angle and bodily activity. Journal of Broadcasting & Electronic Media 17(3): 353–362.

40.

Manovich

(2020) Cultural Analytics. Cambridge, MA: MIT Press.

41.

Margolis

Pauwels

(2011) The SAGE Handbook of Visual Research Methods. London: SAGE.

42.

Martinec

Salway

(2005) A system for image–text relations in new (and old) media. Visual Communication 4(3): 337–371.

43.

Marwick

Boyd

(2011) I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. New Media & Society 13(1): 114–133.

44.

Miller

(2019) Topics and emotions in Russian Twitter propaganda. First Monday. Epub ahead of print 6 May. DOI: 10.5210/fm.v24i5.9638.

45.

Miskimmon

O’loughlin

Roselle

(2014) Strategic Narratives: Communication Power and the New World Order. London: Routledge.

46.

Morris

Counts

Roseway

, et al. (2012) Tweeting is believing? Understanding microblog credibility perceptions. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, Seattle, WA, 11–15 February, pp. 441–450. New York: ACM.

47.

Mueller

(2019) Report on the investigation into Russian interference in the 2016 presidential election. Available at: https://www.justice.gov/archives/sco/file/1373816/download

48.

Mulvey

(1975) Visual pleasure and narrative cinema. Screen 16(3): 6–18.

49.

Mulvey

(2006) Death 24X a Second—Stillness and the Moving Image. London: Reaktion Books.

50.

Otto

Springstein

Anand

, et al. (2019) Understanding, categorizing and predicting semantic image-text relations. In: Proceedings of the 2019 on international conference on multimedia retrieval, Ottawa, ON, Canada, 10–13 June, pp. 168–176. New York: ACM.

51.

Papacharissi

(2015) Affective Publics: Sentiment, Technology, and Politics. Oxford: Oxford University Press.

52.

Parry

(2010) A visual framing analysis of British press photography during the 2006 Israel- Lebanon conflict. Media, War & Conflict 3(1): 67–85.

53.

Pomerantsev

(2019) This Is Not Propaganda: Adventures in the War Against Reality. London: Faber & Faber.

54.

Pritchard

Morgan

(2011) Tourist bodies, transformation and sensuality. In: Bramham

Wagg

(eds) The New Politics of Leisure and Pleasure. London: Palgrave Macmillan, pp. 153–168.

55.

Rasband

(2012) ImageJ: image processing and analysis in Java. Astrophysics Source Code Library. Available at: https://ui.adsabs.harvard.edu/abs/2012ascl.soft06013R/abstract

56.

Rogers

(2013) Digital Methods. Cambridge, MA: The MIT Press.

57.

Romanets

(2017) Virtual warfare: masculinity, sexuality, and propaganda in the Russo-Ukrainian war. East/West: Journal of Ukrainian Studies 4(1): 159–177.

58.

Rose

(2016) Visual Methodologies: An Introduction to Researching with Visual Materials. London: SAGE.

59.

Selvage

(2019) Operation “Denver”: the East German Ministry of state security and the KGB’s AIDS disinformation campaign, 1985–1986 (part 1). Journal of Cold War Studies 21(4): 71–123.

60.

Seo

(2014) Visual propaganda in the age of social media: an empirical analysis of Twitter images during the 2012 Israeli–Hamas conflict. Visual Communication Quarterly 21(3): 150–161.

61.

Seo

(2019) Visual propaganda and social media. In: Baines

O’Shaughnessy

Snow

(eds) The SAGE Handbook of Propaganda. London: SAGE, pp. 126–136.

62.

Siegel

Tucker

(2018) The Islamic State’s information warfare: measuring the success of ISIS’s online strategy. Journal of Language and Politics 17(2): 258–280.

63.

Steel

(2015) Streets to screens: mediating conflict through digital networks. Information, Communication & Society 18(11): 1269–1274.

64.

Stewart

Arif

Nied

, et al. (2017) Drawing the lines of contention: networked frame contests within #BlackLivesMatter discourse. Proceedings of the ACM on Human-Computer Interaction 1(CSCW): 96.

65.

Strömbäck

(2008) Four phases of mediatization: an analysis of the mediatization of politics. The International Journal of Press/Politics 13(3): 228–246.

66.

Tajfel

(1978) Social categorization, social identity and social comparison. In: Tajfel

(ed.) Differentiation Between Social Groups: Studies in the Social Psychology of Intergroup Relations. London: Academic Press, pp. 61–76.

67.

Twitter (2018) Update on Twitter’s review of the 2016 US election. Available at: https://blog.twitter.com/en_us/topics/company/2018/2016-election-update.html

68.

United States Senate Committee (2017) United States Senate Committee on the judiciary, subcommittee on crime and terrorism: testimony of Sean J. Edgett, Acting General Counsel, Twitter, Inc. Available at: https://www.judiciary.senate.gov/imo/media/doc/10-31-17%20Edgett%20Testimony.pdf

69.

Welch

(2013) Images of the Hun: the portrayal of the German enemy in British propaganda in world war I. In: Welch

(ed.) Propaganda, Power and Persuasion: From World War I to Wikileaks. London: I.B. Tauris, pp. 37–64.

70.

Welch

(2016) Persuading the People: British Propaganda in World War II. London: The British Library.

71.

Xia

Lukito

Zhang

, et al. (2019) Disinformation, performed: self-presentation of a Russian IRA account on Twitter. Information, Communication & Society 22(11): 1646–1664.

72.

Xie

Zhang

(2013) White or tan? A cross-cultural analysis of skin beauty advertisements between China and the United States. Asian Journal of Communication 23(5): 538–554.

73.

Zehm

Nobis

Schwabe

(2003) Multiparameter analysis of vertical vegetation structure based on digital image processing. Flora—Morphology, Distribution, Functional Ecology of Plants 198(2): 142–160.

74.

Zhong

Krolik

Mozur

, et al. (2020) Behind China’s Twitter campaign, a murky supporting chorus. The New York Times, 8 June.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.28 MB