The Conservatism of Emoji: Work,Affect,and Communication

Abstract

This piece examines emoji as conduits for affective labor in the social networks of informational capitalism. Emoji, ubiquitous digital images that can appear in text messages, emails, and social media chat platforms, are rich in social, cultural, and economic significance. This article examines emoji as historical, social, and cultural objects, and as examples of skeuomorphism and of technical standardization. Now superseded as explicitly monetized objects by other graphics designed for affective interactions, emoji nonetheless represent emotional data of enormous interest to businesses in the digital economy, and continue to act symbolically as signifiers of affective meaning. We argue that emoji characters both embody and represent the tension between affect as human potential, and as a productive force that capital continually seeks to harness through the management of everyday biopolitics. Emoji are instances of a contest between the creative power of affective labor and its limits within a digital realm in the thrall of market logic.

If you smile through your fear and sorrow

Smile and maybe tomorrow

You’ll see the sun come shining through for you

Nat King Cole, “Smile”

Introduction: It Began With a Smile

Emoji are popular digital pictograms that can appear in text messages, emails, and on social media platforms. These characters are generally understood as a light-hearted, almost comedic form of communication, but they have a rich and complex socioeconomic history that precedes the range of mobile devices where they commonly appear. Beginning with the rise of the iconic “smiley” face in the second half of the twentieth century, the emoji as a cultural form has emerged out of typographic habits, corporate strategies, copyright claims, online chat rooms, and technical standards disputes. As a genial and widespread vernacular form, emoji now serve to smooth out the rough edges of digital life. Further, we argue that to understand affect’s current place within contemporary capitalism we need to consider this face: . These symbols do considerable work to underscore tone, introduce humor, and give individuals a quick and efficient way to bring some color and personality into otherwise monochrome networked spaces of text. Yet emoji also do more than this. Beyond their adorable exuberance, emoji can act as an emotional coping strategy and a novel form of creative expression, even if, in both cases, working within real limits. Emoji create new avenues for digital feeling, while also remaining ultimately in the service of the market.

According to Sarah Ahmed (2010), “affect is what sticks” (p. 29)—to people, places, and objects. And affective labor, an intensification of what sociologist Arlie Russell Hochschild (1979/2012) first termed “emotional labor” some 30 years ago, is a vital engine in the machine of late capitalism. Be it the forced greetings from fast-food workers, casual banter with Uber drivers, a flight attendant’s fixed smile, or the nurse patting a patient’s arm before the needle goes in, these affective impulses lead to forms of “emotional competence,” as Eva Illouz (2007) calls them, that serve to sustain social relations within the logics of economic instrumentalism and efficiency. Often gendered as feminine, affective and emotional forms of labor have long been seen as marginal to production, yet are central to the functioning of contemporary economic systems. “Affective labor, the production and reproduction of life, has become firmly embedded as a necessary foundation for capitalist accumulation,” observes Michael Hardt (1999, p. 99).

Affective labor has been a powerful driver of informational capitalism, in which, according to Hardt, “providing services and manipulating information are at the heart of economic production.” For Hardt, Illouz, and other theorists of affect, the radical challenge of the neoliberal economy is posed primarily by “the affective labor of human conduct and interaction”—or as Hardt calls it, the “other face of immaterial labor” (p. 90). “We increasingly think like computers,” Hardt argues of immaterial labor, “and the interactive model of communication technologies becomes more and more central to our laboring activities” (pp. 94-95). This essay considers emoji as historical and cultural objects, technical constructs, and instances of the techniques of computational control within communication platforms. Above all, these graphic forms are exemplary of the tension between affect as liberating human potential, and as a productive force that the market continually seeks to harness through the commoditization of emotional sociality. Through a highly compressed lexicon, the emoji character set vividly illustrates the constraints on affective labor under informational capital. Alexander Galloway (2006) argues, regarding video games, that “it is precisely those places in culture that appear politically innocent that are at the end of the day the most politically charged” (p. 95). The same can be said of emoji. Where did these characters come from? How do they “stick” to our digital lives? What do they and their successors portend for everyday emotional life under twenty-first century capitalism? The story begins with a smile.

The Smile as Artifact of Capital

The 1963 merger of the State Mutual Life Assurance Company of Worcester, Massachusetts, and Ohio’s Guarantee Mutual Company would be historically unremarkable were it not for its role in the birth of the “smiley” face. The State Mutual Company’s management was in need of an internal public relations campaign to improve morale after the turmoil and job losses prompted by the merger (Honan, 2001). The company commissioned Harvey Ball, a freelance advertising artist, to create something cheery: he offered a grinning yellow face with two dots for eyes and a wide grin. The smiley face was born as part of this corporate morale-building strategy and stamped on buttons and pins: company employees, Ball recalls, “would have pockets full of them” (BBC Radio and Wise Buddha Creative, 2013).

The executives of the newly amalgamated firm loved Ball’s design, and were quick to trumpet its broader economic and ideological significance. “The power of a smile is unlimited,” proclaimed The Mutualite, the company’s internal magazine, in announcing the campaign in January of 1964: “A smile is contagious. It is entertainment and medicine. It is food for friendship.” Company literature also suggested that a smile was “vital to business associations and to society” (Worcester Historical Society, 2013). Employees were encouraged to smile while talking to clients on the phone and while filling out insurance forms. Ball was paid US$240 for the campaign, including US$45 for the rights to his smiley face. He never held a copyright on the image and made no money from his smiley face for the rest of his life.

Yet the symbol itself endured. “It’s just a yellow field with three marks on it,” observed artist Dave Gibbons, who deployed the smiley in Alan Moore’s iconic 1986 graphic novel Watchmen: “It couldn’t be more simple. And so to that degree, it’s empty. It’s ready for meaning” (Stamp, 2013). During the late 1960s, the smiley began to appear on t-shirts, signs, buttons, and other ephemera; by the early 1980s, the image had proliferated, becoming a permanent feature of Western pop cultural representation. The face appeared on children’s pencil cases and underground acid-house record covers alike, in both the United States and around the world. The graphic also became terrain for contests over intellectual property. Because Ball had failed to apply for a trademark on his original design, other entrepreneurs claimed the image for themselves. Two Philadelphia entrepreneurs, brothers Bernard and Murray Spain, filed for copyright on the smiley face in 1971 along with the phrase “have a happy day.” The brothers made millions. In Europe, Franklin Loufrani, a journalist for France Soir, placed the smiley face under a competing copyright claim in 1972 (Stamp, 2013). And in 1989, a New York disk jockey (DJ) named Charlie Almazora claimed that his radio station had first created the smiley in 1963 as a promotional symbol for “the WMCA good guys,” the station’s roster of DJs (Honan, 2001). These battles came to a head in 1997 when the Loufrani family sued Wal-Mart over the corporation’s own claim of ownership over the image. The suit lasted a decade, and was ultimately settled out of court for an undisclosed amount (Stamp, 2013).

The corporate history of the smiley face is a fitting prologue to the dynamics of its digital transmutation. Sianne Ngai (2015) argues that,

its unflinching gaze as we encounter it daily as a cookie, on a price tag, or in a comic book, confronts us in a palpably unsettling way with the radically alienated status of sociality itself under conditions of generalized commodity production. (p. 40)

The original smiley face made happiness, in Sarah Ahmed’s (2010) terms, “a happening”: it crystallized the force of the feeling into an icon that could be simultaneously mobilized in the service of institutional corporate power, transcend that control to become a cultural touchstone, and become recaptured as a form of intellectual property. Sharing a social bond through icons like the smiley means sharing “an orientation towards those objects as . . . good” (p. 29), but also leaving these objects open to systematic commercialization. We argue that emoji, like the smiley, operate within this oscillating dynamic, whereby affect is captured by capital through proprietary cultural representations and subsequently escapes, only to be recaptured through new technocultural forms. And as Sumanth Gopinath (2013) describes in his study on the rise and fall of the mobile ringtone, this dynamic is oppositional: “a series of dialectical processes [of] emergence, popularity, normalization, and waning, and involving content repurposing and cycles of nostalgic revival” (p. xxii).

Michael Hardt (1999) argues that the communality of affect means it is perennially more powerful than the forces of capital attempting to harness it: affective labor, he suggests, “present an enormous potential for autonomous circuits of valorization, and perhaps for liberation” (p. 100). In this reading of what Hardt terms “a biopower from below,” affect’s emancipatory potential is located in the daily habits of individuals collaborating together to create novel forms of cultural expression. This might give us hope for the emoji character set’s creative potential. Yet Gopinath’s work on the ringtone economy cautions against over-valorizing the capacity of affect to fuel an escape from the imperatives of the digital economy.

Emoticons and Emoji as Data Culture

Faces have long been created through the clever arrangement of punctuation marks. In 1948, Popular Mechanics devoted a full-page spread to the creation of “Keyboard Art”—itself a precursor to ASCII art (Carlsson & Miller, 2012). And in 1969, Vladimir Nabokov suggested the utility of expressing feelings via typographical shorthand in an interview with the New York Times: Q:

How do you rank yourself among writers (living) and of the immediate past?

Nabokov:

I often think there should exist a special typographical sign for a smile—some sort of concave mark, a supine round bracket, which I would now like to trace in reply to your question. (Nabokov, 1969)

The idea for such marks in the computational context is often credited to Scott E. Fahlman, a young faculty member in Carnegie Mellon University’s computer science department who suggested a “joke marker” on a university bulletin board (or bboard) on September of 1982 (Kennedy, 2012). There was much wordplay in the world of Carnegie Mellon University’s (CMU) board, but sarcasm and dry humor were often misread. Fahlman (n.d.) first pitched the idea of the “emoticon” in this bboard message:

I propose that the following charac-ter sequence for joke markers:

:-)

Read it sideways. Actually, it is

probably more economical to mark

things that are NOT jokes, given current

trends. For this, use :-(

The practical utility of the emoticon grew along with the increasingly computerized social transactions of everyday life. While in North America emoticons developed as sideways vertical images of the face, Japanese emoticons, known as kaomoji or “face marks,” tended to be set in horizontal profile views: (•‿•) instead of :-). This formal variation stemmed in part from a long history of technological adaptations around the efficient printing of East Asian character sets (Mullaney, 2012). The complexity of horizontal kaomoji also meant an increase in the number of characters needed to produce any particular image. This challenge to speed and clarity was perhaps one of the reasons why emoji were first developed in Japan. By the mid-1990s, Japanese teenagers were commonly using pagers to craft private online social spaces in a country where both housing and phone lines were prohibitively expensive, especially for young workers (Lippit, 1995). Shigetaka Kurita, a designer for Japanese telecom carrier NTT Docomo, was instructed to create new icons for the company’s devices as a way to define its brand and secure customer loyalty in the competitive youth pager market. NTT Docomo had already incorporated a small graphic heart into its pager symbols to popular acclaim (Blagdon, 2013). When Kurita and his colleagues released the resulting set of 12-by-12-pixel graphics, the smiley face was reified at the level of code.

Part of the charm of contemporary emoji usage is that a small set of seemingly banal images can be manipulated to produce such communicative diversity. The messages created with emoji prompt a hermeneutic impulse: just what did the author intend to say with this series of images? Viewers are asked to decode what is being represented, marveling at both the limits and expansiveness of meanings made possible by tiny hands praying, a ghost sticking its tongue out, and a row of eggplants. Man Bartlett and Jacinda Russell’s recent project Emoji Art History played with the power of emoji to remix cultural norms through allusion and simplification. Collected from tweets and instant messages, the piece showcased some of the works produced via the #emojiarthistory hash tag and Tumblr site, emoji versions of well-known works of visual art (Vartanian, 2013). In 2013, New York City’s Eyebeam Art + Technology Center hosted the Emoji Art and Design Show; the event featured works in various visual media drawing on the emoji palette. “Emoji have become an ever-evolving communal form of cryptography,” wrote Jenna Wortham (2013) in a ‘zine accompanying the installation (p. 19). In the same publication, journalist Clive Thompson (2013) suggested that one ancestor to contemporary emoji was the rebus, a puzzle or visual pun in which words or syllables within a sentence are replaced by images that serve as homophones for the missing text. Like emoji, the rebus has a checkered history of both delight and repulsion. “Just as a pun is conventionally met with a groan,” notes linguist Michael J. Preston (1982), “so the rebus is often acknowledged by a statement of disdain, unless, of course, one knows a rebus or two and can respond in kind” (p. 119).

Popular culture now draws deeply from the emoji palette. The world is now home to Emoj.li, the emoji-only social network. Comedian Nick Offerman (2014) appeared on The Conan O’Brian Show to pitch a product line of emoji carved from solid oak, billed as “a more old-fashioned, more personal, more American mode of communication.” In 2013, pop singer Katy Perry released a music video for her single “Roar,” comprising emoji transliterations to the song’s lyrics—a speedy, topical, and inexpensive prelude to the song’s live-action video released later that year. Then in March of 2014, a video producer named Jesse Hill (2014) set Beyoncé’s song “Drunk in Love” to a quick-fire emoji music video transliteration, overlaying characters from Apple’s emoji font in front of a plain white background. The hermeneutic pleasure of the emoji “Drunk in Love” comes not simply from Hill’s witty deployment of unlikely characters to represent particular lines of the song: it also stems from the ways Hill multiplies, cross-cuts, fades, rotates, and arrays emoji to capture and remediate both lyric complexity and the aural rhythm of the piece, through rapidly appearing and disappearing flights of images. Visual leitmotifs for repeated phrases of the music cascade down the screen in parallel to Beyoncé’s voice; the emoji version of a made-up mouth stands in for the singer’s subjectivity, as sexual and insistent as Patricia Quinn’s famous lips in the opening sequence of The Rocky Horror Picture Show (Licci, 2009). Part of the appeal of the emoji “Drunk in Love” stems from how readily it conjures Beyoncé’s celebrity persona out of the emoji character set: emoji are ideal for representing the trappings of celebrity culture, from martinis to paparazzo attacks, all within a normative, cartoon vision of wealth and fame.

Equally virtuosic from a technical perspective is Emoji Dick, artist Fred Benenson’s crowd-sourced, book-length retelling of Herman Melville’s (1851/2010) novel through emoji. Recently accepted into the Library of Congress, Emoji Dick is as much an experiment in creation through crowd-sourced labor as it is an attempt at a new genre of fiction. Benenson used Kickstarter to pay for Amazon Mechanical Turk workers to translate each sentence of Herman Melville’s novel in three different ways, and subsequently paid a second panel of Turkers to vote on the most fitting translation. The end result, according to Benenson (2009), saw “over eight hundred people spend approximately 3,795,980 seconds working to create this book.” Each worker was paid 5 cents per translation, and 2 cents for each vote. Following the tradition of the rebus, Benenson’s aim was purportedly to contrast “really simple, constrained language and classical literature” (Popovich, 2013). Alongside the puzzle of interpreting each sentence from one of the American Renaissance’s greatest and most flamboyantly verbose novels as a string of emoji, the piece was also intended to highlight the current conditions of possibility for digital labor and the arts. Some journalists hailed Emoji Dick as exemplifying the creative possibilities of crowd sourcing, and Benenson agreed: “I think I was super interested in pushing the boundaries of both crowd-sourcing and also of emoji—to see how far you can take it,” he claimed (Popovich, 2013). Despite the “emojional” labor of hundreds of crowd laborers, Benenson received the lion’s share of the credit for the project in the press.

The organizers of the Emoji Art and Design show dubbed emoji “the new visual vernacular” (Weber, 2013), yet the emoji vernacular is slippery and allusive: it draws its potency, and limits, from reference to and dependence on a small cadre of other media and material objects. Emoji Dick and Hill’s “Drunk in Love” are two pinnacles of emoji as cultural expression, contemporary rebuses perfectly executed yet relying on other, richer works as indissociable references. Emoji were supposed to facilitate a wider range of text-based emotional communication—without emoji, Kurita claimed, “you don’t know what’s in the writer’s head”—yet they simplify as much as they diversify digital communication. Media theorist Laura Marks (2010) uses the term “lame infinity” to describe the ways in which digital technology seems capable of vast potential but is instead used to produce a dispiriting kind of sameness. Emoji, too, seem trapped in this bind, a “normcore system of emotion”: the generic basic-wear of digital communication, “a taxonomy of feeling in a grid menu of ideograms” (Crawford, 2014). Emoji can deaden as well as enliven. For all their creative potential, emoji were intended to normalize and then capitalize on the collective strength of affect in human social relations online.

Emoji as Social Technics

The technical history of emoji demonstrates how affect’s power has been harnessed by informational capital, even if the full extent of human creative capacity remains outside its reach. With only 144 pixels each—18 bytes—an emoji must compress the face or object it represents into the most schematic configuration possible to achieve its symbolic effect. Kurita envisioned the creation of “a complete set of 176 twelve-by-twelve-pixel characters that could cover the entire breadth of human emotion” (Blagdon, 2013). At first, NTT Docomo sought to make emoji proprietary—and by extension, create a source of revenue out of the circulation of their users’ feelings and social affects—whenever it was legally or technically possible to do so. Yet in a strange echo of Harvey Ball’s smiley, Docomo was unable to secure a copyright on its emoji characters, allegedly because the individual emoji were considered too simple to be trademarked (Blagdon, 2013).

NTT Docomo nonetheless maintained their emoji as a form of brand difference. Despite being unable to copyright the characters themselves, the original emoji set was designed to display only on Docomo’s proprietary platform, exclusive to phones owned and operated by the company’s subscribers in Japan. The wide popularity of emoji drove uptake of the company’s phones, and by extension helped establish emoji as a desirable feature across the Japanese telecommunications industry. “The good thing about that was that everyone’s emoji were identical,” claimed Kurita—“If each manufacturer had added its own originality to the characters, the emoji would have been all mixed up and inconsistent, even inside Docomo” (Blagdon, 2013). Laura Marks (2010) observes that the pixel itself standardizes meaning: “the indexical truth of the vector [graphic] is replaced by the bland, boot-faced approximations of pixel graphics” (p. 205). The number of possible combinations of black and white pixels in a 12-by-12 grid is 2 to the 144th power: despite a minimal baseline of possible graphical units, a set grid of pixels, like the force of affect, can be recombined and extended in an almost infinite array of patterns and meanings. As Marks (2010) argues, pixel graphics may be an example of lame infinity, but they can still prompt astonishing innovation within the constraints afforded them, inspiring artists to “pursue the infinite variations of the standard form.”

In the late 1990s, NTT Docomo competitors SoftBank and DDI Cellular Group quickly produced rival emoji character sets, which were made available exclusively on their competing handset platforms. All of these emoji were originally encoded in Shift Japanese Industrial Standards or Shift-JIS, the ASCII standard for Japanese characters. NTT Docomo also created versions in the Unicode Standard, the computer industry’s basic worldwide character set administered by the Unicode Consortium (Scherer et al., 2010). Yet, the corporate desire to restrict the legibility of each company’s character set to a particular company platform left the Japanese phone carriers at a disadvantage when Apple introduced the iPhone to Japan in 2008 (Oomori, 2008). Apple’s business model entailed selling a standard hardware and software package to its carriers worldwide. With the introduction of the iPhone by SoftBank, an NTT Docomo competitor, users in Japan demanded a way to use emoji across multiple handset platforms; intrigued, users in other parts of the world also began to deploy hacks and special apps to make Apple’s emoji keyboard, standard in iOS because of the Japanese market, available to them as well. The burgeoning popularity of the character set in the 2000s meant that neither Apple and SoftBank nor NTT Docomo could charge a premium for access to emoji. Still, it was only in 2010 that emoji were formally incorporated into Unicode. Apple’s inclusion of emoji into iOS5 in 2011—in the face of its North American customers hacking into the iPhone emoji keyboard—furthered the spread of the characters’ popularity outside of East Asia (Bennett, 2014). Today, more than 880 emoji have now been accepted by the Consortium and consolidated within the Unicode 7.0 standard, while a further 300 were added in the release of Unicode 8.0 in mid-2015.

While decisions about the emoji code base are centralized by a global technical standard, the particular “look” of emoji characters still depends on the fonts in which they are represented, fonts that are most certainly subject to copyright law. It is possible to display emoji as simple pixilated graphics, but smartphones display emoji in proprietary fonts rich with color and movement, featuring stylistic alterations or even changes in major graphic details and attributes such as object orientation. Emoji are now, in the parlance of the gaming industry, “skinned.” While this visual variation allows for particular styles of inventiveness in emoji font graphics, it also unsettles the universality of emoji characters. Emoji, like other skeuomorphs—linoleum that looks like wood grain, the trash bin on your desktop, the shutter click sound on a digital camera—are objects that recreate functional components as decorative elements. Each Unicode character compresses the multiple typographic components that previously made up emoticons and kaomoji into one unique symbol, while still maintaining a rough pictorial continuity between the forms dictated by the earlier ASCII characters—the new emoji thus retain some of that ASCII genealogy in their 12-by-12-pixel form. Yet, each system creates its own emoji overlays that recast the code in proprietary formats, producing a top-layer of intellectual property and a means to extract value.

Emoji as Affective Labor

The anthropologist Nicholas Gessler (1998) terms skeuomorphs “material metaphors” that persist in computational media in order to “help us map the new onto an existing cognitive structure.” Yet, the affective power of emoji can in part be explained not just by their skeuomorphism, but also by their conceptual plasticity. An emoji, like emoticons or kaomoji, straddles the conceptual line between ideogram and pictogram. Ideograms are symbolic representations of a particular concept or idea; pictograms are ideograms that show a pictorial image of the object being represented. To a greater degree than the emoticon, the utility of an emoji lies in the indeterminacy of its pictographic versus iconographic legibility as a signifier of affect, emotion, or sociality. In designing the original emoji character set, Kurita claimed to have been inspired by common ideographic elements in Japanese popular visual culture, including the conventions of manga comics: “When someone gets an idea [in manga], they have the light bulb,” Kurita explained, “and there were a lot of cases where I used those as a kind of hint and rearranged things” (Blagdon, 2013). Because the meaning of individual emoji is relatively plastic—after all, what intrinsic emotion does that flamenco dancer represent?—emoji use is heavily structured by linguistic and social contexts, and by both cultural and personal conventions (Derks, Bos, & Grumbkow, 2007). Studies have noted, emoticons and emoji not only indicate particular emotions but also general categories of “illocutionary force” within speech, acting as moments of emphasis, de-emphasis, or punctuation (Dresner & Herring, 2010). Frequently serving to complement or complicate speech, emoji can also function as instances of phatic expression: the equivalent of “Hi, how are you” and “I’m fine,” phrases that primarily serve a social role instead of a specific linguistic one (Licoppe, 2008).

Precisely because of their social utility and interpretive flexibility, emoji now straddle the increasingly blurred lines between life and work for many white-collar professionals. Created using Amazon Mechanical Turk, Emoji Dick is one example of this messy intersection of emoji as a cultural aesthetic and as markers in a sociotechnical system of potential exploitation and precarity. The production of capitalist subjectivity through this affective indeterminacy is one of the ways in which capital seeks to co-opt and exploit affective labor. Emoji are useful components for working socially across computational media: they show the importance, and paradoxical invisibility, of affective and social ties across digital structures of work. Emoji are a prophylactic against the visceral traumas of what Melissa Gregg (2010) terms the worker’s “schizophrenic and unpredictable encounter” with a culture of white-collar technical work characterized by a cynical, mediated sociality (pp. 252-253). Much as insurance company employees were once enjoined to be cheerful and more productive through the circulation of the smiley face, contemporary workers must navigate similar challenges with the smiley’s digital descendants. In this context, affect and its circulation is the bedrock of company value. As emoji have spread to smartphone users around the world, they have helped “to sustain the shock . . . of postmodernity” (Negri, 1999, p. 86) for professional relationships conducted, often out of necessity, via platforms like Slack, short message service (SMS), or email.

Emoji are reshaping mediated communication in the personal as well as in the corporate sphere—precisely because our interactions in both arenas are often mediated across the same platforms. The patterns of use for emoji over time between friends and partners can become abstract and cryptic, or can degenerate to become pro forma: just plain basic. In the best case, there is a unique personal subtext to that exchange of a rainbow or the love-heart smile, many layers of unspoken meaning that would be difficult for intelligence analysts or machine-learning algorithm to parse. Nonetheless, this complexity has not stopped institutions from making the attempt, and commercializing emoji sociality in other ways. Applications such as Couple are designed to convey the sentiment of “I’m thinking of you” in as few keystrokes as possible, in what human–computer interaction (HCI) designers call “low-content intimacy signaling” (Kaye, 2005). Like Couple, some programs specifically focus on romantic partnerships; more common, however, are individualized routines of communication via SMS or mobile messenger apps like Snapchat, LINE, WhatsApp, Skype, and FaceTime (Newman, 2015). Emoji and emoticons have become crucial elements in the affective mix of relationship maintenance, sustenance, and continuation.

Now the Apple smart watch brings emoji even closer to the body by deploying animated emoji that signal attention, happiness, or displeasure through the wrist of the wearer, via its haptic feedback capacities. These platforms are capitalizing on emoji use precisely in order to drive users to socialize more frequently: Snapchat recently changed its messaging function to use emoji to visualize the frequency of interactions among users on the app. “Suddenly,” The Washington Post noted, “users can see—in hard, cruel emoji—that their significant other snaps more with somebody else” (Dewey, 2015b). The maintenance of teen social ties, always a fraught experience, has become affective grist for the mill of social media’s bottom line.

Emoji as Resistance and Control

“This face is a symbol of capitalism,” declared Murray Spain of the smiley face: “Our intent was a capitalistic intent . . . our only desire was to make a buck” (BBC Radio and Wise Buddha Creative, 2013). What habits of daily life within the broader frame of neoliberal economics do emoji promote, from the painted nails to the martini glasses? What behaviors, ethics, and values do they normalize and what are they unable to account for? By giving us a visual vocabulary for the neoliberal digital everyday, emoji offer an example of one of the two facets of biopower: what Foucault termed “anatomo-politics.” Foucault (1978) describes the disciplinary nature of anatomo-politics as operating through “the body as a machine . . . the optimization of its capabilities, the extortion of its forces, the parallel increase of its usefulness and its docility, its integration into systems of efficient and economic controls” (p. 139). While the other pole of this articulation of power, the biopolitical regulation of populations, has been of tremendous interest to scholars over the past three decades, the disciplining habituation of anatomo-politics, particularly as mobilized by affect and emotion, takes on particular resonance in our interconnected, digitally mediated present. “It is not only that we acquire good taste through habits,” observes Sarah Ahmed (2010); “rather, the association between objects and affects is preserved through habit” (p. 35). The habits that emoji present as worthy are suggestive, if not of happiness, then at least a kind of consumerist, and highly gendered, gratification. In Katy Perry’s “Roar,” the emoji transliteration of the song’s lyrics also serve as a stark commentary on the basic anatomo-political maintenance of daily life: sleeping, eating, bathing, grooming, and charging our digital devices. The habitual domestic maintenance depicted in the video, still often considered “women’s work,” is heavily featured in the basic emoji character set.

However, progressive activism can change the face of emoji. Within the character set as a whole, there are many white emoji faces, hands, and body parts. Yet only two characters seem to be intended as racially Asian, and none seem to be interpretable as racially African or African-American. The current emoji character set hews “to a very narrow view of culture, one that is mostly white and heterosexual,” notes Mercedes Kraus (2013), one of the collaborators in the Emoji Art and Design Show. Yet as Native American rights activist Aura Bogado (2014b) argues, many online minority groups, including communities of color, have found creative ways to “make even limited digital possibilities vibrant.” Bogado (2014a) points to the example of political emoji art, including a piece through which she uses the emoji character set to narrate the events in Ferguson, Missouri, in the aftermath of the shooting of Michael Brown. Drawing in part on a long history of indigenous nonalphabetic literary traditions (Rasmussen, 2012), this activist emoji art is exemplary of Hardt’s “biopower from below.” Bogado’s work suggests new, emerging ways to tell stories about power, justice, mediation, and state violence—as well as pointedly highlighting the racial homogeneity of the emoji character set. These questions of representation are important, as Sydette Harry (2014) argues, because minority communities “are using tech to survive,” as well as to make their voices heard.

The lack of racial diversity in the emoji set prompted a wider online campaign and outcry. In response to a public petition, Apple announced that it would work with the Unicode Consortium on a technical solution (Kelion, 2014). The result, announced in November of 2014, was a proposal to include a skin tone modifier to Unicode 8.0. The proposal hazards that “emoji characters for people and body parts are meant to be generic,” but admits that, “following the precedents set by the original Japanese carrier images, they are often shown with a light skin tone instead of a more generic (nonhuman) appearance, such as a yellow/orange color or a silhouette.” These modifications to Unicode to increase representations of ethnic and racial diversity are welcome, though they also raise a number of questions about how emoji represents humans.

The proposed schema for skin tone modifiers (five different shades ranging from dark brown to pink) is based on the Fitzpatrick scale, which The Guardian described, somewhat uncritically as “a recognized standard used by dermatologists” (Tan, 2015). According to its author, noted academic dermatologist Thomas B. Fitzpatrick, the Fitzpatrick scale was developed to better “classify persons with white skin [emphasis in the original] in order to select the correct initial doses of ultraviolet A” for a psoriasis treatment. Perceived skin tone was coupled with a personal history questionnaire to determine if some visually “dark” complexions in fact burned more easily than had been thought. “Later, in addition to white-skinned persons,” Fitzpatrick (1988) wrote, “brown- and black-skinned persons were included in the classification” (p. 869). If nothing else, the scale’s systematic default assumptions—Type I for the “whitest” skin, Type VI for the “blackest”—are exemplary of the emoji character set’s indebtedness to established hierarchies of gendered and racialized authority and inequality. Moreover, even in systems that do not support the combined character of a face plus a skin tone modifier, a skin tone modifier itself might be supported on its own. Constituted by a rectangular block of shading, the skin tone modifier would be displayed alongside emoji symbols, such as gas pumps or pieces of fruit, which are not designed to support its inclusion in a combined character (Davis & Edberg, 2014). The addition of standalone skin tone modifiers has the potential to destabilize, for good and ill, a wide range of emoji characters. And what about that “non-human” yellow hue—itself another subtly racializing default—proposed as the generic, unmodified skin tone? According to The Washington Post, the color choice harks all the way back to Harvey Ball’s original smiley face (Dewey, 2015a).

Despite Kurita’s original intentions towards universality, the authors of the emoji diversity proposal suggested that, “it is beyond the scope of Unicode to provide an encoding-based mechanism for representing every aspect of human appearance diversity that emoji users might want to indicate.” “The best approach for communicating very specific human images,” the authors maintain, “or any type of image in which preservation of specific appearance is very important—is the use of embedded graphics.” These images—colloquially known as “stickers”—are simple images that can be shared within instant messaging apps, anything from a panda riding an airplane to an image of Iron Man. Stickers are increasingly supported by smartphone social networks such as LINE, a Japanese mobile messaging app whose popularity in East Asia is built on its proprietary stable of stickers—otherwise known as “next-level emoji” (Byford, 2014). Yet, this proposed solution for improving emoji diversity in fact signals a further evolution in the business models of affective digital communication. Throughout 2014 and 2015, technology magazines like The Verge and Fast Company ran admiring profiles of LINE, noting its ambitions toward entry to the North American market, and the fact that stickers offer unprecedented opportunities for the centralization, analysis, and monetization of affective representation—the company reported revenues of US$656 million in 2014 (McCracken, 2015). Stickers are also a feature of Facebook’s Messenger application and other messaging services (Byford, 2013). Some stickers are free, but many come at a price: up to US$2 to unlock particular sets on LINE. LINE’s business model thus entails selling its own sticker collections and engaging in merchandising deals with other media properties as part of transmedia marketing strategy: LINE’s sticker characters, like Cony the Rabbit and Brown the Bear, have become Japanese pop culture icons in and of themselves. Stickers, crucially, are proprietary to each platform that sells them.

We argue that stickers represent an attempt on the part of social media platforms—seemingly successful in the short run—to re-commodify the affective labor which, at least in part, had been lost with the standardization of emoji into Unicode. Individual users will be able to express themselves within preselected bands of racial, emotional, or otherwise idiosyncratic diversity—but at a new financial cost. Established platforms such as Facebook have sought to include stickers, emoji, and emotional signifiers in their user experience design, to compete with platforms like LINE and with an eye toward the further commoditization and monetization of social affect. Facebook’s sticker strategy, part of what the company terms “compassion research,” appears to involve using stickers as inducements for users to engage more frequently with its in-platform messaging apps (Van Grove, 2013). Ironically, Facebook sticker characters such as the bizarrely evocative Business Fish are beginning to gain cultural currency precisely because they reflect the precarious conditions of digital affective labor: Business Fish’s harried demeanor and melodramatic desperation to succeed in the corporate world are simultaneously descriptive and critical of the current pressure on workers, who are confronted by demands for new technical expertise, increased productivity, and heightened emotional competence at all times (Khan, 2015).

Platforms are also increasingly interested in analyzing emoji as a source of data about their users’ habits, moods, desires, and patterns of behavior. In the spring of 2013, Facebook introduced the ability to choose from a variety of emoji-like feelings as part of a user’s Status Updates. Individuals can register that they feel happy, sad, frustrated, or a variety of other emotions and sensations (for a time including, controversially, fat). Some commentators drew a nostalgic connection with early Web 1.0 blogging and social media platforms like LiveJournal and MySpace, which also once allowed users to set their mood status (Kanalley, 2013). But despite the uproar over Facebook’s emotional contagion study and concern about the manipulation of user’s emotions, it is clear that quantifying, tracking, and manipulating emotions is a growing part of the social media business model. “By selecting your current activity instead of merely writing it out, you structure data for Facebook,” a TechCrunch columnist observed when the feature was rolled out. “If you choose a particular pre-formatted emotion, piece of media, or food, Facebook could potentially use that behavior to pinpoint you with ads” (Constine, 2013). Sentiment-analysis firms like Lexalytics are also working to incorporate emoji into their business models, providing data profiles grounded in emotion and mood to their customers. Lexalytics (2014) promises its clients that these profiles enable companies can “quickly filter through any irrelevant data and get down to what matters most to you”: better monitoring and modulating the flow of consumer desire.

So emoji have, in some sense, become the victims of informational capital in their turn. Alongside stickers, the spread of mobile communication away from pagers and simple SMS platforms to technologies such as AOL Instant Messenger (AIM), Skype, and Twitter has meant stability in the embedded graphic design of emoji-like, or “emoji-lite” images of smiley faces, ghosts, and cups of coffee, despite the fact that the underlying technical constraints of the 12-by-12 grid have been removed from these graphics. Skype and Facebook now present users with a palette of symbols functionally and aesthetically similar to emoji—skeuomorphs in their turn—but which are also graphics interchange format (GIF) files, not Unicode characters. While emoji persist as universally interoperable, social media users are already becoming accustomed to this wider, more colorful, and often more affectively expressive sets of images displayable by high-quality smartphone screens: an animated GIF keyboard, allowing users to insert moving pixel images into their messages, is now available for Apple’s iOS 8 (Bell, 2014). These shifts repeat the process of affective transformation and cooption of earlier digital forms, following the patterns Sumanth Gopinath observed with the mobile ringtone. The expansion of informational capital necessarily entails the constant shoring-up of the same affective, emotional, and social ties it must simultaneously exploit, in order to enable, in Negri’s words, “control of an immeasurable productive reality” composed of our own everyday affective bits and bytes.

Conclusion: The Conservatism of Emoji

Emoji, like the original smiley, could be understood as expressions of “cruel optimism.” Affect theorist Lauren Berlant (2011) defines this state as an affective zone in which “the object/scene that ignites a sense of possibility actually makes it impossible to attain the expansive transformation for which a person or a people risks striving” (p. 2). Emoji, like that original smiley, are prophylactic—they help people in digital environments cope emotionally with the experience of building and maintaining social ties within hierarchical technological platforms and unjust economic systems that operate far outside of their control. Yet the emancipatory potential of emoji is restricted by their industrial and commercial limitations. Like the smiley before them, emoji help affective laborers to, in Berlant’s words, “emit desire and identification with the affective ties of collegiality” even as the logic of capitalism constantly undercuts those social bonds (p. 218). Emoji offer us more than just a cute way of “humanizing” the platforms we inhabit: they also remind us of how informational capital continually seeks to instrumentalize, analyze, monetize, and standardize affect.

Representations of feeling in general, and happiness in particular, are often painted across the exterior of moneymaking ventures. Emoji are an exuberant form of social expression but they are also just another means to lure consumers to a platform, to extract data from them more efficiently, and to express a normative, consumerist, and predominantly cheery world-view. Where can resistance be found? Berlant suggests paying attention to “the messy dynamics of attachment, self-continuity, and the reproduction of life that are the material scenes of living on” (p. 15). In this light, emoji should be understood both as a rear-guard action to enable sociality in digital networks and also the means to quantify, measure, signal, and control affective labor, and reinforce existing regimes of inequality and exploitation. Yet particular appropriations of the character set, such as emoji political art, showcase the possibilities for affective expression in new forms of digital living and communal cultural production. As Berlant observes, “we understand nothing about impasses in the political without having an account of the production of the present” (p. 4). In order to find other similar strands of emancipatory potential in affect—whose “production, value, and subjectivity” is for Negri (1999), indestructible (p. 88)—we must first understand its contemporary trajectories, successes, and failures. As a blip in the continuing evolution of platform languages, emoji may be remembered as a conservative, conventional, but ultimately necessary step along the way.