Abstract
Young adults and college students who receive limited information about certain facets of sexuality, such as how to initiate sex and communicate consent, may turn to television and film for models. In fact, previous research suggests that young adults seek out media content such as television and film to obtain information about sexuality particularly in the absence of other sources of information. As such, it is important to understand the types of actions and behaviors modeled in mainstream media with regard to sexual activity. The overarching goal of the current study was to examine how sexual consent is depicted in mainstream film. However, we found there was limited information to guide the methodological design of such a study. Drawing on previous research, we developed a unique set of analytic procedures for conducting such a specific content analysis. The current article details our methodological approach including (1) how we developed our analytic procedures, (2) our rationale for developing these procedures, and (3) justification for deviating from previous procedures. This article concludes with a discussion regarding limitations of our study and lessons learned for conducting similar studies in the future.
Numerous studies have examined depictions of sexual content, including sexual behavior, in mainstream media as well as the potential influence such sexual content has on young people’s attitudes and beliefs toward sex and their sexual behaviors (e.g., Bleakley, Hennessy, Fishbein, & Jordan, 2008; Collins et al., 2004; Fisher, Hill, Grube, & Gruber, 2004; Kunkel et al., 2007; Pardun, L’Engle, & Brown, 2005). Given the high frequency with which young adults consume media, it is important to understand media’s role as a “sexual socializer” (Manganello et al., 2010, p. 364) for young people. Media’s role as a sexual socializer may be particularly salient for behaviors such as sexual consent communication, a topic rarely addressed in sexuality/health education curricula in public schools. As such, the initial purpose of the current study was to examine and document how sexual consent communication is portrayed among characters engaging in sexual activity in mainstream films.
While developing the analytic procedures for this study, we expanded our scope to include a range of factors associated with sexual activity. Sorsoli, Ward, and Tolman (2008) note that “paying close attention to the nuances in content seems particularly necessary when attempting to link viewing practices to complex social behaviors such as sexuality” (p. 149). Therefore, the overall purpose of the current study was to document how mainstream films depict several aspects of sexuality including orgasm, use of protective methods (e.g., condoms, birth control pill), use of sexual enhancement products, as well as several facets of consent communication. In order to do this, the four coauthors (referred to as “the research team” going forward) viewed 50 popular, American films released during 2013. Drawing on previous literature and using an inductive approach, we developed unique methodological procedures to assess our variables of interest.
Although there is a substantial body of literature describing analytic approaches to examine sexual content/sexual behavior in media (e.g., Bleakley, Jamieson, & Romer, 2012; Fisher et al., 2004; Nalkur, Jamieson, & Romer, 2010; Sorsoli et al., 2008) and conducting content analyses with media (e.g., Riffe, Lacy, & Fico, 2014), we found these procedures insufficient to guide this specific study. In the current article, we describe previous research in which sexual content in media were analyzed, our rationale for developing unique methodology, and a detailed description of our unique methodological approach.
Review of Literature: Content Analysis in Media
Content analyses have been conducted in several studies to assess depictions of sexual content or sexual behaviors in a variety of media. For example, Bleakley et al. (2012) assessed the occurrence of sexual content and the extent to which main female and male characters engaged in sexual behavior in top-grossing films in the United States released between 1950 and 2006. In this study, researchers defined sexual content as kissing on the lips, nudity, sexual behavior, and sexual intercourse. They also used a 4-point rating system to assess the explicitness of sexual content.
Similarly, Fisher and colleagues (2004) conducted a content analysis to assess the extent to which sexual content was depicted in popular television programming for youth and the nature of such content. In their study, the prevalence and frequency of sexual behaviors, sexual talk, and sexual responsibility were analyzed. They defined sex as “any depiction of sexual activity, sexually suggestive behavior, or any talk about sexuality or sexual activity” (p. 534). They provided explicit definitions for eight sexual behavior categories, four categories for sexual talk, and three categories for sexual responsibility. Additionally, like Bleakley et al. (2012), Fisher et al. also analyzed the explicitness of the sexual content using a 5-point scale.
Manganello and colleagues’ (2010) comparison of adolescents’ perceptions of the presence or absence of sexual content on television programming to those of researchers also provided some helpful guidance for the development of our coding procedures. In their study, Manganello et al. compared young adults’ and researchers’ perceptions of sexual content using modified versions of three different coding schemes. The coding schemes compared were developed by Kunkel and colleagues (2003, 2007), Kunkel, Eyal, Finnerty, Biely, and Donnerstein (2005), and Ward (1995) and based on the integrative model, a theory that incorporates constructs from health-behavior theories such as Theory of Planned Behavior (Fishbein & Yzer, 2003). Each of the three coding schemes used different variables and coding categories; we were able to draw strengths from the three schemes as well as identify pitfalls to help develop and refine our coding procedures based on our specific research aims. Similar to Bleakley et al. (2012) and Fisher et al. (2004), all three of the content analyses compared in this study examined the presence of various sexual content (e.g., sexual behaviors, sexual talk, portrayal of character’s sexuality, pregnancy/sexually transmitted infections (STI) prevention mechanisms). Although these studies provided a more comprehensive list of variables worth analyzing in mainstream film, to meet our research goals, a different methodological approach was required.
In addition to reviewing content analysis studies related to sexual content in media, we also sought out content analysis studies examining other variables in media such as violence (e.g., Collins & Carmody, 2011; Monk-Turner et al., 2004), gender (e.g., Collins, 2011; England, Descartes, & Collier-Meek, 2011; Neuendorf, Gore, Dalessandro, Janstova, & Synder-Suhy, 2010; Lindgren & Lelievre, 2009; Sorsoli et al., 2008), health and sexual health (e.g., Hether & Murphy, 2010; Hust, Brown, & L’Engle, 2008), and romantic relationships (e.g., Hefner & Wilson, 2013). Many of these studies provided general guidance and reinforced recommendations worth following in our analyses. Such recommendations included: clearly articulating operational definitions of variables/coding categories, adequately training coders, conducting reliability assessments, and using previous research to develop a strong theoretical and conceptual foundation for the study and analytic procedure (e.g., Manganello et al., 2010; Neuendorf, 2011; Potter, 2008; Sorsoli et al., 2008; Riff et al., 2014).
The Current Article
Bleakley et al.’s (2012) and Fisher et al.’s (2004) studies were ideal initial selections to provide important guidance when constructing the methodological procedures and codebook for our study as they represent systematic, methodologically rigorous assessments of two large samples of films (N = 855) and television programs (N = 1,276). Indeed, we drew on the coding categories provided by Fisher et al. (2004) as a foundation for our list of sexual behaviors and mimicked Bleakley and colleagues’ (2012) use of main characters for coding purposes. However, the goal of these two studies was to assess the extent to which sexual content occurred during segments of media; minimal additional analyses were conducted to contextualize the sexual content. It is likely the case that contextualizing the sexual content was out of the scope of these studies given the large sample size of film and television segments being analyzed. Because we were interested in documenting depictions of consent communication, significant contextualization of the sexual content was required. Thus, we were limited in the extent to which we could rely on these procedures exclusively.
Despite the comprehensive search, previous methodologies in isolation were insufficient to address our research questions because our research questions focused on assessing how consent is depicted in films, whereas prior media content analyses primarily examined whether certain behaviors (e.g., sexual behaviors, sexual talk) or factors (e.g., sexual responsibility, romantic themes) were present or absent. In the current article, we have provided a detailed account articulating our methodological approach including: (1) how we developed our analytic procedures, (2) our rationale for developing these procedures, and (3) justification for modifying or deviating from previous procedures. Additionally, we describe our analytic approach to conducting intercoder reliability (ICR) and present our reliability scores on all of the variables analyzed in the study. The article concludes with a discussion regarding limitations of our study and lessons learned for conducting similar future studies.
Methodological Approach
Coding Procedures: Development, Implementation, and Justification
We utilized a multistep, inductive process to develop the coding scheme for the current study; this section describes the development process and provides justification for the unique procedures utilized. The inductive process outlined below took place over the course of 3 months spanning over 100 hr of preparation time.
The research team members consisted of four sexuality researchers with backgrounds in public health. Three of the four were doctoral students in a public health PhD program at the time of the study; the remaining member completed her doctoral work in public health and is a faculty member in a public health department. All research team members had conducted content analyses of sexual behaviors prior to the time of this study; however, none of these content analyses involved media.
Step 1: Literature review
First, authors conducted an extensive review of literature as described above. We reviewed articles in which content analysis was utilized to examine sexual, romantic, and/or violent content in mainstream media such as films and television programming. The review of literature provided guidance regarding: (1) how sexual behaviors have been defined, (2) how to characterize the relationship status of characters, and (3) the extent to which specific variables of interest (e.g., sexual consent cues) had previously been examined in the literature. To our knowledge, the latter variable of interest, sexual consent cues, has yet to be examined in previous research.
We utilized the sexual behavior categories outlined by Fisher et al. (2004) to guide the occurrence of sexual behavior in films. However, we substantially revised their initial list; we added additional sexual behaviors (oral sex—e.g., kissing or licking of the genitals; manual sex—e.g., touching of genitals) and differentiated between receptive and performative behaviors. We added additional behaviors because previous research suggests that the types of cues utilized to communicate consent may differ based on the sexual behavior being engaged in (e.g., Hall, 1998; Jozkowski, Peterson, Sanders, Dennis, & Reece, 2014). Additionally, the research team theorized that consent cues likely differ depending on who was performing the behavior and who was receiving the behavior. For example, a person wanting to receive oral sex may use different cues to communicate their desire than a person who wants to perform oral sex (see Table 1 for the full list of the sexual behaviors coded for in the current study). Note that when the “other behaviors of a sexual nature” code was selected, the research team recorded the specific behavior characters engaged in for future analysis.
Units of Analyses Coded.
Note. MSW = Men having sex with women; WSW = Women having sex with women; MSM = Men having sex with men.
Justification for deviating from previous literature
Because the initial purpose was to examine sexual consent communication, specific factors that influence consent communication also needed to be considered. For example, previous research suggests that young adults endorse the notion that once a person has consensually engaged in sexual behavior with a partner, the partner likely expects that sexual intercourse will continue; this concept is known as sexual precedence theory (Shotland & Goodstein, 1992). If sexual precedence is assumed, viewers may perceive that characters with a sexual history require less explicit or different consent cues compared to characters engaging in sex as novel partners. Additionally, Humphreys (2007) found that individuals involved in a romantic relationship are perceived as requiring less explicit cues to communicate consent compared to partners about to engage in sexual activity for the first time. However, Foubert, Garner, and Thaxter (2006) found that college fraternity men reported feeling more comfortable using more explicit cues to ask a romantic partner for consent as opposed to a novel partner because the fear of rejection was lower with the romantic partner (i.e., they assumed that their romantic partner would be more likely to consent to sex). The men in Foubert et al.’s (2006) study also indicated feeling more confident in their ability to accurately interpret their romantic partners’ nonverbal cues compared to a novel partner. Considering these findings, it may be the case that characters’ portrayals of consent differ based on sexual history and relationship status. For these reasons, the research team concluded that it was necessary to record data related to characters’ sexual histories and relationship status.
Similar to the construction of the sexual behavior variable, we utilized the relationship status categories outlined by Fisher et al. (2004) as a starting point for defining our relationship status variable. However, we added three additional categories to their list: (1) the characters had met before in romantic context; (2) the relationship context is unknown, unclear, or uncertain; and (3) transactional sexual relationship (e.g., sex worker, prostitute). We coded characters’ relationship to the individual they were engaging in sexual activity with and noted any other relationship details in the narrative (e.g., if a character was married to someone else, not to the person they were engaging in sexual behavior with in a particular scene, they were not coded as married; their marital status with another person was noted in the narrative). Additionally, we created a variable to assess characters’ sexual history. For this variable, there were three categories: (1) there is a history of vaginal–penile intercourse (for a man and woman), penile–anal intercourse (for two men), and oral sex (for two women); (2) individuals do not have a sexual history of vaginal–penile, penile–anal, or oral sex; and (3) we are uncertain if the characters have a sexual history. These two variables (relationship status and sexual history) along with our variable assessing the sexual behavior being engaged in by the characters during a particular sex scene were the first variables to be articulated in the team’s coding manual (i.e., a document articulating descriptions and operational definitions of variables we intended to code for when viewing films; see Table 1 for a description of each variable and the codes utilized to assess each variable).
Rationale for analyzing films over television
The decision to record data related to sexual history and relationship status in turn influenced how we selected our media genre. The research team made the strategic decision to assess mainstream films as opposed to television programming (see Table 2 for a complete list of the films included in the sample). With the exception of sequels, the plot lines of films are generally self-contained. In other words, viewers are introduced to a set of characters and circumstances at the start of a film and are expected to make sense of these characters’ behaviors and interactions throughout the film. Viewers are generally not expected to have previous knowledge or understanding of characters’ dispositions, relationships, and so on prior to the start of the film but will learn this information as the film progresses. Alternatively, television shows that span several episodes and seasons carry plot lines, relationships between characters, and histories between characters from episode to episode. Therefore, unless one viewed an entire series from a particular television program, it would be nearly impossible to account for all the relationship nuances that occur between characters in specific episodes.
Film Genres (According to IMDb).
Note. N = 50. IMDb = Internet Movie Database.
aIMDB also indicted the genre “Mystery” for Now You See Me. bIMDB also indicated the genre “Sport” for 42.
Without the ability to fully assess characters’ sexual history and relationship status in television programming in a manageable way, we concluded that our consent data would be incomplete. As such, we opted to examine the portrayals of sexual encounters and consent negotiations in mainstream films. That being said, there were eight films in our sample which were sequels to previous films (Hunger Games: Catching Fire, Grown-ups 2, The Hangover Part 3, Anchorman: The Legend Continues, The Hobbit: the Desolation of Smaug, Insidious Chapter 2, Star Trek: Into Darkness, Thor 2: The Dark World). For these films, the relationship statuses and sexual histories of characters appearing in previous films were not considered in the current study; the research team only utilized the information provided in the specific film being viewed to assess these variables. However, the relationship status and sexual histories from previous films in succession were generally made apparent to viewers early in the sequel film.
Step 2: Sex and the City and initial development of consent cues
After completing the literature review, the research team used an inductive process to begin developing operational definitions of codes to analyze characters’ consent communication cues. Although the research team decided to use full-length films for our sample, we intentionally selected specific television programming for consent code development because these particular television shows provided a greater opportunity for exposure to scenes involving sexual activity in a shorter period of time than could be viewed in a full-length film. Television shows airing on cable networks, such as HBO, were selected for this initial coding development because there are more frequent and more explicit sexual activities shown on such cable networks.
To begin the inductive development process, each author individually viewed an entire season of HBO’s Sex and the City series (Star & King, 1998); each team member viewed a different season to increase diversity of media being viewed. Each time a scene involving sexual behavior occurred in an episode, the team member wrote a detailed narrative of the events involved in the scene including verbatim dialogue. The research team decided to record partnered sexual activities that involved two or more people. As such, we did not include masturbation in our analyses. We rationalized that consent communication for a solo-sexual behavior was not relevant and, therefore, should not be included in the current study.
After each team member viewed their respective season of the show, all four team members met to discuss their narratives of the sex scenes. Based on the narratives and discussion, the research team saw a number of consistencies between the consent themes developed via the inductive process and those cues/behaviors derived from Hickman and Muehelenhard’s (1999) study. Similar to Hefner and Wilson’s (2013) use of interpersonal relationship research to develop their coding categories, we decided to base our codes for the consent communication variable on five of Hickman and Muehlenhard’s seven subscales for defining our consent coding scheme. These five subscales were: (1) direct verbal cues, (2) direct nonverbal cues, (3) indirect verbal cues, (4) indirect nonverbal cues, and (5) no response cues (see Table 3 for specific definitions and examples of these codes). We modified Hickman and Muehlenhard’s language slightly. This modification was made because the research team believed the terms explicit and implicit provided a more accurate description of the cues compared to direct and indirect, respectively.
Operational Definitions and Examples for Consent and Refusal Codes (Based on Hickman & Muehlenhard, 1999).
The two subscales excluded were direct refusals and intoxication signals. The direct refusals subscale was accounted for in our coding of refusals (described below). Cues emerging on Hickman and Muehlenhard’s (1999) subscale for intoxication signals (e.g., saying “I’m really drunk” or “I’m feeling a little drunk”) did not emerge in our inductive process. Additionally, we accounted for the characters’ intoxication level via another variable, substance use, in which we analyzed whether characters consumed alcohol and/or used other drugs (e.g., see Table 1) prior to or during sexual activity. Thus, Hickman and Muehlenhard’s (1999) cues for intoxication signals were not included as a consent code in our study.
Step 3: The To Do List and the development of refusal signal cues
The research team then began applying the initial coding manual to films. We individually viewed The To Do List (Todd & Carey, 2013) and applied the five-code consent coding scheme described above. This film was strategically selected because of the diversity in sexual behaviors depicted in the film. Two important developments were uncovered at this stage. First, we realized that a coding scheme for characters who refused sexual advances needed to be developed as we observed situations in this film in which one character initiated sexual activity and, in response, another character refused. To accomplish this, we utilized a similar framework for refusals as suggested by our inductive process for consent and Hickman and Muehlenhard’s (1999) study: explicit verbal refusal, implicit verbal refusal, explicit nonverbal refusal, and implicit nonverbal refusal (see Table 3). Second, we realized that the coding categories needed to be clarified and refined. For both indicators of consent and refusals, we developed specific operational definitions and supplemented these definitions with specific examples from the media we had previously viewed for the creation of the coding manual. An additional category called “no consent shown” was also added to the consent coding scheme to account for scenes involving sexual behavior in which the scene begins with the sexual behavior already in progress such that it was impossible to determine how characters communicated consent. See Table 3 for the operational definitions of the consent and refusal codes created and utilized in the current study. These descriptions and definitions were included in the team’s coding manual.
While viewing The To Do List, the research team realized that a range of other factors related to sexual behavior, but different than consent, may be interesting to document. As such, additional variables along with descriptions and operational definitions were added to the coding manual. These variables included: (1) initiator/gatekeeper of the sexual behavior, (2) occurrence of orgasm during sexual behavior, (3) method of contraception or other forms of protection utilized during sexual behavior, (4) presence of sexual enhancement products (e.g., vibrators, lubricant, lingerie) during sexual behavior, and (5) substance use (e.g., alcohol, cocaine) of characters prior to/during sexual behavior (see Table 1 for a description of each variable and codes utilized to assess each variable).
Although viewing and coding The To Do List helped to refine the consent codes, it was clear that our codebook was not yet comprehensive, thus more development was necessary. As such, the research team decided to go back to viewing television programming, so we could maximize exposure to scenes including sexual behavior in a shorter time period to further refine the codebook. The research team viewed and individually coded several other media including episodes of HBO’s series True Blood (Harris et al., 2008) and Sex and the City (Star & King, 1998). Each team member individually analyzed the scenes and applied their own codes. Each team member was allowed to view scenes as many times as she believed necessary in order to fully analyze the scene. This process helped to further refine the coding manual.
Specifically, it was determined at this stage that multiple consent codes could be applied to a particular scene depending on what cues, behaviors, or words were utilized by characters to communicate consent as interpreted by the research team members. Therefore, when assessing a scene for consent cues, coders recorded one or more cues they interpreted the characters using to communicate consent for a particular behavior. For example, if a coder interpreted that a character engaged in an implicit nonverbal cue followed by an explicit verbal cue to communicate consent, the coder would record 4 → 1 as the consent code. If two cues occurred at the same time, for example, an explicit verbal cue and explicit nonverbal cue, the coder would use a slash to indicate the codes: 1/3. If multiples of the same type of cue occurred consecutively, all cues were collapsed into a single code. For example, if characters said “I want to have sex with you! [Pause]. Let’s have sex!” it would be coded as one explicit verbal cue despite being two separate statements. Therefore, both the type and order of consent cues were important; the coders assessed for agreement on both of these criteria when comparing codes and discussing discrepancies.
Coding separately procedure and shortfalls
For some of the True Blood (Harris et al., 2008) and Sex and the City (Star & King, 1998) episodes, the team members coded in different locations and at different times and then arranged a time to meet to discuss discrepancies. For other episodes, the team members individually coded in the same room, at the same time, and immediately discussed discrepancies after each scene had been coded. The research team quickly realized that coding films in separate places/times and meeting afterward to address discrepancies posed several challenges. First, some team members interpreted certain scenes and behaviors as being worthy and appropriate of coding, whereas other team members did not. For example, scenes where sexual behavior did not specifically occur (because the behavior was refused or the characters were interrupted) were sometimes overlooked by one or two team members. It was also difficult to recall pertinent information from the scenes when meeting for discussion; this made it challenging to resolve discrepancies in coding.
Coding together procedure and final decision
For media that were viewed together, the team members met in the same room and viewed the media at the same time, and then individually coded and recorded their responses on individual coding sheets. The group determined, by consensus, when a scene that included actual or possible sexual behavior requiring analysis took place, and then applied codes for each of the variables of interest (see Table 1) individually. After the team members had adequately viewed and coded the scene, the team members immediately compared codes and discussed discrepancies. For situations in which discrepancies existed, the team repeatedly viewed the scene as a group to come to consensus.
The research team made an intentional decision to use the latter procedure in which team members coded films individually, but at the same time, in the same location. Given that the purpose of this study was to assess depictions of sexual consent communication in films, it was necessary for all coders to agree on the occurrence of sexual behaviors that would require coding of consent or refusals cues. As such, it was determined that for the current study the research team would view media in the same location at the same time, but each coder would individually assess and code scenes requiring analysis by recording their codes for all variables of interest on an individual coding sheet. After each coder completed their analysis, the team members discussed discrepancies and came to consensus before moving on in the film. The research team rationalized that this would be the most pragmatic way to approach coding films so that scenes requiring analysis could be determined as a team and discrepancies could be addressed immediately.
It was also determined that the research team would create a rotating schedule in which only three of the four team members would code a given film so a consensus could be determined with at least two out of three coders reaching agreement. After consensus among the coders was reached, the team created a master coding sheet which included the agreed upon codes for all of the variables being assessed (see Table 1). Each individual coders’ separate coding sheet was then utilized for ICR. We kept record of each coder’s assessment for each scene for reliability purposes but utilized the master coding sheet to record all final codes included in our data set.
Because the research team decided to code films together, during viewing, specific roles were assigned to each of the three coding team members. One individual was responsible for transcribing the narratives of the scenes being analyzed (i.e., specific details of the scene and verbatim dialogue of characters involved in the scene). A second individual was responsible for pausing and rewinding the film so that the research team members could view each scene multiple times. Finally, the third person was responsible for recording the final list of agreed-upon codes and points of discrepancy by the coders for reliability purposes on a master coding sheet.
Step 4: Sex Drive, Don Jon, and coding procedure implementation and refinement
After development of a more refined coding manual and coding procedures via Step 3, the research team selected films to pilot. Similar to The To Do List, two films, Sex Drive (Levy, Morgenstein, Morris, Nelson, & Anders, 2008) and Don Jon (Bergman & Gordon-Levitt, 2013) were strategically selected because of their frequency of sexual content. During this process, additional consent variables were added to the coding manual: precommunication of interest in sexual behavior, perception of sexual behavior as consensual versus nonconsensual, and location of consent communication. These variables are described in Table 1 as well as in detail below. Additionally, the research team decided only to code sexual behaviors that occurred among characters portrayed in the film. Such behaviors could occur in a character’s fantasy, dream, or memory as well as in the “reality” of the film. However, sexual behaviors occurring in pornography that might be playing in the background of a particular scene were not analyzed in this study; this decision was based on Bleakley et al.,’s (2012) coding procedures.
Step 5: My awkward sexual adventure and final procedures
After completing Step 4, the coding manual and procedures seemed to be finalized. In order to assess for initial reliability, the research team viewed another strategically selected film in its entirety (My Awkward Sexual Adventure; Hagopian, Chernick, & Garrity, 2012) and several scenes from another film (The Counselor; Scott, Wechsler, Schwartz, Schwartz, & Scott, 2013) in order to apply our refined coding procedures and examine the extent to which the team was consistent in coding. Based on our ease of coding and high rate of agreement, it was determined that our coding manual was comprehensive and our procedures were well articulated; thus we started analyzing films from our sample.
Selection Criteria and Justification for Film Sample
Our overarching goal for the study was to examine the ways in which sexual consent communication is displayed in mainstream films. We were interested in films viewed by college students due to the high rates of sexual assault experienced by college women (e.g., Cantor et al., 2015; Daigle, Fisher, & Cullen, 2008; Fisher, Cullen, & Turner, 2000; Kilpatrick, Resnick, Ruggiero, Conoscenti, & McCauley, 2007; Krebs, Lindquist, Warner, Fisher, & Martin, 2007, 2009) and the national attention focused on college students, sexual assault, and sexual consent (e.g., Anderson, Svrluga, & Clement, 2015; Office for Civil Rights, 2011; White House, 2014; White House Task Force to Protect Students From Sexual Assault, 2014).
The current peer-reviewed literature does not provide strong guidance regarding how to select samples of films to review. Some studies utilized top-grossing films that stretched over several decades (Bleakley et al., 2012; Hefner & Wilson, 2013; Nalkur et al., 2010). For these studies, coders typically viewed segments of the film looking for violent and sexual content. Other researchers selected specific segments of film or television programming to assess with little justification (Fisher et al., 2004; Manganello et al., 2010). Because our research questions and methodological approach deviate substantially from procedures utilized in past studies, we needed narrower selection criteria for selecting our sample.
In deciding how to select a sample of media, we considered the diversity of media available to us, our research questions, and the audience we aimed to generalize to as suggested by others conducting content analysis with media (e.g., Bridges, 2010; Manganello & Fishbein, 2008; Ward, Gorvine, & Cytron, 2001). We arrived at our sample of films through a multistep, strategic process. First, we compiled lists of films released in 2013. We narrowed our sample to include only films that were spoken in English. Next, we limited our sample to films with a Parental Guidance (PG-13) rating or higher as such films are more likely to contain sexual activity (Motion Picture Association of America Inc., 2007). Because we aimed to understand what messages might be conveyed to college students via mainstream films regarding consent communication, we selected films that were most popular among this demographic. To do this, we provided a list of all films with a rating of PG-13 or higher that were in English to a convenience sample of college students (N = 120). We asked if they had previously viewed the film (response options: yes/no). We cross-referenced the most popular films according to our sample (i.e., those films which had the highest record of students indicating they had viewed the film) with a list of top-grossing films according to the Internet Movie Database to assess different film genres. This process resulted in a diverse list of films (n = 50) with many film genres represented (see Table 2).
Variables Included in the Current Study
A total of 11 specific variables associated with characters’ engagement in sexual activity were collected for each scene in which sexual activity occurred for all 50 films (see Table 1). In addition to these 11 variables, we also recorded the demographic characteristics (assumed gender, approximate age range, and assumed race/ethnicity unless specifically stated) of the characters who engaged in sexual activity. The characters’ names, or a description of the character if the name was not provided, were recorded as well. Each scene in which sexual behavior occurred for each character was labeled with a unique identification number (see Table 1 for a description of numbering the scenes and characters). If characters engaged in multiple sexual behaviors (e.g., kissing, performative oral sex, and then vaginal–penile intercourse) during a particular scene, each behavior, along with the other variables (e.g., initiator/gatekeeper, protection, use of enhancement products, consent/refusal cues) included in the coding manual were coded for each behavior. Scenes were also subsequently described in a textual narrative including verbatim dialogue between characters preceding sexual behavior. It was decided to include the textual narrative so that we could reference this information during data analysis stages and include additional nuances that may be relevant to the sexual behavior and/or consent cues but would not be captured in our quantitative codes.
Table 1 names each variable and provides a brief description of the variable as well as the response options that coders could select from when coding each variable. The scene ID, character names, and demographic characters are transparent. Other variables such as initiator/gatekeeper, sexual behavior, relationship status, and sexual history were defined based on previous research as articulated above and in Table 1. Four additional variables (orgasm, protection, use of enhancement products, and substance use) were also coded. For all four of these variables, coders recorded their presence or absence (e.g., whether or not a character experienced orgasm during the scene; if condoms or other forms of birth control/protective mechanisms were utilized for relevant behaviors; whether or not a character used a sexual enhancement product during their sexual behavior; whether or not the character had used psychoactive substances prior to or during sexual behavior). The research team also assessed if the variable was depicted or merely mentioned (orgasm, protection, and use of enhancement products) and what type was used (use of enhancement products and substance use). Table 1 provides additional descriptions of these variables.
Consent variables were created specifically for this study. Four specific variables related to consent were coded for each sexual behavior: (1) precommunication of interest in sexual behavior, (2) perception of sexual behavior as consensual or nonconsensual, (3) location of initial consent communication, and (4) consent and refusal codes. Some of these variables are not intuitive. Therefore, we have provided additional explanation regarding these variables due to their importance and relevance to the current study. Below we have described the first three variables, Table 3 includes additional information regarding the operational definitions and specific examples of the consent and refusal cues utilized as codes in the current study.
Precommunication of interest in sexual behavior: This variable refers to a situation in which the viewer is provided information about a character’s willingness to engage in sexual activity that the potential sexual partner would not be privy to. For example, if the viewer can hear an internal monologue from a character in which that character expressed his/her desire to have sex with another character, the viewer would be privy to this information; such information may influence what the viewer may interpret regarding consent. However, that character’s potential sexual partner would not be privy to this information, so, in theory, it should not influence consent negotiation between individuals. This variable makes a distinction between what is possible in real-life sexual situations where we cannot hear another person’s internal monologue compared to what viewers would be exposed to in film. We thought it was important and relevant to record this information because it may influence viewers’ perspectives on consent.
Perception of sexual behavior as consensual or nonconsensual: The consent codes created for this study were designed to describe the cues/behaviors characters who engaged in sexual behavior used to communicate consent to their respective partner(s). In addition to defining and recording the consent cues, we were also interested in the coders’ perceptions of the sexual behavior as consensual or nonconsensual. We conceptualized consensual sex as sexual behavior in which all individuals participating freely agreed, verbally or nonverbally, to the behaviors engaged in. Nonconsensual sex was defined as sexual behaviors that at least one character did not agree to, was forced to engage in, or coerced into.
This variable was created because we thought it would be interesting to compare the extent to which the research team consistently interpreted a behavior as consensual (or not consensual) regardless of the types of consent cues that were utilized by characters to communicate consent (or refusal). Through the refinement process, we decided that we would make consent determinations at the event level and not the individual level. That is, if a man grabbed a woman’s butt after she told him not to touch her, we coded the entire event as nonconsensual as opposed to coding it as consensual for the man and nonconsensual for the woman. After all, if one person in a dyadic sexual encounter does not agree/consent to the behavior(s), the sexual encounter is nonconsensual.
Furthermore, while refining the coding manual in Step 3 above, the research team encountered scenes that had a distinct change in tone with regard to consent. Some scenes began with a person refusing sexual behavior that was progressing and then ended with the same character happily engaging in the behavior (e.g., they finally “gave in to temptation”). Other scenes began consensually but, because of some unexpected circumstance, ended nonconsensually (e.g., a blindfolded woman learns she is not having sex with whom she thought she was having sex with). Therefore, additional codes of “began consensual and ended as nonconsensual,” “began as nonconsensual and ended as consensual,” and “unsure” were added to the coding manual.
3. Location of consent communication: Some previous research suggests that young adults perceive consent as a process in which several subtle nonverbal or verbal cues are considered collectively and interpreted as consent to a particular sexual behavior like sexual intercourse (e.g., Beres, 2014; Humphreys, 2004; Authors, redacted). Specifically, Jozkowski and Hunt (2013) found that college students perceive this process of consent communication as beginning in social settings such as at parties, in a bar, or at social gatherings. In fact, both Beres (2010) and Jozkowski and Hunt (2013) found that young adults perceived that the transition home from a social setting (e.g., bar) to a private residence to be an indicator of consent. As such, we accounted for this potential perception of consent negotiation (i.e., consent as a process) by coding the “location” for which it seemed the characters in film began initiating the consent cues articulated by Beres (2010) and Authors (redacted). Specifically, Authors (redacted) utilize the terminology “outside the bedroom” to refer to situations in which college students perceived consent negotiation beginning in a public space or in the context of a social setting, generally a different physical location from where sexual behavior actually occurs. Alternatively, the term “inside the bedroom” refers to consent negotiation that begins in a private space or where the actual sexual activity ultimately takes place. For situations in which the scene started with characters already engaged in sexual behaviors, the research team used the code “not applicable” as prior negotiations of consent were not shown to the viewer. In such instances, the no consent shown consent code was utilized.
Intercoder Reliability
There were 486 behaviors over 189 scenes. Due to the subjective nature of observational data and the use of multiple coders, it was necessary for us to assess the level of agreement between the coders across the variables. In previous content analysis studies, it was common for researchers to train students or other individuals to code the media (e.g., Bleakley et al., 2012; Fisher et al., 2004; Hefner & Wilson, 2013). Indeed, Neuendorf (2011) states “there is nothing more important to valid and reliable measurement process than coder training” (p. 283). In the current study, only the four research team members coded the films. The decision to only have the research team members code was realized during the construction of the codebook given the complexity of the coding scheme. As such, the research team members were highly trained in the analytic procedures consistent with recommendations (e.g., Neuendorf, 2011).
Common practice for assessing reliability in content analyses is to compare a random sample of 10–20% of at least two coders’ codes, aiming for agreement at least 70% of the time (e.g., Neuendorf, 2011). Our study deviates from previous research in that three of the four research team members coded each film; thus, the reliability assessments compared the responses for each variable between all three coders for 100% of the films included in the sample. As such, our reliability assessments resulted in more comprehensive estimations of coding consistency because we analyzed all codes for each research team member rather than utilizing a subset of codes to estimate reliability as previous researchers have done.
ICR scores were calculated for all observed variables, except character demographics, sexual behavior, and consent and refusal cues, to assess the consistency between coders. ICR was not calculated for sexual behavior because it was important for the coders to come to consensus about a behavior occurring or not prior to coding each scene; thus, the coders came to a consensus on the behavior(s) for each scene prior to analyzing scenes for the remaining variables. Similarly, the coders came to a consensus on the character demographics eliminating the need to calculate ICR for those variables.
We determined that calculating Light’s κ (Light, 1971) to measure ICR for all other variables, with the exception of the consent/refusal cues variable, was most appropriate because data were nominal, and we had more than two coders coding each film. Light’s κ was calculated by conducting Cohen’s κ in SPSS Version 23.0 (IBM Corporation, 2015) for all coder pairs then computing the arithmetic mean for each variable. For example, the Cohen’s κs for each coder pair on the precommunication variable was .871, .824, and .847; thus, Light’s κ was .85 [(.871 + .824 + .847) ÷ 3]. All ICR coefficients were above .60 indicating substantial agreement with the majority. In fact, the ICR for most of the variables was above .81 demonstrating “almost perfect” agreement according to Landis and Koch (1977, p. 165).
Calculating a traditional ICR computation for the consent/refusal cues variable was not appropriate because the variable violated the assumption that each coder was required to have an equal number of observations for a given variable. In other words, Coder A could observe and enter the code “3 → 4,” Coder B could record the code “2 → 3/4,” and Coder C could report the code “4” as the consent/refusal cues variable for a single scene making the number of observations among the coders unequal and violating this important assumption of traditional ICR calculations. Therefore, the research team developed a percentage of agreement calculation for the consent/refusal cues variable that measured the degree of agreement between the coders and the master coding sheets for each film. The percent agreement calculation was completed by dividing the total number of observations for each of the three coders that matched the master coding sheets by the total number of all possible observations among the three coders. For example, if the master coding sheet read “3/4” for a given scene and the three coders reported the above cues, the percentage of agreement for that given scene would be 83% (5 ÷ 6). This is because Coders A and B both reported two codes that matched the master coding sheet for that scene, Coder C only reported one code that matched the master coding sheet, and there were a total of six codes reported between Coders A, B, and C for that scene.
Table 4 includes the ICR coefficients and percentage of agreement for all appropriate variables. The high levels of agreement likely reflect the thoroughness of the analytic procedures developed during this study and the refined detail of the codebook. We anticipate that such level of detail in the methodological development for this study could make replication straightforward in the future.
Intercoder Reliability for all Variables.
Limitations and Lessons Learned for Future Research
In this article, we aimed to articulate our coding procedures so that individuals interested in replication or conducting similar research may use this information as a framework for guiding such studies. Like every study, however, there are important limitations to note. Below, we outlined limitations we confronted during this study and provide suggestions for how others may consider addressing and overcoming such limitations in future work.
Diversify Coding Teams
The research team for the current study consisted of White non-Hispanic, highly educated (master’s degree or PhD), cisgendered women under the age of 35 who work in the same academic discipline. Although members of the team grew up in very different geographical locations, it is likely that the cultural lens applied to the coding for the project was highly similar between coders. This may have helped achieve consistent ICR. However, we believe the lack of cultural diversity in coder perspectives is a limitation of the study. We suggest that future research specifically seek out coders that will bring diverse perspectives to the project.
Avoid Coder Fatigue
Typical films viewed for the current study lasted between one and a half to 3 hours; the coding process almost always took far longer than screen time alone. During our study, we found that coding sessions lasting longer than 3 hours resulted in fatigue issues that raised concern of increased error. To remedy this, each week, three coders spent 2–3 hr/day, 2–4 days/week, coding films. The number of coding sessions and length of the sessions depended on the length of the film as well as the number of scenes in the film that required analysis. The research team determined that they would analyze one film per week. As an additional mechanism to ease coder fatigue, the four-person research team rotated coding responsibilities on a monthly basis. As such, each coder was afforded one week “off” during the 4-week cycle such that three coders coded each film. We found this methodological approach to be helpful and endorse the practice for future studies.
All Coders Should View Every Film
Due to the practice of having an “off-week” for each coder, we encountered another issue when we began analyzing narration data (i.e., the recorded descriptions of scenes including sexual behavior and the verbatim dialogue preceding and during such scenes) collectively as a group at the conclusion of the study. Some coders were not familiar with specific scenes of interest because the film had been coded during their off-week. Therefore, we recommend that during a coder’s off-week, he/she still view the film, though this can be done separate from the coding team. We recommend this practice so that even coders who did not analyze a particular film scheduled during their off-week still be familiar with the film in general including important scenes involving sexual behavior or other themes.
Allow Substantial Time for Codebook Generation and Refinement
Although we detail a five-step process for refining our codebook above, the number of hours that were required to create the final codebook far exceeded our initial expectations. Generating many of the procedures ourselves with limited guidance from previous studies was a very time-intensive task. Future researchers generating their own procedures should expect to dedicate many person-hours to a study of this magnitude. It is our hope that, through using our process as a guide in future studies, front-end preparation for a similar analysis can be greatly reduced.
Coding Procedure Modifications are Likely to Happen
Sometimes codebook modifications happen even after extensive preparation. The decision to make modifications will most likely depend on the importance of the variable to the research question. For example, while coding our films we encountered sexual behavior that occurred between main characters and sex workers. We then had to add an additional relationship status category of “transactional sex” to our codebook because we believed that our existing codes would not capture important nuances related to transactional sex. Because previous literature suggests that relationship status may affect the types of consent cues that people use (e.g., Humphreys, 2007), modifying this variable was important for our study. Future researchers may find it necessary to go back and recode scenes that had been previously coded or make a decision to leave the original coding procedures intact.
Conclusion
Given the vast viewership mainstream films have and the sexual content found in these films, studying sexual health variables in film can be a worthwhile avenue for research. This article provides a detailed description of how we developed our analytic procedures and offers several suggestions of methodological choices that future researchers may make while analyzing sexual health depictions in films. Through detailing the sample selection, variable creation, and codebook refinement processes, as well as rationale for each of these, it is our intention to not only make our procedures as transparent as possible but also to create a guide other researchers can utilize. It is our hope that, in doing this, other researchers may be able to minimize preparation time for such studies by building upon our methodology in a manner that suits their research interests.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.