#ec: Findings and implications from a quantitative content analysis of tweets about emergency contraception

Abstract

Twitter, a popular social media, helps users around the world quickly share and receive information. The way in which Twitter frames health issues – especially controversial issues like emergency contraception (EC) – can influence public opinion. The current study analyzed all English-language EC-related tweets from March 2011 (n = 3535). Variables measured user characteristics (e.g. gender), content (e.g. news, humor), Twitter-specific strategy (e.g. retweet), and certain time periods (e.g. weekends). The analysis applied chi-square and regression analyses to the variables. Tweets most frequently focused on content related to news (27.27%), accessing EC (27.27%), and humor (25.63%). Among tweets that were shared, however, the most common content included humor, followed by personal/vicarious experience. Although only 5.54% of shared tweets mentioned promiscuity, this content category had the strongest odds for being shared (OR = 1.51; p = 0.031). The tweet content with lowest odds of being shared were side effects (OR = 0.24; p < 0.001), drug safety (OR = 0.44; p < 0.001), and news (OR = 0.44; p < 0.001). Tweets with the greatest odds of having been sent on a weekend sought advice (OR = 1.94; p = 0.012), addressed personal or vicarious experience (OR = 1.91; p < 0.001), or contained humor (OR = 1.56; p < 0.001). Similar patterns occurred in tweets sent around St. Patrick’s Day. Only a few differences were found in the ways in which male and female individuals discussed EC on Twitter. In particular, when compared to males, females mentioned birth control (p = 0.002), EC side effects (p = 0.024), and issues related to responsibility (p = 0.003) more often than expected. Study findings offer timely and practical suggestions for public health professionals wanting to communicate about EC via Twitter.

Keywords

Twitter emergency contraception social media content analysis agenda-setting

Background

Social media use continues to increase among most age groups and ethnicities around the world, with an estimated 2.55 billion people accessing it by 2017.¹ Twitter is one of fastest growing social networking sites, with use almost doubling since November 2010.² Approximately 18% of adults online use Twitter, with the greatest proportion of use among 18–29 year-olds.² There is an ever-increasing user base with over 255 million active users who send approximately 500 million tweets a day.³ Twitter is also truly a global networking site, with support of over 35 languages and 77% accounts being outside the United States.³

With such a wide breadth and depth of reach, Twitter has the potential to quickly spread information around the world. Social media, including Twitter, has a number of uses in public health, including tracking illnesses over time as well as exploring the knowledge, beliefs, and attitudes of health-related behaviors.^4–6 Further, doctors and other health professionals may use social media to share important health information with patients via social media.⁷ Although there has been research on the effectiveness of public health messages in social media, little research is available specific to how matters of sexual health are presented.⁵

Emergency contraception

Emergency contraception (EC) – also referred to as the “morning-after pill” and the “day after pill” – is a birth control method that prevents pregnancy after unprotected sex.^8–10 There are several methods of EC available, including several pill options as well as intrauterine devices (IUD).⁸ The effectiveness of EC methods depends on the time taken after unprotected sex, whether a women is in the fertile phase of her menstrual cycle, as well as the method itself.⁸ For example, EC pills which contain only progestin (levonorgestrel) are 89% effective if taken within three days after unprotected sex, whereas EC pills which contain Ulipristal acetate pills have been found to be 85% effective up to 120 hours, or five days, after unprotected sex.^8,11 The copper IUD, on the other hand, is 99% effective if inserted within five days post unprotected sex.⁸ Studies have also repeatedly found EC to be safe, with minimal side effects.^8,11–15

Although EC is internationally supported, access to EC varies across the globe. EC pills containing levonorgestrel are on the World Health Organization’s (WHO) essential medicines list and EC is discussed in major family planning publications and guidelines.¹⁶ There are 148 countries that have at least one registered EC product and 20 countries that provide EC directly over the counter.¹⁷ However, one study found that a little over half of the public sectors in low-resource countries offer EC.¹⁸ In addition, analysis of demographic health surveys from 45 countries showed gaps exist around the world in women’s knowledge about EC.¹⁶ Knowledge about EC ranged from only 35% in Latin America to as little as 11% in Asia.¹⁶ Furthermore, women with greater education and wealth, as well as women who lived in urban areas, reported higher rates of knowledge about EC.

Even if EC is available, perceptions about EC may play a key role in accessibility and utilization. In a study which interviewed health care workers in India, Senegal, and Nigeria (n = 66), providers were found to withhold information about EC and even refuse to provide it due to negative perceptions about the drug.¹⁹ The majority of providers in the study did not support EC being available over the counter or providing EC in advance of need. Providers believed such provisions would lead to increased sexual risk-taking.¹⁹ Sexual risk-taking can be defined as any sexual activity which increases the risk of an individual contracting a sexually transmitted infection (STI) or experiencing an unintended pregnancy.²⁰ This negative perception of EC is one held by some health care workers and some of the general public.²⁰ However, scientific evidence demonstrates that greater accessibility to EC does not increase sexual risk-taking.

Information posted on social media platforms, such as Twitter, can serve to address common knowledge gaps as well as to generate dialogue about issues such as the accessibility, availability, and utilization of EC. Using Twitter to begin to understand the public’s perception about EC can help identify more effective strategies to increase political and public support of EC. Such support may ultimately foster a more receptive environment for EC and has the potential to influence utilization.

Media priming and agenda-setting

Identifying the ways in which traditional media effects are applied in social media may create opportunities for organizations to more effectively promote public health messages. Two relevant media effects have been identified which help to explain ways in which media may influence perceptions or behaviors. These are media priming and agenda-setting. Both demonstrate the way in which the presentation and temporality of information can influence decision-making.²¹

Media priming is the residual, sometimes unintentional effect of media exposure, which may affect judgments, perceptions, and behavior.²² This concept is generally discussed in tandem with agenda-setting theory, which suggests that emphasis of topics in mass media is related to how the public perceives the importance of these topics.²³ This emphasis makes the issues easier to recall, or more salient, so the public places greater importance on the topics.

The effects of media priming and agenda-setting can be substantial, and previous research supports the notion that greater exposure to mass media can affect perceptions and stereotypes by reactivating previously formed biases.^24–26 For example, the more an individual encounters a specific position in the news about a particular issue, the greater significance the individual may place on this position when forming opinions about the issue. Previous studies have suggested that the frequency and ways in which topics, including those about health, are presented in the media have the potential to affect people’s perceptions and opinions.^24–27 Both media priming and agenda-setting theories have been used to explain this effect, emphasizing that public opinion may be significantly influenced not so much by the content of messages in the media, but rather by the frequency of the messages. This concept is especially relevant when looking at social media platforms such as Twitter, which encourage the sharing and reposting of messages. Better understanding what content is most frequently posted and shared between users may provide insight into potential influences on public perception about specific issues. Public health practitioners, in particular, may be able to leverage the potential effects of media priming and agenda-setting by promoting the dissemination and repetition of specific health messages that encourage healthy behaviors.

The purpose of the current study, which analyzed tweets from March 2011, was to explore the ways in which EC is presented on Twitter and provide insight to public health practitioners and other professionals interested in fostering greater uptake of EC when needed. Such insight can help inform the development of more effective and appropriate public health messages which employ social media such as Twitter.

Methods

Sample

The initial study sample consisted of all English-language tweets posted in March 2011 containing the words “emergency contraception,” or “morning-after pill,” or “day after pill.” All tweets, regardless of whether or not they were repeats, were included in order to have a better estimation of the complete census of tweets related to EC that were posted during the time period of interest. The tweets were collected through an online social media listening tool, which captured all English-language tweets containing the specific keywords during the study time period. Tweets were manually downloaded on a daily basis to ensure that a true census of all relevant tweets was included. March 2011 was chosen as the sampling time frame because International Women’s Day and an international online discussion within a listserv of public health professionals with an interest in EC took place during this month. The listserv discussion was hosted by the International Consortium for Emergency Contraception (ICEC) and WHO Department of Reproductive Health and Research. This two-week discussion forum, which engaged 1224 public health professionals from 106 countries, discussed the international EC landscape, current barriers to EC use, and strategies to increase EC access.²⁸ The discussion forum did not include a strategic Twitter presence. The timeframe of the current study was selected in order to provide the discussion forum organizers with a glimpse as to the general conversation about EC occurring in the public Twittersphere during the same month as their discussion.

Data collection

A content analysis was conducted on the tweets using 22 content categories coded by 16 individual coders. The coding guide was informed by the current clinical guidelines and research literature about EC as well as input from EC experts from the ICEC and the American Society for Emergency Contraception. Coders participated in a total of three training sessions, each lasting approximately 1–1.5 hours. Trainings incorporated instruction about specific content categories as well as time to practice and discuss the coding instructions. In between each training session, coders individually practiced coding and then sent their practice coding to the first author. The second and third group training session included discussions about the discrepancies and questions that arose as a result of the practice coding. These discussions led to clarifications and modifications to the coding guide, based on group consensus. After the final group training session, the coding scheme was finalized. At this point, coders received instruction not to discuss coding decisions with each other. This was to ensure that the coding decision process was standardized, remained free of further modifications, and could be replicated. Such assurances enhance the reliability of content analysis results.²⁹

Variables

The final coding guide included variables which measured content categories related to sender characteristics, use of Twitter-specific tools, use of specific words, and focus of tweet content (see Table 1 for definitions and example tweets). Three variables in the current study described characteristics of the users who sent the tweets. Coders referred to the username, profile image, and profile text to make informed decisions regarding sender characteristics. If the tweet user did not have an active profile, the tweet was coded as such. Tweets were coded for whether the tweet was sent by an individual or non-individual. Examples of non-individuals included organizations, media agencies, pharmacies, clinics, or other groups of individuals. The individual classification was all-inclusive of individual people and could include the general public as well as individuals who might hold strong professional/personal opinions about EC (e.g. clinicians, reproductive health/religious conservative advocates). When coding for whether the tweet user was male or female, only tweets from individuals were coded and a code was provided for cases where the gender was unclear. Coders captured the location for each tweet by copying the exact user-specified location in the user profile.

Table 1.

Emergency contraception tweet content category definitions and examples.

Content	Definition	Example tweet
Sender characteristics
Individual	Sender of tweet is an individual	N/A (assessment made by looking at user profile)
Gender	Sender of tweet is male/female	N/A (assessment made by looking at user profile)
Twitter-specific tools
Hashtag	Tweet uses # symbol to categorize messages	A morning after pill three days later #thingsmoreusefulthantrevorgillies
Retweet	Tweet uses “RT@username” in order to forward specific tweet to others	Dam didn’t we use da day after pill lol … stop with ur April fools joke RT @Amach128 I’m pregnant. Yay!
Link	Hyperlink to external site	How well does emergency contraception work? http://tinyurl.com/2fsh3wp
Reply	Tweet uses “@username” at beginning of tweet in direct response to another tweet	@Abortion_Rights: Go @estellehart’s mum! –> Good news: Morning after pill now free from pharmacies in Wales, including for under 16s bit.ly/fvW0Ti
Mention	Tweet uses “@username” anywhere in tweet to tag another user in the tweet	Go @estellehart’s mum! –> Good news: Morning after pill now free from pharmacies in Wales, including for under 16s bit.ly/fvW0Ti
Use of specific words in tweet
Birth control	Tweet uses specific words (i.e. family planning, birth control)	I dont see how u can get preggo wit birth control and the day after pill … all these sources to go to, and yet u still preggo #SMH
Side effects	Tweet uses phrase “side effects”	i’ll deal with the side effects of the morning after pill #TypeSex
Promiscuity	Tweet uses specific words (i.e. prostitute, bitch, skank, slut)	We’ve all taken the morning after pill … It’s kinda like vitamins around here. Just a nice way of calling everyone sluts
Abortion	Uses specific words (i.e. abortion, abort)	the “morning after pill” aka breakfast in bed #anti-abortion
Focus of tweet content
EC	Tweet is about emergency contraception (EC)	@ILyAD I have none and no accidents. But I have got that morning after pill due to CRAZY NIGHTS Lmao
News	Tweet addresses specific news stories related to EC	Morning-After Pill Causes Upheaval at FDA http://goo.gl/fb/MZDpY
Personal or vicarious experience	Tweet addresses an individual’s experience with EC, whether personal or vicarious	Would like to thank the guys who left me a $40 tip last night. It financed my morning after pill this morning. #waiting
Seeking advice	Tweet addresses seeking advice regarding EC	Can you mix roofies with morning after pill in alcohol? Asking for Saturday night.
Access	Tweet addresses issues regarding the process of obtaining EC	Pharmacists in Wales will be able to hand out the morning-after pill free of charge from today
Humorous	Tweet includes humorous content	Morning-after pill free in Wales from today but how do you get the sheep to swallow it?
Mechanism of action	Tweet addresses the way in which EC works biologically to prevent pregnancy	RT 3x@IamHuman_G: The morning after pill interrupts your menstrual cycle, & altho it is FDA approved, u can only take it ONCE in ur lifetime!
Effectiveness	Tweet addresses how well EC works is at preventing pregnancy	The Morning After Pill is not an abortion pill. It is a high dosage birth control pill that should be taken 1-5 days after unprotected sex.
Drug safety	Comment about safety concerns/issues of the EC drug for the woman taking EC	Morning-After Pill as Safe as Birth Control, Studies Find: The morning-after pill may be safe and effective as a … http://bit.ly/feBwKW
EC responsibility	Makes or counters the claim about irresponsible behavior as a result of increasing access to EC	They are going to give the morning after pill to under 15 year olds. Why not teach girls morals and self-respect

A total of five variables indicated whether a Twitter-specific tool was used. First, the link variable identified whether the tweet included a link to a URL address, which connects people to external content. Second, coders assessed whether tweets included a hashtag (#), a tool which allows the sender to mark keywords or topics. Coders also measured whether a tweet included a retweet (RT) or a direct reply to another tweet, proxies for the interactive sharing of information. Finally, the mention variable captured any time @username appeared anywhere in the body of the tweet. Both RT and replies are considered mentions; however, not all mentions are retweets or replies. Because the mention variable encompasses all of the Twitter-specific tools used to share other tweets, it served as the variable which measured whether a tweet was shared.

In addition to sender characteristics and the use of Twitter-specific tools, variables also captured content related to EC. One set of variables identified the presence of certain words often associated with EC (e.g. birth control, promiscuous, side effects, abortion). For these particular variables, coders did not interpret the meaning the tweet, but rather searched for the presence of specific words. Another set of variables specific to EC assessed the focus of the tweet’s content. When make coding decisions about the focus, coders analyzed both the actual content of the tweet as well as any included URL links. These variables measured content such as whether or not tweets included EC-related news stories, humor, individual experience with EC, advice, etc. (See Table 1 for a listing of all variables.) In addition, two new variables were created using the date each tweet was sent, one to capture whether a tweet was sent during the weekend and the second to capture a tweet sent around the St. Patrick’s Day weekend.

Data analysis

The initial sample of tweets consisted of 4049 tweets which were randomly and evenly distributed among the 16 coders. A subset of 228 randomly selected tweets was coded by all coders to assess reliability of the quantitative variables. No reliability coefficients were calculated for location, given its qualitative nature. The reliability data set was selected by taking a random sample of 5% of the full data set.³⁰ Inter-coder reliability was calculated across all 16 coders using percentage agreement and Cohen’s kappa (see Table 2). Based on accepted standards,²⁹ 80% was used as a threshold in this study to determine whether percent agreement between coders was acceptable. Cohen’s kappa, a more conservative measure of reliability which adjusts for chance, was selected as a second measure to assess reliability across coders. In analyzing the Cohen’s kappa coefficient, values greater than 0.75 were deemed excellent agreement beyond chance, values between 0.40 and 0.75 were deemed fair agreement beyond chance, and values below 0.40 were deemed poor agreement.³¹

Table 2.

Reliability of content analysis of emergency contraception (EC) tweets.

Content category	% agreement	Cohen’s kappa
EC	0.96	0.66
Individual	0.93	0.85
Male	0.92	0.87
Hashtag	0.93	0.86
Mention	0.92	0.86
Retweet	0.93	0.85
Reply	0.90	0.71
Link	0.95	0.91
Birth control	0.95	0.60
Promiscuous	0.95	0.70
Side effects	0.96	0.69
Abortion	0.95	0.70
News	0.90	0.78
Access	0.83	0.62
Humorous	0.80	0.55
Personal or vicarious experience	0.76	0.48
Effectiveness	0.83	0.58
Drug safety	0.92	0.67
Seeking advice	0.93	0.57
Mechanism of action	0.91	0.55
EC responsibility	0.91	0.60

This process identified three coders who performed inconsistently compared to the other coders. As a result, in order to maintain the highest level of integrity, their coded tweets were dropped from the final dataset. A total of 3750 tweets, coded by 13 coders, remained in the dataset. Of these, 94.3% (n = 3535) addressed EC specifically and, therefore, comprised the final dataset. Given that the data was randomly divided across all coders and that the initial dataset was a census of all EC tweets during March 2011, the final dataset remained representative of EC tweets from that time period.

Researchers used Stata12.0 to analyze the content of the tweets. Descriptive statistics calculated the general frequency distribution across content categories. Chi-square statistics compared differences in tweet content categories. For content categories that reached statistical significance in chi-square analyses, logistic regression tested the strength of associations. The study explored specific differences of interest. First, the study investigated whether tweets were sent by individuals or not as well as by males versus females. Such insight could be instrumental in identifying possible missed opportunities for communicating public health messages about EC to individuals. This study also explored differences in tweets that were shared versus those that were not. As mentioned previously, Twitter enables users to quickly share information and experiences. The more a message about a topic is shared, the more importance users may place on this message, thus influencing perceptions and opinions.²⁶

Finally, this study investigated whether individuals discussed EC differently during weekends and holidays (i.e. St. Patrick’s Day). Alcohol consumption is associated with high-risk sexual activity that may put a woman at risk for unintended pregnancy.³² Studies have also found associations between drinking and specific time periods – mainly weekends and certain holidays.^33,34 Furthermore, in a Twitter content analysis of drinking on the weekends and holidays, researchers found that there was a higher prevalence of tweets citing problem drinking posted on the weekends and on holidays.⁶ Because there may be greater risk during these time periods for unprotected sex, women may experience greater need for EC information and services and therefore may turn to social media for information and guidance about EC.

Results

Table 3 summarizes the descriptive statistics across all content categories for tweets about EC (n = 3535). Users who tweeted in English about EC during March 2011 represented 49 different countries (see Table 4). Among EC tweets with users who had active profiles (n = 2885), 83.81% (n = 2418) were from individuals. Among tweets sent by individuals with an active profile and where gender could be identified (n = 2297), 50.15% (n = 1152) were sent by females and 49.85% (n = 1145) were sent by males. More than three out of every four EC tweets (n = 2850) used a Twitter-specific tool, the most common being mention (40.88%) followed by hashtag (35.41%). Among the other content categories investigated, content related to specific news stories and content regarding EC access were the most common (both 27.27%), followed by content including humor (25.63%).

Table 3.

Distribution of trends for tweets about emergency contraception (EC) (n = 3535).

Content category		n (%)
Sender characteristics
Individual	Individual	2418 (68.46)
	Non-individual	467 (13.21)
	Unable to assess	644 (18.22)
Gender of individual	Female	1152 (32.59)
	Male	1145 (32.39)
	Gender unclear	121 (3.42)
Use of Twitter-specific tools
Hashtag	Yes	1252 (35.41)
Hashtag	No	2283 (64.58)
Mention	Yes	1445 (40.88)
Mention	No	2090 (59.12)
Retweet	Yes	983 (27.81)
Retweet	No	2552 (72.19)
Reply	Yes	489 (13.83)
Reply	No	3044 (86.11)
Link	Yes	1193 (33.75)
Link	No	2342 (66.25)
Use of specific words
Birth control	Used specific words	225 (6.36)
Birth control	Did not use specific words	3310 (93.64)
Promiscuous	Used specific words	143 (4.05)
Promiscuous	Did not use specific words	3391 (95.93)
Side effects	Used specific words	53 (1.50)
Side effects	Did not use specific words	3482 (98.50)
Abortion	Used specific words	98 (2.77)
Abortion	Did not use specific words	3435 (97.17)
Focus of tweet content
News	Addressed content	964 (27.27)
News	Did not address content	2570 (72.70)
Access	Addressed content	964 (27.27)
Access	Did not address content	2569 (72.67)
Humorous	Addressed content	906 (25.63)
Humorous	Did not address content	2629 (74.37)
Personal or vicarious experience	Addressed content	852 (24.10)
Personal or vicarious experience	Did not address content	2683 (75.90)
Effectiveness	Addressed content	428 (12.11)
Effectiveness	Did not address content	3107 (87.89)
Drug safety	Addressed content	356 (10.07)
Drug safety	Did not address content	3179 (89.93)
Seeking advice	Addressed content	96 (2.72)
Seeking advice	Did not address content	3439 (97.28)
Mechanism of action	Addressed content	84 (2.38)
Mechanism of action	Did not address content	3451 (97.62)
EC responsibility	Addressed content	26 (0.74)
EC responsibility	Did not address content	3509 (99.26)
Time period tweet sent
Weekend	Saturday or Sunday	556 (15.73)
Weekend	Monday–Friday	2979 (84.27)
St. Patrick’s Day	17 March 2011–20 March 2011	344 (9.73)
St. Patrick’s Day	Other days in the month	3191 (90.27)

Table 4.

Countries represented in English-language tweets related to emergency contraception, March 2011.

Region	Countries
Africa	• Botswana	• Namibia
	• Egypt	• Nigeria
	• Ghana	• South Africa
	• Kenya	• Tunisia
	• Mauritius
Americas	• Bahamas	• Jamaica
	• Brazil	• Mexico
	• Canada	• St. Lucia
	• Chile	• United States
	• Colombia	• Venezuela
	• Dominican Republic
Asia	• Bahrain	• Malaysia
	• India	• Pakistan
	• Japan	• Philippines
	• Kazakhstan	• Singapore
	• Korea	• Thailand
	• Lebanon	• United Arab Emirates
Europe	• Belgium	• Italy
	• Denmark	• Netherlands
	• France	• Romania
	• Germany	• Spain
	• Greece	• Switzerland
	• Holland	• Ukraine
	• Ireland	• United Kingdom
Oceania	• Australia	• New Zealand
	• Indonesia

Note: Regions based on United Nations designations.

Results from the chi-square analyses indicated differences between individuals and non-individuals in eight of the 13 content categories (see Table 5). Compared to non-individuals, a greater percentage of EC-related tweets sent by individuals mentioned promiscuity, a personal or vicarious experience, or had a humorous tone. For example, 25.96% of tweets sent by individuals addressed a personal or vicarious experience, compared to only 4.93% of tweets sent by non-individuals (p < 0.001). Similarly, 29.14% of tweets sent by individuals had a humorous tone compared to only 2.57% of tweets by non-individuals (p < 0.001). When examining the logistic regression results, similar patterns emerged (see Table 5), although the strength of associations varied. The strongest association for a tweet being sent by an individual was for promiscuity (OR = 23.47; 95% CI = 3.27–168.46; p = 0.001), followed by humorous content (OR = 15.59; 95% CI = 8.73–27.83; p < 0.001). On the other hand, EC tweets specifically addressing birth control, access, effectiveness, drug safety, and news had greater odds of being sent by non-individuals.

Table 5.

Differences in tweet content about emergency contraception (EC), individuals versus non-individuals.

	Chi-square analysis				Logistic regression^a
Content category	Non-individual n (%)	Individual n (%)	Total n (%)	Chi-square; p-value	Odds ratio	95% confidence interval	p-value
Birth control	48 (10.28)	145 (5.99)	193 (6.69)	11.51; 0.001	0.56	0.39–0.78	0.001
Promiscuous	1 (0.21)	116 (4.8)	117 (4.06)	21.13; <0.001	23.47	3.27–168.46	0.002
Side effects	12 (2.57)	34 (1.41)	46 (1.59)	3.38; 0.066	0.73	0.59–0.90	0.003
Abortion	12 (2.58)	62 (2.56)	74 (2.56)	0.0002; 0.99
Access	154 (32.98)	637 (26.35)	791 (27.43)	8.62; 0.003
EC responsibility	1 (0.21)	22 (0.91)	23 (0.80)	2.39; 0.12
Mechanism of action	14 (3.00)	60 (2.48)	74 (2.56)	0.42; 0.52
Effectiveness	105 (22.48)	266 (11.00)	371 (12.86)	46.11; <0.001	0.43	0.33–0.55	<0.001
Drug safety	126 (26.98)	178 (7.63)	304 (10.53)	159.92; <0.001	0.22	0.17–0.28	<0.001
News	304 (65.10)	530 (21.92)	834 (28.91)	355.06; <0.001	0.15	0.12–0.19	<0.001
Personal or vicarious experience	23 (4.93)	628 (25.96)	651 (22.56)	99.16; <0.001	6.71	4.40–10.21	<0.001
Seeking advice	8 (1.71)	73 (3.02)	81 (2.81)	2.44; 0.12
Humorous	12 (2.57)	705 (29.14)	717 (24.84)	148.05; <0.001	15.59	8.73–27.83	<0.001

Regression analysis conducted only for variables which reached statistical significance in chi-square analysis.

As previously stated, tweets sent by individuals were almost evenly distributed between males and females. When comparing tweet content by gender via a chi-square analysis, however, only a few differences surfaced (see Table 6). Tweets specifically mentioning birth control comprised 7.12% of all tweets sent by females, compared to only 4.19% of tweets sent by males (p = 0.002). Females also tweeted more about side effects than males (p = 0.024). In addition, 1.56% of tweets sent by females, compared to 0.35% of tweets sent by males, discussed claims about whether or not having access to EC increases irresponsible behavior (p = 0.003).

Table 6.

Differences in tweet content about emergency contraception (EC), males versus females.

	Chi-square analysis				Logistic regression
Content category	Female n (%)	Male n (%)	Total n (%)	Chi-square; p-value	Odds ratio	95% confidence interval	p-value
Birth control	82 (7.12)	48 (4.19)	130 (5.66)	9.21; 0.002	0.57	0.40–0.82	0.003
Promiscuous	64 (5.56)	50 (4.37)	114 (4.97)	1.71; 0.19
Side effects	23 (2.00)	10 (0.87)	33 (1.44)	5.12; 0.024	0.43	0.20–0.91	0.028
Abortion	27 (2.34)	34 (2.97)	61 (2.66)	0.87; 0.35
Access	299 (25.95)	307 (26.86)	606 (26.41)	0.24; 0.62
EC responsibility	18 (1.56)	4 (0.35)	22 (0.96)	8.90; 0.003	0.22	0.07–0.65	0.006
Mechanism of action	31 (2.69)	27 (2.36)	58 (2.53)	0.26; 0.61
Effectiveness	131 (11.37)	107 (9.34)	238 (10.36)	2.54; 0.11
Drug safety	84 (7.29)	72 (6.29)	156 (6.79)	0.913; 0.34
News	235 (20.40)	243 (21.22)	478 (20.81)	0.236; 0.063
Personal or vicarious experience	302 (26.22)	305 (26.64)	607 (26.43)	0.0527; 0.82
Seeking advice	35 (3.04)	33 (2.88)	68 (2.96)	0.049; 0.83
Humorous	338 (29.34)	344 (30.04)	682 (29.69)	0.14; 0.71

Among all EC-related tweets, 41% (n = 1457) were shared. Humorous tweets made up the largest percentage of tweets that users shared, followed by tweets containing personal or vicarious experience. There were statistically significant differences among seven of the 13 content categories when comparing whether a tweet was shared or not (see Table 7). Among tweets that were shared, 31%, compared to only 21.91% of tweets that were not shared, had a humorous tone (p < 0.001). Similarly, among shared EC-related tweets, 28.51% mentioned a personal or vicarious experience, compared to only 21.05% of tweets not shared (p < 0.001). Table 7 also presents the odds of the statistically significant content categories being shared by individuals. Although only 5.54% of shared tweets mentioned promiscuity, this content category had the strongest odds for having been shared (OR = 1.51; 95% CI = 1.04–2.20; p = 0.031). The other two content categories which with the strongest associations for having been shared were birth control (OR = 1.37; 95% CI = 1.01–1.86; p = 0.046) and personal or vicarious experience (OR = 1.23; 95% CI = 1.03–1.47; p = 0.024). The tweet content with lowest odds of being shared were side effects (OR = 0.17; 95% CI = 0.07–0.45; p < 0.001), drug safety (OR = 0.24; 95% CI = 0.17–0.34; p < 0.001), and news (OR = 0.44; 95% CI = 0.37–0.54; p < 0.001).

Table 7.

Differences in tweet content about emergency contraception (EC), shared versus not shared.

	Chi-square analysis				Logistic regression^a
Content category	Shared n (%)	Not shared n (%)	Total n (%)	Chi-square; p-value	Odds ratio	95% confidence interval	p-value
Birth control	98 (6.78)	127 (6.08)	225 (6.36)	0.71; 0.40	1.37	1.01–1.86	0.046
Promiscuous	80 (5.54)	63 (3.02)	143 (4.05)	13.98; <0.001	1.51	1.04–2.20	0.031
Side effects	5 (0.35)	48 (2.30)	53 (1.50)	22.01; <0.001	0.17	0.07 – 0.45	<0.001
Abortion	42 (2.91)	56 (2.68)	98 (2.77)	0.16; 0.69
Access	375 (25.97)	589 (28.20)	964 (27.29)	2.13; 0.14
EC responsibility	11 (0.76)	15 (0.72)	26 (0.74)	0.02; 0.88
Mechanism of action	37 (2.56)	47 (2.25)	84 (2.38)	0.36; 0.55
Effectiveness	139 (9.62)	289 (13.83)	429 (12.11)	14.22; <0.001	0.80	0.63–1.01	0.065
Drug safety	45 (3.11)	311 (14.88)	356 (10.07)	130.60; <0.001	0.24	0.17–0.34	<0.001
News	216 (14.95)	748 (35.81)	964 (27.28)	187.34; <0.001	0.44	0.37–0.54	<0.001
Personal or vicarious experience	412 (28.51)	440 (21.05)	852 (24.10)	25.99; <0.001	1.23	1.03–1.47	0.024
Seeking advice	39 (2.70)	57 (2.73)	96 (2.72)	0.003; 0.96
Humorous	448 (31.00)	458 (21.91)	906 (25.63)	37.03; <0.001	1.19	0.996–1.41	0.059

Regression analysis conducted only for variables which reached statistical significance in chi-square analysis. Each logistic regression model measured the odds for a tweet having been shared and also controlled for whether the tweet was sent by an individual or not.

Although weekends (Saturday and Sundays) represented 26% of days in March 2011, only 15.72% of EC tweets were sent on a weekend. Weekend content differed from non-weekend content for six content categories (see Table 8). Compared to the rest of the week, a smaller percentage of tweets sent on the weekend addressed drug safety (p < 0.001), news (p < 0.001), or birth control (p = 0.036). A greater percentage of tweets sent on the weekend contained humor (p < 0.001), addressed personal or vicarious experience (p < 0.001), or sought advice (p = 0.012). These same content categories had the greatest odds of being sent on the weekend (see Table 8). Tweets with the lowest odds of occurring during the weekend were those about news (OR = 0.15; 95% CI = 0.10–0.23; p < 0.001) or drug safety (OR = 0.23; 95% CI = 0.12–0.42; p < 0.001).

Table 8.

Differences in tweet content about emergency contraception (EC), weekends versus not on weekends.

	Chi-square analysis				Logistic regression^a
Content category	Non-weekend n (%)	Weekend n (%)	Total n (%)	Chi-square; p-value	Odds ratio	95% confidence interval	p-value
Birth control	130 (6.45)	15 (3.73)	145 (5.99)	4.38; 0.036	0.64	0.39–1.04	0.074
Promiscuous	101 (5.01)	15 (3.73)	116 (4.80)	1.20; 0.27
Side effects	26 (1.29)	8 (1.99)	34 (1.41)	1.19; 0.28
Abortion	51 (2.52)	11 (2.74)	62 (2.56)	0.06; 0.81
Access	526 (26.10)	111 (27.61)	637 (26.35)	0.39; 0.53
EC responsibility	21 (1.04)	1 (0.25)	22 (0.97)	2.33; 0.13
Mechanism of action	50 (2.48)	10 (2.49)	60 (2.48)	0.0001; 0.99
Effectiveness	228 (11.25)	39 (9.70)	267 (11.00)	0.83; 0.36
Drug safety	168 (8.33)	10 (2.49)	178 (7.36)	16.78; <0.001	0.23	0.12–0.42	<0.001
News	508 (25.20)	22 (5.47)	530 (21.92)	76.20; <0.001	0.15	0.10–0.23	<0.001
Personal or vicarious experience	480 (23.80)	148 (36.82)	628 (25.96)	29.55, <0.001	1.91	1.53–2.38	<0.001
Seeking advice	53 (2.63)	20 (4.98)	73 (3.02)	6.31; 0.012	1.94	1.17–3.24	0.011
Humorous	556 (27.57)	149 (37.06)	705 (29.14)	14.65, <0.001	1.56	1.25–1.96	<0.001

Regression analysis conducted only for variables which reached statistical significance in chi-square analysis.

Each logistic regression model measured the odds for a tweet having been sent on a weekend and also controlled for whether the tweet was sent by individual or not.

Similar patterns to the weekend variable were found for tweets sent around St. Patrick’s Day (see Table 9). Among all EC tweets sent in March 2011, 9.73% occurred in the days around St. Patrick’s Day. Tweets with the greatest odds of occurring around St. Patrick’s Day were those that sought advice (OR = 2.45; 95% CI = 1.41–4.25; p = 0.001), shared personal or vicarious experience (OR = 1.59; 95% CI = 1.22–2.08; p = 0.001), or contained humor (OR = 1.47; 95% CI = 1.12–1.92; p = 0.005). Tweets with the lowest odds of having been sent around St. Patrick’s Day were those which discussed news (OR = 0.17; 95% CI = 0.10–0.27; p < 0.001) or drug safety (OR = 0.19; 95% CI = 0.08–0.43; p < 0.001).

Table 9.

Differences in tweet content about emergency contraception (EC), St. Patrick’s Day weekend (17 March 2011–20 March 2011) versus other days in March.

	Chi-square analysis				Logistic regression^a
Content category	Other days in March n (%)	St. Patrick’s Day weekend n (%)	Total n (%)	Chi-square; p-value	Odds ratio	95% Confidence interval	p-value
Birth control	138 (6.38)	7 (2.72)	145 (5.99)	5.45; 0.019	0.40	0.19 – 0.82	0.012
Promiscuous	109 (5.04)	7 (2.72)	116 (4.80)	2.71; 0.10
Side effects	30 (1.39)	4 (1.56)	34 (1.41)	0.047; 0.83
Abortion	55 (2.54)	7 (2.72)	62 (2.56)	0.030; 0.86
Access	574 (26.57)	63 (24.51)	637 (26.35)	0.50; 0.48
EC responsibility	22 (1.02)	0 (0.00)	22 (0.91)	2.64; 0.10
Mechanism of action	55 (2.54)	5 (1.95)	60 (2.48)	0.34; 0.56
Effectiveness	250 (11.56)	16 (6.23)	266 (11.00)	6.64; 0.01
Drug safety	172 (7.96)	6 (2.33)	178 (7.36)	10.65; 0.001	0.19	0.08 – 0.43	<0.001
News	514 (23.74)	16 (6.23)	530 (21.92)	41.38; <0.001	0.17	0.10 – 0.27	<0.001
Personal or vicarious experience	540 (24.98)	88 (34.24)	628 (25.96)	10.26; 0.001	1.59	1.22–2.08	0.001
Seeking advice	57 (2.64)	16 (6.23)	73 (3.02)	10.11; 0.001	2.45	1.41–4.25	0.001
Humorous	612 (28.31)	93 (36.19)	705 (29.14)	6.91; 0.009	1.47	1.12–1.92	0.005

Regression analysis conducted only for variables which reached statistical significance in chi-square analysis.

Each logistic regression model measured the odds for a tweet having been sent around St. Patrick’s Day weekend and also controlled for whether the tweet was sent by an individual or not.

Discussion

With the advent of social media, the public is now a part of the news-making and agenda-setting process. Social media sites like Facebook and Twitter are increasingly being used to share and discuss news and information.^35,36 With a specific focus on EC, the current study builds on other studies analyzing the intersection between health topics and social media.^{4,6,7,37–46} In terms of frequency of tweets, it is interesting to note that the vast majority of tweets about EC were sent by individuals, rather than non-individuals. This finding suggests that there may be missed opportunities for public health organizations to engage individuals in dialogue about EC via Twitter. In addition, tweets were sent by a near equal number of males and females and there were few differences in the type of content posted by males versus females. While many would assume that women would likely talk more about EC since it is a method of family planning that they consume, this finding suggests men and women seem to be talking about EC via Twitter in similar amounts and ways. In addition, the fact that men discuss EC as often as women suggests that future reproductive health programs or campaigns addressing issues about EC should consider not solely focusing on women but rather developing strategies to engage men as well.

Media priming and agenda-setting theory indicate that the frequency and ways in which media present a certain topic can influence perceptions and judgments.^24,26,47 The utility of this theoretical perspective has been applied before to other health topics and is supported by the findings of the current study.^27,48,49 The content categories with the highest frequency were tweets which discussed a specific news story, discussed EC access, mentioned a personal or vicarious experience with EC, or had a humorous tone. The content categories which had greater odds of being shared included specifically mentioning promiscuity, discussing a personal or vicarious experience, specifically mentioning birth control, or having a humorous tone. As stated above, news was one of the most frequent content categories identified in the EC-related tweets. This finding is consistent with other studies which found that one of the main uses of Twitter has been to report news.^4,35,41,50 Several study findings, however, lead to some important implications for public health organizations wanting to reach individuals with messages about EC. In particular, although news was one of the most frequent content categories among EC tweets, they had lower odds of being sent by an individual, being shared, or being sent either during the weekend or around St. Patrick’s Day. These findings suggest that for organizations interested in developing a social media strategy to promote EC among individuals, focusing on the news angle of EC may end up missing the mark in terms of reaching and influencing individuals.

The current study found that incorporating humor as well as expressing a personal or vicarious experience was a prominent approach for communicating about EC via Twitter. EC-related tweets which included a humorous tone also had greater odds of having been sent on the weekend or around St. Patrick’s Day. The fact that individuals sent almost all of the humorous EC tweets suggests a missed opportunity for public health organizations to utilize humor in social media messaging related to EC during weekends and holidays. Similarly, tweets with personal or vicarious experience also had greater odds of being sent on weekends and around St. Patrick’s Day, as well as being shared with others. The process of disclosing personal experiences on Twitter may be therapeutic for some users as well as creating a community or forum in which users can discuss and seek advice on health issues, such as EC.⁴¹ These findings also complement previous research which documented social media’s role for normalizing and de-stigmatizing discussions about sexual health and ultimately for increasing uptake and sharing of sexual health information, especially among youth.⁵¹ Given the current study findings, organizations may want to strategically craft tweets to include elements which are more relevant to individuals, such as those expressing a personal or vicarious experience. For example, instead of focusing on specific news stories or general facts about EC, organizations should consider involving the intended audience to develop and send tweets in which they disclose personal or vicarious experience about accessing or using EC.

Although the current study tried to apply as much rigor as possible, five limitations exist. First, findings only provide a snapshot from a month in 2011 of how EC is framed on Twitter. During several weeks of this month, as mentioned previously, an online discussion of public health professionals specific to EC occurred. This online discussion took place through an email listserv and did not have a strategic Twitter presence. The study’s first author decided to do a study of Twitter that would overlap with the same time period as the online discussion to identify what were the discussions that were occurring in the Twittersphere at the same time. It is not clear, however, the effect that the online listserv discussion may have had on Twitter. As a result, future studies should assess content over a longer time period in order to track changes in content over time and also to see whether what was observed in the current study was affected by the listserv discussion. When discussing Twitter use around weekends and holidays, the one-month time frame only captured tweets posted on a few weekends and one holiday. Future studies including other holiday weekends may shed greater insight as to whether the trends found in the current study were generalizable to holidays in general or were unique to the days surrounding St. Patrick’s Day. St. Patrick’s Day, nevertheless, may provide some level of representation of increased drinking around the holidays as the day is specifically associated with heavier drinking.⁵² In addition, the one-month time frame does not allow exploring whether there has been an evolution in the way in which EC is framed on Twitter among both individuals and non-individuals. Although the cross-sectional nature of the current study is limited in its ability to discuss trends or shifts over time, the study nevertheless provides insight that remains relevant today.

Second, because the current study only looked at English-language tweets, results cannot be generalized to the way in which EC is framed via Twitter in other languages. If all languages were analyzed, Twitter could become an even more valuable tool in understanding EC usage in a global context, including in some countries where discussions in the public sphere about EC are not socially acceptable. Despite this limitation, results are still applicable in a global context since the English-language tweets came from many different countries and regions of the world. Furthermore, the geographic reach represented in this study did include locations in which there are gaps in EC availability and access.¹⁷

Third, it was challenging to gather characteristics for all users. Users are free to post whatever information they choose on their profile or within the tweet, including the location from where they are tweeting. As a result, many tweets had ambiguous or missing location details. Furthermore, in a small percentage of tweets, coders were not able to determine from the user profile whether the user was male or female. The large sample size of the study, nevertheless, provided enough statistical power to yield interesting findings by gender.

Fourth, Twitter limits tweets to 140 characters, which made for challenges regarding assessing the tone, intention, or meaning of particular tweets. For example, the current study was not able to assess whether content was used in a positive or negative context. Being unable to determine valence of tweets, however, may be less important for the purposes of this study since media priming indicates that judgments and perceptions may be formed based solely on sheer level of exposure to a message, regardless of whether that message is positive or negative. In addition, the challenges in assessing tone or meaning due to the short length of tweets may have negatively impacted the reliability of the dataset when controlling for agreement by chance. Nevertheless, this study applied a well-tested coding guide, a rigorous reliability assessment, and no discussion between coders which, in the end, yielded a valid dataset. Moreover, the percent agreement across coders was acceptable to high, which, given the large number of coders, is less likely have occurred due to chance compared to studies which only use two coders.³⁰

Finally, the way in which the current study was able to investigate retweeting behavior was somewhat limited in scope. Due to constraints in the software used to extract EC-related tweets and resource availability, the coders were not able to investigate the way in which social networks of individual Twitter users may have influenced retweeting behavior. Studies suggest that factors like similarity to the user posting the tweet as well as one’s followers on Twitter may affect retweeting behavior.^53,54 Future studies, therefore, should incorporate ways to measure and capture a more comprehensive picture of retweeting behavior. Nevertheless, research literature stresses the importance of the focus and content of a tweet for retweeting behavior.^54–59 Moreover, the function of sharing information via Twitter can also help to establish new linkages and form new relationships, as individuals find commonality and shared interests based on the content of shared tweets.⁵⁵ Therefore, exploring whether or not tweets are retweeted by type of content still provides value for organizations wanting to figure out how they may be able to encourage greater sharing of tweets that they generate.

In spite of the above limitations, two important implications emerged from study findings. First, the increasing use of social networking sites, including Twitter, provides an opportunity to share messages about EC and other public health issues with the public. From current trends of growth and reach, it appears as though Twitter is not going away any time soon. As a result, it would greatly benefit public health organizations to determine ways to strategically integrate Twitter as part of a comprehensive mass media communication strategy. In addition to the current study, further studies should explore the ways in which individuals actually use Twitter to communicate and learn about public health topics, especially those that are controversial in nature such as EC.

Second, the current content analysis suggests that an effective organizational strategy to using Twitter may want to explicitly involve the public in the creation and dissemination of public health messaging. In particular, organizations should encourage people to share messages which incorporate the content which has greater relevance to individuals, such as personal experience or humor. From a message framing and priming perspective, the more a message is shared, the greater prominence this message will have in the public’s mind. As a result, if organizations can develop messaging strategies that encourage individuals to share their own stories and experiences related to EC, this sharing of information may increase the importance of the message for other individuals who may still be forming their own opinion or perception.

In conclusion, the current content analysis of EC tweets is an important first step in understanding the potential role of Twitter to address this topic. Findings from this study highlight the need for future studies to continue gathering information on public sentiment about EC, both inside and outside the United States. At the same time, study findings suggest that Twitter may serve as an additional communication tool to strategically engage society in discussions about health topics, including those more controversial in nature such as EC. Although Twitter may not yet be universally used across all socioeconomic groups, it still remains an important social media platform to incorporate into an organization’s overall mass media communication strategy about public health issues. The more dialogue that can be fostered about EC through various media, including social media, the more likely it is that an environment will be created that normalizes the use of EC.

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors declare that there is no conflict of interest.

Guarantor

Tilly Gurman

Ethical approval

The current study was not considered human subjects research as the study, consisting of analyzing publicly available tweets, did not involve the collection of data through intervention or interaction with individuals, nor did it handle any private information.

Contributorship

The study’s first author conceived the study, developed study protocol, trained data collectors, and led the data analysis process. Both study authors were involved in data analysis. The study’s second author wrote the first draft of the manuscript. The study’s first author revised the first draft and prepared it for publication. Both authors reviewed and edited the manuscript and approved the final version of the manuscript.

Peer-review

This paper was reviewed by Megan Lim, Burnet Institute, Centre for Population Health; Ulrike Deetjen, Oxford Internet Institute; and one other reviewer who has chosen to remain anonymous.

Acknowledgments

We would like to thank the larger research team – Sayed Alam, Isabel Barillas, Kristina Beall, Christopher Burton, Chiara Davis, Jenna Frkovich, Holly Greb, Lauren Hall, Kelly Healy, Christine Iversen, Britta Mason, Sara Miner, Kristin Roha, Shafiqullah Stanikzai, Regan Trappler, and Janna VanHoven – for their energy and effort. Without them, this research would not have been possible.

References

eMarketer. Social networking reaches nearly one in four around the world, www.emarketer.com/Article/Social-Networking-Reaches-Nearly-One-Four-Around-World/1009976 (2013, accessed 6 December 2013).

Brenner J and Smith A. 72% of online adults are social networking site users. Washington, DC: Pew Research Center. Available at: http://www.pewinternet.org/files/old-media//Files/Reports/2013/PIP_Social_networking_sites_update_PDF.pdf (August 2013, accessed 28 December 2015).

Twitter. About Twitter, https://about.twitter.com/company (2015, accessed 15 November 2013).

Chew

Eysenbach

. Pandemics in the age of Twitter: Content analysis of tweets during the 2009 H1N1 outbreak. PloS ONE 2010; 5: e14118.

Dredze M. How social media will change public health. IEEE Intelligent Systems 2012; 27: 81–84.

West

Cougar Hall

Prier

. Temporal variability of problem drinking on Twitter. Open J Prev Med 2012; 02: 43–48.

Hawn

. Take two aspirin and tweet me in the morning: How Twitter, Facebook, and other social media are reshaping health care. Health Aff 2009; 28: 361–368.

Stewart F, Trussell and Van Look JP. Emergency contraception. In: Hatcher RA, Trussell J, Stewart F, et al. (eds) Contraceptive Technology: Eighteenth Revised Edition. New York NY: Ardent Media, 2004, pp. 279–303.

Princeton University Office of Population Research & Association of Reproductive Health Professionals. What is emergency contraception? http://ec.princeton.edu/emergency-contraception.html (2015, accessed 15 November 2013).

10.

Croxatto

Devoto

Durand

. Mechanism of action of hormonal preparations used for emergency contraception: A review of the literature. Contraception 2001; 63: 111–121.

11.

Jadav

Parmar

. Ulipristal acetate, a progesterone receptor modulator for emergency contraception. J Pharmacol Pharmacother 2012; 3: 109–111.

12.

Glasier

. Emergency postcoital contraception. N Engl J Med 1997; 337: 1058–1064.

13.

Raymond

Halpern

Lopez

. Pericoital oral contraception with levonorgestrel: A systematic review. Obstet Gynecol 2011; 117: 673–681.

14.

von Hertzen

Piaggio

Van Look

. Emergency contraception with levonorgestrel or the Yuzpe regimen. Task force on postovulatory methods of fertility regulation. Lancet 1998; 352: 1939.

15.

Rodriguez

Godfrey

Warden

. Prevention and management of nausea and vomiting with emergency contraception: A systematic review. Contraception 2013; 87: 583–589.

16.

Westley

Kapp

Palermo

. A review of global access to emergency contraception. Int J Gynecol Obstet 2013; 123: 4–6.

17.

International Consortium for Emergency Contraception. Emergency contraception status and availability, www.cecinfo.org/country-by-country-information/status-availability-database/ (2015, accessed 28 December 2015).

18.

USAID|DELIVER PROJECT, Task Order 4. 2012. Measuring Contraceptive Security Indicators in 2011. Arlington, Va.: USAID|DELIVER PROJECT, Task Order 4.

19.

Brady M, Khan ME, Ahonsi B, et al. Providers’ and key opinion leaders’ attitudes, beliefs, and practices concerning emergency contraception: A multicountry study in India, Nigeria, and Senegal, www.popcouncil.org/uploads/pdfs/2012RH_ECBriefMulticountry.pdf (2012, accessed 13 December 2015).

20.

Weiss DC, Harper CC, Speidel JJ, et al. Does emergency contraception promote sexual risk-taking? Bixby Center for Global Reproductive Health. San Francisco: University of California, April 2008. Available at: http://bixbycenter.ucsf.edu/.

21.

DeCoster

Claypool

. A meta-analysis of priming effects on impression formation supporting a general model of informational biases. Pers Soc Psychol Rev 2004; 8: 2–27.

22.

Appel

. A story about a stupid person can make you act stupid (or smart): Behavioral assimilation (and contrast) as narrative impact. Media Psychol 2011; 14: 144–167.

23.

Scheufele

Tewksbury

. Framing, agenda setting, and priming: The evolution of three media effects models. J Comm 2007; 57: 9–20.

24.

Arendt

. Dose-dependent media priming effects of stereotypic newspaper articles on implicit and explicit stereotypes. J Comm 2013; 63: 830–851.

25.

Dixon

. Black criminals and white officers: The effects of racially misrepresenting law breakers and law defenders on television news. Media Psychol 2007; 10: 270–291.

26.

Kim

S-H

Han

Choi

D-H

. Attribute agenda setting, priming and the media’s influence on how to think about a controversial issue. Int Comm Gaz 2012; 74: 43–59.

27.

Lee

Len-Ríos

. Defining obesity: Second-level agenda setting attributes in black newspapers and general audience newspapers. J Health Comm 2014; 19: 1116–1129.

28.

International Consortium for Emergency Contraception. Emergency contraception: How far have we come? What’s new? What’s next? Online discussion forum, 2nd March–16th March 2011, final report, www.cecinfo.org/custom-content/uploads/2012/12/ICEC-IBP-Online-Forum-on-EC-REPORT.pdf (2011, accessed 28 December 2015).

29.

Neuendorf

Reliability for content analysis. In: Jordan

Kunkel

Manganello

(eds). Media messages and public health: A decisions approach to content analysis, 1st edition. New York, NY: Routledge, 2008, pp. 67–87.

30.

Potter

Levine-Donnerstein

. Rethinking validity and reliability in content analysis. J Appl Comm Res 1999; 27: 258–284.

31.

Banerjee

Capozzoli

McSweeney

. Beyond kappa: A review of interrater agreement measures. Can J Stat 1999; 27: 3–23.

32.

Patrick

Maggs

. Does drinking lead to sex? Daily alcohol–sex behaviors and expectancies among college students. Psychol Addict Behav 2009; 23: 472–481.

33.

Del Boca

Darkes

Greenbaum

. Up close and personal: Temporal variability in the drinking of individual college students during their first year. J Consult Clin Psychol 2004; 72: 155–164.

34.

Mäkelä

Martikainen

Nihtilä

. Temporal variation in deaths related to alcohol intoxication and drinking. Int J Epidemiol 2005; 34: 765–771.

35.

Purcell K, Rainie L, Mitchell A, et al. Understanding the participatory news consumer: How internet and cell phone users have turned news into a social experience. Pew Research Center, March 2010.

36.

Eysenbach

. Medicine 2.0: Social networking, collaboration, participation, apomediation, and openness. J Med Internet Res 2008; 10: e22.

37.

Hull

Schmittel

. A fumbled opportunity? A case study of Twitter’s role in concussion awareness opportunities during the Super Bowl. J Sport Soc Issues 2015; 39: 78–94.

38.

Belair-Gagnon

Mishra

Agur

. Reconstructing the Indian public sphere: Newswork and social media in the Delhi gang rape case. Journalism 2014; 15: 1059–1075.

39.

Freberg

Palenchar

Veil

. Managing and sharing H1N1 crisis information using social media bookmarking services. Publ Relat Rev 2013; 39: 178–184.

40.

Donelle

Booth

. Health tweets: An exploration of health promotion on twitter. Online J Issues Nurs 2012; 17: 4.

41.

Sullivan

Schneiders

Cheang

. ‘What’s happening?’ A content analysis of concussion-related traffic on Twitter. Br J Sports Med 2012; 46: 258–263.

42.

Liu

Kim

. How organizations framed the 2009 H1N1 pandemic via social and traditional media: Implications for U.S. health communicators. Publ Relat Rev 2011; 37: 233–244.

43.

Scanfeld

Larson

. Dissemination of health information through social networks: Twitter and antibiotics. Am J Infect Contr 2010; 38: 182–188.

44.

Bull

Levine

Black

. Social media-delivered sexual health intervention: A cluster randomized controlled trial. Am J Prev Med 2012; 43: 467–474.

45.

Guse

Levine

Martins

. Interventions using new digital media to improve adolescent sexual health: A systematic review. J Adolesc Health 2012; 51: 535–543.

46.

Young

Szekeres

Coates

. Sexual risk and HIV prevention behaviours among African-American and Latino MSM social networking users. Int J STD AIDS 2013; 24: 643–649.

47.

Meriläinen

Vos

. Human rights organizations and online agenda setting. Corp Comm Int J 2011; 16: 293–310.

48.

Jarlenski

Barry

. News media coverage of trans fat: Health risks and policy responses. Health Comm 2013; 28: 209–216.

49.

Dixon

Warne

Scully

. Agenda-setting effects of sun-related news coverage on public attitudes and beliefs about tanning and skin cancer. Health Comm 2014; 29: 173–181.

50.

Java A. Why we twitter: Understanding microblogging usage and communities. In: Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis – WebKDD/SNA-KDD ’07. New York, USA: ACM (Association for Computing Machinery), 2007, pp.56–65.

51.

Byron

Albury

Evers

. “It would be weird to have that on Facebook”: Young people’s use of social media and the risk of sharing sexual health information. Reprod Health Matters 2013; 21: 35–44.

52.

Freeman D. St. Patrick’s Day no excuse to go overboard on Guinness, expert says. CBS News. 17 March. Available at: http://www.cbsnews.com/news/st-patricks-day-no-excuse-to-go-overboard-on-guinness-expert-says/ (2011, accessed 28 December 2015).

53.

Boyd D, Golder S and Lotan G. Tweet, tweet, retweet: Conversational aspects of retweeting on Twitter. HICSS-43. IEEE: Kauai, HI, 6 January 2010.

54.

Wang

Zuo

Wang

. A multidimensional nonnegative matrix factorization model for retweeting behavior prediction. Math Probl Eng 2015; 2015: 1–10.

55.

Ahn

Park

J-H

. The structural effects of sharing function on Twitter networks: Focusing on the retweet function. J Inform Sci 2015; 41: 354–365.

56.

Abdullah

Nishioka

Tanaka

. User’s action and decision making of retweet messages towards reducing misinformation spread during disaster. J Inform Process 2015; 23: 31–40.

57.

Prestin

Lee

. What do people like to “share” about obesity? A content analysis of frequent retweets about obesity on Twitter. Health Comm 2016; 31: 193–206.

58.

Rudat

Buder

. Making retweeting social: The influence of content and context information on sharing news on Twitter. Comput Hum Behav 2015; 46: 75–84.

59.

Kyumin

Mahmud

Chen

. Who will retweet this? Detecting strangers from Twitter to retweet information. ACM Transactions on Intelligent Systems and Technology 2015; 6: 1–25.