Evaluation of a Mobile Phone App for Providing Adolescents With Sexual and Reproductive Health Information,New York City,2013-2016

Abstract

The New York City (NYC) Department of Health and Mental Hygiene released the Teens in NYC mobile phone application (app) in 2013 as part of a program to promote sexual and reproductive health among adolescents aged 12-19 in NYC. The app featured a locator that allowed users to search for health service providers by sexual health services, contraceptive methods, and geographic locations. We analyzed data on searches from the Where to Go section of the app to understand the patterns of use of the app’s search functionality. From January 7, 2013, through March 20, 2016, the app was downloaded more than 20 000 times, and more than 25 000 unique searches were conducted within the app. Results suggest that the app helped adolescents discover and access a wide range of sexual health services, including less commonly used contraceptives. Those designing similar apps should consider incorporating search functionality by sexual health service (including abortion), contraceptive method, and user location.

Keywords

adolescent health contraception evaluation health communications health promotion information technology sexual health

In the United States, rates of pregnancy,^1
–3 sexually transmitted infections (STIs),^4,5 and human immunodeficiency virus (HIV) infection^6,7 are higher among adolescents than among other age groups. Although traditional media (eg, television, film, radio, and music)⁸ have featured campaigns to promote adolescent sexual health, new media (eg, mobile phones, mobile applications [apps], and social networking sites) have not yet been effectively used for this purpose.^8,9 Many adolescents feel that sexual health information is important¹⁰ and are receptive to using mobile technology (including mobile phones) to access it.^{10

–19}

Mobile phones are a promising technology for engaging adolescents in accessing information about sexual health topics. Mobile phone ownership is common in the United States, particularly among adolescents. In 2015, 73% of teenagers aged 13-17 owned a mobile phone, of whom 91% used their mobile phone to access the internet.²⁰ The use of mobile technology for health promotion offers privacy,^{9,10,13,14,21
–23} access to personalized information,^{8
–10,14,24} and convenience,^18,21,22 making it a valuable way to provide accurate information about sexual health to adolescents.^{15,17,18,24
–26}

In collaboration with the Bronx Teens Connection Youth Advisory Board, the New York City (NYC) Department of Health and Mental Hygiene (DOHMH) created the Teens in NYC mobile phone app in 2013. This app was an extension of a paper-based resource, launched in 2007, that listed health care providers in NYC that met best practices in sexual health care for adolescents.²⁷ The app had 3 main sections: Where to Go, What to Get, and What to Expect (Figure A).²⁸

Figure.

Graphic user interfaces of Teens in NYC mobile application (app) (A) home page and (B) Where to Go section, used in 28 503 separate searches, New York City, February 27, 2013, through March 20, 2016. The home page is where users navigate to 1 of 3 main sections of the app: Where to Go (where to access sexual health services), What to Get (descriptions of contraceptive methods), and What to Expect (videos of adolescents accessing sexual health services). In the Where to Go section, users search for health care provider listings by selecting sexual health services (eg, birth control or gynecology) or contraceptive methods (eg, condoms) and by indicating their geographic location of interest (eg, current location).

The Where to Go section of the Teens in NYC app allowed users to find providers of sexual health services or contraceptive methods. Users were prompted to select from 3 menus (services, methods, and geography) on the same screen. Users could specify services or methods (each via drop-down menus) and indicate their geographic location of interest (using a particular address and/or borough or a current location) (Figure B). On each drop-down menu for services and methods, they could choose 0, 1, or >1. However, they had to make at least 1 selection in either the services menu or the methods menu. On the services drop-down menu, users could select “gold star” service to find providers who offered certain sexual health care services and contraceptive methods on-site for free.

The Where to Go section therefore allowed for the tailoring of information to the user’s needs and preferences, an approach that others have reported to be a key to the success of new media initiatives,^{8
–10,14,24} including initiatives focused on contraceptive services.^29,30 The other 2 sections of the app supported the search functionality. The What to Get section provided brief descriptions of each contraceptive method, and the What to Expect section featured videos²⁹ of adolescents accessing sexual health services. We analyzed search data from the Where to Go section of the Teens in NYC mobile app to understand patterns of use of a mobile phone app that contained a provider locator for local sexual health care services.

Methods

We obtained a data set in Microsoft Excel from the NYC DOHMH information technology department that included all searches conducted using the app from February 27, 2013, through March 20, 2016 (NYC DOHMH, unpublished data). This data set consisted of a separate row for each search conducted and individual columns for the following variables of interest that users could select during a search: geographic location (current location, borough, or address), corresponding borough (when current location or address was searched), various sexual health services (birth control or gynecological service, STI testing and treatment, pregnancy testing, abortion service, HIV testing, gold star service, mental health counseling, and lesbian/gay/bisexual/transgender/queer [LGBTQ]–specific service), and various contraceptive methods (condom, oral contraceptive pill, emergency contraception, injectable contraception, female condom, hormonal intrauterine device, copper intrauterine device, contraceptive implant, vaginal ring, and contraceptive patch). We calculated percentages for variables with discrete outcomes (eg, frequency of selecting a particular service) and means and medians for variables with continuous outcomes (eg, number of services selected).

The information technology department also provided us with access to data on the number of apps that were downloaded from App Annie’s free online app monitoring tool,³⁰ as well as Apple’s iTunes Connect internet-based tool.³¹ We obtained data on the number of downloads from the dates of the app’s launch (January 7, 2013, for Apple and January 14, 2013, for Android) through March 20, 2016, and we exported that data set (unpublished data) to Microsoft Excel.

Because both our search and download data sets included no personally identifiable user information, we were unable to link those data sets to determine how many users conducted searches. Instead, we used the number of app downloads to estimate the number of potential users of the app’s Where to Go section.

By accepting the Terms of Service for the Teens in NYC mobile app, users consented to anonymous data collection. NYC DOHMH privacy restrictions for adolescent-focused services prohibited the use of any existing analytics packages that would store data on users offsite or collect unique user information. For each Where to Go search performed, the app sent deidentified search information directly to a secure database at NYC DOHMH. No data were stored on users’ phones. The NYC DOHMH Institutional Review Board determined that this study did not meet the definition of research and was deemed exempt.

We compared the distribution of searches by corresponding borough of current location with the overall distribution of residents aged 12-19 by NYC borough by using data from the 2010 US Census.³² We also compared the contraceptive methods searched with 2015 data from a representative sample of NYC public high school students in the Youth Risk Behavior Survey (YRBS).³³ The 2015 YRBS data describes contraceptive methods used at last sex among students who had sex in the past 3 months.

Results

We identified 22 137 downloads of the Teens in NYC mobile app from January 7, 2013, through March 20, 2016, and 28 503 unique searches in the app’s Where to Go section from February 27, 2013, through March 20, 2016. The geographic location of interest for searches was the user’s current location in 17 544 (65%) searches, a borough in 2885 (10%) searches, and an address in 1330 (5%) searches. Geographic selection data were missing in 5744 (21%) searches. Current locations used in searches came from all 5 NYC boroughs and were nearly proportional to the distribution of residents aged 12-19 across the boroughs (Table 1).

Table 1.

Distribution of location of Teens in NYC mobile app^a users who performed a search on the app and chose “current location” as location of interest,^b by borough, from February 27, 2013, through March 20, 2016, and location of residence of adolescents aged 12-19 in NYC,^c by borough

Borough	Location of Users Who Performed a Search on Teens in NYC Mobile App, No. (%) of Searches	Location of Residence of Adolescents Aged 12-19 in NYC, No. (%)
Bronx	3510 (19)	175 265 (21)
Brooklyn	4757 (26)	265 321 (33)
Manhattan	3779 (20)	112 366 (14)
Queens	5052 (27)	214 044 (26)
Staten Island	446 (2)	51 675 (6)
Missing or outside NYC	1000 (5)	NA
Total	18 544 (100)	818 671 (100)

Abbreviations: app, application; NA, not applicable; NYC, New York City.

^aReleased by NYC Department of Health and Mental Hygiene in 2013 as part of a program to promote sexual and reproductive health among adolescents aged 12-19 in NYC.

^bLocation refers to the borough in which the user was when he or she performed the search. User could choose among 3 options for location of interest: current location (18 544 [65%] searches), borough (2885 [10%]) searches, and address (1330 [5%]) searches. Geographic selection data were missing in 5744 (20%) searches. “Current location” was default option.

^cBased on data from US Census 2010.³² Numbers were weighted for sample nonresponse.

Among all 28 503 searches of the Where to Go section of the Teens in NYC mobile app, 10 656 (37%) were for both 1 or more sexual health service and 1 or more contraceptive method, 10 134 (36%) were for only 1 or more sexual health service, and 5931 (21%) were for only 1 or more contraceptive method. Data on sexual health service and contraceptive method selections were missing in 1782 (6%) searches. Among all searches, 20 790 (73%) involved the selection of at least 1 sexual health service. A mean of 2 and median of 1 unique sexual health services were searched. A sexual health service was selected in searches 45 110 times. Of these, birth control or gynecological services were searched 12 509 (28%) times, STI testing and treatment 8902 (20%) times, pregnancy testing 7674 (17%) times, abortion services 7494 (17%) times, HIV testing 3874 (9%) times, gold star services 2201 (5%) times, mental health counseling 1607 (4%) times, and LGBTQ-specific services 849 (2%) times.

Among all 28 503 searches, 16 587 (58%) involved the selection of at least 1 contraceptive method. Of note, 2414 (8%) searches included the selection of condom or female condom with another contraceptive method. A mean of 2 and a median of 1 unique contraceptive methods were searched. Contraceptive method was selected in searches 26 539 times. Of these, condom was searched 6630 (25%) times. The most frequently searched methods (condom and oral contraceptive pill) were also the most commonly used methods in the 2015 YRBS. However, whereas 52% of the contraceptive methods searched were for other methods, only 19% of students in the 2015 YRBS reported using other methods (Table 2).

Table 2.

Distribution of contraceptive methods searched on the Teens in NYC mobile app^a from February 27, 2013, through March 20, 2016, and of contraceptive method used at last sex by NYC public high school students aged 12-19 who had sex in the past 3 months and used contraception^b

Contraceptive Method	Contraceptive Method Searched by Users of Teens in NYC App, No. (%) of Searches	Contraceptive Method Used at Last Sex Among Those Who Used Contraception, Weighted No. (%)
Condom	6630 (25)	19 000 (62)
Oral contraceptive pill	6002 (23)	5000 (16)
Other method^c	13 907 (52)	6000 (19)^c
Contraceptive patch	1135 (4)	—
Copper intrauterine device	1170 (4)	—
Vaginal ring	1166 (4)	—
Contraceptive implant	1208 (5)	—
Hormonal intrauterine device	1356 (5)	—
Female condom	1779 (7)	—
Injectable contraception	2443 (9)	—
Emergency contraception	3650 (14)	—
Not sure	NA	1000 (3)
Total	26 539 (100)	31 000 (100)

Abbreviations: —, not available; app, application; NA, not applicable; NYC, New York City.

^aReleased by NYC Department of Health and Mental Hygiene in 2013 as part of a program to promote sexual and reproductive health among adolescents aged 12-19 in NYC.

^bBased on data from 2015 Youth Risk Behavior Survey³²; data were weighted to the NYC public high school population.

^c“Other method” in the 2015 Youth Risk Behavior Survey included other hormonal birth control (eg, injectable, ring, implantable), withdrawal, and other methods, such as intrauterine device.³³

Lessons Learned

This case study demonstrated the importance of making information available to adolescents about where to access all US Food and Drug Administration–approved contraceptive methods, as well as STI and other family planning services, including abortion services, which are often excluded from public education. The results suggest that the mobile app reached NYC adolescents nearly proportionately to their population distribution across the 5 NYC boroughs. One of the exceptions, the overrepresentation of current location searches in Manhattan, may have been related to a large number of adolescents traveling from other boroughs or surrounding suburbs to or through this central borough for school or other reasons.

We observed that searching for providers of sexual health services or contraceptive methods by current location was more popular than searching for them by selecting a borough, perhaps because current location was the default search option. Before the study, we had hypothesized that selecting a borough would be more popular than current location, postulating that adolescents may want to avoid going to a sexual health care provider close to their home or school. Future research should explore whether selecting a borough would be more popular if it were more prominently displayed in an app, perhaps in a map. Whereas individual providers could be mapped in the Teens in NYC mobile app, users were unable to interact directly with a map of providers by borough or by all of NYC.

The comparison of Teens in NYC mobile app search data on contraceptive methods with 2015 YRBS data indicates that the mobile app may have helped adolescents discover a wider range of contraceptive methods than those currently in use. To further support users in finding these methods, we structured the mobile app so that users who searched for contraceptive methods received results listing providers who prescribed and dispensed contraceptives. This decision was informed by Bronx Teens Connection Youth Advisory Board members, who advised that having to visit a pharmacy would be a barrier to accessing contraceptives.

Of note, we emphasized dual protection (eg, using condoms with another contraceptive method) to address pregnancy and STI prevention in the app. We set up the app’s What to Get section to display phrases such as “my birth control + his condoms” or “use with condom every time you have sex” when users accessed this section. In addition, similar messages appeared within the narrative videos²⁹ in the What to Expect section. Yet, despite these messages, only 8% of searches included the selection of both condoms and another contraceptive method. However, this low number of searches may not fully reflect the impact of these messages because easy access to condoms in NYC public high school health resource rooms and other settings (eg, pharmacies and convenience stores) potentially reduced the need for users of the Teens in NYC app to select condoms in their searches.

Downloads of the Teens in NYC mobile app were driven by media coverage after a May 2013 public launch, as well as by online and transit advertisements. Continuous marketing to reach those entering adolescence and to create exposure when adolescents need sexual health services would likely have further expanded the reach of the Teens in NYC mobile app. This strategy could be used by others who are developing similar apps. Indeed, the Teens in NYC mobile app has inspired apps with similar functionality, including TMI Georgia³⁴ (formerly gPower³⁵) in Georgia and Oakland Teens³⁶ in California. Yet, although it would be feasible for more cities and states to launch their own mobile apps, limited resources and the common functionalities required for locally based apps suggest that a national mobile app, particularly one that could be adapted to local needs, may be a more efficient use of limited resources.

Our analysis provides valuable information for those building similar apps. Our observations suggest the importance of allowing for searches: (1) by a range of contraceptive methods and sexual health services, including less widely available services such as abortion services, to serve a broad array of needs; (2) by user’s current location, to provide service information with a convenient point of reference; and (3) by contraceptive methods that are not only prescribed but also dispensed at provider locations, to support ease of access, particularly to less commonly used contraceptive methods.

Limitations

This analysis had several limitations. First, because of NYC DOHMH privacy restrictions for adolescent-focused services, we were unable to use a more robust app analytics software package. As a result, we could not collect data on individual users, and we were unable to group searches by individual user or to evaluate common metrics (eg, user demographic characteristics, user navigation) through the app. Although we did not have an analytics package that assessed the ages of users, a user survey suggested that 77% of users were adolescents (data not shown). Second, at the time this article was finalized, the Teens in NYC mobile app was not available in either Apple or Android app stores, primarily because of a lack of resources needed to update the app to meet current platform operating system specifications.

Conclusion

Although NYC DOHMH is working to identify and secure the resources needed to update and maintain the Teens in NYC mobile app, it is developing a sexual health mobile app that will reach beyond the teen audience, address limitations of analytics, and ensure the broadest reach by expanding the content available for those of different genders and sexual orientations. This evaluation of the Teens in NYC mobile app suggests that mobile apps with search functionality are a valuable way to deliver sexual health service information to adolescents.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Teens in NYC mobile app was funded by New York City tax-levy dollars.

References

Trussell

Henry

Hassan

Prezioso

Law

Filonenko

. Burden of unintended pregnancy in the United States: potential savings with increased use of long-acting reversible contraception. Contraception. 2013;87(2):154–161.

Biggs

Karasek

Foster

. Unprotected intercourse among women wanting to avoid pregnancy: attitudes, behaviors, and beliefs. Womens Health Issues. 2012;22(3):e311–e318.

Romero

Pazol

Warner

. Reduced disparities in birth rates among teens aged 15-19 years—United States, 2006-2007 and 2013-2014. MMWR Morb Mortal Wkly Rep. 2016;65(16):409–414.

Frerich

Garcia

Long

Lechner

Lust

Eisenberg

. Health care reform and young adults’ access to sexual health care: an exploration of potential confidentiality implications of the Affordable Care Act. Am J Public Health. 2012;102(10):1818–1821.

Centers for Disease Control and Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Division of STD Prevention. Sexually transmitted disease surveillance 2015. https://www.cdc.gov/std/stats15/STD-Surveillance-2015-print.pdf. Accessed December 5, 2017.

Kasedde

Kapogiannis

McClure

Luo

. Executive summary: opportunities for action and impact to address HIV and AIDS in adolescents. J Acquir Immune Defic Syndr. 2014;66(suppl 2):S139–S143.

Centers for Disease Control and Prevention. Diagnoses of HIV infection in the United States and dependent areas, 2015. HIV Surveillance Rep. 2015;27:1–114. http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html. Accessed December 5, 2017.

Collins

Martino

Shaw

Influence of New Media on Adolescent Sexual Health. Santa Monica, CA: RAND Corporation; 2011. https://www.rand.org/pubs/working_papers/WR761.html. Accessed December 5, 2017.

Guse

Levine

Martins

. Interventions using new digital media to improve adolescent sexual health: a systematic review. J Adolesc Health. 2012;51(6):535–543.

10.

Boyar

Levine

Zensius

TECHsex USA: Youth Sexuality and Reproductive Health in the Digital Age. Oakland, CA: ISIS, Inc; 2011. https://www.yth.org/wp-content/uploads/YTH-youth-health-digital-age.pdf. Accessed December 5, 2017.

11.

Jones

. Soap opera video on handheld computers to reduce young urban women’s HIV sex risk. AIDS Behav. 2008;12(6):876–884.

12.

Levine

McCright

Dobkin

Woodruff

Klausner

. SEXINFO: a sexual health text messaging service for San Francisco youth. Am J Public Health. 2008;98(3):393–395.

13.

Cornelius

St Lawrence

. Receptivity of African American adolescents to an HIV-prevention curriculum enhanced by text messaging. J Spec Pediatr Nurs. 2009;14(2):123–131.

14.

Cornelius

Cato

St Lawrence

Boyer

Lightfoot

. Development and pretesting multimedia HIV-prevention text messages for mobile cell phone delivery. J Assoc Nurses AIDS Care. 2011;22(5):407–413.

15.

Gold

Lim

Hellard

Hocking

Keogh

. What’s in a message? Delivering sexual health promotion to young people in Australia via text messaging. BMC Public Health. 2010;10:792.

16.

Gold

Aitken

Dixon

. A randomised controlled trial using mobile advertising to promote safer sex and sun safety to young people. Health Educ Res. 2011;26(5):782–794.

17.

Gold

Lim

Hocking

Keogh

Spelman

Hellard

. Determining the impact of text messaging for sexual health promotion to young people. Sex Transm Dis. 2011;38(4):247–252.

18.

Lim

Hocking

Hellard

Aitken

. SMS STI: a review of the uses of mobile phone text messaging in sexual health. Int J STD AIDS. 2008;19(5):287–290.

19.

Lim

Hocking

Aitken

. Impact of text and email messaging on the sexual health of young people: a randomised controlled trial. J Epidemiol Community Health. 2012;66(1):69–74.

20.

Lenhart

. Teens, social media, and technology overview 2015. http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015. Accessed December 5, 2017.

21.

Crutzen

Peters

Portugal

Fisser

Grolleman

. An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: an exploratory study. J Adolesc Health. 2011;48(5):514–519.

22.

Perry

Kayekjian

Braun

Cantu

Sheoran

Chung

. Adolescents’ perspectives on the use of a text messaging service for preventive sexual health promotion. J Adolesc Health. 2012;51(3):220–225.

23.

Preston

Walhart

O’Sullivan

. Prompting healthy behavior via text messaging in adolescents and young adults. Am J Lifestyle Med. 2011;5(3):247–252.

24.

Cole-Lewis

Kershaw

. Text messaging as a tool for behavior change in disease prevention and management. Epidemiol Rev. 2010;32:56–69.

25.

Jones

Eathington

Baldwin

Sipsma

. The impact of health education transmitted via social media or text messaging on adolescent and young adult risky sexual behavior: a systematic review of the literature. Sex Transm Dis. 2014;41(7):413–419.

26.

Cornelius

Appiah

. Using mobile technology to promote safe sex and sexual health in adolescents: current practices and future recommendations. Adolesc Health Med Ther. 2016;7:43–57.

27.

New York City Department of Health and Mental Hygiene. Sexual and reproductive health care best practices for adolescents and adults. June 2016. http://www1.nyc.gov/assets/doh/downloads/pdf/ms/srh-clinical-guide.pdf. Accessed December 5, 2017.

28.

Health 2.0. Teens in NYC: Sexual Health 2.0. 2016. http://health2con.com/tvshow/teens-in-nyc-sexual-health-2-0. Accessed December 5, 2017.

29.

New York City Department of Health and Mental Hygiene. Teen sexual health. www1.nyc.gov/site/doh/health/health-topics/teen-sexual-health.page . Accessed March 2, 2018.

30.

App Annie. App Annie Free Version. https://www.appannie.com. Accessed March 2, 2018.

31.

Apple, Inc. Itunes Connect. http://https//itunesconnect.apple.com. Accessed March 2, 2018.

32.

New York City Department of City Planning. Decennial census—Census 2010: total population by age and sex. 2010. www1.nyc.gov/site/planning/data-maps/nyc-population/census-2010.page. Accessed March 2, 2018.

33.

New York City Department of Health and Mental Hygiene. Youth Risk Behavior Survey, 2015. https://a816-healthpsi.nyc.gov/epiquery/YRBS/yrbsIndex.html. Accessed December 5, 2017.

34.

Georgia Campaign for Adolescent Power and Potential. TMI Georgia app. 2017. https://itunes.apple.com/us/app/tmi-georgia/id816161302?mt=8andhttps://play.google.com/store/apps/details?id=com.ydo.tmiga&rdid=com.ydo.tmiga. Accessed March 2, 2018.

35.

Georgia Campaign for Adolescent Power and Potential. gPower app. 2015. https://play.google.com/store/apps/details?id=com.gpower.app&hl=en. Accessed March 2, 2018.

36.

Oakland Unified School District. Oakland Teens app. 2015. https://itunes.apple.com/us/app/oakland-teens/id970376848?mt=8andhttps://play.google.com/store/apps/details?id=com.prolificinteractive.hot&hl=en. Accessed March 2, 2018.