Test-Retest Reliability of Self-Reported Substance Use and Sexual Risk Behavior Among at-Risk Adolescents

Abstract

Adolescents often engage in behaviors such as substance use and risky sexual activity that can lead to negative health and psychological consequences for themselves and others. Accurate measurement of these behaviors in surveys is challenging given that the behaviors are often viewed as undesirable and/or are illegal, so it is important to test the psychometric properties of instruments used to assess adolescent risk behaviors. The current study aimed to assess the test-retest reliability of a widely used measure of youth risk-taking behavior, the Youth Risk Behavior Survey (YRBS). A sample of 156 at-risk adolescents aged 16–18 years (81% male; 61% White) completed the YRBS retrospectively across intervals ranging from 3 to 12 days during their stay in a residential program at which they were under close supervision and had limited ability to engage in new risk behaviors. Participants were asked to complete the YRBS based on their “typical” (pre-program) behavior at both administrations, which were 10–14 weeks into their stay. The reliability of responses was assessed using kappa and weighted kappa analyses. Findings indicate moderate to substantial reliability for nearly all items, suggesting that at-risk youth reliably reported their engagement in health risk behaviors across multiple administrations and supporting the psychometric strength of the YRBS measure for use with this population.

Keywords

test-retest reliability Youth Risk Behavior Survey risk behavior substance use sexual behavior adolescents

Survey data suggest that adolescents often engage in health risk behaviors (e.g., substance use, risky sexual behaviors) which can lead to negative physical and emotional consequences for themselves and others (Achhab et al., 2016; McPherson et al., 2013). The leading causes of death among 15- to 19-year-olds in the U.S. are unintentional injuries (36.3%), intentional self-harm (22.9%), and assault (16.9%; Heron, 2019). These deaths, many of which may be preventable, have been linked to health risk behaviors including drinking alcohol, engaging in unprotected sexual activities, smoking cigarettes, and carrying weapons (Achhab et al., 2016; Brener et al., 2003; Kann et al., 2018; Rosario et al., 2014). Substance use is a particularly important contributor to health risk behaviors as it can lower inhibitions and increase the likelihood that youths will engage in other dangerous behaviors, such as driving while intoxicated or riding in a vehicle where the driver is intoxicated (Yellman et al., 2020), and being involved in sexual violence or nonconsensual sexual activity (Basile et al., 2020). It is therefore important to determine the utility of instruments designed to collect information about self-reported substance use and other health risk behaviors among adolescents by evaluating the psychometric properties of these measures. This is an important first step to understanding the larger phenomenon of adolescent health risk behavior and determining ways to mitigate its effects.

Health risk behaviors related to substance use and unprotected sex are of particular concern given their high prevalence rates among adolescents (Johnston et al., 2020). In the 2019 Monitoring the Future survey of over 42,000 youths, the most commonly used substances among adolescents within the last 30 days were alcohol (18.2%), marijuana (15.6%), and vaping nicotine (18.1%; Johnston et al., 2020). Some adolescents begin engaging in substance-related risk-taking behaviors in early adolescence (ages 12–13 years), with prevalence of use increasing with age across substances (Charles et al., 2017; Johnston et al., 2020; Substance Abuse and Mental Health Services Administration, 2017). For instance, in the 2019 Monitoring The Future adolescent survey, rates of 30-day alcohol (29.3%), marijuana (22.3%), and vaping nicotine (25.5%) use among 12^th graders were higher than rates of use among eighth graders (7.9%, 6.6%, and 9.6%, respectively; Johnston et al., 2020). Individuals who begin using substances at an earlier age are at an increased risk of developing substance use disorders, with one national study indicating that 14 years is the median age of onset for a DSM-IV-TR diagnosis of a substance abuse (Swendsen et al., 2012). The prevalence of youth engaging in risky sexual behaviors (e.g., multiple sexual partners, no contraceptive use) is also high, which may contribute to long-term health consequences including unintended pregnancy, sexually transmitted infections, or HIV/AIDS (Szucs et al., 2020). Additionally, risky sexual behaviors may increase the likelihood of sexual violence victimization during childhood and adolescence (Basile et al., 2020). Importantly, the prevalence rates of health risk behaviors are not evenly distributed in the population. At-risk adolescents, which include adolescents who have a) participated in delinquent activity, b) been charged with a crime, c) dropped out of school, or d) come from socioeconomically disadvantaged backgrounds (Boyce et al., 2008; Mason et al., 2010; Wang & Fredricks, 2014) are more likely to engage in these behaviors than are youths from other backgrounds. Thus, determining the psychometric properties of self-report measures of these behaviors among at-risk youths is especially important to understanding the nature of their health risk behaviors.

Health risk behaviors are typically measured via retrospective self-report questionnaires that cannot typically be verified in a way that is ethical, feasible, or cost-effective (Rosenbaum, 2009). However, the accuracy of these self-reports may be compromised due to several factors. Some health risk behaviors are sensitive in nature; therefore, many respondents may not be willing to explicitly report them (Norwood et al., 2016). In addition, some behaviors may be difficult to accurately recall, leading to an underreporting of behaviors (Nyitray et al., 2010). Recall difficulty may be especially significant when respondents are asked to remember their substance use, as being under the influence of substances may impair the respondent’s memory (Norwood et al., 2016). Finally, some behaviors may be purposefully under- or over-reported by adolescents because engaging in such behaviors is considered socially undesirable or desirable, respectively (Norwood et al., 2016). Such factors can make it difficult to properly interpret and apply results from health risk behavior self-report measures.

Evaluating the properties of measures of adolescent self-reported health risk behaviors is useful for many reasons. For example, it can increase confidence in data collected on the prevalence of, and trends in, health risk behaviors, which can inform health policies and practices at national, regional, and international levels (Bae et al., 2010; Brener et al., 2003). Additionally, results based on these measures can assist in identifying target populations who may benefit from prevention and intervention efforts (Baheiraei et al., 2012). One way to examine the confidence that researchers can have in self-reported health risk behaviors is to examine the test-retest reliability of assessment instruments. Test-retest reliability is the extent to which a measure yields consistent results when administered across two timepoints (Polit, 2014). It is often assessed using intra-class coefficients for continuous data and kappas or weighted kappas for ordinal and dichotomous data collected across timepoints (Cicchetti et al., 1990; Polit, 2014). For measures asking about lifetime substance use and other historical information, responses should be similar when the measure is given multiple times within a relatively short interval.

Test-retest reliability has been examined across multiple health risk behavior studies among adolescents, with far more research focusing on measures of substance use than those that assess risky sexual behavior. One measure of substance use that has been examined in this manner is the CRAFFT, which is a 6-item screener named for the mnemonic that covers the topics in the six items. It is used to help identify adolescents who are at-risk for engaging in problematic substance use (Levy et al., 2004). Results from prior studies indicate that the CRAFFT has acceptable (i.e., ICC >0.70; De Vet et al., 2011; κ > .41; Cohen, 1960) test-retest reliability among adolescents in outpatient primary care settings when assessed at 1-week time intervals (ICC= 0.93; Levy et al., 2004). Other substance use measures that have been tested in similar studies include the Form 90 Drug and Alcohol (ICC = 0.71; Slesnick & Tonigan, 2004), the Problem Oriented Screening Instrument for Teenagers (ICCs= 0.72-0.88; Knight et al., 2001), and the Substance Abuse Subtle Screening Inventory (ICCs = 0.81-0.92; Feldstein & Miller, 2007), all of which have demonstrated acceptable test-retest reliability in adolescent samples assessed across 1- and 2-week time intervals. Though less research exists on test-retest reliability of risky sexual behaviors, previous research indicates that adolescents tend to provide adequately reliable information regarding risky sexual behaviors. Using the Sexual Risk Behavior Assessment Schedule for Homosexual Youths interview, Schrimshaw and colleagues (2006) found that sexual minority youths reliably (ICCs = 0.77-0.97) reported risky sexual behaviors after a 2-week period. In another study, the authors found that adolescent females reliably reported information about risky sexual behaviors over a 1- to 2-week period (ICC = 0.74; Sieving et al., 2005). Similar results have been found when testing sexual risk behavior reporting among racial and ethnic minority adolescent samples (ICCs = 0.65-1.00, Sneed et al., 2001).

In terms of more comprehensive measures of engagement in high-risk behaviors, the Adolescent Risk-Taking Questionnaire demonstrated acceptable test-retest reliability on most scales when adolescents were tested after a 1-week recall period (rs = 0.35-0.80; Gullone et al., 2000). Flisher and colleagues (2004) also found that adolescents reliably (κs ≥ .41) recalled information regarding various risk behaviors when asked in a dichotomous (i.e., “yes” or “no”) format.

One of the most common and well-established self-report measures of health-risk behaviors among adolescents in the United States is the Youth Risk Behavior Survey (YRBS), a health risk assessment measure that was developed in 1989 by the Centers for Disease Control and Prevention (CDC; Kolbe et al., 1993). The YRBS focuses on risky behaviors that typically develop during adolescence and young adulthood. Sample behaviors include tobacco use, drug use, alcohol use, self-harming behaviors, and risky sexual behaviors. Respondents provide information on their engagement in the behavior, including the frequency, age of onset, and consequences of the behavior. The YRBS has demonstrated acceptable or greater test-retest reliability (i.e., kappa mean scores at 58.1% or higher) among community middle (Zullig et al., 2006) and high school (Raghupathy & Hahn-Smith, 2012) samples when assessed at 2-week intervals. In addition, an adapted version of the YRBS used in Korea demonstrated moderate to excellent test-retest reliability on a majority of items assessed (Bae et al., 2010). However, another study found greater stability in estimates over time for items related to sex, drugs, alcohol, and tobacco as compared to other risk behaviors (e.g., weight control behaviors; Rosenbaum, 2009). Other research has found that several items on the YRBS significantly differed across a 2-week period (Brener et al., 2002). Taken together, these findings provide promising indicators of the test-retest reliability of the YRBS but also suggest a need for more research on this measure, especially in diverse populations that have not been well-represented in previous studies. Expanding research can help determine the confidence with which the YRBS can be used to accurately estimate the prevalence of risk behaviors in a variety of adolescent populations.

The present study aims to evaluate the test-retest reliability of the YRBS among at-risk adolescents over a short period of time (3–12 days). Specifically, the study compares responses to 32 health-risk related questions involving five types of risk behaviors (i.e., alcohol use, marijuana use, other drug use, tobacco use, and risky sexual behaviors) using kappa (dichotomous items) and weighted kappa (ordinal items) analyses. Kappa and weighted kappa were chosen as they are appropriate analyses for these dichotomous and ordinal data (vs. ICC; Cicchetti et al., 1990). Based on prior literature and expectations for sound psychometrics, it is hypothesized that all YRBS items will demonstrate acceptable or better test-retest reliability.

Methods

Participants

Participants in this study were adolescents in a National Guard Youth ChalleNGe Program site located in the Southeastern United States (Youth ChalleNGe Academy; YCA). Individuals attending this program are considered “at-risk” for negative outcomes associated with early termination from high school, such as disruptive and deviant behaviors as well as impairment in their ability to maintain relationships and function within a society (Bombac, 2009). This program does not accept individuals with felony charges or current drug use, and though many individuals attending this program indicate clinical or subclinical behavioral and emotional problems, the difficulties are not as severe as what is seen in juvenile justice or psychological treatment facilities (Charles et al., 2021). Data for the current study were collected during spring 2019 (N = 62) and fall 2019 (N = 94). The combined sample consisted of 156 adolescents from 16 to 18 years old, with an average age of 16.72 (SD = .66). Participants in this sample were mostly male (80.8%), and mostly Caucasian (61.4%). More descriptive demographic information can be seen in Table 1.

Table 1.

Demographic Variables.

	M	SD
Age	16.72	.66
Age	n	%
Sex
Boys	126	88.1%
Girls	17	11.9%
Race
White/Caucasian	86	61.4%
Black/African-American	36	25.7%
Other/Multiracial	18	12.9%
Ethnicity
Hispanic/Latino/a	6	4.3%
Non-Hispanic/Latino/a	134	95.7%

Measures

Demographics

Participants were asked basic demographic questions including their age, gender, ethnicity, arrest history, and who they lived with before enrolling in YCA.

Youth Risk Behavior Survey

The YRBS is a self-report survey developed by the Centers for Disease Control and Prevention (CDC) to collect data on high-risk adolescent behaviors that may lead to death, injury, and other negative outcomes for the individual or society (Centers for Disease Control and Prevention, 2018). The YRBS includes six categories of risky behaviors that are monitored across national and local levels, and three categories (alcohol and other drug use, tobacco use, and risky sexual behaviors) were used in this study. Items reflected the frequency, amount, and age of first experience relevant to alcohol use (e.g., ‘how often do you have at least one drink of alcohol?‘), other drug use (e.g., ‘how old were you when you tried marijuana for the first time’; ‘during your life, how many times have you used any form of cocaine, including powder, crack, or freebase?‘), tobacco use (e.g., ‘have you ever tried cigarette smoking, even one or two puffs?‘), and risky sex behavior (e.g., ‘the last time you had sexual intercourse, did you or your partner use a condom?‘). Thirty-two items were considered in this study, some items included dichotomous response options (e.g., yes/no) and others used ordinal response options (e.g., frequency of an event). Table 2 indicates which items are dichotomous and were analyzed using kappa and which ones are ordinal and were analyzed using weighted kappa.

Table 2.

Test-Retest Reliability Across Health Behaviors on the YRBS.

YRBS Items	Kappa (k)	Time 1% endorsing	Time 2, %endorsing
Tobacco	.61
Ever tried cigarettes*	.73	71.8%	65.6
Age first smoked whole cigarette <13 years	.64	40.5%	33.6%
Smoked >5 cigarettes per day on the days smoked	.70	55.2%	27.3%
Tried to quit smoking cigarettes within past 12 months*	.60	26.9%	25.0
Uses smokeless tobacco more than once per week	.56	21.2%	26.8
Smokes cigars or cigarillos more than once per week	.43	36.8%	26.8
Alcohol	.61
Ever drank alcohol*	.62	74.2%	68.8
Age first drank alcohol <13 years	.52	29.1%	23.9
Drinks alcohol more than once per week	.62	26.1%	15.7
Has ≥5 alcoholic drinks more than once per week	.66	23.2%	16.3
Marijuana	.77
Ever used marijuana*	.77	74.2%	68.8
Age first used marijuana <13 years	.78	22.6%	22.1
Uses marijuana more than once per week	.76	48.4%	43.8
Other drugs	.51
Ever used cocaine*	.54	21.4%	16.2
Ever used inhalants*	.52	20.8%	17.0
Ever used heroin*	.46	12.4%	11.2
Ever used methamphetamines*	.58	16.2%	16.3
Ever used ecstasy*	.60	23.4%	17.0
Ever used hallucinogenic drugs*	.58	30.5%	22.9
Ever used synthetic marijuana*	.45	29.9%	24.8
Ever used steroids*	.36	13.6%	9.8
Ever used prescription drugs (without prescription)*	.55	45.5%	32.0
Ever injected illegal drugs*	.41	10.5	9.2
Sexual behavior	.67
Ever had sexual intercourse*	.70	78.7%	74.7
Age first had sexual intercourse <13 years	.67	24.8%	20.7
Have had ≥4 sexual partners total	.75	62.0%	61.8
Have had ≥4 sexual partners within past 12 months	.60	25.6%	26.1
Used drugs/alcohol before sex in most recent experience*	.52	44.6%	43.1
Used drugs/alcohol before sex in most recent experience*	.72	48.3%	44.8
Used a condom in most recent experience*	.52	65.3%	64.7
Used pregnancy prevention in most recent experience*	.79	5.2%	4.5
Have had both male and female sexual contact*	.75	11.3%	8.4
Total YRBS	.63

Note: The underlined kappa on the first line of each subsection represents the mean kappa for that category. *denotes dichotomous items that were analyzed using kappa. Remaining items were ordinal items analyzed using weighted kappa. The Time 1/2 percentages reflect % endorsing the higher risk categories on the ordinal scale.

Procedure

This study received approval from the Institutional Review Board of the corresponding author’s institution. Informed consent to recruit participants and conduct the study at YCA was obtained from the program director, who serves as guardian ad litem for adolescents during their stay. All youths who were residents at the time of data collection were eligible to participate in the study. Groups of residents who shared living quarters and general schedules were brought to a classroom where trained research assistants explained the study procedures and goals to the entire group. This was repeated for the multiple groups in residence at times convenient for each group’s schedule. Participants provided informed assent (younger than 18) or consent (18 years old) before participation. Participation was voluntary, and youths’ status in the program was unaffected by their decision to participate in the study. Program staff did not receive results for specific participants and information about residents who did or did not participate in the study was not shared with the program. Data were collected in group testing sessions via computers equipped with Qualtrics survey software over an approximately 2-week period as part of a larger battery of self-report measures. The mean length of time that youths had been in the program at the first testing session was 12.4 weeks (range: 10–14 weeks). The YRBS survey was administered twice, with the amount of time between administrations ranging from three to 12 days.

Results

The sample consisted of 156 participants who were administered the YRBS survey on two separate days with a range of three to 12 days (M = 8.28, SD = 3.13) between administrations. Data were screened for missing or problematic values and to determine if there were any outliers. No issues were observed. Kappa (16 dichotomous items) and weighted kappa (16 ordinal items) analyses were conducted in SPSS 25 to determine the test-retest reliability estimates from each item, each category, and across all items measured (Cohen, 1968). To examine whether testing comparisons between measures completed within a very short window differed from those with a slightly longer test-retest window, analyses were conducted without participants who only had 3 days between administrations (n = 24; remainder of participants had 6–12 days between administrations). The reliability of items was similar (e.g., same classification of level of agreement) when participants with 3 days were included as compared to when they were excluded. As such, all participants were retained.

Information about the prevalence of high-risk behavior is presented in Table 2 along with the reliability data described below. Ordinal items (e.g., age at first use) were collapsed into two groups of responses based on being relatively higher versus lower risk behavior. The percentages in the Time 1/Time 2 columns in Table 2 thus reflect the number of participants who endorsed the ordinal responses deemed higher risk (e.g., use <13 years old).

Reliability of the YRBS

The test-retest reliability estimates of all items used from the YRBS survey are displayed in Table 2. Dichotomous items were analyzed using kappa and ordinal items were analyzed using weighted kappa. None of the items had kappas between .81 and 1.00 (almost perfect agreement). However, there were 15 items (48.9%) that had kappas between .61 and .80 (substantial agreement) and 16 items (50.0%) that had kappas between .41 and .60 (moderate agreement). One item (“During your life, how many times have you taken steroid pills or shots without a doctor’s prescription?”) had fair agreement (i.e., kappa between .21 and .40; κ = .36). The average kappa across all items was in the substantial agreement range (κ = .63).

Reliability estimates were also examined by each category. Overall, items related to marijuana use were most reliably reported (κ = .77) followed by the sexual behaviors category (κ = .67), the cigarette and tobacco category (κ = .61), and the alcohol category (κ = .61). All of these categories had test-retest reliability in the substantial agreement range. The items related to other drug use had moderate test-retest reliability (κ = .51).

All items within the sexual behaviors category had kappas ranging from .52 to .79 indicating moderate to substantial agreement. Within the sexual behaviors category, the most consistently reported items related to the sex of partners with whom they have had sexual contact during their life (κ = .79), how many people they have had sexual intercourse with during their life (κ = .75), and which sexual orientation best describes them (κ = .75).

Within the cigarettes/tobacco category, items had kappas ranging between .43 and .73, which indicates moderate to substantial agreement. The most consistently reported items within the cigarettes/tobacco category were related to whether they have ever tried cigarette smoking (κ = .73), how many cigarettes they smoke per day on the days they smoke (κ = .70), and how old they were the first time they smoked a whole cigarette (κ = .64).

Items in the alcohol category had kappas ranging from .52 to .66 indicating moderate to substantial agreement. Within the alcohol category, the most consistently reported item was how often they have five or more drinks of alcohol in a sitting (κ = .66) and the least consistently reported item reported was about their age at first drink (κ = .52). Items within the marijuana category had very similar reliability values (κs = .76-78). Within the other drugs category, items had kappas ranging between .36 to .60 indicating fair to moderate agreement. The items that were most consistently reported within this category related to how many times within their life they have used ecstasy or MDMA (κ = .60), hallucinogenic drugs (κ = .58), and methamphetamine during their life (κ = .58). The least reliably reported item related to steroid use (κ = .36).

Discussion

The current study assessed the test-retest reliability of three categories (i.e., alcohol and other drugs, tobacco, and risky sexual behaviors) of the Youth Risk Behavior Survey (YRBS; CDC, 2018) on a sample of “at-risk” adolescents (ages 16–18) in a National Guard Youth ChalleNGe Program site located in the southeastern United States. Overall, findings indicate moderate to substantial test-retest reliability for all items except one about lifetime steroid use, suggesting the at-risk youth reliably reported health risk behaviors over short intervals. The average kappa across all measures was in the substantial agreement range. An examination of each category revealed that all items within the tobacco, alcohol, and sexual behaviors categories displayed moderate to substantial agreement, whereas items in the marijuana category were all in the substantial range and those in the other drugs category were in the fair to moderate range. Overall, the test-retest reliability of the YRBS was acceptable in this study, though not as high for other drug use as it was for other categories of risk behavior. This indicates that adolescents in the YCA program were relatively consistent in how they reported a range of risk behaviors.

Based on these findings, the YRBS demonstrated test-retest reliability that is comparable to that of similar risk behavior assessment instruments such as the CRAFFT (Levy et al., 2004), Form 90 Drug and Alcohol (Slesnick & Tonigan, 2004), Substance Abuse Subtle Screening Inventory (Feldstein & Miller, 2007), and Sexual Risk Behavior Assessment Scale for Homosexual Youths (Schrimshaw et al., 2006). It should be noted that although all of these scales have been found to have at least adequate test-retest reliability, the statistical tests used to determine this (ICCs) in prior studies of these instruments are not appropriate for use with the dichotomous and ordinal data produced by the YRBS so the values cannot be compared directly across studies. All but one item in this study had κs ≥ .41, indicating moderate or substantial reliability, and this is comparable to a study examining the test-retest reliability of a yes/no dichotomous risk behavior survey examined among adolescents (Flisher et al., 2004). Notably, the YRBS performed as well as other risk behavior measures while being tested in a unique, residential sample with greater engagement in risk behaviors than the general population. Additionally, the test-retest reliability estimates for the YRBS found in the current study were similar to those found when the YRBS was administered to community middle (mean κ = .63; Zullig et al., 2006) and high school students (mean κ = .58; Raghupathy & Hahn-Smith, 2012), as well as a national sample of adolescents in the United States (median tetrachoric correlation = .87; Rosenbaum, 2009). Therefore, results support the notion that the YRBS can successfully be implemented as a tool to assess for a variety of risk behaviors in settings where such behaviors are likely more prevalent than in the community. This is important to consider given that the YRBS is a comprehensive measure of risk behaviors and its use may be more cost-effective than the use of multiple narrowband instruments.

In the current study, higher test-retest reliability was found among items assessing risky sexual behaviors, tobacco, alcohol, and marijuana use relative to those assessing other drug use. There are a few potential explanations as to why participants may have reported this type of information more consistently across time points. First, although confidentiality was explained to all adolescents before they participated, some individuals may have felt reluctant to disclose information regarding use of “hard drugs” that, if known by others, could result in serious legal repercussions (Brener et al., 2003). Second, research has shown that substance use during adolescence may affect brain development and cognition (Squeglia et al., 2009) so it is possible that adolescents who use a wider variety of substances or who use stronger substances have genuine difficulty recalling their use. Third, participants may have been confused by some items. For example, two of the items with the lowest test-retest reliability in this study were “During your life, how many times have you taken steroid pills or shots without a doctor's prescription?” and “During the past 30 days, on how many days did you smoke cigars, cigarillos, or little cigars?” Participants may have been unsure what these less commonly used substances were or whether they had used them, so they answered differently at different timepoints. Additionally, any disagreement between the two timepoints would have a stronger impact on reliability statistics for these low base rate behaviors than they would for behaviors that were more common in the sample.

This study evaluated test-retest reliability of the YRBS using kappa and weighted kappa estimates across a short recall period (3–12 days). Additionally, the sample was fairly large and more racially diverse than those in previous studies on assessing risk behavior. The study also involves an “at-risk” sample of primarily male adolescents, which is an important group to study, though it is also the case that results may not generalize to other, different populations. Another limitation of this study is that self-reported results on the YRBS could not be verified and may have been subject to reporting biases. Despite these limitations, the results of this study suggest that the YRBS generally demonstrates adequate test-retest reliability and can be used with confidence in populations like the one in this study. Future research should continue to focus on the psychometrics of measures of self-reported health risk behaviors so that researchers are able to use the most reliable and valid assessments to identify individuals and target populations who may benefit from prevention and intervention efforts. Additionally, it would be beneficial for future studies to examine whether the terminology used for different substances on the YRBS match the names that adolescents are familiar with and whether all substances and methods used by adolescents (e.g., vaping) are adequately covered by this measure. It is possible that updating the items to more closely match adolescents’ experiences would improve test-retest reliability as well as the validity of results.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Nora E. Charles

References

Achhab

Y. E.

Ammari

A. E.

Kazdouh

H. E.

Najdi

Berraho

Tachfouti

Lamri

El Fakir

Nejjari

(2016). Health risk behaviours amongst school adolescents: Protocol for a mixed methods study. BMC Public Health, 16(1209), 1–6. https://doi.org/10.1186/s12889-016-3873-4

Bae

Joung

Kim

J. Y.

Kwon

K. N.

Kim

Y. T.

Park

S. W.

(2010). Test-retest reliability of a questionnaire for the korea youth risk behavior web-based survey. Journal of Preventive Medicine and Public Health, 43(5), 403–410. https://doi.org/10.3961/jpmph.2010.43.5.403

Baheiraei

Hamzehgardeshi

Mohammadi

M. R.

Nedjat

Mohammadi

(2012). Psychometric properties of the Persian version of the youth risk behavior survey questionnaire. Iranian Red Crescent Medical Journal, 14(6), 363–370.

Basile

K. C.

Clayton

H. B.

Rostad

W. L.

Leemis

R. W.

(2020). Sexual violence victimization of youth and health risk behaviors. American Journal of Preventive Medicine, 58(4), 570–579. https://doi.org/10.1016/j.amepre.2019.11.020

Boyce

W. F.

Davies

Gallupe

Shelley

(2008). Adolescent risk taking, neighborhood social capital, and health. Journal of Adolescent Health, 43(3), 246–252. https://doi.org/10.1016/j.jadohealth.2008.01.014

Brener

N. D.

Billy

J. O.

Grady

W. R.

(2003). Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. Journal of Adolescent Health, 33(6), 436–457. https://doi.org/10.1016/S1054-139X(03)00052-1

Brener

N. D.

Kann

McManus

Kinchen

S. A.

Sundberg

E. C.

Ross

J. G.

(2002). Reliability of the 1999 youth risk behavior survey questionnaire. Journal of Adolescent Health, 31(4), 336–342. https://doi.org/10.1016/S1054-139X(02)00339-7

Centers for Disease Control and Prevention . (2018). Youth Risk Behavior Survey Questionnaire. www.cdc.gov/yrbs

Charles

N. E.

Bullerjahn

M. R.

Barry

C. T.

(2021). Understanding at-risk youths: Average PAI-A scores and their associations with impulsivity-related constructs. Journal of Personality Assessment, 103(1), 33–47. https://doi.org/10.1080/00223891.2019.1692855

10.

Charles

N. E.

Mathias

C. W.

Acheson

Dougherty

D. M.

(2017). Preadolescent sensation seeking and early adolescent stress relate to at-risk adolescents’ substance use by age 15. Addictive Behaviors, 69, 1–7. https://doi.org/10.1016/j.addbeh.2017.01.005

11.

Cicchetti

D. V.

Showalter

McCarthy

(1990). A computer program for calculating subject-by-subject kappa or weighted kappa coefficients. Educational and Psychological Measurement, 50(1), 153–158. https://doi.org/10.1177/0013164490501018

12.

Cohen

(1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177%2F001316446002000104

13.

Cohen

(1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220. https://psycnet.apa.org/doi/10.1037/h0026256

14.

De Vet

H. C.

Terwee

C. B.

Mokkink

L. B.

Knol

D. L.

(2011). Measurement in medicine: A practical guide. Cambridge university press.

15.

Feldstein

S. W.

Miller

W. R.

(2007). Does subtle screening for substance abuse work? A review of the Substance Abuse Subtle Screening Inventory (SASSI). Addiction, 102(1), 41–50. https://doi.org/10.1111/j.1360-0443.2006.01634.x

16.

Flisher

A. J.

Evans

Muller

Lombard

(2004). Brief report: Test–retest reliability of self-reported adolescent risk behaviour. Journal of Adolescence, 27(2), 207–212. https://doi.org/10.1016/j.adolescence.2001.10.001

17.

Gullone

Moore

Moss

Boyd

(2000). The adolescent risk-taking questionnaire: Development and psychometric evaluation. Journal of Adolescent Research, 15(2), 231–250. https://doi.org/10.1177/0743558400152003

18.

Heron

(2019). Deaths: leading causes for 2017. National Vital Statistics Reports, Vol. 68. https://www.cdc.gov/nchs/data/nvsr/nvsr68/nvsr68_06-508.pdf

19.

Johnston

L. D.

Miech

R. A.

O’Malley

P. M.

Bachman

J. G.

Schulenberg

J. E.

Patrick

M. E.

(2020). Monitoring the future national survey results on drug use 1975-2019: Overview of key findings on adolescent drug use. Institute for Social Research, The University of Michigan. http://www.monitoringthefuture.org//pubs/monographs/mtf-overview2019.pdf

20.

Kann

McManus

Harris

W. A.

Shanklin

S. L.

Flint

K. H.

Queen

Lowry

Chyen

Whittle

Thornton

Lim

Bradford

Yamakawa

Leon

Brener

Ethier

K. A.

(2018). Youth risk behavior surveillance — United States, 2017. MMWR Surveillance Summary 67(SS-8), 1–114. http://dx.doi.org/10.15585/mmwr.ss6708a1

21.

Knight

J. R.

Goodman

Pulerwitz

DuRant

R. H.

(2001). Reliability of the Problem Oriented Screening Instrument for Teenagers (POSIT) in adolescent medical practice. Journal of Adolescent Health, 29(2), 125–130. https://doi.org/10.1016/S1054-139X(00)00206-8

22.

Kolbe

L. J.

Kann

Collins

J. L.

(1993). Overview of the youth risk behavior surveillance system. Public Health Reports, 108(Suppl 1), 2–10.

23.

Levy

Sherritt

Harris

S. K.

Gates

E. C.

Holder

D. W.

Kulig

J. W.

Knight

J. R.

(2004). Test-retest reliability of adolescents’ self-report of substance use. Alcoholism: Clinical and Experimental Research, 28(8), 1236–1241. https://doi.org/10.1097/01.ALC.0000134216.22162.A5

24.

Mason

W. A.

Hitch

J. E.

Kosterman

McCarty

C. A.

Herrenkohl

T. I.

Hawkins

J. D.

(2010). Growth in adolescent delinquency and alcohol use in relation to young adult crime, alcohol use disorders, and risky sex: A comparison of youth from low- versus middle-income backgrounds. Journal of Child Psychology and Psychiatry, 51(12), 1377–1385. https://doi.org/10.1111/j.1469-7610.2010.02292.x

25.

McPherson

K. E.

Kerr

Morgan

McGee

Cheater

F. M.

McLean

Egan

(2013). The association between family and community social capital and health risk behaviours in young people: An integrative review. BMC Public Health, 13(971), 1–13. https://doi.org/10.1186/1471-2458-13-971

26.

Norwood

M. S.

Hughes

J. P.

Amico

K. R.

(2016). The validity of self-reported behaviors: Methods for estimating underreporting of risk behaviors. Annals of Epidemiology, 26(9), 612–618. https://doi.org/10.1016/j.annepidem.2016.07.011

27.

Nyitray

A. G.

Harris

R. B.

Abalos

A. T.

Nielson

C. M.

Papenfuss

Giuliano

A. R.

(2010). Test–retest reliability and predictors of unreliable reporting for a sexual behavior questionnaire for US men. Archives of Sexual Behavior, 39(6), 1343–1352. https://doi.org/10.1007/s10508-009-9522-6

28.

Polit

D. F.

(2014). Getting serious about test–retest reliability: A critique of retest research and some recommendations. Quality of Life Research, 23(6), 1713–1720. https://doi.org/10.1007/s11136-014-0632-9

29.

Raghupathy

Hahn-Smith

(2012). Reliability of the high school Youth Risk Behavior Survey when administered online. International Quarterly of Community Health Education, 32(2), 135–148. https://doi.org/10.2190%2FIQ.32.2.d

30.

Rosario

Corliss

H. L.

Everett

B. G.

Reisner

S. L.

Austin

Buchting

F. O.

Birkett

(2014). Sexual orientation disparities in cancer-related risk behaviors of tobacco, alcohol, sexual behaviors, and diet and physical activity: Pooled Youth Risk Behavior Surveys. American Journal of Public Health, 104(2), 245–254. https://doi.org/10.2105/ajph.2013.301506

31.

Rosenbaum

J. E

. (2009). Truth or consequences: The intertemporal consistency of adolescent self-report on the Youth Risk Behavior Survey. American Journal of Epidemiology, 169(11), 1388–1397. https://doi.org/10.1093/aje/kwp049

32.

Schrimshaw

E. W.

Rosario

Meyer-Bahlburg

H. F.

Scharf-Matlick

A. A.

(2006). Test–retest reliability of self-reported sexual behavior, sexual orientation, and psychosexual milestones among gay, lesbian, and bisexual youths. Archives of Sexual Behavior, 35(2), 220–229. https://doi.org/10.1007/s10508-005-9006-2

33.

Sieving

Hellerstedt

McNeely

Fee

Snyder

Resnick

(2005). Reliability of self reported contraceptive use and sexual behaviors among adolescent girls. The Journal of Sex Research, 42(2), 159–166. https://doi.org/10.1080/00224490509552269

34.

Slesnick

Tonigan

(2004). Assessment of alcohol and other drug use by runaway youths: A test-retest study of the Form 90. Alcohol Treatment Quarterly, 22(2), 21–34. https://doi.org/10.1300/J020v22n02_03

35.

Sneed

C. D.

Chin

Rotheram-Borus

M. J.

Milburn

N. G.

Murphy

D. A.

Corby

Fahey

(2001). Test-retest reliability for self-reports of sexual behavior among thai and korean respondents. AIDS Education and Prevention, 13(4), 302–310. https://doi.org/10.1521/aeap.13.4.302.21429

36.

Squeglia

L. M.

Jacobus

Tapert

S. F.

(2009). The influence of substance use on adolescent brain development. Clinical EEG and Neuroscience, 40(1), 31–38. https://doi.org/10.1177%2F155005940904000110

37.

Substance Abuse and Mental Health Services Administration . (2017). Key substance use and mental health indicators in the United States: Results from the 2016 national survey on drug use and health. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/

38.

Swendsen

Burstein

Case

Conway

K. P.

Dierker

Merikangas

K. R.

(2012). Use and abuse of alcohol and illicit drugs in US adolescents: Results of the National Comorbidity Survey–Adolescent Supplement. Archives of General Psychiatry, 69(4), 390–398. https://doi.org/10.1001/archgenpsychiatry.2011.1503

39.

Szucs

L. E.

Lowry

Fasula

A. M.

Pampati

Copen

C. E.

Hassaini

K. S.

Steiner

R. J.

(2020). Condom and contraceptive use among sexually active high school students—youth Risk Behavior Survey, United States, 2019. US Department of Health and Human Services, Centers for Disease Control and Prevention. https://www.cdc.gov/mmwr/volumes/69/su/pdfs/su6901a2-H.pdf

40.

Wang

M. T.

Fredricks

J. A.

(2014). The reciprocal links between school engagement, youth problem behaviors, and school dropout during adolescence. Child Development, 85(2), 722–737. https://doi.org/10.1111/cdev.12138

41.

Yellman

M. A.

Bryan

Sauber-Schatz

E. K.

Brener

(2020). Transportation risk behaviors among high school students—youth Risk Behavior Survey, United States, 2019. US Department of Health and Human Services, Centers for Disease Control and Prevention. https://www.cdc.gov/mmwr/volumes/69/su/pdfs/su6901a9-H.pdf

42.

Zullig

K. J.

Pun

Patton

J. M.

Ubbes

V. A.

(2006). Reliability of the 2005 middle school Youth Risk Behavior Survey. Journal of Adolescent Health, 39(6), 856–860. https://doi.org/10.1016/j.jadohealth.2006.07.008