Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care

Abstract

Aim:

This cross-sectional study investigates the psychometric properties of the Swedish edition of the Adolescent Drug Abuse Diagnosis (ADAD), and specifically examines the internal consistency and construct validity of five life domains reported by female adolescents in special residential care in Sweden (N = 780; M_age = 16 years old).

Methods:

Principal component analysis and entropy-based analysis were used to test construct validity.

Conclusion:

Results indicate that ADAD may be able to reliably distinguish between areas that are important targets for intervention.

Keywords

Adolescent Drug Abuse Diagnosis instrument (ADAD)construct validity girls reliability young women youth-reported scores

This article examines the psychometric properties of a number of life areas within a widely used assessment entitled the Adolescent Drug Abuse Diagnosis instrument (ADAD; Friedman & Utada, 1989). The Swedish version of ADAD (Söderholm Carpelan, Hermodsson, & Öberg, 1997) is used in special residential treatment/care settings in Sweden. Sound assessment information lays the foundation for case conceptualisation and treatment planning (National Board of Health and Welfare, 2015). Therefore, professionals involved in special residential care should work to accurately and efficiently understand the severity and extent of problems that youth present with, as well as document youth resources and challenges in important life areas (e.g., family, other relationships, in the community; Forkby & Höjer, 2011; National Board for Institutional Care, 2010b). Inefficient and/or inaccurate assessment results and practices can harm youth. For example, tailored, effective treatments can be delayed or not delivered at all due to a lack of insight into the risks, problems, and strengths of young people who come into treatment (Forkby & Höjer, 2011; Olsson & Fridell, 2018). The assessment and practical challenges in this type of treatment are many, and the overall aim of assessment should ultimately be based on the benefit this activity provides to the young person (National Board for Institutional Care, 2010b; Söderholm Carpelan & Hermodsson, 2004).

ADAD was designed to meet the aforementioned challenges and the purpose of the present study was to advance what is known about the psychometric properties of ADAD, namely examine internal consistency reliability and construct validity, with a focus on adolescent girls’ reports of their own problems, which, we argue, is of importance in its own right. It is time to consider how well ADAD functions as a channel for young people to describe their more specific symptoms and adjustment problems. This is an issue that appears to have been missing in earlier ADAD literature. The approach may offer a needed way to prioritise young people’s voices in their own assessment, as well as offer a complement to ADAD composite scores, which have some limitations (e.g., Börjesson, Armelius, & Östgård-Ybrandt, 2007; Chinet, Plancherel, Bolognini, Holzer, & Halfon, 2007; Jansson, 1999; Ybrandt, Börjesson, & Armelius, 2008).

Special residential treatment in Sweden

Special residential treatment provides interventions and general care to young people (Janson, 2004; Ministry of Health and Social Affairs [SOU 2000:77]). Youth in special residential care are between the ages of 13 and 20 years, and there are presently 23 special residential homes for young people in Sweden (National Board for Institutional Care, 2010a). The National Board of Institutional Care (SiS) is a Swedish government agency that provides this care. Youth come into special residential homes in diverse ways. Most of the admissions to this type of treatment are compulsory and are made in accordance with the Care of Young Persons Act (SFS 1990:52). In these cases, it has been determined that there is substantial risk to the young person’s health and development in relation to substance abuse/addiction, criminal activity, or violent and disruptive behaviour. Some of these homes care for young people who have committed serious criminal offences and have been sentenced under the Care of Young Offenders Act (SFS 1998:603) (Department of Justice Ministry of Health and Social Affairs, 2000). A minority of youth voluntarily come into this type of treatment under the Social Services Act (SFS 2001:453. For more on special residential care in Sweden see Forkby and Höjer, 2011).

Adolescent Drug Abuse Diagnosis instrument (ADAD)

ADAD has been used in care and treatment settings in several European countries including Sweden as well as in Canada and the United States of America (USA, applications include general clinical assessment, treatment planning, and outcome evaluation). ADAD was originally developed in the USA as a structured interview to be used with young people who experienced drug addiction and associated problems (Friedman & Utada, 1989). The instrument has been translated into Swedish (Söderholm Carpelan et al., 1997) and French (Bolognini et al., 2001). Further, a European coordinating group (Czobor, Bacskai, Öberg, Haack, & Gerevich, 2011) has made a revised edition of the instrument available (i.e., the European Adolescent Assessment Dialogue, EuroADAD). The original ADAD is diverse in terms of its content, response options, and scoring alternatives. The Swedish edition of ADAD is regularly used in Swedish special residential treatment at the time of enrolment, when treatment/care is concluded, and at a follow-up (post-treatment) interview. ADAD is available in paper and computerised formats. In Sweden, ADAD serves as a tool for treatment planning and evaluation and method development and should facilitate follow-up with young people as they leave treatment. Within the Swedish government (i.e., SiS), ADAD interviews have also been used for planning activities. All interview data are saved in a research database and are available (in an anonymous format) to research projects with ethical permission.

Regarding content, the original ADAD (Friedman & Utada, 1989) was partially based on the Addiction Severity Index (McLellan et al., 1992). ADAD consists of 150 questions concerning nine life problem areas: physical health (also called medical), school, employment, social relations, family and background, psychological health, criminality (also called legal), as well as alcohol and drug use. Responses that can be given on ADAD include dichotomous (yes or no), Likert-type, and open-ended response options.

The time frame for questions includes problems in the last 30 days, last three months, and lifetime (i.e., have you ever). For alcohol and drug use and the criminality area there was an additional time frame used, “typical month last year”.

One important purpose of ADAD is to let the youth express their concerns and needs for assistance as they perceive them and thus become more involved in their own care. ADAD scoring is sometimes complex in that several different types of scores are generated for each area. For each ADAD area, based on a four-point scale, young people rate how worried or concerned they are with their own problems and life conditions in the past 30 days (youth report) and perceptions of their need for help/treatment (youth report). The interviewer also estimates the young person’s need for help/treatment based on a 10-point scale (interviewer report). Finally, the interviewer’s estimation of the accuracy of the youth’s report of problems is also recorded (e.g., due to not understanding a question or if factually inaccurate information is given).

Another scoring option is to use an ADAD composite score that consists of the sum of youth responses to an item subset (i.e., a selected group of items) within an area and some of these items are recoded or weighted in the process. A three-member panel of researchers/clinicians determined the weighting of the items through a consensus procedure. The panel considered both the clinical significance and the importance of items in the selected subset of items that make up the composite score as reflected by item correlations with interviewer’s severity ratings (Friedman, & Utada, 1995).

The ADAD composite scores have been criticised (e.g., Börjesson et al., 2007; Chinet et al., 2007; Jansson, 1999; Ybrandt et al., 2008). For example, the studies by Börjesson et al. (2007) and Ybrandt et al. (2008) compared ADAD composite scores between a normative and antisocial group of youth and results indicated that differences between the groups were small as measured by the composite scores, but not as measured by the interviewer and youth reports on ADAD. For the normative group, the composite score also showed higher values in the physical health, social, family and alcohol areas than interviewer ratings and adolescent ratings.

In relation to problems with ADAD composite scores, Jansson (1999) noted potential undesirable statistical properties for these scores. Distribution of composite values can easily be skewed, when a youth must agree to a large number of options in one of the checklists in order to get a high score in the final composite value. For example, the drug use checklist contains 11 different drugs and to get maximum value in the composite drug use score all 11 drugs must be used two or more times a day for a period of 30 days. Thus, it is unlikely that a young person will get a maximum value on the composite drug use score. This can lead to an underestimation of drug problems. Thus, Jansson (1999) and Ybrandt et al. (2008) suggested that composite scores on ADAD may not provide the most accurate indication of youth problems. Chinet et al. (2007) also noted that the ADAD composite scores in different areas also have the disadvantage of having different ranges in varied areas, thereby making the area composite scores difficult to combine, if, for example, factor analysis identified a higher order factor structure across scale/area scores. ADAD is an assessment tool that has been widely used and it has important real-life implications for many young people in treatment and care, and therefore it should continue to be developed through systematic research studies so that it has the best possible psychometric qualities (i.e., reliability and validity) as well as the ability to predict important outcomes.

Prior studies of ADAD’s internal consistency reliability

In terms of psychometric properties, the two indicators of most relevance to the present study are reliability at the item-scale/area level and construct validity of item-scales/areas as examined by factor analysis. Most reports of internal consistency of the different ADAD areas have concerned the composite score or the interviewer severity ratings. The youth report is the focus of present study. A summary of what is known about internal consistency reliability with the composite and interview severity score is given here, as some arguments about the multidimensional design of ADAD are evident in this research literature and relevant to the present study, even though this literature does not focus solely on youth-reported ADAD items, which is the focal point of the present study.

In the original validation study of ADAD, Friedman and Utada (1989) argued that Cronbach’s alpha (a typical index of internal consistency reliability among items on the same scale) was not the most appropriate way to investigate the reliability of ADAD. Internal consistency is a measure of homogeneity, but all items within an ADAD area are not intended to measure the same construct. In the ADAD validation study, Friedman and Utada’s (1989) sample consisted of adolescents in drug treatment in different parts of the USA and results indicated that Cronbach’s alphas of the composite scores ranged from .66 in the social relations area to .85 in the family area. A Swiss study with a clinical sample using the French edition of ADAD reported alpha values for the composite scores that ranged from .47 in the social relations area to .87 in the family area (Bolognini et al., 2001). In a Swedish study, alpha values for composite scores in a normative adolescent sample ranged from .57 in the social relations area to .85 in the family area, and within a Swedish clinical sample alpha values ranged from .32 in the school area to .84 in the drug and family areas (Börjesson et al., 2007). Across the aforementioned studies the evidence base for ADAD scales is mixed, with some scales demonstrating less than optimal reliability (e.g., social relations) while other ADAD scales have shown very good reliability (e.g., family) as indexed by Cronbach’s alpha.

Variation in the aforementioned results concerning the reliability of the ADAD composite scores could be a function of differences across studies in terms of the national/cultural context, refinements in the ADAD instrument made over time, type of reliability index used in a particular study (Cronbach’s alpha and/or McDonald’s omega), or study sample characteristics (e.g., normative and/or clinical youth samples). Thus, the extant research literature indicates that the internal consistency reliability of some ADAD scales is unclear, and further psychometric development of ADAD scores is warranted.

Prior studies of ADAD’s construct validity

Construct validity allows for greater insight into what a survey is actually measuring from the standpoints of theory as well as study participants and their life experience. Friedman and Utada (1989) reported that they had conducted separate factor analyses for ADAD items in each area, but the specific results of these factor analyses were not reported in the original validation study. Other indications of construct validity for ADAD come from a Swedish study (Börjesson et al., 2007). In this case, construct validity was tested using principal component analysis on the interviewer severity ratings. Results showed that one factor was composed of social, criminality, and alcohol and drug area items and another factor consisted of physical health, school, family, and psychological health area items (Börjesson et al., 2007).

In sum, most of what is known about ADAD scale reliability and factor structure (an indicator of construct validity) concerns composite scores or interviewer severity ratings and the psychometric properties of the youth-report ADAD scale/area scales have been under-studied and are not widely reported in the research literature. Further, other novel contributions of this study are that the tests of construct validity (i.e., dimensional analyses) are made at the item level within several ADAD life problem areas, and the analyses are not on severity ratings or composite scores that represent one area each (i.e., a summary score). Thus, this study aims to address the aforementioned gaps in the research literature regarding youth-reported ADAD responses internationally as well as within the Swedish ADAD research literature. The original ADAD continues to be used in special residential care in Sweden and studies that may be able to refine and make this instrument more efficient and useful are needed as points of information that can inform practice and treatment in this type of care.

Study aims

In light of the need to focus on the perspectives of young people and to provide more scoring options for the use of ADAD, the purpose of this study was to examine the internal consistency reliability and construct validity of five life domains in youth-reported ADAD scores with a nationally derived sample of Swedish adolescent girls in special residential treatment.

In this study, the focus was on areas that most directly concerned the participants themselves, in contrast to relational problems that might involve the adjustment problems of significant others. Based on this focus, the analyses reported in this article concern five ADAD areas: physical health, psychological health, legal/criminality, and alcohol and drug problem areas. In each area, items that reflect participants’ own perceptions of their current adjustment problems were included in the analyses. This measurement focused study is part of wider programme of research that concerns the adjustment and prognosis of Swedish adolescent girls and young women in special residential treatment (Klingstedt, Olsson, Ferrer-Wreder, & Wångby-Lundh, 2018; Wångby-Lundh, Klingstedt, Bergman, & Ferrer-Wreder, 2018).

Method

Participants

The study sample was derived from all adolescent girls and young women in Sweden who were admitted to special residential homes from 1997 to 2000. Among the 1168 girls and young women who were admitted to special residential treatment during this time period, 57% were admitted because of socially destructive behaviour, 40% because of drug abuse, 31% because of criminality, and 15% because of alcohol abuse. Of the participants in the sample, 86% were admitted for at least one of these behaviours. In a minority of cases, there were other (additional) reasons for admission to special residential treatment including: psychological problems (28%), school problems (34%) and/or family problems (34%).

The mean age of the participants in the study, at entry into care, was 16.05 years (SD = 1.79). Twenty-seven percent of the study participants were between 12 and 14 years old, 42% were between 15 and 16 years old, and 31% were 17 to 20 years old. Eighty-two percent of the study participants were born in Sweden. The school problem area included participants aged 13 to 16 years old (n = 538) as in Sweden, school is not compulsory for children older than 16 years of age.

Study dropout

Of the 1168 admitted young women, 388 (33%) were not included in the SiS research database. Many were never interviewed at intake into special residential treatment due to administrative problems with the implementation of the interview procedure at these homes (Söderholm Carpelan & Hermodsson, 2004). For 22 of the 388 young women, there was a documented refusal to be included in the research database. There was no significant association between being included in the SiS research database or not and reason for commitment to treatment, except for school problems (36% versus 29%, p < .05). There was no significant association between inclusion in the research database and number of reasons for referral to treatment.

Measures

This study’s main instrument is the Swedish version of the ADAD which is similar to the original American edition of the instrument (Friedman & Utada, 1989) and includes all items from the original ADAD. All changes in the Swedish version of ADAD were approved of by one of the authors of the original instrument, Alfred Friedman (Söderholm Carpelan et al., 1997).

Item selection

The criteria for ADAD item inclusion in the present study’s analyses were that the item should reflect participants’ current adjustment problems. For physical and psychological problems, the timeframe was “in the last 30 days” as reported by youth themselves (youth-reported ADAD items only). ADAD item inclusion for the alcohol and drug area was reflected with the response alternative “typical month last year” and the response alternative for criminality “past 12 months” was chosen.

Items reflecting a lifetime or other long-term time frame, such as “experienced serious depression ever in life” were excluded from the present analyses (exceptions are noted in the measures section). The young persons’ ratings of their worries about their problems and their need for help/treatment were also excluded from the analyses. The rationale for excluding these items was that high ratings in some circumstances might be considered benign. For example, to be concerned over one’s criminal behaviour might be better than not being worried at all. Some questions were also excluded from the analysis because of a high non-response rate (> 50%). Based on the item inclusion criteria, the items listed below remained in the analyses and were grouped into the respective four areas that had been used in prior ADAD studies. Below is a description of each original ADAD area and details about which items remained in the analysis.

Physical health

The physical health area in ADAD consists of 14 questions, one of which has the form of a checklist. This checklist includes 25 physical health problems (e.g., eating problems, frequent headaches, etc.). Four items concerning sexuality and pregnancy and an item concerning HIV/AIDS were excluded from the analyses. Twenty of the checklist items and five of the remaining non-checklist physical health ADAD items were included in the analyses (25 items in total). Examples of items from this area are described in Table 1 and ESM (Electronic Supplemental Material) Table A. For the scale, to make the data levels more comparable across items in the analyses, the items with continuous scales were recoded into categorical variables (see notes in Table 1).

Table 1.

Factor loadings for an oblique principal component analysis of items measuring physical health problems and the entropy-based match. Factors with three or more items loading > .40.

Item	Factor			entropy-based analysis
Item	I	III	VIII	entropy-based analysis
Eating problem^a	.696			X
Nausea/vomiting^a	.612			X
Often stomach-ache^a	.447			X
How many times past year have you visited a doctor?^b	.432
Are you sick often?^a		–.803		X
Frequent colds^a		–.747		X
How would you rate your physical health?^c		–.507
Frequent headaches^a		–.455
Fainting spells^a			.618
Pounding heart^a			.490	X

Notes. ^a0 = no; 1 = yes. ^b0 = 0 times; 1 = 1–4 times; 2 = 5 or more times; combined with D7 When was the last time you stayed in a hospital for physical or medical problems?; 19 young women having stayed in hospital the last year before admission were coded “1–4 times”. ^cReversed.

Psychological health

The psychological health area in ADAD consists of eight questions, three of which have the form of checklist (i2, i31, and i32). Two checklists and one additional question were included in the analysis (see Table 2 and ESM Table B for example items from this area). The first checklist contained 41 psychological and emotional reactions and symptoms, while the second checklist included 10 more severe symptoms. In the Swedish version of the ADAD, one item was added to the second checklist: i3j Have you experienced eating problems during the past 30 days? Almost all items were dichotomous (categorical) and the only item that had a continuous scale was recoded into a categorical variable (see specific note in Table 2).

Table 2.

Factor loadings for an oblique principal component analysis of items measuring psychological health problems and the entropy-based match. Factors with three or more items loading > .40 and index reliability.

Item	Factor				entropy-based analysis	Standardised item Cronbach’s alpha
Item	I	II	III	IV	entropy-based analysis	Standardised item Cronbach’s alpha
Index 1. Lack of belonging						.82
Feel you don’t belong or fit in	.731				X
Feel lonely even when you are with people	.725				X
Feel lonely	.648				X
Feel that people are unfriendly or dislike you	.612				X
Feel like others are against you	.576				X
Feel very self-conscious	.456				X
Feel inferior to others	.429				X
Index 2. Aggressive behaviour						.68
React by slamming doors, etc.		.771			X
Do angry things you can’t control		.749			X
Get into fights or arguments easily		.569			X
Trouble controlling violent behaviour past 30 days		.500			X
Index 3. Para-suicidal problems						.80
Suicide attempt			.707		X
Serious thoughts of suicide^b			.769		X
Feel like injuring or hurting yourself			.669		X
Feel you would be better off dead			.582		X
Having feelings of worthlessness			.421		X
Index 4. Anxiety/depression						.76
Serious anxiety/tension past 30 days				.743	X
Serious depression past 30 days				.664	X
Feel anxious or worried a lot				.568	X
Feel blue or depressed				.414	X
Psychological or emotional problems past 30 days^a				.533	X

Notes. ^a0 = 0 days; 1 = 1–7 days; 2 = 8–13 days; 3 = 14–23 days; 4 = 25 or more days.

Criminality

The criminality area in ADAD consists of 18 questions. Four of these questions are checklists (J91–4), assessing the youth’s involvement in 17 separate offence categories. One of these checklists, considering criminal activities in the past 12 months and two additional questions were selected for analysis (see Table 3 and ESM Table C for example items). For each separate offence category, young people report the number of times they have been involved in the activity. Before the analyses, these reports were recoded into response categories, the same as were used in a national Swedish study (Ring, 1999; see the specific notes in Table 3).

Table 3.

Factor loadings for an oblique principal component analysis of items measuring current criminality. Factors with three or more items loading > .40 and index reliability.

Item	Factor		Standardised item Cronbach’s alpha
Item	I	II	Standardised item Cronbach’s alpha
Index 1. Property and drug offences			.86
How much money did you make from illegal activities, past year?^b	.876
How many times involved in receiving stolen goods, past 12 months?^a	.775
How many times involved in drug sales or trafficking, past 12 months?^a	.764
How many times involved in burglary/larceny, past 12 months?^a	.724
How many days in the past 30 have you engaged in illegal activities?^c	.693
How many times involved in shoplifting during, 12 months?^a	.526
How many times been involved in auto theft, past 12 months?^a	.501
How many times been involved in illegal possession of weapon, past 12 months?^a	.473
How many times have involved in robbery, past 12 months?^a	.419
Index 2. Vandalism and assault			.78
How many times been involved in violence against civil servant, past 12 months?^a		.567
How many times been involved in disorderly conduct behaviour, past 12 months?^a		.758
How many times been involved in assault, past 12 months?^a		.748
How many times been involved in graffiti, past 12 months?^a		.734
How many times been involved in vandalism, the past 12 months?^a		.724

Notes. ^a0 = 0 times; 1 = 1–2 times; 2 = 3–5 times; 3 = 6–10 times; 4 = 11 times or more. ^b0 = 0; 1 < 1000 Swedish crowns; 2 < 1000–5000 Swedish crowns; 3 < 5000–10,000 Swedish crowns; 4 = 10,000 Swedish crowns or more. ^c0 = 0; 1 = 1 day; 2 = 2–7 days; 3 = 8–21 days; 4 = 22–30 days.

Alcohol and drug use/abuse

The alcohol and drug use/abuse area in ADAD includes 32 questions. Four of these questions are checklists concerning 16 different drugs (K1) including alcohol. The checklist concerning frequency of use in a typical month during the last year and five other questions were included in the analyses (see Table 4 and ESM Tables D and E for example items). Continuous scales were recoded into categories (see specific notes in Table 4). The frequency distribution was considered when the responses to these questions were recoded.

Table 4.

Factor loadings for an oblique principal component analysis of items measuring current alcohol and drug use/abuse. Factors with three or more items loading > .40 and index reliability.

Item	Factor				Standardised item Cronbach’s alpha
Item	I	II	III	IV	Standardised item Cronbach’s alpha
Index I. Drugs that stimulate the central nervous system					.72
How much money would you say you spent during the past 30 days on drugs?^d	.827
How often, inject amphetamines during a “typical” month in the past year?^a	.691
How often, using medicines during a “typical” month in the past year?^a	.539
How many days have you used more than one substance in the past months?^d	.507
Index II. Alcohol use/abuse					.76
How much did you usually drink per day in the past month on the days you drank alcohol?^b		.819
How many times have you gotten drunk in the past month?^c		.808
How often, using alcohol during a “typical” month in the past year?^a		.740
How much money would you say you spent during the past 30 days on alcohol?^e		.728
Index III. Heroin and similar drugs					.63
How often, inject heroin during a “typical” month in the past year?^a			.778
How often, using heroin/opiates (not injected) during a “typical” month in the past year?^a			.763
How often, using painkillers during a “typical” month in the past year?^a			.497
Index IV. Abuse of party drugs					.76
How often, using ecstasy during a “typical” month in the past year?^a				.884
How often, using cocaine during a “typical” month in the past year?^a				.705
How often, using hallucinogens during a “typical” month in the past year?^a				.612
How often, using amphetamines during a “typical” month in the past year?^a				.559
How often, using marijuana/hashish during a “typical” month in the past year?^a				.487

Notes. ^a0 = 0; 1 = once per month or less; 2 = 2–4 times a month; 3 = 2–3 times a week; 4 = 4–6 times a week; 5 = once per day; 6 = twice or more per day. ^b0 = 0; 1 = 1 drink; 2 = 2 drinks; 3 = 3 drinks; 4 = 4 drinks; 5 = 5 drinks or more. ^c0 = 0; 1 = 1 time; 2 = 2 times; 3 = 3 times; 4 = 4 times; 5 = 5 times or more. ^d0 = 0; 1 = 1–4 times; 2 = 5 times or more. ^e0 = 0; 1 > 50 Swedish crowns; 2 = 51–100 crowns; 3 = 101–200 crowns; 4 = < 200 crowns. ^e0 = 0; 1 = > 500 crowns; 2 = 501–1000 crowns; 3 = 1001–5000 crowns; 4 = < 5000 crowns.

Procedure

For the analyses conducted here, the data are cross-sectional. The Swedish National Board of Institutional Care (SiS) research database provided anonymous (to the research team) ADAD item responses. The young women were interviewed by trained SiS personnel at the time of their intake into treatment and gave their consent for their data being included in the SiS research database (if participants were younger than 15 years old, parents also provided consent to have their child’s ADAD responses in the SiS research database). This study’s ethics approval was obtained from the Swedish National Board of Health and Welfare.

Statistical analysis plan

Many of the ADAD items that were included in this study had a dichotomous response format, which is less suited for factor analysis. Thus, two methods were used to examine ADAD’s construct validity for the youth-reported items: principal component (PCA) and entropy-based analyses (i.e., both analyses provide a dimensional analysis of ADAD). PCA is distinct in comparison to other factor analytic methods but can fulfil a similar aim to factor analysis. For the sake of brevity, across the dimensional analysis (i.e., PCA and entropy-based analysis), we use the term factor (instead of component when referring to PCA results) from this point forward.

Construct validity: Principal component analysis (PCA)

For each of the four areas, a separate exploratory PCA with an oblique rotation was performed at the item level. All factors with an eigenvalue greater than one were examined and item loadings greater than .40 on a factor were retained on that factor.

Construct validity: Entropy-based analysis

For two areas, psychological and physical problems, an additional method was applied to test the construct validity. This method used entropy-based measures of associations and structural plots to identify functional associations between nominal items (Frank, 1999; Lorenc, 2003). The variables in these areas were mainly at the nominal level and entropy-based analysis is suitable for this type of data. By measuring the strength of the dependency between the variables, it is possible to see which variables measure similar constructs and which variables measure something that is different from the others. The association limit was set at .100. The entropy-based measure for association for nominal and ordinal data is presented in Appendix A.

Development of problem indexes

In the development of the indexes of physical health and psychological health problems, the results of both the PCA and entropy-based analyses were used. Indexes were only developed for methodologically stable dimensions, that is, dimensions that were identified both in the PCA and the corresponding entropy-based analysis. Items loading greater than .40 on a factor were evaluated for inclusion in the corresponding index only if the same item was not included in a different dimension in the entropy-based analysis. To improve the reliability of the indexes, only items with a corrected item-total correlation greater than .30 were kept in the final version of the indexes.

For criminality and alcohol and drug use/abuse, the indexes were based on the results of the PCAs. All items loading greater than .40 on a factor were considered for inclusion in a corresponding index, but only those items that had a correlation of equal to or greater than .30 with the rest of the items in the index were included in the final version.

A global index was also developed for each ADAD area. All items included in the dimensional analyses (PCAs and entropy-based analyses) of a particular area were evaluated for inclusion in the global index for that area. All items with corrected item-total correlations of equal to or greater than .30 were kept in the final version of the global index.

Results

Missing values

Missing values at the item level in each area ranged from 5% to 11%.

Physical health problems

In the physical health area, three of the factors had three or more items with loadings >.40. New factors were: Factor I, Eating and stomach problems which included four items of which three had a counterpart in the entropy-analysis. The internal consistency of an index with these items was low, Cronbach’s alpha = .56. The two other factors were Factor III, Frequent health problems and Factor VIII, Convulsions and fainting spells. No index was created for Factor III and VIII, because only a few corresponding items were found in the entropy analysis (See Table 1 and ESM Table A).

Based on these results, it was not possible to construct specific indexes with sufficient internal consistency in the physical health area, based on the PCA and entropy-based analysis. Based only on reliability analysis, a global scale for ADAD could be created in the physical health area. Fifteen of 26 items had item-total correlations > .30 and the resulting index had a standardised item alpha = .78. These items are identified in ESM Table A.

Psychological health problems

The PCA resulted in seven factors that had three or more items with loadings > .40, and four of these factors had corresponding components in the entropy-based analysis: Indexes were constructed from the items included in these four factors: Index I (Factor I), Lack of belonging; Index II (Factor II), Aggressive behavior; Index III (Factor III), Anxiety/depression; Index IV (Factor IV), Para-suicidal problems.

All four indexes had at least adequate Cronbach's alpha coefficients. Items that are included in the indexes with acceptable Cronbach's alpha coefficients (i.e., for this area .68 or higher alphas) are presented in Table 2.

Criminality

Two of four factors had loadings > .40 in three or more items: Factor I, Property and drug offences, and Factor II, Vandalism and assault (See Table 3). The index based on the first factor had a Cronbach’s alpha = .86 and the index based on the second factor had an alpha = .78. The global index (See ESM Table C), based on an item analysis of all items included in the dimensional analysis, included 16 items and had an alpha = .87.

Alcohol and drug use/abuse

Four factors had three or more items loading > .40: Factor I, Drugs that stimulate the central nervous system; Factor II, Alcohol use/abuse; Factor III, heroin and similar drugs; and Factor IV, Abuse of party drugs. The corresponding indexes had Cronbach’s alpha values of .72, .76, .63, and .76, respectively (See Table 4). A fifth index was arrived at through collapsing Factor I and III into an index of hard drug abuse (alpha = .75). The global index of alcohol use/abuse, the same as Factor II above, had an internal consistency of .76 (see ESM Table D). A global index of drug abuse based on an item analysis of all items included in the dimensional analysis was also constructed, consisting of 11 items with a Cronbach’s alpha value of .82.

Discussion

Standardised assessment instruments such as the ADAD are an important aspect of clinical practice. Reliance on these assessment instruments for clinical decision-making places demands on service providers to understand an assessment’s psychometric properties, including an instrument’s strengths and weaknesses. This study presents an analysis of the ADAD’s construct validity and internal consistency in four youth-reported areas: physical health, psychological health, legal/criminal, and alcohol and drug problems. Principal component analysis and entropy-based analyses, which provide tests of ADAD’s construct validity, uncovered a series of parsimonious dimensions and sub-dimensions within the identified areas.

From a practical standpoint, if the present study results are replicated in other similar study samples including more contemporary cohorts of Swedish adolescent girls in special residential care, it may be efficient to shorten ADAD measurement within certain selected areas where a reliable index can be identified. For example, one such area could be physical health, where a factor structure was not evident but the original 25 items could be reduced to a 15-item global scale that had good reliability.

Other study results showed a multidimensional factor structure with acceptable to good reliability in three ADAD areas of psychological health problems, criminality, and alcohol/drug use. For example, regarding psychological health problem items a four-factor structure was identified with factors related to lack of belonging (α = .82), aggressive behaviour (α = .68), para-suicidal problems (α = .80), and anxiety/depression (α = .82) (global psychological health area α = .92). Further results indicated a two-factor structure for items in the criminality area related to property and drug offences (α = .86), vandalism and assault (α = .78) (global criminality area (α = .87), as well as a four-factor structure for items in the alcohol and drug use area concerning use of drug stimulants (α = .72), alcohol use/abuse (α = .76), heroin and similar drugs (α = .63), and party drugs (α = .76) (global drug use/abuse area [α = .82] and Factors I and III possible to combine into a hard drug use scale).

As described in the introduction, direct comparisons regarding the results of prior published research studies that have addressed ADAD’s scale reliability and factor structure have been conducted with interview severity ratings or composite scores (e.g., Börjesson et al., 2007; Czobor et al., 2011) and the present study used only youth-reported scores on the ADAD. Given this important difference, the present study adds to what is known about ADAD’s psychometric strengths and limitations.

Overall, in terms of key strengths, these findings indicate that if the youth-report items are used, several conceptually meaningful multidimensional factors can be identified that have acceptable to good reliability. In terms of limitations, it appears that the physical health area items can use additional refinement. The indicators that were identified as having not acceptable or poor reliability, i.e., aggressive behaviour and heroin and similar drug use, had the lowest alpha values and also the shortest scales with four and three items, respectively (Table 2 and Table 4). These scales could potentially be improved by adding more items to these areas, as some indicators contain few items which may have had an impact on the internal consistency in these areas.

Moving from strengths and limitations of ADAD as used in this study to the limitations and contributions of the present study itself, several key study limitations should be noted. For example, the full range of all ADAD items (youth-reported) were not included in the study analyses and only particular areas were included. Other scores were not considered in this study such as composite scores or interviewer severity ratings. These could be viewed as limitations. However, there was an intention to focus on the perspective of the young person by the use of the youth-reported scores.

There are also several limitations regarding the generalisability of these findings. This study was conducted during a particular time period, and study results would only be relevant to Swedish adolescent girls in special residential care during that time. The replication and extension of the present study findings are needed with other more contemporary cohorts of girls in special residential care in Sweden. Because special residential care is rooted in a Nordic child welfare context, these results regarding the use of ADAD in the Swedish special residential care context are most relevant to Nordic countries and may not generalise to the use of ADAD in other child welfare and care contexts in other parts of the world.

This study also only included adolescent girls and not boys. Young people placed in compulsory care in Sweden are an especially vulnerable group. Within the institutional setting in Sweden, young women may be especially vulnerable as institutional care in Sweden was largely developed as a response to criminal behaviour among adolescent boys (Berit, 2000). Gender has historically been important to how problems are defined in young people and the responses society has toward these problems (Hamreby, 2004; Lundström & Sallnäs, 2003; Mattson, 2010). Understanding how gender may impact instrument validity is important for clinical care. Gender-specific interpretations may be inconsistent with the intent of scale developers, placing a greater burden on the assessor to understand how these interpretations may impact instrument validity. In addition, gender differences in an instrument’s construct validity may point to theoretical shortcomings in an instrument’s development. In this study, we found that the Swedish version of the ADAD may reliably distinguish between several important areas for intervention with Swedish youth in special residential care. However, further investigation critically comparing gender differences in relation to the use of this scale are needed.

Despite the noted limitations and need for further research studies on this topic, there are also several study contributions of note. As a main study conclusion, the study results provisionally indicated that ADAD (with youth-reported scores) may be able to distinguish between a series of areas (e.g., particularly psychological health, criminality, and alcohol/drug use abuse) that are likely to be important targets for intervention. Valid and reliable assessment procedures within special residential care in Sweden are necessary in order to lay the foundation for further treatment, as the problem configurations for this population appear to be complex and the long-term outcomes of youth in this type of care may be connected to initial problem profiles (Wångby-Lundh et al., 2018).

Thus, assessment challenges in Swedish special residential care are further complicated by the nuanced nature of the problems that young people in this type of treatment experience. As noted, the particular configurations of problems of youth in care are likely to be complex (Wångby-Lundh et al., 2018) and knowing the severity and array of these problems clearly matters as an essential guide for designing and gauging the likely effectiveness of treatment or other care (i.e., does the young person experience high levels of drug use and criminality and have few relational resources, or does the young person experience moderate levels of a diversity of problems such as alcohol abuse, school dropout, joblessness along with elevated internalising problems such as depression, but has relational resources to draw upon).

Indeed, the value of this particular ADAD scoring approach is exemplified by two subsequent studies with the present study sample (Wångby-Lundh, et al., 2018; Klingstedt et al., 2018), in which subscales found in this study were used to identify through an advanced cluster analysis approach (a person-oriented approach; Bergman & Magnusson, 1997; Magnusson, 1988) subgroups of participants with different problem configurations (subgroups of participants with different types, numbers, and severity of problems across different ADAD domains). It should be noted that the scales in these subsequent person-oriented analyses included but also went beyond the subscales identified in this study. Further, in our wider programme of research (Wångby-Lundh et al., 2018; Klingstedt et al., 2018), the identified problem configurations at intake into care predicted subsequent adjustment in this study sample. Thus, at the foundation of this line of inquiry are, in part, the present study’s results.

With regard to additional study contributions, it should also be reiterated that the alternative approach to scoring ADAD and scale development that was illustrated in this study, also highlights the young person’s view of her own problems and adjustment. More scoring alternatives for ADAD that are psychometrically sound and ecologically valid for a particular national care context are also of use to practitioners working in special residential care within Sweden. Having different scoring alternatives also allows for more flexible use of ADAD, for example, a shortened and targeted scale-based edition of ADAD could be used as an outcome evaluation index (short-term pre and post testing) in a targeted intervention/treatment that aims to improve discrete problems or combinations of problems. In special residential care in Sweden, the aim is to use assessment and the young person’s perspective (i.e., wishes and voice) to inform treatment and to provide an empirical anchor that serves as a benchmark to gain insight into changes in the young person over time, with the ultimate aim of moving the long-term life prospects of youth in care in a more positive and beneficial direction (Olsson & Fridell, 2018). The reliability and validity of assessment procedures are foundational to achieving this goal.

Supplemental material

Supplemental Material, ESM_Table_A - Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care

Supplemental Material, ESM_Table_A for Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care by Marie-Louise Klingstedt, Margit Wångby-Lundh, Tina Olsson and Laura Ferrer-Wreder in Nordic Studies on Alcohol and Drugs

Supplemental material

Supplemental Material, ESM_Table_B - Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care

Supplemental Material, ESM_Table_B for Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care by Marie-Louise Klingstedt, Margit Wångby-Lundh, Tina Olsson and Laura Ferrer-Wreder in Nordic Studies on Alcohol and Drugs

Supplemental material

Supplemental Material, ESM_Table_C - Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care

Supplemental Material, ESM_Table_C for Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care by Marie-Louise Klingstedt, Margit Wångby-Lundh, Tina Olsson and Laura Ferrer-Wreder in Nordic Studies on Alcohol and Drugs

Supplemental material

Supplemental Material, ESM_Table_D - Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care

Supplemental Material, ESM_Table_D for Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care by Marie-Louise Klingstedt, Margit Wångby-Lundh, Tina Olsson and Laura Ferrer-Wreder in Nordic Studies on Alcohol and Drugs

Supplemental material

Supplemental Material, ESM_Table_E - Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care

Supplemental Material, ESM_Table_E for Reliability and construct validity of five life domains in the adolescent drug abuse diagnosis instrument in a sample of Swedish adolescent girls in special residential care by Marie-Louise Klingstedt, Margit Wångby-Lundh, Tina Olsson and Laura Ferrer-Wreder in Nordic Studies on Alcohol and Drugs

Footnotes

Acknowledgement

We want to thank the National Board of Institutional Care for supplying data and funding this study. We are also very grateful to Associate Professor Ingegerd Jansson who took part in designing this study, and gave us invaluable support with the entropy-based statistical analyses.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This study received funding from the National Board of Institutional Care.

ORCID iD

Marie-Louise Klingstedt

Supplemental material

Supplemental material for this article is available online.

Appendix A

In this section, we present an entropy-based measure of association for nominal and ordinal data. The entropy of a nominal and ordinal variable X is defined as

where p_i is the probability that X will take the value i. The probabilities are estimated by the observed relative frequencies in the data. Note that $H (X) \geq 0$ , and will be equal to 0 if X has a one point-distribution, that is when X is a constant. The entropy will take its maximum value when X is uniformly distributed over all possible values.

Because the entropy depends only on the probabilities of different values on X, and not on the values themselves, $H (X)$ is suitable for nominal data. The entropy can be interpreted as a measure of dispersion, that is as a measure of the uncertainty in X.

Equivalently, the entropy of two variables X and Y is given by

where $p_{i j}$ is the probability that X will take the value i and Y will take the value j.

The association between X and Y is given by

where X is the independent variable and Y is the depending variable. The measure is interpreted as the relative reduction in uncertainty of Y by conditioning on X. This means that, if there is an association between X and Y, we already know something about Y by knowing the value of X. $A (X, Y)$ is a measure of the size of this knowledge.

The value of $A (X, Y)$ is between 0 and 1. $A (X, Y)$ takes the value 0 if there is no association between X and Y, that is X and Y are independent and we will gain no extra knowledge about Y by knowing the value of X. If $A (X, Y)$ = 1, there is a complete dependency between X and Y, and by knowing X we can completely determine the value of Y.

A is an asymmetric measure, can have that $A (X, Y)$ $\neq A (Y, X)$ . Association measured by entropy can be either linear or non-linear. The measure does not give a direction of the association. (Note that association including any nominal, non-binary, variable can not have a direction.)

A convenient computer program is available for using the method described here see Lorenc (2003).

References

Bergman

L. R.

Magnusson

(1997). A person-oriented approach in research on developmental psychopathology. Development and Psychopathology, 9, 291–319.

Berit

(2000) De besvärliga flickorna - ungdomshemmets disciplin ur ett genusperspektiv [The problematic girls - the discipline of the youth home from a gender perspective]. I Ingrid Sahlin & Malin Åkerström (Red.) Det lokala våldet [In The local violence]. Stockholm: Liber.

Bolognini

Plancherel

Laget

Chinet

Rossier

Cascone

… Halfon

. (2001). Evaluation of the Adolescent Drug Abuse Diagnosis instrument in a Swiss sample of drug abusers. Addiction, 96, 1477–1484.

Börjesson

Armelius

B.-Å.

Östgård-Ybrandt

(2007). The psychometric properties of the Swedish version of the Adolescent Drug Abuse Diagnosis (ADAD). Nordic Journal of Psychiatry, 61, 225–232.

Chinet

Plancherel

Bolognini

Holzer

Halfon

(2007). Adolescent Substance-use assessment: Methodological issues in the use of the ADAD (Adolescent Drug Abuse Diagnosis). Substance Use and Misuse, 40, 405–425.

Czobor

Bacskai

Öberg

Haack

M.-J.

Gerevich

(2011). The European Adolescent Assessment Dialogue (EuroADAD): A psychometric evaluation. European Addiction Research, 17, 302–315. doi:10.1159/000329989

Department of Justice Ministry of Health and Social Affairs. (2000). Omhändertagen - Samhällets ansvar för utsatta barn och unga [Taken care of: Society’s responsibility for vulnerable children and young people] (Official Reports of the Swedish Government 2000:77). Retrieved from https://www.regeringen.se/rattsliga-dokument/statens-offentligautredningar/2000/08/sou-200077/

Forkby

Höjer

(2011). Navigations between regulations and gut instinct: The unveiling of collective memory in decision-making processes where teenagers are placed in residential care. Child & Family Social Work, 16, 159–168. doi:10.1111/j.1365-2206.2010.00724.x

Frank

(1999). Entropy – a neglected quantity in elementary statistics. Stockholm, Sweden: Department of Statistics, Stockholm University.

10.

Friedman

A. S.

Utada

(1989). A method for diagnosing and planning the treatment of adolescent drug abusers (The Adolescent Drug Abuse Diagnosis [ADAD] Instrument). Journal of Drug Education, 19, 285–312.

11.

Friedman

A. S.

Utada

(1995). Composite scores for the ADAD (Adolescent Drug Abuse Diagnosis: A structured interview instrument). Formulas for developing composite scores of measuring change and treatment outcome (Unpublished manuscript). Philadelphia, PA: Belmont Center for Comprehensive Treatment.

12.

Janson

C-G

. (2004). Youth Justice in Sweden. In Tonry & Doob (Eds.), Youth Crime and Youth Justice: Comparative and Cross-National Perspectives (pp. 391–441). Chicago, IL, USA: The University of Chicago Press.

13.

Jansson

(1999). Problem i sammanfattande mått i ASI [Problems in composite scores in ASI] (Allmän SiS-rapport 2001:10). Stockholm, Sweden: National Board of Institutional Care.

14.

Kerstin

Hamreby, K

. (2004). Flickor och pojkar i den sociala barnavården. Föreställningar om kön och sociala problem under 1900-talet (Doktorsavhandling) [Girls and boys in Social Child Care, perceptions of gender and social problems during the 20th century (Doctoral dissertation)]. Umeå universitet, Institutionen för socialt arbete. Retrieved from http://umu.diva-portal.org/smash/get/diva2:143349/FULLTEXT01.pdf

15.

Klingstedt

M-L.

Olsson

Ferrer-Wreder

Wångby-Lundh

(2017). Relationship quality, well-being and, externalizing problems: The Prospective importance of behavior profiles among adolescent girls and young women who experienced care in special residential homes. J Nordic Psychology, 69, 1–30.

16.

Klingstedt

M.-L.

Olsson

Ferrer-Wreder

Wångby-Lundh

(2018). Relationship quality, well-being and, externalizing problems: The prospective importance of behavior profiles among young women who experienced care in special residential homes. Nordic Psychology, 70(1), 47–70. doi:10.1080/19012276.2017.1339626

17.

Lorenc

(2003). EMA, a program for calculation of entropy-based measures of association. Stockholm, Sweden: Department of Statistics, Stockholm University.

18.

Lundström

Sallnäs

(2003). Klass, kön och etnicitet i den sociala barnavården. [Class, gender and ethnicity in Social Child Care] Socialvetenskaplig tidskrift, 10, 193–213.

19.

Magnusson

(1988). Individual development from an interactional perspective: A longitudinal study. Hillsdale, NJ: Lawrence Erlbaum.

20.

Matsson

(2010). Intersektionalitet i socialt arbete. Teori, reflektion och praxis. [Intersectionality in social work. Theory, reflection and practice]. Malmö: Gleerups Utbildning AB.

21.

McLellan

A. T.

Kushner

Metzger

Peters

Smith

Grissom

… Argeriou

. (1992). The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment, 9, 199–213.

22.

National Board for Institutional Care. (2010a). Årsredovisning 2010 [Annual Report 2010]. Stockholm, Sweden: National Board for Institutional Care.

23.

National Board for Institutional Care. (2010b). ADAD – Adolescent Drug Abuse Diagnosis manual. Stockholm, Sweden: National Board for Institutional Care.

24.

National Board of Health and Welfare. (2015). Utreda barn och unga. Handbok för socialtjänstens arbete enligt socialtjänstlagen [Investigate children and young people. Handbook for social work according to the Social Services Act]. Stockholm, Sweden: National Board of Health and Welfare. Retrieved from http://www.socialstyrelsen.se/Lists/Artikelkatalog/Attachments/19644/2015-1-9.pdf

25.

Olsson

T. M.

Fridell

(2018). The five-year costs and benefits of extended psychological and psychiatric assessment versus standard intake interview for women with comorbid substance use disorders treated in compulsory care in Sweden. BMC Health Services Research, 18, 1–13.

26.

Ring

(1999). Hem och skola, kamrater och brott [Home and school, friends and crimes] (Unpublished doctoral dissertation). Department of Criminology, Stockholm University.

27.

Socialdepartementet. (2000). Omhändertagen - Samhällets ansvar för utsatta barn och ung (Statens offentliga utredningar, 2000:77) [Taken care of – Society’s responsibility for vulnerable children and young people. Official Reports of the Swedish Government, 2000: 77]. Retrieved from https://www.riksdagen.se/sv/dokument-lagar/dokument/statens-offentliga-utredningar/sou-2000-77-d2_GOB377d2/html

28.

SFS 1990:52 Swedish Codes of Statutes. (1990). Care of Young Persons (special provisions) Act Department of Justice.

29.

SFS 1998:603 Swedish Codes of Statutes. (1998). Care of Young Offenders Act Department of Justice.

30.

SFS 2001:453 Swedish Codes of Statutes. (2001). The Social Services Act.

31.

Söderholm

Carpelan, K.

Hermodsson

(2004). ADAD och utvecklingen av ett dokumentationssystem [Swedish ADAD, the development of a documentation system for Institutional Care]. Nordisk Sosialt Arbeid, 24, 110–123.

32.

Söderholm

Carpelan, K.

Hermodsson

Oberg

(1997). Manual ADAD-intervju. [Swedish ADAD Interview - Manual]. Stockholm, Sweden: National Board for Institutional Care.

33.

Wångby-Lundh

Klingstedt

M-L.

Bergman

L.R.

Ferrer-Wreder

(2018). Swedish adolescent girls in special residential treatment: A person-oriented approach to the identification of problem syndromes. Nordic Psychology, 70(1), 17–46. doi:10.1080/19012276.2017.1323663

34.

Ybrandt

Börjesson

Armelius

B.-Å.

(2008). The Adolescent Drug Abuse Diagnosis composite scores in Swedish normal and antisocial adolescents. Substance Use & Misuse, 43, 1411–1423. doi:10.1080/10826080801925465

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.02 MB

0.01 MB