Abstract
Objective
This study evaluated the effectiveness of academic screening measures in relation to parent-reported diagnoses.
Design
Multicenter, retrospective cohort study including structured interviews, questionnaires, and chart reviews.
Setting
Six North American cleft centers.
Participants
Children (n = 391) with cleft lip and/or palate, ages 8 to 10 years of age (192 male) and their guardians were recruited during regular clinic visits.
Main Outcome Measures
Parent and child ratings on the Pediatric Quality of Life Inventory (PedsQL) School Scale, child report on CleftQ School Scale, parent report on the Adaptive Behavior Assessment System-Third Edition Functional Academics (ABAS-FA) Scale and Child Behavior Checklist (CBCL) School Competency Scale, parent interview, and medical chart review.
Results
Risk for concerns ranged from 12% to 41%, with higher risk reflected on the CBCL-SC compared to other measures. Males with cleft palate were consistently at the highest risk. Only 9% of the sample had a parent-reported diagnosis of a learning or language disability. Ratings from the ABAS-FA and CBCL-SC had the highest utility in identifying those with language and/or learning concerns.
Conclusions
As cleft teams work to develop standardized batteries for screening and monitoring of patients, it is important to evaluate the effectiveness of measures in identifying those at highest risk. When screening for language and learning disorders, questions related to potential academic struggles, such as increased school effort or increased school distress, are most useful. Referrals for follow-up evaluation are recommended for those identified at high risk.
Introduction
The diagnosis of a cleft of the lip and/or palate (CL/P) requires multidisciplinary collaboration for the diverse needs of patients across their life span. Experts in surgery, dentistry, orthodontics, otolaryngology, nursing, genetics, speech-language pathology, psychology, and other fields are often brought together to assist with care and management. Internationally, cleft teams have called for a more standardized approach to screening a variety of clinically relevant areas of functioning. 1 Regional working groups (eg, EuroCleft, 2 AmeriCleft, 3 ScandCleft, 4 and the Clinical Standards Advisory Group [CSAG] 5 ) started the process of developing standardized metrics within disciplines. Recently, larger collaborative efforts have been made to create a standard collection of measures to be given at specific scheduled time points for purposes of assessing etiologies, treatments, outcomes, and impacts of cleft across disciplines (eg, the International Consortium for Health Outcomes Measurement [ICHOM] 6 and the Cleft Collective, 7 a UK-based longitudinal cohort study). Such batteries would minimize the burden on families and enhance cross-institutional collaboration.
These projects take a multidisciplinary and developmental approach, with specific metrics measured in infancy, toddlerhood/preschool, school-aged, adolescence, and adulthood. Key domains of focus shared across these projects include dental and oral health, speech/communication, appearance, psychosocial development, and cognition/vocational milestones. The focus of the current paper is on the measurement of cognition/vocational milestones and how this translates into screening for academic risk and possible learning disabilities.
Decades of research have documented a higher rate of learning concerns in children with CL/P, even among those without an associated syndrome.8,9 This increased risk has been identified through case/control10–12 and population-based studies.13–15 Research has focused heavily on language and reading giventhe nature of the cleft and its impact on both speech and hearing. Although results are mixed depending on the method of measurement, inclusion/exclusion criteria, and comparison groups, most have found an increased risk for language-based learning disorders (eg, dyslexia11,16), with the highest risk among boys with palatal involvement.17,18 Studies evaluating neuropsychological patterns related to academic underachievement have found impairments in reading to be associated with deficits in phonological awareness 19 and auditory working memory.11,12
Early identification and intervention for language and learning disorders is associated with the best outcomes for students. 20 In their 2011 review, Rimrodt and Lipkin 21 expressed the importance of early screening to identify increased learning effort (eg, acting out in class, taking longer to complete work, demonstrating resistance or reluctance to do school work) and school distress (eg, frequent failures, punishments for misbehavior, disengagement) among patients, before they result in school failure (eg, retention, expulsion, dropping out). Using a medical model, recommendations for screening, obtaining relevant histories, and referral for further diagnostic testing were made.
Given that cleft care teams provide care for children from birth through adulthood, screening for cognitive issues has become a recognized service that approved clinics are expected to provide. 22 However, clinic visits are lengthy with multiple specialists evaluating the child's functioning across many areas. Despite the need for appropriate academic screening, current suggestions for standard protocols fall short; with an understandable emphasis on psychosocial concerns that unfortunately omit questions related specifically to academic struggles. ICHOM 6 utilizes subscales from the self-report CLEFT-Q, 23 including the School subscale that asks questions related to general psychosocial functioning within the school setting. Similarly, up to 5 years of age, the Cleft Collective resource 24 includes the parent-report Ages and Stages Questionnaire, 3rd Edition 25 (ASQ-3) and ASQ Social Emotional 26 (SE) questionnaires. After the age of 5 years, the parent-report Strength and Difficulties Questionnaire 27 (SDQ), and self-report Harter Self-Perception Profile28,29 (HSPP) are administered. Parents also provide responses to the Pediatric Quality of Life (PedsQL) Healthcare Satisfaction 30 and Family Impact 31 modules, but these are specific to perceptions of healthcare and parent/family functioning rather than child-specific functioning. The ASQ-3 measures general cognition through 5.5 years of age; the SDQ measures behavioral/emotional functioning. The HSPP looks at student's self-perception across different areas, including academics; this is the only measure thus far that provides information on school distress, one of the 2 screening areas recommended by Rimrodt and Lipkin. 21
The Americleft Psychosocial Outcomes Project 32 was established around the same time as ICHOM and the Cleft Collective. As the primary aim of this project was limited to psychosocial and cognitive outcomes rather than being multi-disciplinary, there was room to expand on some of the measures given. Although there is some overlap with ICHOM (ie, CLEFT-Q), additional measures related to academic functioning (ie, Adaptive Behavior Assessment System, 3rd Edition 33 [ABAS-3], the Child Behavior Checklist 34 [CBCL], and PedsQL Inventory 35 ) were also completed, along with the parental report of diagnosed psychosocial, behavioral, and language-learning disabilities. The ABAS-3 is an adaptive, parent report questionnaire and measures the general ability to use reading, writing, and math in daily life. The CBCL measures internalizing and externalizing symptoms, but also includes an Academic Competence scale that evaluates school attendance, performance across subject areas, use of special services, and identified disabilities. The PedsQL Inventory includes the School Functioning subscale, which obtains both parent and child-response related to issues with focus and school absences (see Table 1).
Common Screening Measures and Constructs They Assess.
Abbreviations: PedsQL School, Pediatric Quality of Life Inventory, School; ABAS-FA, Adaptive Behavior Assessment System, 3rd Edition, Functional Academics; CBCL-SC, Child Behavior Checklist, Academic Competence; ASQ-3, Ages and Stages Questionnaire, 3rd Edition; SDQ, Strengths and Difficulties Questionnaire; HSPP, Harter Self-Perception Profile.
The ASQ-3, SDQ, and HSPP were not included in the present study, but are included here for reference.
The purpose of the current study was to leverage data from the Americleft Psychosocial Outcomes Project to compare common measures included in screening batteries for school-aged patients with CL/P. Specific research aims were to (1) identify patterns of academic risk among children with CL/P across the different measures and (2) evaluate the effectiveness of each screening measure in identifying potential academic issues based on parent report of diagnoses. Based on a review of items, it was hypothesized that the CleftQ-School and PedsQL-School scales were more measures of psychosocial functioning within the school setting and would not be predictive of academic risk. However, the CBCL-School and ABAS-3 Functional Academic measures incorporated aspects of both increased learning effort and school distress, so would be more predictive of academic risk. Understanding the level of academic risk and utility of different screening measures will be beneficial for cleft teams working to improve screening and referral of patients, ensuring the earliest identification and intervention for potential issues.
Materials and Methods
Recruitment and Assessment
Children (ages 8-10 years old) with oral clefts and their families were approached and invited to participate by trained research staff during a visit to one of 6 cleft centers in North America (Shriners Hospitals for Children, Chicago; Nationwide Children's Hospital, Columbus; the Hospital for Sick Children, Toronto; the University of Iowa, Iowa City; Lancaster Cleft Palate Clinic, Lancaster; and Children's Medical Center, Dallas). Exclusion criteria were: (1) complex craniofacial syndromes; (2) other comorbid medical issues; and (3) children and parents who spoke languages other than English or Spanish. Children with Pierre Robin sequence who did not require a tracheostomy and those with Van der Woude syndrome were eligible to participate. Parents/Guardians provided informed consent and children provided written or verbal assent. Questionnaires were completed via paper and pencil or online REDCap administration (using a tablet) during the clinic visit. If questionnaires were not completed before the end of the clinic visit, postage-paid envelopes were provided for paper submission or a REDCap link was emailed to the parent/guardian for online submission. Children were compensated with a small gift (value of $5) for their participation. Data collection occurred from 2016 to 2019. All procedures were approved by each site's Institutional Review Board. 32
Measures
The Americleft protocol included parent/guardian report and self-report questionnaires designed to obtain information on medical/treatment history, psychosocial functioning, and academic functioning. To limit the influence of repeated measures, when 2 parents responded to questionnaires, only the first rating was used in analyses. The ratings used were predominately completed by mothers (n = 322; 82%). The following measures from the protocol were included in the current analysis.
Pediatric quality of life inventory generic core scales (PedsQL)
The PedsQL 35 is a widely used measure of health-related quality of life that asks parents and children to report the degree to which the child experiences issues in functioning across physical, emotional, social, and school domains. The School subscale (5 items; approximately 1 min to complete) specifically evaluates a child's ability to focus and keep up with schoolwork as well as amount of school missed due to illness or doctor visits. Parent (PR) and child report (CR) for the PedsQL-School scores were transposed to Z-scores using normative data 36 ; higher values indicate better functioning.
CLEFT-Q
The CLEFT-Q 23 is a self-report measure developed for use in persons with CL/P ages 8 to 29 years. The School subscale (10 items; approximately 2 min to complete) was utilized for the current study. It evaluates if a child has friends, feels accepted by peers and teachers, participates in activities, and feels happy and safe at school. CLEFT-Q School subscores were transposed to Z-scores using normative data, 37 where higher values indicate better functioning.
Adaptive behavior assessment system, 3rd edition (ABAS-III)
The ABAS-III 33 is a measure of child functioning based on parent report. Items from the Functional Academics subscale (ABAS-FA; 23 items, approximately 9 min to complete) were used to obtain parents’ ratings of the frequency of learning challenges for their child. Questions focus on a child's ability to use basic reading, writing, and math skills in daily life. Standard scores are calculated with a normative mean of 10 and a standard deviation of 3; higher values indicate better functioning. To ease comparison across measures, ABAS-FA standard scores were transposed to Z-scores.
Child behavior checklist (CBCL)
The CBCL 34 is a parent report measure of children's competencies and psychological adjustment. The competency scales focus on school (attendance, special services, problems, disabilities), social (number of peer friendships, frequency of visits, quality of interactions), and activities (number of sports, activities, chores, and level of performance) functioning. The psychosocial adjustment portion assesses internalizing and externalizing problems, attention problems, and thought problems. The current study used the School Competency T-Score (CBCL-SC; 4 items, approximately 2 min to complete). The normative mean is 50 with a standard deviation of 10; higher values indicate better functioning. Again, T-Scores were transposed to Z-scores for comparison across measures.
Medical history and chart review
Parents completed a structured interview to identify their child's treatments received since birth including behavioral and learning diagnoses, speech therapy, other developmental therapies, educational services/interventions, and family socioeconomic status (SES) using a Hollingshead scale 38 (where higher values indicate higher SES). Self-reported diagnoses (ie, learning/language disorders, attention disorders, behavioral disorders, mood disorders, and autism spectrum disorder [ASD]) were the variables of interest.
Analyses
Participants (a-priori descriptive statistics)
Univariate analysis was used to evaluate potential group differences in age and SES. When homogeneity of variance was violated, Welch Statistic was utilized. Pearson χ2 was used to evaluate group differences on categorical variables (ie, cleft type, sex, and adoption status). Main effects were considered significant at α < .05, and interactions were considered significant at α < .10. Partial eta squared was calculated to estimate effect size with the following interpretations: small effect = .01, medium effect = .06, and large effect >.13. 39 All analyses should be interpreted in light of uneven group sizes (particularly over-representation of participants with CLP vs those with CL and CP).
Patterns of risk across each measure
First, univariate ANOVAs were run to evaluate the main and interaction effects of sex and cleft type on each screening measure (ie, CLEFT-Q School, PedsQL-School, ABAS-FA, and CBCL-SC). Next, for the CLEFT-Q School, ABAS-FA, and CBCL-School, scores were coded as “not at risk” if the standard score was above the 16th Percentile and as “at risk” if the standard score was at or below the 16th Percentile. For the PedsQL-School, the risk was defined based on published cutoff scores. 36 Pearson χ2 analysis was utilized to evaluate the main effects of sex and cleft type on the rate of participants in the at-risk range.
Effectiveness of identifying academic risk
Incidence of parent-reported diagnoses from clinical interview were tabulated for learning and language disorders (ie, expressive/receptive language disorder and specific learning disabilities in reading [dyslexia], expressive writing [dysgraphia], and mathematics [dyscalculia]), attention deficit/hyperactivity disorder (ADHD), behavioral disorders (ie, conduct disorder and disruptive behavior disorder), mood disorders (ie, anxiety and depression), and ASD. Pearson χ2 analysis was utilized to evaluate the main and interaction effects of sex and cleft type on incidence. Finally, estimates of sensitivity (true positive rate), specificity (true negative rate), and predictive values were obtained to evaluate the potential clinical utility of each screening measure.
Results
Participants (A-priori Descriptive Statistics)
Nine hundred and ninety-three children (ages 8-10 years old) with oral clefts and their families were screened for participation. Of those screened, 228 were not eligible for participation: 66% had another comorbid medical issue, 28% had a craniofacial syndrome, and 6% spoke a language other than English or Spanish. Of those eligible for participation, 414 (76% of eligible) were enrolled. Reasons for declining the study included: time constraints (43%), active decline (28.7%), missed appointment (14.9%), and other (13.1%). Of those enrolled, a total of 311 had complete data and 391 had at least partial data (CLEFT-Q School n = 324, PedsQL School CR n = 341, PedsQL School PR n = 384, ABAS-FA n = 387, and CBCL-SC n = 380). Participant inclusion across the 6 centers was as follows: Shriners (n = 160), Nationwide (n = 118), Hospital for Sick Children (n = 38), Iowa (n = 30), Lancaster (n = 25), and Dallas (n = 20).
Of children included in this study, 192 (49%) were male and 260 (66%) had cleft of the lip and palate (CLP). There were 72 (18%) children with cleft palate (CP) and 59 (15%) were born with cleft lip (CL). Participants were more likely to be female (χ2 (1) = 10.89, P < .001). There were no sex (F(1, 385) = 0.98, P = .322; ƞ2 = .003) or cleft type (F(2, 385) = 0.71, P = .490; ƞ2 < .01) differences in child age. There were no sex (F(1, 372) = 2.32, P = .129; ƞ2 < .01) or cleft type differences (F(2, 372) = 0.55, P = .577; ƞ2 < .01) for SES. The majority of participants were Caucasian (56.3%); 28.4% reported as Asian, 9.9% as Hispanic/Latinx, 1.3% as African American, 1.3% as Native American, and 2.8% as multiracial. One hundred and eleven participants (28%) were adopted (see Table 2). As there were no significant differences on outcome measures for those who were versus were not adopted, all participants were analyzed together. Distribution across grade levels was as follows: first = 7, second = 91, third = 125, fourth = 120, fifth = 46, and sixth = 2.
Participant Demographics.
Abbreviation: SES, socioeconomic status; higher values reflect higher SES.
Patterns of Risk Across Each Measure
Responses on the PedsQL School CR, ABAS-FA, and CBCL-SC were moderately and negatively skewed (skewness (SE) = −0.69 (0.13), −0.57 (0.12), and −0.64 (0.13), respectively), with a higher proportion of responses reflecting better functioning. Responses on the CLEFT-Q School and PedsQL School PR were approximately symmetric (skewness (SE) = −0.49 (0.14), −0.46 (0.13), respectively). Scores on the CLEFT-Q School and ABAS-FA scale were predominately in the average to the high average range, scores on both the child and parent report of the PedsQL-School were predominately within the average range, and scores on the CBCL-SC were in the low average to average range (see Figure 1).
Mean (95% ci) z-score for screening measures by sex and cleft type. Abbreviations: CBCL-SC, Child Behavior Checklist, Academic Competence; PedsQL School, Pediatric Quality of Life Inventory, School; ABAS-FA, Adaptive Behavior Assessment System, 3rd Edition, Functional Academics; CR, child report; PR, parent report.
On the CLEFT-Q School, there were no significant sex or cleft type main effects or interactions. For the PedsQL-School, there was a significant main effect of sex where females scored higher than males on both child and parent report (mean difference (standard error) = 0.30 (0.13); F(1, 335) = 5.58, P = .019; ƞ2 = .02 and mean difference (standard error) = 0.28 (0.12); F(1, 378) = 5.81, P = .016; ƞ2 = .02, respectively). For the parent report, there was also a significant main effect of cleft type (F(2, 378) = 4.46, P = .012; ƞ2 = .02), where participants with CP scored significantly lower than those with CL (mean difference (standard error) = −0.49 (0.17), P = .009). This effect did not reach significance for child report, though the same pattern was present (F(2, 335) = 2.55, P = .008; ƞ2 = .02; CP versus CL = −0.41 (1.81), P = .074). On the ABAS-FA, there was a significant sex-by-cleft type interaction (F(2, 381) = 2.54, P = .080; ƞ2 = .01). Males had lower scores than females but this difference was only significant for participants with CP (F(1, 69) = 8.07, P = .006; ƞ2 = .11). For the CBCL-SC, there were no significant main or interaction effects of sex or cleft type (Figure 1 and Table 3).
Z-Scores and Percentage Coded as “At Risk” Across Sex and Cleft Type. a
F-Statistics are presented for univariate ANOVAs and Chi-square Statistics are presented for categorical risk data.
P < .05.
P < .001.
P < .10 (only noted for interactions).
When scores were coded based on risk, total risk was the lowest for the ABAS-FA = 12%, and CLEFT-Q School = 17%. Total risk was higher on the CBCL-SC = 29%, PedsQL-School parent report = 39%, and PedsQL-School child report = 41%. Chi-squares were run to assess potential sex and cleft type main effects for each measure. There was a significant main effect of sex for the PedsQL-School (child and parent report; χ2 (1) = 4.44, P = .035 and χ2 (1) = 6.37, P = .012, respectively) and the CBCL-SC (χ2 (1) = 4.00, P = .046), where risk was higher for males compared to females. There was also a significant main effect of cleft type for PedsQL-School Parent Report, where those with CP had the highest risk compared to those with CL (χ2 (2) = 9.89, P = .007). There were no significant main effects for the CLEFT-Q School and the ABAS-FA (see Table 3).
Effectiveness of Identifying Academic Risk
Incidences of diagnoses based on parent report are provided in Table 4. A formal diagnosis of either a learning or language disorder was reported in 9% of the full sample. There were no significant sex (χ2 (1) = 0.27, P = .607) or cleft type (χ2 (2) = 1.17, P = .558) differences in rates of learning or language diagnoses. Among learning/language diagnoses, expressive and receptive language disorders were diagnosed in 5% of participants, followed by reading (4%), writing (3%), and math (2%) disorders. Among other diagnoses, attention deficits were the most frequently reported (16%); males were nearly 3 times more likely to have a diagnosis than females (χ2 (1) = 19.12, P < .001) which is slightly higher than differences in the general population. 40 Behavioral and mood disorders were reported in 3% of the participants and ASD was reported in 1% of participants.
Percent of Parent-Reported Diagnoses.
Abbreviation: LD, learning disorder.
Estimates of performance (ie, sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) for each measure in identifying participants with a learning and/or language disorder diagnosis are presented in Figure 2.
Calculation of sensitivity, specificity, and predictive values. Abbreviations: CBCL-SC, Child Behavior Checklist, Academic Competence; PedsQL School, Pediatric Quality of Life Inventory, School; ABAS-FA, Adaptive Behavior Assessment System, 3rd Edition, Functional Academics; CR, child report; PR, parent report.
When evaluating the effectiveness of screening measures, PPV (percentage of those deemed “at risk” who were true positives) and NPV (percentage of those deemed “not at risk” were true negatives) are the key variables of importance. 41 As hypothesized, both the ABAS-FA and CBCL-SC had higher PPV (30.4% and 16.5%, respectively) and NPV (94.3% and 95.1%, respectively) compared to the CLEFT-Q School and PedsQL-School child and parent report (PPV = 3.6%, 7.2%, and 10.8%, respectively; NPV = 91.4%, 92.0%, and 92.1%, respectively). This indicates that the ABAS-FA and CBCL-SC had higher accuracy in identifying those who truly were “at risk” and “not at risk.”
Sensitivity (ie, percentage of those who were true positives out of those who have a diagnosis) and specificity (ie, percentage of those who were true negatives out of those who do not have a diagnosis) are also helpful metrics to consider. The CBCL-SC was the only measure that had both sensitivity and specificity over 50% (58.1% and 73.4%, respectively). Although the ABAS-FA had the highest specificity (90.8%), less than half of those who had a diagnosis were identified as “at risk” (sensitivity = 42.4%).
Post Hoc Analyses
A review of the PedsQL-School and CBCL raised questions about the potential utility of specific items. For the PedsQL-School, 3 items could be related to increased school effort (ie, paying attention in class, forgetting things, and keeping up with school work) and the other items are related to attendance. To determine whether responses on the first 3 items may be more sensitive to potential learning concerns, mean responses were calculated for the first 3 items (PedsQL-School Effort) and the last 2 items (PedsQL-School Attendance) where lower scores reflected less concern. Since cut-points for items are not available, 2 repeated measures ANOVAs (one for parent report and one for child report) were run with the diagnosis of a language/learning disability as the between-subjects independent variable and scores on the PedsQL-School Effort versus Attention as the within-subjects variable. For parent-report, there was a significant diagnosis*item interaction (F(1, 373) = 6.66, P = .010; ƞ2 = .02), where responses on PedsQL-School Effort were rated with more problems than items on the PedsQL-School Attendance, and the difference was more profound for those with a language/learning disability. Neither the diagnosis*item interaction (F(1, 330) = 1.29, P = .257; ƞ2 < .01) nor the main within-subject effect between items (F(1, 330) < 0.01, P = .963; ƞ2 < .01) were significant for child-report.
The CBCL-SC is composed of items that measure performance across classes, receipt of services, grade retention, and general problems in school. Clinically, it is of importance to professionals when issues are occurring (ie, poor grades, retention, and problems in school), but receipt of services is not endorsed. For the current sample, of those with a diagnosed language/learning disorder (n = 34), 38% were not reported to be currently receiving services.
Discussion
Internationally, efforts are being made to develop a consensus for psychological screening and evaluation of children with CL/P. Given the importance of academic screening, the current paper sought to provide information on academic risk and evaluate the utility of different screening measures in the identification of risk for academic achievement.
Ratings across 4 of the measures evaluated (CLFT-Q School, PedsQL-School child and parent report, and ABAS-FA) were all at or above the average range. In contrast, ratings on the CBCL-SC were in the low average to average range. These differences are likely related to what each measure is evaluating. The CLFT-Q School and PedsQL-School measure adjustment and functioning within a school setting. The ABAS-FA measures the general ability to use reading, writing, math, and organizational skills within daily life. Whereas, items on the CBCL-SC flag potential issues with low school performance and need for academic services. The findings presented here suggest that, in general, students with CL/P are not experiencing significant psychosocial issues in the school setting and are able to use academic skills in daily life. However, they demonstrate higher risk in regard to low academic achievement and higher need for services. The differences across these measures reflect the importance of what questions are being asked to ensure that concerns are adequately being evaluated.
Although ratings were at or above the average range for most children with CL/P, males were at the highest risk for issues. In fact, males with CP were at the highest risk across all measures (ranging from 29% to 82% in the “at risk” group). Even the measure that yielded the lowest risk (CLEFT-Q School) is more than double the expected percentage of risk found within the general population. 40 This supports previous research that males with CP are at a higher risk for language and learning concerns.17,18
Using parental report as a marker, only 9% had a known language or learning diagnosis. However, without direct assessment of academic achievement, the accuracy of parent-reported language or learning diagnoses cannot be determined. Previous work evaluating reading among males with CL/P found that while only 4% of the sample had a diagnosis of dyslexia, 37% had reading performance that fell within the impaired range. 11 Additionally, more subtle learning disabilities may not yet have been identified for participants in early elementary school. As stated earlier, while screening of academic issues is required for ACPA-approved cleft clinics, the method in which this screening is achieved is not standardized. Some centers may ask brief questions regarding school performance, where others have on-site psychologists who conduct more formal screening and assessment. Ensuring that screening efforts are adequately identifying those at greatest risk and in need of follow-up assessment is a critical need.
Analysis of screening measures is a complex matter, with consideration needed for not only predictive values, sensitivity, and specificity but also to prevalence, the cost of screening and diagnostics, as well as the impact of false positives and false negatives. For language and learning disorders, there is evidence of higher prevalence (up to 40%-60% for subgroups) and neuropsychological assessment required for diagnostics is costly. Although the monetary cost related to a possible false positive is high, the long-term impact of a false negative is immeasurable for a student. This means that maximizing true positives (higher PPV) and true negatives (higher NPV) is of utmost importance.
In comparing the screening measures utilized in this study, the ABAS-FA and CBCL-SC had the highest PPV and NPV. Both were negatively skewed to a significant degree. For the ABAS-FA, this resulted in 50% of participants falling at or above the 84th Percentile (above average), reflecting a possible ceiling effect for this measure. On the CBCL-SC, only 14% of participants were scored at the ceiling level. Finally, while specificity was higher for the ABAS-FA (high ability to identify those who were not reported to have a language/learning diagnosis), sensitivity was higher for the CBCL-SC (moderate ability to identify those who had a reported learning/language disorder).
As noted by Rimrodt and Lipkin, 21 screening for learning disabilities must be able to identify increased learning effort and school distress among patients, ideally before they result in school failure. Routine academic screening by the cleft team can be achieved by asking these key questions to explore whether there are any issues with school, peers, performance, or homework completion. A clinical review of individual items from the CBCL-SC could be useful in determining the need for referrals, particularly if a student's score is indicative of school performance concerns, repeating a grade, or school problems, but is not yet receiving special services. However, as demonstrated by the current study, asking these questions alone is not sufficient in identifying students who may require intervention. Additional screening for specific learning disabilities may be needed. For example, researchers in Sweden have been testing a Short Dyslexia Scale that requires parent response to seven screening questions. They reported that 22% of participants with CL/P fell below the 10th percentile. 42 In clinical practice, those with concerning screening results could then be used to trigger a referral to a team or local psychologist for a neuropsychological evaluation or collaboration with the child's school to advocate for further evaluation and/or service provision.
Despite the large, multicenter nature of this study design, the findings reported have limitations. The generalizability of the findings is limited by the lack of racial and ethnic diversity within the sample. Our sample was not powered to evaluate the potential influence of key socio-demographic variables (eg, socioeconomic status, adoption status) that are associated with the incidence of language and learning concerns as well as access to appropriate diagnostic and intervention services. Additionally, parent-reported diagnoses were used as an indicator for learning/language disability rather than medically confirmed diagnoses or actual measures of skill/performance. It is possible that parents misreported diagnoses or that children who truly had a learning/language disability had not been identified or diagnosed. Future prospective work should follow children from identification and referral through treatment/intervention to assess factors of timing and type of intervention on outcomes. Information on the effectiveness of different measures in the identification of academic risk helps guide cleft teams in determining what screening methods to use and how to best address their patient's needs.
Conclusion
As cleft teams work to develop standardized batteries for screening and monitoring of patients throughout life, it is of importance that work be done to assess the success of those measures in identifying those at the highest risk. For the screening of language and learning disorders, this means ensuring that questions related to potential academic struggles, such as increased school effort or increased school distress, are asked and referrals for follow-up evaluation are made for those identified at high risk. By following patients and tracking the results of follow-up diagnostics, cleft teams will be able to best ensure that our efforts are truly meeting the needs of our patients.
Footnotes
Acknowledgments
The authors would like to extend our sincerest gratitude to the families who gave their time to participate in this study. This work was supported by a grant through the Cleft Palate Foundation (CPF); a training grant through the National Institute of Dental and Craniofacial Research (grant number K23DE024511); and the National Center for Advancing Translational Sciences of the National Institutes of Health (grant number UL1TR002537).
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Center for Advancing Translational Sciences, National Institute of Dental and Craniofacial Research, the Cleft Palate Foundation, (grant number UL1TR002537, K23DE024511).
