Validation of an Ultrashort Screening Tool for Specific Learning Disorder Among School-Going Children.

Abstract

Background:

Specific learning disorder (SLD) is a neurodevelopmental disorder characterized by impairment in reading, written expression, and arithmetic. It can be identified during early school days itself. There is a lack of a screening tool for SLD that is properly validated and easy to administer. This study is an attempt to validate an ultrashort tool for screening SLD.

Methods:

This cross-sectional study was done at Government Medical College Ernakulam, South India. Children aged 6–16 years attending Psychiatry Outpatient Department were recruited by consecutive sampling. The self-administered screening tool, known as Ernakulam Learning Disorder (ELD) tool, was given twice within a gap of 2 months, and the results were compared with the clinical diagnosis, which was taken as the gold standard. The sensitivity, specificity, positive predictive value, negative predictive value, and kappa of the screening tool were calculated.

Results:

The sample consisted of children aged 8–16 years. The sensitivity and specificity of the tool were found to be 98.2 (95% confidence interval [CI] = 90.4–100) and 87.5 (CI = 76.8–94.4), respectively. The positive predictive value was 87.3 (CI = 76.5–94.4) and the negative predictive value was 98.2 (CI = 90.6–100). The kappa value was 0.96 (CI = 0.92–1.01).

Conclusions:

ELD tool seems highly useful as a screening instrument for detecting SLD; it is simple, easy to remember, and easy to administer.

Key Messages:

There is sufficient evidence of validity for this tool to be used for screening SLD routinely in all school-going children.

Specific learning disorder (SLD) is a neurodevelopmental disorder characterized by impairment in reading, written expression, and arithmetic. ¹ It can occur alone or in combination. Those with SLD usually have a normal intelligence although they exhibit academic difficulties. ¹ The prevalence of SLD varies from 5% to 15%. ¹ A recent study conducted in Kerala, Ernakulam Learning Disorder (ELD) study, showed a prevalence of 16.49%. ² Since SLD is a neurodevelopmental disorder and has onset during the early ages of life, it can be identified during the primary classes itself.^1,3 Lack of timely identification of the disorder and inadequate training lead to poor academic performance and several behavioral problems in the child. ³ Although a comprehensive assessment is the ideal method for diagnosing SLD, the high prevalence necessitates an accurate but brief screening tool for the same. Although several tools are available for screening SLD, most of them are not properly validated and have multiple questions for each domain of SLD, making them lengthy and time consuming to administer. Single-item or ultrashort screening questionnaires have been developed for depression, alcohol and drug use disorders, and neurocognitive disorders for use in primary care.^4–7 In this study, our aim is to validate an ultrashort screening tool named as ELD tool for SLD consisting of a single question for each domain of learning disorder, which is less time consuming and easy to administer.

Materials and Methods

This was a cross-sectional study. The participants were recruited from September 2019 to December 2019, using consecutive sampling. The study population was the school-going children aged 6–16 years attending the Psychiatry Outpatient (OP) Department of Government Medical College, Ernakulam. Children with visual, hearing, or locomotor impairments that interfere with the assessment, children accompanied by parents or guardians from whom a valid consent could not be obtained, children from whom assent could not be obtained, and children not accompanied by a parent or guardian who is involved in studies were excluded.

The sample size was calculated using the formula 4pq/d²; d is precision, and it was taken as 10. The sample size required for the study was 106, taking the prevalence of SLD in our OP as 40% (as it is a center where children with poor scholastic performance are referred from the entire district); 10% of the sample size was taken extra, expecting some improperly filled up pro forma and poor data quality.

Study Procedure

The study had approval from the Institutional Ethics Committee. We developed four questions addressing the core features of SLD using focus group discussions with teachers, parents of children with SLD, and specialists in the field, thus ensuring the face validity and content validity. The construct validity was not assessed as it is an ultrashort tool. The tool, developed and validated in Malayalam, is a self-report instrument to be answered by a person involved in the child’s study. Pretesting of the tool was done in a small sample of children with SLD to correct the errors in language, and none of them were included in the subsequent study. An English version of the tool was also developed (Appendix 1).

After taking informed consent from the parent or guardian and assent from the child, the Malayalam version of the tool was given to the person involved in the child’s studies. A pro forma to collect the basic sociodemographic details of the child was also given. The clinical assessment to check whether the child has a diagnosis of SLD as per the DSM 5 criteria was done within 2 months, by a psychiatrist who has more than eight-year experience in the field and was blind to the results of the screening test. At that time, the tool was again given to evaluate the test–retest reliability.

IQ assessment was done using Malin’s Intelligence Scale for Indian Children. It has 11 subsets, classified into Verbal and Performance subsets. The test–reset reliability is 0.91, and concurrent as well as congruent validity has also been established. This tool has been widely used in the Indian context for assessing intellectual abilities in children. This was used to identify children with intellectual disability (ID) and borderline intelligence. ⁸

Statistical Analysis

The data were entered into Excel sheet, and statistical analysis was done using R statistical software. The categorical data were summarized as proportions. The sensitivity, specificity, positive predictive value, negative predictive value, and kappa statistic for test–retest reliability of the tool were calculated.

Results

The sample consisted of 120 children aged 8–16 years. Their median age was 15 years (interquartile range 2). The majority (20.83%) of the sample consisted of children aged 14 years. Boys constituted the majority (70%). Children from families below the poverty line formed 60.83% of the sample. In the sample, 46.67% had SLD, 23.33% had ID, 21.67% had borderline intelligence, and 8.33% were others, which included normally developing children who had no diagnosis and children with a diagnosis other than SLD, ID, or borderline intelligence (Table 1).

Table 1.

Distribution of Sociodemographic Features of the Sample

Variables	Frequency N = 120	Percentage
Age in Years’
8	3	2.5
9	2	1.67
10	5	4.17
11	4	3.33
12	9	7.5
13	11	9.17
14	25	20.83
15	42	35
16	19	15.83
Sex
Male	84	70
Female	36	30
Religion
Hindu	47	39.17
Christian	43	35.83
Muslim	30	25
Socioeconomic status
Above poverty line	47	39.17
Below poverty line	73	60.83
Diagnosis
Specific learning disorder	56	46.67
Intellectual disability	28	23.33
Borderline Intelligence	26	21.67
Others	10	8.33

Table 2.

Classification of the Subjects Depending on the Diagnosis and Test Results

	SLD as per Clinician	No SLD as per Clinician	Total
Screening test positive	55 (98.21%)	8 (12.5%)	63 (52.5%)
Screening test negative	1 (1.79%)	56 (87.5%)	57 (47.5%)
Total	56	64	120

SLD: specific learning disorder.

Table 3.

The Domains of Questionnaire and the Frequency of Response

Domain	Test		Retest
	Yes	No	Yes	No
Global impression on intelligence	63	57	61	59
Global impression on reading	53	10	51	10
Global impression on writing	60	3	58	3
Global impression on arithmetic skills	50	13	48	13

The sensitivity of the tool was found to be 98.2 ( 95% confidence interval (CI) = 90.4–100). The specificity was 87.5 (CI = 76.8–94.4). The positive predictive value was 87.3 (CI = 76.5–94.4), and the negative predictive value was 98.2 (CI = 90.6–100).

The tool’s test–retest reliability was found using kappa statistics. The kappa value was 0.96 (CI = 0.92–1).

Discussion

This ultrashort tool for screening SLD has very high sensitivity. This means that it is a very good instrument to screen SLD. The specificity of the tool is very good, but we need to do further confirmation of the diagnosis clinically or with another instrument. The tool consists of a total of four questions with “Yes” or “No” answers and takes less than 5 min for screening. The first question is a screening question, and if the answer to that question is “Yes,” then only the next three questions are attempted, which are global questions pertaining to reading, writing, and arithmetic (Table 4). A score of 1 is given for each question, and if the score is ≥1, the child is considered as screened positive for SLD. If the answer to the first question is “No,” then the child has ID or borderline intelligence. The questionnaire was able to detect 100% of children with ID. So this tool can be used to screen children with ID. For the purpose of the study, SLD was diagnosed only in children with IQ ≥90. The tool has a high negative predictive value. It was able to identify 100% of the children with no diagnosis of SLD, ID, or borderline intelligence.

The kappa coefficient indicated a highly significant correlation between the test findings when the test was done at two different time points. This indicates that the tool has good test–retest reliability.

Several tools are used for screening and diagnosing SLD, but most of them are not properly validated. Specific Learning Disability—Screening Questionnaire, meant to be used by teachers, is not properly validated. When compared to this tool, our tool is validated and less time consuming. ⁹

Dyslexia Assessment for Languages of India is a comprehensive screening tool that has a junior screening tool (5–7 years) and a middle screening tool (8–10 years) and is available in four different languages. This tool is time consuming and can be used for children up to 10 years only, whereas our tool can be used for children up to 16 years.

Another tool for screening children with SLD, which is validated in almost the same population as the tool under study, has 26 questions pertaining to the various domains of SLD, with a sensitivity of 100. But its specificity is very low, and the reliability has not been tested. ¹⁰ In contrast, our tool has reasonable sensitivity and specificity of 98.2 and 87.5, respectively.

Maharashtra State Council of Educational Research and Training has devised a screening tool containing ten questions meant to be answered by teachers who have taught the student for at least 3 months and who have attended a training workshop on SLD. Our tool has four very simple questions only, which can be answered by any person who is involved in the child’s study. Besides, no training is needed and it is validated, too.

Cognitive Therapy Clinic Learning Disability Assessment Test is another screening tool for SLD. This assesses nine core learning disability domains with multiple questions in each domain and can be used by trained personnel only. This test has a Cronbach’s alpha value of 0.65. In comparison to this tool, our tool is easy to administer, there is no need for any trained personnel to administer it, takes very less time to administer, and has high reliability.

In summary, compared to the existing tools, our tool has very good validity parameters. It is less time consuming and easy to administer. The tool is self-administered and needs no trained professionals to administer it. This tool can be used irrespective of the board as it is based on the opinion of the person involved in the child’s study and not on any syllabus-based questions. This tool can be used as a screening tool to detect SLD; it is valid and has high test–retest reliability. This tool is unique because it is ultrashort; no other tool is as short as this one, and it has a screening question unlike other tools used for screening SLD which helps to differentiate SLD from low intelligence and has good validity parameters.

Limitations

The study was conducted in a tertiary care hospital where children are referred for various reasons, and this may have affected the validity parameters of the tool, especially the positive predictive value, so further work is needed to establish validity and reliability in a wider population, and particularly in those at risk for SLD in the general population. Nonetheless, there is sufficient evidence of validity for it to be used routinely in all children considered at risk of SLD. The tool is administered to a person involved in the child’s study, so only if such a person is available, this tool can be used. The tool does not assess the motivational factors of the child to study, which is important in conditions associated with poor scholastic performance.

Conclusions

This tool appears to be very useful for screening SLD in collaboration with school authorities and parents. It is valid with good sensitivity, specificity, positive predictive value, negative predictive value, and test–retest reliability. The tool is able to identify children with ID from SLD with accuracy. It is self-administered and takes a little time to administer. The tool helps in the early detection of children with SLD, and if given to the person involved in the child’s study by the school authorities, it can pick up the affected children, and such screening should be made mandatory in schools. Intervention facilities must be provided, and we need further research to develop tools that can assess the interventions given to children with SLD.