Abstract
We examined the test–retest reliability of the Child Occupational Self-Assessment (COSA). Fifty-two children ages 6–12 yr completed the COSA on two separate occasions 7–14 days apart. Participant data were analyzed using intraclass correlation coefficients (ICCs). Test–retest reliability was good for total Competence and Value scores (ICC2,1 = .72–.77) and poor to good across category scores (ICC2,1 = .44–.78). These findings suggest that the children’s perceptions of their abilities and the value they placed on their everyday activities as reflected in the test items were fairly consistent over a short period of time.
Keywords
The Child Occupational Self-Assessment (COSA; Keller, Kafkes, Basu, Federico, & Kielhofner, 2006) is a self-report assessment tool and outcome measure designed to empower children during the therapy process by capturing their perceptions regarding their own sense of competence in and the importance they place on their everyday activities. The COSA contains 24 items and the following three follow-up questions:
What are two other things you are really good at that we didn’t talk about today?
What are two other things you have a big problem with that we didn’t talk about today?
Is there anything else that is important to you that we didn’t get to talk about? Would you like to tell me? (Keller et al., 2006).
Children are asked to rate each item using two Likert scales: Competence (I have a big problem doing this, I have a little problem doing this, I do this OK, or I am really good at doing this) and Value (Not really important to me, Important to me, Really important to me, or Most important of all to me; Keller et al., 2006). To help children accurately rate each item, emoticons accompany both scales.
The COSA has undergone extensive psychometric testing using Rasch analysis. Studies demonstrate that the COSA has high internal validity and good content, structural, and substantive validity (Keller & Kielhofner, 2005; Kramer, Kielhofner, & Smith, 2010). The COSA was designed for children age 8 yr and older who have adequate cognitive abilities for self-reflection and planning. Studies have shown that it can be used with younger children and children with intellectual disabilities, but their response patterns may differ from those of their older or nondisabled peers (Kramer et al., 2010; Kramer, Smith, & Kielhofner, 2009). Moreover, younger children and children with intellectual disabilities may benefit from simplified 2-point Likert scales to enhance the utility of the assessment (Kramer et al., 2009).
Clinically, the COSA can be used in all areas of pediatric practice in conjunction with other assessments and has been found to promote collaboration between occupational therapy practitioners and children in generating client-centered individualized education program (IEP) goals (Harney & Kramer, 2007). By ensuring the child’s priorities are included in the IEP process, the COSA contributes to the best practices set forth by the Individuals With Disabilities Education Improvement Act of 2004 (IDEA 2004; Pub. L. 108–446). According to IDEA 2004 (§ 300.321), children must be included in the IEP process whenever possible, and they must be invited to attend the IEP meeting if the purpose of the meeting is to discuss postsecondary goals. When children cannot attend their own IEP meeting, their preferences and interests must be considered by the IEP team (IDEA 2004, § 300.321).
In addition to generating client-centered IEP goals, we also suggest using the COSA as an outcome measure. Before an outcome measure can be used with the utmost confidence, it needs to have established sensitivity and reliability. To our knowledge, no studies have examined whether the COSA can detect changes in children’s perceptions after an intervention (sensitivity) or whether children’s perceptions of their abilities on the COSA are stable over a short period of time (test–retest reliability).
The purpose of the current study was to examine the test–retest reliability of the COSA for school-age children. Although the COSA manual recommends administration to children between ages 8 and 13 yr, we chose to include children ages 6 to 13 yr in our data collection. Clinically, the COSA can be used with children younger than 8 yr if the clinician deems they have the cognitive ability for self-reflection and planning. Other studies of child self-report measures have found younger children to be less reliable than older children (Dowell & Ogles, 2008; Mellor, 2004); thus, a second aim of this study was to examine whether any age-related differences occurred in test–retest reliability.
Method
Procedures
This research was approved by the State University of New York Downstate Medical Center institutional review board. Participants were recruited from the community and a private school. Written caregiver permission and child assent were obtained for all study participants. Data collection took place in classrooms at the private school and in children’s homes. Data were collected by the last three authors of this study (Crook, MacSaveny, and McLaughlin) and three occupational therapy graduate students. All data collectors were female and were trained in the administration of the COSA by the first author of this study (Ohl).
After caregiver permission and child assent were obtained, the COSA was administered to each participant twice by the same data collector 7–14 days apart. A demographic questionnaire was completed by each caregiver at the onset of the study.
Scoring and Data Analysis
The completed COSA forms were scored according to the guidelines outlined by Keller and Kielhofner (2005). Competence ratings were given scores of 1 (I have a big problem doing this), 2 (I have a little problem doing this), 3 (I do this OK), or 4 (I am really good at doing this). Value ratings were given scores of 1 (Not really important to me), 2 (Important to me), 3 (Really important to me), or 4 (Most important of all to me). A total score was calculated for both the Competence and Value scales. Additional Competence and Value scores were calculated by grouping COSA items according to the following categories: Activities of Daily Living/Instrumental Activities of Daily Living (ADLs/IADLs; Items 1–6 and 8), Motor and Praxis Skills (Items 9 and 23–25), Cognitive Skills (Items 7, 10, 11, 16, and 17), Emotional Regulation Skills (Items 20–22), and Communication and Social Skills (Items 12–15, 18, and 19). To ensure accuracy, all forms were scored twice. The data were analyzed using IBM SPSS Statistics (Version 20; IBM Corporation, Armonk, NY).
Test–retest reliability was estimated using intraclass correlation coefficients (ICCs). Based on the ICC findings of other child self-report measures (Bult, Verschuren, Gorter, Jongmans, Piškur, & Ketelaar, 2010; Dowell & Ogles, 2008; Imms, 2008; Mellor, 2004), we considered ICC values ≥ .70 to be good, values between .50 and .70 to be moderate, and values < .50 to be poor. A point biserial correlation coefficient was used to examine the relationship between the participants’ age and a difference score, which was derived by calculating the difference in their initial COSA scores (pretest) and follow-up COSA scores (posttest).
Results
A convenience sample of 52 children ranging in age from 6 to 12 yr (mean [M]= 8.56, standard deviation [SD] = 1.59) participated in this study. The majority of participants were identified by their caregivers as White (73.1%), attended public schools (57.7%), and did not receive any related services (82.7%) at the time of the study. Table 1 provides additional demographic characteristics.
Participant Characteristics (N = 52)
Note. ADHD = attention deficit hyperactivity disorder; — = not applicable.
Descriptive data on the Competence and Value total and category pretest and posttest scores and ICCs are provided in Table 2 and Table 3. ICCs for the Competence (ICC2,1 = .717) and Value (ICC2,1 = .772) total scores were good. The Competence category scores (ADLs/IADLs, Motor and Praxis Skills, Cognitive Skills, Emotional Regulation Skills, and Communication and Social Skills) ranged from poor to moderate (ICC2,1 = .442–.699), and the Value category scores ranged from moderate to good (ICC2,1 = .578–.779). Emotional Regulation Skill scores were the lowest for both Competence (ICC2,1 = .442) and Value (ICC2,1 = .578) scales.
Competence Scores: Descriptive Data and ICCs (N = 52)
Note. ADLs/IADLs = activities of daily living/instrumental activities of daily living; ICC = intraclass correlation coefficient; M = mean; SD = standard deviation.
Value Scores: Descriptive Data and ICCs (N = 52)
Note. ADLs/IADLs = activities of daily living/instrumental activities of daily living; ICC = intraclass correlation coefficient; M = mean; SD = standard deviation.
Additional analyses were conducted to determine whether the children’s age was related to the difference between their pretest and posttest scores. We did not find a significant relationship between age and difference scores across the total Competence (r = –.148, p = .296) and Value (r = .074, p = .604) scores or the following categories: ADLs/IADLs, Motor and Praxis Skills, Emotional Regulation Skills, and Communication and Social Skills. We did, however, find a weak statistically significant relationship between age and the Cognitive Skills Competence difference scores (r = –.317, p = .022), indicating that older children had a smaller change in their scores than did younger children.
Discussion
We examined the test–retest reliability of the COSA using total Competence and Value scores and category Competence and Value scores (ADLs/IADLs, Motor and Praxis Skills, Cognitive Skills, Emotional Regulation Skills, and Communication and Social Skills). Our findings indicate that the children’s responses to the COSA items remained fairly consistent over a 2-wk period, with total Value and Competence scores (ICC2,1 = .717–.772) having greater consistency than category scores (ICC2,1 = .442–.779). The lowest ICCs were found under the Emotional Regulation Skills category for both Competence (ICC2,1 = .442) and Value (ICC2,1 = .578) scores, which is not surprising given that emotions are generally more variable than physical and cognitive performance.
We also examined the relationship between age and change in pre- and posttest scores, hypothesizing that more variability would occur in scores among younger children. On the whole, our findings did not support our hypothesis. Young children’s perceptions of their abilities and the value they placed on everyday tasks were just as reliable as older children’s perceptions and values, with one exception: We found a weak statistically significant correlation between age and Cognitive Skills Competence difference scores, indicating that younger children’s perceived competence on cognitive tasks fluctuated more than older children’s. Given the relative weakness of the correlation, further study is needed.
Limitations and Future Research
This study was limited by a small sample of geographic convenience. More research is needed to determine whether the COSA is sensitive enough to detect change in children’s performance postintervention. Additional research is also needed to examine whether children’s perceptions of their competence adequately matches their abilities on other standardized assessments.
Implications for Occupational Therapy Practice
The results of this study indicate that when using the COSA as an outcome measure, occupational therapy practitioners can be reasonably certain that the changes they detect in a child’s perceived Competence and Value are not due to the child’s fluctuating perception of his or her own abilities. However, further study is needed.
Footnotes
Acknowledgments
We thank Joyce Sabari for her support and guidance and Hana Abdelrasoul Ennab, Tamar Blumenkranz, Natalia Zlotnikov, and Annabelle Wu for their assistance with data collection. We extend special thanks to the children and families who participated in the study.