Exploration of Caregiver Interrater Agreement and Test-Retest Reliability on the Infant Cleft Observer Outcomes (iCOO)

Abstract

Background

Caregiver and observer-reported measures are frequently used as outcomes for research on infants and young children who are unable to report on their own health. Our team developed the Infant with Clefts Observation Outcomes Instrument (iCOO) for infants with cleft lip with or without cleft palate. This exploratory study compared test-retest and interrater reliabilities to inform whether differences in caregiver perspective might affect the iCOO.

Methods

This study is a secondary analysis comparing caregiver interrater agreement to test-retest reliability. Twenty-five pairs of caregivers completed the iCOO before surgery, 1 week later for test-retest reliability, 2 days after surgery, and 2 months after surgery. Reliability was assessed using intraclass correlations (ICCs) and t-tests were used to compare ratings between caregivers.

Results

Infants had cleft lip (28%) or cleft lip and palate (72%). Primary caregivers were predominantly mothers (92%) and secondary caregivers were predominantly fathers (80%). Test-retest reliability met psychometric standards for most items on the iCOO (81%-86% of items). Caregiver agreement on the iCOO items was lower than test-retest reliability (33%-46% of items met psychometric standards). Caregivers did not systematically differ in whether they rated infants as healthier or less healthy than the other caregiver (5%-16% of items had statistically significant differences).

Conclusions

Caregivers used the measure consistently, but had different experiences and perceptions of their infant’s health and functioning. Future studies are needed to explore mechanisms for the differences in test-retest and interrater reliability. Whenever possible, the same caregiver should provide ratings of the infant, including on the iCOO.