Content Validation of the Communication Inventory Disability–Observer Reported (CID-OR)

Abstract

CDKL5 deficiency disorder is a rare and severe developmental and epileptic encephalopathy that has profound effects on communication. It is essential that communication be measured accurately for upcoming gene therapy trials. The Communication Inventory Disability–Observer Reported (CID-OR) was developed from a framework of communication derived from parent/caregiver interview data (n = 23), in consultation with disability and communication experts, and after reviewing concepts in existing measures. In this study, parents and caregivers (n = 21) took part in 1-hour semistructured think-aloud online interviews. A directed content analysis was conducted using both deductive and inductive coding. Findings suggest that CID-OR was comprehensive, comprehensible, and relevant. Multiple small adjustments were made to improve clarity and representativeness and reduce caregiver burden based on feedback in the data. This content validity study is an important element in the process of developing an effective measure of communication of people with CDKL5 deficiency disorder.

CDKL5 deficiency disorder is a rare, complex, and severe developmental and epileptic encephalopathy. CDKL5 deficiency disorder is associated with early onset seizures, substantial global developmental delays, and multiple comorbid conditions, including profound communication impairments. ¹ The main functional domains of gross and fine motor, eating, and communication are severely impacted, with abilities affected in multiple ways and to varying degrees.^2–4 For example, approximately 25% walk and fewer will attain a pincer grasp, ³ and a little more than 25% are enterally fed. ² Body language (96%), facial expressions (94%), early sounds (91%), and simple gestures (89%) have been reported as the most commonly used modes of communication, compared with single words (8%) or multiple sentences (8%). ³ Limitations of gross, fine, and oral motor functioning each have the potential to restrict the use of nonsymbolic modes of communication such as gestures and whole-body movements, and symbolic modes of communication including augmentative and alternative communication (AAC) devices and sign language. ⁵ Other features of CDKL5 deficiency disorder, such as cortical visual impairment, which occurs in the majority (∼75%) of people with CDKL5 deficiency disorder, ⁴ imposes additional challenges for communication. Cortical visual impairment can reduce capacity for communication by limiting access to visual cues and ability to use many forms of augmentative and alternative communication, which primarily rely on vision. ⁶

Communication impairments have significant impacts on people with CDKL5 deficiency disorder and their families. For example, caregivers have reported that communication impairments can impose considerable difficulties engaging with their child and understanding their needs. ⁷ Nevertheless, people with CDKL5 deficiency disorder can use a range of communication modes to express different purposes and meanings affording understanding and value to day-to-day interactions with their communication partner. ⁸

New clinical trials for disease-modifying gene therapies necessitate outcome measures that are relevant to people with CDKL5 deficiency disorder and their families to understand their efficacy. One such outcome measure is impacts on communication. Precise and valid outcome measures of the communication of people with CDKL5 deficiency disorder are needed to use communication as an outcome measure. Validation data that demonstrate reliability and reproducibility and represent the spectrum of challenges relevant for the population are limited for observer-reported communication measures used for individuals with neurodevelopmental disorders. ⁹ One measure, the Communication and Symbolic Behavior Scales–Developmental Profile (CSBS-DP ITC), ¹⁰ which comprises of an infant toddler checklist and a parent questionnaire, demonstrates evidence of sensitivity and validity in some developmental and epileptic encephalopathy conditions ¹¹ and Rett syndrome. ¹² However, we found a floor effect for the Communication and Symbolic Behavior Scales–Developmental Profile infant toddler checklist in CDKL5 deficiency disorder, indicating that it might not capture the more subtle forms of communication used by those with more severe impairments. ¹³

We previously explored parent perspectives on how people with CDKL5 deficiency disorder communicate and developed a framework of communication for CDKL5 deficiency disorder. ⁸ Combining this framework with expert clinical opinion and experience, and concepts from existing measures, we developed a draft measure of communication called the Communication Inventory Disability–Observer Reported (CID-OR). Evaluating content validity is an important component of measure development to ensure that the measure reflects the issues and needs of the population of interest. ¹⁴ This study describes the process of developing CID-OR, assessing its content validity in terms of clarity, comprehensiveness, and relevancy, and documents the changes that were made in response to caregiver and clinician feedback, prior to psychometric testing.

Methods

Figure 1 summarizes the overall processes of developing and validating CID-OR using a modified version of the Patient-Report Outcome (PRO) Development Process. ¹⁴ The content validity phase Figure 1 is described in this article and the measurement property psychometrics phase activities (below the dotted line) are outlined in a separate article where the psychometric properties of CID-OR were assessed (J. Downs et al, unpublished data [2025]). The University of Western Australia Human Research Ethics Committee provided ethical approval (2019/RA/4/20/6198). The Standards for Reporting Qualitative Research ¹⁵ and the COSMIN Study Design checklist for patient-reported outcome measurement instruments were followed. ¹⁶

Figure 1.

CID-OR development process.

Participants

The sample comprised parents and caregivers of people with CDKL5 deficiency disorder who were identified through the International CDKL5 Deficiency Disorder Database. ¹⁷ Twenty-three families from the previous qualitative study that explored how people with CDKL5 deficiency disorder communicate ⁸ were invited to participate in the current study and 18 (78%) agreed to participate. An additional 3 families were recruited from the International CDKL5 Disorder Database, ¹⁷ 2 of whom were selected to further explore the perspectives of families with very young children (aged 2 and 3 years old), resulting in a total of 21 participating families.

Information power was considered when determining the necessary sample size. ¹⁸ This concept posits that the richer and more relevant information a sample has, the fewer participants are needed. It also takes into account the breadth of the study aim, phenomenon specificity, application of theory, dialogue quality, and analysis type. ¹⁸ It was estimated that a sample of approximately 20 participants would sufficiently achieve information power for this study.

Procedure

Item Development

Literature on communication in people with CDKL5 deficiency disorder and other developmental and epileptic encephalopathies was reviewed before commencing the measure development process. Although the available research is informative,^3,9,11,19 measure validation and particularly content validation was limited, indicating the gap to be addressed in the current study.

The communication framework developed from interviews with parents of people with CDKL5 deficiency disorder was the basis for CID-OR. ⁸ The qualitative data were organized into three overarching categories: purpose and meaning, mode of communication, and reciprocal exchanges. In addition to reciprocal exchanges, 4 categories were identified within purpose and meaning, including indicating understanding, demonstrating preferences, expressing emotions, and social connections. Concepts within items from established communication measures (Communication and Symbolic Behavior Scales infant toddler checklist, Communication and Symbolic Behavior Scales parent questionnaire 10, Communication Matrix, ²⁰ Vineland ²¹ ) were mapped to concepts within these 5 categories. Consistent with the qualitative framework, the final draft measure contained concepts identified in the following measures: Communication and Symbolic Behavior Scales infant toddler checklist (n = 7), Communication and Symbolic Behavior Scales parent questionnaire (n = 3), Communication Matrix (n = 7), and Vineland (n = 7).

These common concepts were shaped into new items for CID-OR to achieve clarity and consistency, ensuring each item addressed a single concept. An additional 13 novel items not identified in the available measures were created to comprehensively address the meaning within each of the categories in the qualitative data. The final draft measure comprised 35 items.

Additionally, 6 speech pathologists working within the fields of communication and disability reviewed the measure. In combination with the content validation interview data, their feedback informed the changes to the draft CID-OR.

Measure Structure

The introduction to CID-OR described the purpose of the questionnaire and provided definitions for each response category. Parents and caregivers were advised they should answer questions to reflect how their child communicated in the last 30 days. A screener question asked about the use of aided communication, sign language, and/or spoken words. If a parent selected 1 or more of these modes, additional response options were presented for each item.

Items asked about the purpose of different communicative acts (eg, to make a choice, to show affection). Each item was accompanied by examples derived from the interview data set and represented a range of communication complexities. ⁸ For each item, the consistency of communication was measured using a 5-point Likert scale (not yet, hardly ever, sometimes, often, consistently). Parents and caregivers were then asked how the purpose was communicated with options for both symbolic and nonsymbolic forms of communication. Four nonsymbolic options were presented, including (1) limb movements, (2) body and head movements, (3) facial and eye expressions and movements, and (4) vocalizations. If parents indicated that their child used symbolic communication, they were additionally presented with relevant symbolic communication mode options, including augmentative and alternative communication, sign language, and spoken word(s). For each, options included (1) not using the symbolic communication mode for this purpose; (2) using single words, icons, or signs; (3) 2 in combination; (4) 3 to 4 in combination; or (5) using multiple sentences. Finally, an open-ended question offered participants the opportunity to provide additional information. We anticipated that this additional question would be useful in clinical settings.

Content Evaluation Interviews

As an initial check, the draft measure was presented to 2 parents of individuals with CDKL5 deficiency disorder who took part in brief informal discussions to evaluate face validity. Evaluations were positive and the parents did not suggest changes at this point.

A “think-aloud” interview schedule was developed to collect content validation data while participants responded to each item. ²² The schedule was guided by the summary presented in Epstein and colleagues’ work ²³ based on Charters ²² framework. For each item, participants were asked to (1) verbalize their thoughts while answering the question, (2) answer why they responded in that way, and (3) how they would phrase the question using their own words. Finally, participants were asked about the clarity, comprehensiveness, and relevancy of the measure overall.

Participants were sent a link to the draft measure in REDCap^24,25 if they wished to familiarize themselves with the materials before the interview. Interviews were conducted by JK (PhD in psychology with disability research experience) via video call and were audio recorded, after informed consent was provided. The measure was shared on the screen for both the researcher and participant to see. The introduction was read, and feedback was requested. The items were answered verbally and recorded in the online survey by the researcher before questions about the items were asked. Field notes were taken throughout interviews, and raw additive item scores were calculated.

Analysis

Interview recordings were transcribed and checked by JK against the recording for accuracy. A directed content analysis was conducted by JK using NVivo. Coding commenced after transcript familiarity was achieved. Deductive (ie, predetermined) codes were developed categorizing information in terms of comprehensibility, relevance, and suggested wordings for each item, the introduction, screener question, and the overall measure. Inductive (ie, data derived) codes were created during the analysis process when deductive codes could not comprehensively describe the data. Codes for each item were summarized in a spreadsheet that was reviewed by the research team (JK, JD, JS, PJ) and used to assess and modify measure content.

Trustworthiness

Coding was undertaken by one researcher (JK), reducing the confirmability of the findings. However, it was not considered necessary for additional researchers to code the data because of the directed nature of the coding and the frequent and extensive reflective discussions within the research team. This collaborative approach suggests that analytic interpretations were shared by the research team. Additionally, field notes were taken in interviews to improve contextualization and interpretation, and an audit trail was maintained to increase dependability.

Results

Twenty-one parents and caregivers of people with CDKL5 deficiency disorder participated in this study. The individuals with CDKL5 deficiency disorder who were discussed in interviews were between the ages of 2 and 31 years (median = 15 years) and predominantly female (n = 17). Families resided in North America (n = 13), Australia (n = 5), and Europe (n = 3). See Table 1 for further description of the sample.

Table 1.

Sample Characteristics of the Children With CDKL5 deficiency disorder (N = 21).

Age	n
Age
1-7 y	7
8-12 y	2
13-17 y	6
≥18  y	6
Gender
Female	17
Male	4
CSBS - DP Infant Toddler Checklist Score a
0-10	7
11-20	5
21-30	2
31-40	3
41-50	2
≥51	2
CVI
Yes	15
No	6
Seizure frequency b
None	6
Monthly/yearly	1
Weekly	5
1-4 daily	6
≥5 daily	2
Seizure medication b
0	2
1	4
2	6
≥3	8
Mutation group
No functional protein	8
Missense/inframe with catalytic domain	3
Truncation from aa172 to aa781	5
Truncation after aa781	5
Walk 10 steps
No assistance needed	10
Can with assistance	5
Unable	6
Gastrostomy
Oral	18
Enteral	2
Both	1

Abbreviations: CSBS-DP ITC, Communication and Symbolic Behavior Scales–Developmental Profile infant toddler checklist; CVI, cortical visual impairment.

a

The most current CSBS-DP ITC available collected in previous research. ⁸

b

Missing data for 1 participant.

The changes made to CID-OR were informed by a systematic review of participant responses. Minor changes were made in the introduction section to enhance readability. The screener question and open-ended question remained unchanged. Each participant's responses on items informed the wording and examples. Overall, parents reported that the measure was comprehensive in its coverage of how people with CDKL5 deficiency disorder communicate.

Items

Questions

Of the 35 original items, 21 were altered and 14 remained unchanged. Changes were made to broaden the scope of the question (n = 9; eg, the question about affection was changed to accepting as well as showing affection), improve clarity (n = 7; the question about play was rephrased to improve readability), and enhance specificity (n = 5; eg, additional information included to specify that communicating pain should not include reflexive reactions). Additionally, emphasis was added to 2 questions by underlining specific words to differentiate them (ie, 1- and 2-step directions).

Question Examples

Examples were changed and/or added for all items. Participants expressed that the examples helped them understand items; however, some did not feel that their child's capacity was represented. In response to this feedback, we ensured that examples were relevant to someone with limited motor abilities and cortical visual impairment for all items. For example, for the item “Does your child respond to their name?” examples were altered to include the child ceasing movement and noises or turning to look at the parent. These ways of responding were identified in the qualitative data and can be demonstrated by person irrespective of motor or cortical visual impairment status.

Response Categories

One of the response categories for the consistency of each communicative purpose was renamed in response to participant feedback. One participant identified that “not yet” did not accurately reflect the abilities of those who were not able to communicate a purpose in the last 30 days but had been able to previously. Consequently, “not yet” was changed to “not currently.” Accordingly, the definition for “not currently” in the introduction was altered to include both communication that has never been observed or has not been observed in the last 30 days.

Mode Options

Nonsymbolic mode options were also changed in response to the data and feedback from communication experts. Facial movements and eye expressions that had been presented together were separated into 2 distinct measures of mode. This aimed to enhance capacity to document nonsymbolic communication and identify potential differences in the use of nonsymbolic modes in the presence of cortical visual impairment. The wording of the symbolic mode categories for augmentative and alternative communication, sign language, and spoken words were adjusted to improve readability and consistency across these symbolic mode categories.

Item Grouping

The introductions to the social connection and 2-way exchange categories were adjusted to improve readability and provide additional context in response to participant feedback. The order of the 5 categories was changed after reflecting on the calculated scores. We identified that scores tended to be lower in the first category, “Indicating Understanding,” compared to the second, “Demonstrating Preferences.” The order of category was swapped to avoid presenting parents with a generally more challenging and potentially disheartening category at the onset of the measure. Item order within each of the categories was also changed in response to raw scores to avoid presenting lower-scoring items first and to improve the conceptual flow.

Discussion

CID-OR was based on our framework of communication for people with CDKL5 deficiency disorder ⁸ and informed by existing measures and feedback from clinical disability and speech-language experts. A set of principles underpinned the development of CID-OR. First, receptive and expressive communication were observed together because of their close linkage when communication is severely impaired. Second, communication equates to an expression of a message that is given intentionally or perceived as intentional by the communication partner, such that purpose and mode occur simultaneously. Third, multiple symbolic and nonsymbolic modes need to be understood by communication partners. Finally, spoken words, sign language, and augmentative and alternative communication methods of symbolic communication were considered equally. Using think-aloud methods to assess content validity, parents and caregivers generally reported that CID-OR was comprehensive in its coverage of how their child communicates and whether comprehensible and relevant. However, changes were made to improve the content validity, in terms of comprehensiveness (eg, additional examples to represent how more children communicate), their comprehensibility (eg, alter item wording to clarify meaning), and relevancy (eg, changing the response option from “not yet” to “not currently”). Parent and caregiver data were incorporated to be sensitive to parent self-esteem, improve clarity, and capture and represent a broader scope of abilities. This aimed to ensure that CID-OR has optimal capacity to safely and accurately measure communication in upcoming disease-changing gene therapy trials.

Caregiver burden and self-esteem were considered throughout the development and content validity evaluation stages. These accommodations included providing sensitive and inclusive response options and examples (eg, “not currently” is more inclusive and optimistic than “no”), altering the wording and order of items, category order, and use of screening, which resulted in only relevant mode options being presented. These strategies are consistent with best practice methodologies for scale development in disability conditions where there is potential to reduce parent esteem. ²⁶ These are important considerations as research suggests that parents of children with disability, including those with CDKL5 deficiency disorder, face many mental health challenges. ²⁷ For example, having a child with CDKL5 deficiency disorder has been associated with a greater impact on parents’ emotional well-being than having a child with other disabilities (Rett syndrome, Down syndrome) or without a disability. ²⁸ Measures and interventions should always value and prioritize the well-being of participants.

Following content validation, preliminary evaluation of the psychometric properties of CID-OR suggested that the measure is reliable and valid (n = 184; J. Downs et al, unpublished data [2025]). A confirmatory factor analysis identified a single-factor model, and one item was removed after reviewing item loadings. Internal consistency and goodness-of-fit statistics were good (excluding the root mean square error of approximation statistic), and test-retest reliability was high (ICC 0.966). Although CID-OR and the Communication and Symbolic Behavior Scales infant toddler checklist were highly correlated (r = 0.89), the right skew (ie, floor effect) was reduced with CID-OR compared with the Communication and Symbolic Behavior Scales–Developmental Profile infant toddler checklist.

Strengths and Weaknesses

A strength of this study is that 18 of the 21 participants who provided content validity data in this study were interviewed in the previous qualitative study that explored the communication of people with CDKL5 deficiency disorder. It was valuable to understand how the families whose experiences informed the framework for CID-OR evaluated its validity. Further, the process of refining the measure using think-aloud methods aimed to empower parents and avoid burdening them with irrelevant and disheartening questions.

This study has some limitations. The perspectives of communication and disability experts on the relevance and comprehensiveness of CID-OR made a valuable contribution to the final measure. Although extensive notes were taken, these meetings were informal, not recorded, and not systematically analyzed. However, this measure is parent-report and so these perspectives were the most relevant in this context. We also cannot review the recall period of 30 days because it was not specifically discussed in interviews.

Conclusion

We developed CID-OR because there was no available validated and accepted measure of communication for CDKL5 deficiency disorder or similar developmental and epileptic encephalopathies. CID-OR was informed by our framework of communication for people with CDKL5 deficiency disorder, existing communication measures, additional parent and caregiver perspectives, and disability and communication expert advice. During think-aloud interviews, parents generally reported that the measure was clear, relevant, and comprehensive. However, data-driven adjustments were made throughout to improve the measure. Content validation is promising and forms as a foundation for all other checks on reliability and validity. We suggest that CID-OR may have utility in clinical trials and practice for CDKL5 deficiency disorder as well as other conditions with similarly impaired communication abilities, such as in developmental and epileptic encephalopathies. Future research should review the comprehensiveness, comprehensibility, and relevancy of CID-OR in other conditions. Content validity is central to the development of outcome measures. We propose that our methods optimize the suitability of CID-OR for parent-report of communication with individuals with CDKL5 deficiency disorder, enabling confidence in interpreting evidence of reliability and validity.