Abstract
Background:
Comprehensive outcome measurement in pediatric palliative care focusing on the entire unit of care, that is, the affected child and its family, is crucial to depict treatment effects. Despite its increasing relevance, no appropriate multidimensional outcome measures exist for the largest patient group in this field, namely children with severe neurological impairments.
Aim:
The aim of this study was to develop and validate a family-centered multidimensional outcome measure for pediatric palliative care patients with severe neurological impairment that encompasses the entire unit of care.
Design:
Based on results of a qualitative study, the questionnaire was developed by consensus-based generation of questions. It was validated in a multicenter prospective study employing exploratory and confirmatory factor analyses as well as reliability and item analyses.
Setting:
A total of 11 pediatric palliative care teams across Germany aided in the recruitment of study participants. Questionnaires were answered by 149 parents of children with severe neurological impairment and 157 professional caregivers.
Results:
The exploratory factor analysis of parent data revealed a 6-factor structure of the questionnaire representing: symptoms, the child’s social participation, normalcy, social support, coping with the disease and caregiver’s competencies. This structure was verified by a confirmatory factor analysis of professional caregiver data. Five separate items regarding siblings, partners, and further symptoms not applicable for all patients were added to ensure full multidimensionality.
Conclusion:
A validated outcome tool for severely neurologically impaired pediatric palliative care patients, the FACETS-OF-PPC, now exists. Due to its concise length and appropriate psychometric properties, it is well suited for clinical application.
Keywords
Reliance on patient-centered outcome measures is increasing in clinical practice and research.
Utilizing patient-centered outcome measures may improve the quality of services and the patients’ outcomes.
Appropriate outcome measures for pediatric palliative patients with severe neurological impairment, who constitute the largest patient group in pediatric palliative care, have not yet been developed.
The
A parent and professional caregiver version of the FACETS-OF-PPC has been developed and validated.
Up to now, the FACETS-OF-PPC is able to validly and reliably display the patient’s and the family’s current situation. As soon as future research demonstrates its responsiveness to change, it may be utilized for progress monitoring and effectiveness studies.
Introduction
Within palliative care, the use of patient-reported outcome measures in clinical practice and research becomes increasingly important. Such measures may improve the quality of the provided services as well as the patients’ outcomes. 1 To this date, a variety of outcome measures has been developed for adult patients, several of which are multidimensional, considering physical, psychological, social and spiritual aspects relevant in palliative care.1–3 In contrast to this, no appropriate validated multidimensional outcome measure exists for pediatric patients in developed countries.4,5 The sole focus on adult patients is worrisome, as worldwide more than 20 million children are affected by life-limiting conditions, approximately 8 million of whom require specialized pediatric palliative care. 6
Developing appropriate outcome measures for children with life-limiting diseases is an enormous challenge due to several reasons. Generally, the population is very heterogeneous regarding their developmental age and underlying condition.7–9 Affected children often show long survival rates with greatly varying disease trajectories. 10 Frequently they are affected by severe neurological impairment that renders verbal communication impossible.7,9 Utilizing proxy-reports as the mode of data collection becomes inevitable in such cases. Etkind et al. 11 thus suggested that the term patient-centered outcome measure should be used for instruments developed for populations that depend on proxy-reports. Additionally, due to the high involvement of the child’s family, not only the child, but all family members must be regarded as constituting the unit of care. 12 A large variety of treatment objectives targeting not only the child but the entire unit of care must thus be taken into account. 12
Only one multidimensional outcome measures for pediatric palliative care has yet been developed—the APCA Children’s Palliative Outcome Scale 13 (African C-POS). It consists of a self-report and a proxy-report and is thus applicable to verbal as well as non-verbal children. However, it is unclear whether the African C-POS is applicable to children from other regions, as it was built on consensus of local experts in Africa.14,15 It has recently been translated into French and validated in a monocentric study in Belgium, 16 however, the study sample did not include children with severe neurological impairment. Thus, both the original and the translated version of the African C-POS may not be applicable to children with congenital and neurological disorders, who constitute the largest patient group in developed countries for children younger than 16 years. 8
Based on a previous study that identified relevant aspects of pediatric palliative care for children with severe neurological impairment and their families, 17 this study aims to develop and validate a multidimensional family-centered outcome measure for this population. A parent and professional caregiver version will be created. During the validation, special attention will be paid to shortening the questionnaire in order to make it easily applicable, even for burdened families and professional caregivers with limited resources.
Methods
Development of the questionnaire
The development of the questionnaire was based on the relevant aspects of pediatric palliative care for children with severe neurological impairment and their families that were identified in a qualitative study by Ribbers et al. 17
Phase 1—Literature review
A comprehensive literature review in PUBMED was conducted to identify any existing items assessing the relevant aspects identified in the prior qualitative study (see Supplemental Material 1 for search terms). Items were considered appropriate if they were validated for the population in question. Titles and abstracts of all 163 identified articles were screened. Full texts were assessed if abstracts were inconclusive. In line with previous studies and reviews,4,13 our literature search found neither outcome measures nor individual items validated for this patient group.
Phase 2—Formulation of items
Due to the lack of existing measures, items were newly generated by S.R. and M.R. The wording of the items was inspired by the language used by parents and professional caregivers in the interviews conducted by Ribbers et al. 17 A total of 67 items were generated.
Phase 3—Expert panel discussion
The created items were discussed with regard to suitability and comprehensibility with an expert panel consisting of 4 physicians, 1 psychologist, 3 nurses providing pediatric palliative care in an inpatient or outpatient setting and 5 researchers active in this field. A total of three panel discussions were conducted. In the first two meetings, items that needed revision or should be eliminated altogether (e.g. those that did not constitute an outcome in itself but rather referred to the quality of the service (e.g. referring to socio-legal advice provided by the PPC team) or those not generally applicable to all families (e.g. those referring to financial difficulties)) were identified. In a third panel discussion revised items were presented and discussed until consensus was reached. An instrument with 53 items for parents and professional caregivers resulted. It was named the
Psychometric validation of the FACETS-OF-PPC
Design
The validation study was conducted as a multicenter prospective study.
Setting
Study participants were recruited in 11 pediatric palliative care institutions throughout Germany. In order to ensure service-independent suitability of the measure, heterogeneity of institutions was strived for (n = 1 pediatric palliative care unit, n = 3 children’s and adolescents’ hospices, and n = 7 pediatric palliative home care teams).
Participants
All German-speaking parents of a non-verbal child aged 0–25 years with a life-limiting condition and severe neurological impairment were eligible for study participation. Families in acute crises due to the child’s current health status were not approached for study participation. Professional caregivers had to care for children with the above-described characteristics in order to participate.
Recruitment and data collection
Data were collected between February and August 2019. A professional caregiver responsible for study supervision in the respective institution verbally informed eligible parents about the study. Interested parents obtained all study material in an unsealed prepaid return envelope. A professional caregiver was allowed to participate if parents gave written consent. They too received an envelope containing all study material. Participants were asked to fill out all documents, including written consent forms, seal them in the envelope and send it to the study coordinator (S.R.). No cost incurred for them.
Measures
The
Analysis
Parent data were analyzed in IBM SPSS 26.0 using exploratory factor analysis employing principal axis factoring to examine the factor structure of the FACETS-OF-PPC. Oblique rotation was used, as this allows for latent factors to be correlated and usually results in simple solutions. The number of latent factors was determined using the scree test 18 and parallel analysis, 19 the latter carried out in R 20 with the “psych” package. 21 Because of listwise deletion, only questionnaires without missing data were included in the analysis. Items not generally applicable to all participants were excluded from the main analysis (i.e. two items concerning siblings, two items concerning life partners of the caregiver, one item concerning additional symptoms), as they would systematically reduce the sample with complete data, potentially rendering analysis impossible. In order to shorten the measure, items with factor loadings <.4 were deleted.
The suitability of the discovered factor structure for the professional caregiver version, and thus factorial validity, was tested using confirmatory factor analysis in IMB SPSS AMOS 26.0. The recommendations by Schermelleh-Engel et al. provided to evaluate the model fit were applied: χ2/df (<3 = acceptable, <2 = good), Comparative Fit Index (CFI: >0.95 = acceptable, >0.97 = good), and Root Mean Square Error of Approximation (RMSEA, <0.08 = acceptable; <0.05 = good). 22
For both FACETS-OF-PPC versions, Cronbach’s α was employed as a measure of internal consistency (Cronbach’s α of between 0.7 and 0.9 is desirable). 23 The item analysis included item range, difficulty and discrimination and was based on data of the entire sample. Lastly, content and convergent validity of the FACETS-OF-PPC were determined. Convergent validity was assessed by comparing scale scores with global ratings collected in addition to the FACETS-OF-PPC.
Ethics
The study was approved by the Ethics Committee of the Witten/Herdecke University, Germany (ID 190/2018) and was conducted in accordance with the ethical principles specified in the Declaration of Helsinki.
Results
A total of N = 149 parents and N = 157 professional caregivers participated in the study (see Table 1(a) and (b)).
Characteristics of parents and their children (N = 149 a ).
N varies due to missing values.
Indicated by parents and professional caregivers, summarized according to ICD-10 categories, all children suffer severe neurological impairments.
Characteristics of professional caregivers (N = 157 a ).
N varies due to missing values.
Within pediatric palliative care.
Percentages exceed 100, as several work settings/professions may be applicable.
Descriptive analysis of the evaluation questionnaire showed that study participants rated the FACETS-OF-PPC as adequate regarding its length, as well comprehensible and indicated that most items are relevant (see Table 2).
Descriptive results of the analysis of the evaluation questionnaire.
N varies due to missing values.
Scale ranges from 1 (far too short)—5 (far too long).
Scale ranges from 1 (very comprehensible)—5 (very incomprehensible).
Scale ranges from 1 (all items are relevant)—4 (no item is relevant).
Parent version—Exploratory factor analysis
Data of n = 80 parents were included in the exploratory factor analysis. A dropout analysis revealed no significant differences in demographic characteristics between participants included in or excluded from the factor analysis. Data fulfilled the necessary criteria for analysis: The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.61, just above the recommended value of .6. Bartlett’s test of sphericity was significant (X2(1128) = 2323.6, p < 0.05), as desired. According to the scree plot and parallel analysis, a 6-factor structure best classified the data (see Table 3). This factor solution accounted for 51.1% of the total variance (see Table 4). A total of 11 items were eliminated as they had factor loadings <0.4.
Factor loadings and communalities based on the principal-axis analysis with oblimin rotation (n = 80).
Factor loadings <0.4 are suppressed.
Item was deleted as Cronbach’s alpha improved without it.
Descriptive statistics for the six factors identified by principal axis factoring with oblimin rotation for parent data (n = 80).
After deletion of items without which Cronbach’s α is improved.
Analysis based on data of all 149 parents, N varies due to missing values.
Data of all 149 parents were included in the subsequent analyses, in order to maximize external validity. Internal consistency of the factors as indicated by Cronbach’s α was moderate to good (see Table 4). Three more items were eliminated as Cronbach’s α improved if the item was excluded (see Tables 3 and 4). The item analysis (see Supplemental Material 3) showed that 30 items used the full range of the rating scale, whereas 4 did not use the lower range of the scale. Medium to high item difficulty was demonstrated by rather high item means. Corrected item-total correlations were acceptable to good, indicating adequate item discrimination.
Overall, the analysis resulted in 34 items constituting six factors. Based on the results, the factors were labeled: “symptoms,” “child’s social participation,” “normalcy,” “social support,” “coping with the disease,” and “caregiver’s competencies” (see Table 3).
Professional caregiver version—Confirmatory factor analysis
The discovered factor structure was tested for caregivers using confirmatory factor analysis with maximum likelihood estimation. All fit indices except for the Comparative Fit Index indicated an acceptable fit (X2/df = 2.00, RMSEA = 0.08, CFI = 0.78). As it was desired to create parent and professional caregiver versions of the FACETS-OF-PPC that consist of the same questions, one poor fit index was tolerated with all others being acceptable to good. Additionally, Cronbach’s α indicated moderate to good values—0.70 for “symptoms,” 0.69 for “child’s social participation,” 0.92 for “normalcy,” 0.77 for “social support,” 0.74 for “coping with the disease,” and 0.85 for “caregiver’s competencies.”
Item analysis of the professional caregiver data (see Supplemental Material 2) show that 24 items used the full range of the applied rating scale, whereas 10 did not use the lower range of the rating scale. As in the parent version, item difficulty was found to be medium to high and item discrimination was acceptable to good.
Additional items
The five items concerning siblings, life partners and further symptoms that were not generally applicable to the entire sample (see Table 5), were included independently of the subscales in both questionnaire versions. Consideration of the entire unit of care and multidimensionality were thus ensured. These additional items must be evaluated descriptively. As they are not incorporated into the identified scales of the questionnaire, they do not affect the tool’s psychometric properties.
Items not generally applicable to all participants that were added irrespective of identified subscales.
1 (completely disagree)—6 (completely agree).
1 (not present)—6 (very pronounced).
Validity of the FACETS-OF-PPC
As content validity is typically judged based on expert ratings, it was assumed for the developed outcome measure because items have been built based on the interviews conducted with experts by Ribbers et al., 17 that is, parents and professional caregivers. Further indicators of adequate content validity are the parents’ and professional caregivers’ ratings of the questionnaire’s comprehensibility and relevance shown in Table 2. Factorial validity was demonstrated by the replicability of the factor structure for parent data with professional caregiver data. Convergent validity was appropriate, as indicated by significant correlations between scale scores and global ratings thought to be influenced by the respective items (e.g. the child’s quality of life is influenced by his/her symptoms and social participation) (see Table 6).
Correlations of scale scores with global ratings.
For empty fields, correlations were not applicable.
Items were rated on a scale from 1 (very bad)—10 (very good).
Items were rated on a scale from 1 (very pronounced)—10 (not present).
p < 0.05. **p < 0.01.
Discussion
To address the lack of outcome measures for the largest patient group of pediatric palliative care, this study developed and validated a family-centered multidimensional outcome measure for pediatric palliative care, the FACETS-OF-PPC, that is specifically targeted to patients affected by congenital and neurological disorders with severe neurological impairment and their parents. It is the first multidimensional outcome measure for the said population and is specifically designed to focus on aspects relevant to the entire unit of care.
The exploratory factor analysis of parent data of the FACETS-OF-PPC revealed six factors: “symptoms,” “child’s social participation,” “normalcy,” “social support,” “coping with the disease,” and “caregiver’s competencies,” that are comprised of a total of 34 items and jointly accounted for 51.1% of the variance. The factor structure could be confirmed with a confirmatory factor analysis on professional caregiver data indicating a high stability and relevance of factors across target groups and hence factorial validity. The identified factors formed subscales of the FACETS-OF-PPC that showed good to excellent internal consistency. Content and construct validity of the FACETS-OF-PPC were adequate. Overall, it demonstrates appropriate psychometric properties. Five additional items regarding siblings, life partners and further symptoms were included independently of the subscales in both questionnaire versions to ensure multidimensionality.
The FACETS-OF-PPC complies with the recommendations of the EAPC White Paper on outcome measurement. 1 It has been validated for a relevant population—children with congenital and neurological conditions and severe neurological impairment, constituting the largest patient group under the age of 16 in developed countries. 8 Before applying the outcome measure to children with other underlying conditions, it must be validated in a corresponding patient sample. It demonstrates appropriate psychometric properties and, according to the ratings of parents and professional caregivers (see Table 2), is sufficiently concise. Furthermore, both versions of the FACETS-OF-PPC rely on proxy reports. We have refrained from developing a self-report version, simply because the population in question cannot self-report due to the severe neurological impairments associated with their conditions. 9
As recommended, 1 the FACETS-OF-PPC is designed to be multidimensional, assessing the physical, psychological and social outcomes of care. The spiritual domain was not assessed as an outcome. Best et al. 24 present conflicting information on whether or not parents wish for the care team to discuss spirituality. It appears as if spirituality needs to be acknowledged and respected, but not made a subject. As spirituality has not been identified as a relevant outcome by Ribbers et al., 17 we interpret the consideration of spirituality as an indicator of the quality of care, not as an outcome of care. Should the FACETS-OF-PPC ever be complemented by items assessing the quality of the provided care, consideration of religion and spirituality must unequivocally be incorporated.
Contrary to the African C-POS14,15 and its French translation 16 that contain only two items assessing pain and other symptoms, the FACETS-OF-PPC assesses the most common symptoms of the patient population 25 individually. Beyond that, it contains an additional item inquiring about further relevant symptoms. This allows a more detailed assessment of the child’s current physical condition. Moreover, it focusses on the entire unit of care. Additional to the questions focusing on the child, the caregiver and the family as a whole, four items were included that explicitly concern the child’s siblings and life partners of the sick child’s caregiver. Such a focus on all members of the unit of care is indispensable if the FACETS-OF-PPC is thought to contribute to the provision of high quality pediatric palliative care tailoring the need of everyone involved. 10
For the validation of the FACETS-OF-PPC, numerous modes of service provision were included, that is, inpatient care provided by the palliative care unit and hospices, outpatient care provided by specialized pediatric palliative home care teams. Such heterogeneity of the institutions results in the applicability of the outcome measure irrespective of the mode of service provision. Pediatric patients may utilize a variety of different care setting throughout their disease trajectory. 26 Thus, it is of essential importance that the developed outcome measure can be used independently of the current care setting, as recommended by Bausewein et al. 1 This allows for a continuous monitoring of the child’s and his/her family’s well-being.
The remaining recommendations listed in the White Paper that have not yet been addressed relate to the application of the outcome measure in clinical practice and must be considered in the future accordingly. At this point in time, the outcome measure has been shown to be able to display the patient’s and family’s situation at a specific point in time. As outcomes, however, refer to change, future research needs to critically evaluated whether the instrument is responsive to change in order for the tool to qualify as an actual outcome measure. 1
Future research could furthermore examine whether it is feasible for families to indicate which items are currently the most relevant to them. This would enable individualizing the questionnaire on a descriptive level, without impacting the ability to evaluate it in a standardized way. Special attention could then be paid to the change over time in items most relevant to the family’s current situation.
Strengths/limitations
As shown in previous literature, one of the most striking limitations of research in pediatric palliative care is the recruitment of a sufficiently large sample for analysis. 13 According to the rule of thumb of 10 participants per items for exploratory factor analyses, our sample should have consisted of 520 parents and professional caregivers, respectively. Reaching this number in this patient population in a manageable time is impossible. However, even though larger samples are preferred, McNeish 27 demonstrated that reliable results may be obtained with smaller samples as well.
We did manage to include families from numerous and diverse regions throughout Germany as multiple institutions recruited participants for this study. This ensures applicability of the outcome measure independently of geographical region or care setting and thus favors the generalizability of the results. Even though the study furthermore included participants with various nationalities, the FACTS-OF-PPC has only been tested with families that live in Germany and have proficient knowledge of the German language. Hence, before applying the FACETS-OF-PPC with patients from other countries, future research must validate it in corresponding samples.
Last, it must be noted that institutions may not have recruited families solely according to the specified inclusion criteria but might apply their own judgement on whether or not families are too vulnerable to participate. This is known as gate-keeping and is rather prominent in palliative care research. 28
What this study adds
The current study resulted in the first multidimensional family-centered outcome measure for pediatric palliative care for children with severe neurological impairment, the FACETS-OF-PPC. It assesses the physical, psychological and social aspects relevant to the entire unit of care and demonstrates good psychometric properties.
Supplemental Material
sj-pdf-1-pmj-10.1177_0269216320975626 – Supplemental material for Development and psychometric validation of the family-centered multidimensional outcome measure for pediatric palliative care targeted to children with severe neurological impairmentis—A multicenter prospective study
Supplemental material, sj-pdf-1-pmj-10.1177_0269216320975626 for Development and psychometric validation of the family-centered multidimensional outcome measure for pediatric palliative care targeted to children with severe neurological impairmentis—A multicenter prospective study by Sophie Pelke, Julia Wager, Benedikt B. Claus, Boris Zernikow and Mandira Reuther in Palliative Medicine
Supplemental Material
sj-pdf-2-pmj-10.1177_0269216320975626 – Supplemental material for Development and psychometric validation of the family-centered multidimensional outcome measure for pediatric palliative care targeted to children with severe neurological impairmentis—A multicenter prospective study
Supplemental material, sj-pdf-2-pmj-10.1177_0269216320975626 for Development and psychometric validation of the family-centered multidimensional outcome measure for pediatric palliative care targeted to children with severe neurological impairmentis—A multicenter prospective study by Sophie Pelke, Julia Wager, Benedikt B. Claus, Boris Zernikow and Mandira Reuther in Palliative Medicine
Supplemental Material
sj-pdf-3-pmj-10.1177_0269216320975626 – Supplemental material for Development and psychometric validation of the family-centered multidimensional outcome measure for pediatric palliative care targeted to children with severe neurological impairmentis—A multicenter prospective study
Supplemental material, sj-pdf-3-pmj-10.1177_0269216320975626 for Development and psychometric validation of the family-centered multidimensional outcome measure for pediatric palliative care targeted to children with severe neurological impairmentis—A multicenter prospective study by Sophie Pelke, Julia Wager, Benedikt B. Claus, Boris Zernikow and Mandira Reuther in Palliative Medicine
Footnotes
Acknowledgements
We would like to acknowledge all PPC institutions aiding in the recruitment of parents and professional carers for the study: Kinder Palliativ Team Gießen, SAPV Augsburg, SAPV Dresden, KinderPaCT Berlin, KinderPaCT Hamburg, Kinderhospiz Syke, “Der Weg nach Hause” Bielefeld, “Kinder- und Jugendhospiz Bethel” Bielefeld, SAPV Datteln, Station Lichtblicke Datteln
Authors’ contributions
S.R., J.W., B.Z., and M.R.: contributed to study design; S.R. and M.R.: data acquisition; S.R., B.B.C., and M.R.: data analysis; S.R. and M.R.: drafted the article; J.W., B.B.C., and B.Z.: critical revision of the article; S.R., J.W., B.B.C., B.Z, and M.R.: approved the manuscript version to be published.
Data availability
The corresponding datasets of this study are available from the corresponding author on reasonable request.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval and informed consent
The study was approved by the Ethics Committee of the Witten/Herdecke University, Germany (ID 206/2017) and was conducted in accordance with the ethical principles specified in the Declaration of Helsinki.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Federal Ministry of Education and Research [grant number 01GY1717].
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
