Abstract
PURPOSE:
The Measure of Processes of Care for Service Providers (MPOC-SP) is a valid and reliable instrument to measure the professionals’ perception of the extent to which they apply the principles of family-centered services in care for children with disabilities. This study aimed to evaluate the validity and reliability of the Korean translation of the MPOC-SP (Korean MPOC-SP).
METHODS:
The Korean MPOC-SP was completed by 132 rehabilitation service providers in 5 provinces in South Korea. Analyses for internal consistency, construct validity, and test-retest reliability were performed.
RESULTS:
The estimates of internal consistency (Cronbach’s alpha) of the four scales of the Korean MPOC-SP ranged from 0.67 to 0.92. All the scales correlated highly with the other scales (r ranging from 0.61 to 0.77). In addition, all the items exhibited high item-total correlations (rs ranging from 0.40 to 0.83). Three scales had moderate to good reliability with ICCs ranging from 0.57 to 0.78; the ‘providing general information (PGI)’ scale showed low reliability (ICC 0.22).
CONCLUSION:
The Korean MPOC-SP can be considered a valid instrument for group-level research purposes with acceptable internal consistency, but caution is warranted regarding the low test-retest reliability of the PGI scale.
Keywords
Introduction
Family-centered service (FCS) is an approach to healthcare delivery wherein service providers collaborate with families to embrace their unique features and needs [1, 2]. In FCS, service providers and families develop partnerships, share information, and make shared decisions in a respectful and supportive manner [3, 4]. FCS not only benefits children by improving child development, psychological adjustment, performance of functional tasks, and goal achievement but also benefits parents by augmenting parent engagement with the intervention, parental emotional well-being, and satisfaction with the healthcare service [3, 5–9]. Accordingly, FCS can significantly contribute to positive child and family outcomes and is considered the best practice in early intervention and pediatric rehabilitation [1, 10].
FCS is widely adopted by service providers and healthcare organizations; therefore, several measures have been developed to evaluate the family-centeredness of healthcare services provided to children and their families. These measures include the following: Enabling Practices Scale [11]; the Family-Centered Program Rating Scale [12, 13], the Family-Provider Relationship Instrument [14]; and the Measure of Processes of Care [15–17]. Among these measures, the Measure of Processes of Care (MPOC) is a well-validated and reliable tool that is most widely used [1, 19]. Additionally, the MPOC has different versions to measure the family-centeredness of services from the perspective of both parents and service providers.
The original version of the MPOC is a 56-item questionnaire (MPOC-56) that assesses the parents’ perception of the extent of the family-centeredness of the services provided to their children. A shorter and thus, more user-friendly version (MPOC-20) consists of a 20-item questionnaire derived from the MPOC-56. Both MPOC-56 and MPOC-20 have demonstrated good psychometric properties [15, 16] and have been translated and validated for use in many countries, including Japan, the Netherlands, Norway, Slovenia, and Sweden [20–24]. The Measure of Processes of Care for Service Providers (MPOC-SP) is a 27-item self-assessment tool for service providers that measures the extent of the family-centeredness of their services [17]. The MPOC-SP also has good internal consistency, validity, and test–retest reliability, and likewise is validated for use in different countries and with various populations [25–28].
As FCS emphasizes collaborative processes between service providers and families [2, 18], it is important to reflect on the various points of view of families and service providers for evaluating and subsequently improving the quality of FCS. Parents and service providers may perceive the implementation of family-centered behaviors differently. Thus, understanding both parents’ and service providers’ perceptions can provide valuable insights on the practice of FCS. This information can help identify a discrepancy in perceptions of the process of healthcare service, develop training or educational programs, and improve the quality of service.
Since FCS is gaining ground in Korea, it is important to evaluate the family-centeredness of healthcare services using a psychometrically sound measure. Because the MPOC was used internationally, it was deemed appropriate to develop a Korean version of the MPOCs. The MPOC-56 and MPOC-20 have been translated into Korean, and a previous study has evaluated their psychometric properties and provided evidence of good construct and concurrent validity, internal consistency, and test–retest reliability [29]. This study aimed to validate the Korean translation of the MPOC-SP by assessing internal consistency, some aspects of validity, and test-retest reliability.
Methods
Participants
A convenience sample of rehabilitation service providers was recruited through hospitals, rehabilitation clinics, and community healthcare centers from the provinces of Seoul, Gyeong-gi, In-cheon, Daegu, and Gyeong-sang in South Korea. A package was sent to service providers through program managers at each individual institution. Each included an invitation letter, a consent form, a demographic questionnaire, and the Korean MPOC-SP. In order to obtain retest information, one institution was asked to keep a list of names and respondent numbers. All service providers on the list received another MPOC-SP with their unique identification number three weeks after the initial assessment. The questionnaires were sealed in a separate envelope, collected at each center, and then handed to the researcher in person or by post. This study was approved by the Institutional Review Board of the U1 University in South Korea. Data were collected between April 2014 and March 2016.
Korean translation of the MPOC-SP
The MPOC-SP is a self-reported measure for pediatric service providers that assesses the extent of family-centeredness of the services provided by them [17]. The MPOC-SP is based on the MPOC-56 and often used in conjunction with the parent-completed MPOC. The MPOC-SP contains 27 items categorized into one of the following four scales: 1) showing interpersonal sensitivity (SIS, 10 items); 2) providing general information (PGI, 5 items); 3) communicating specific information about the child (CSI, 3 items); and 4) treating people respectfully (TPR, 9 items). Response options range from 1 (‘not at all’) to 7 (‘to a very great extent’) with 0 standing for ‘not applicable’. The items scored 0 are considered invalid. A scale score is calculated as the mean rating of the items in each scale, such as adding the valid responses for the items in the scale and dividing the sum by the number of valid items. Therefore, the scale score can range from 1 to 7.
The original Canadian MPOC-SP was translated into Korean following the guidelines provided by the CanChild Centre for Childhood Disability Research that developed the MPOCs [30]. Translation was performed by the authors. Subsequently, eleven service providers (5 physical therapists, 4 occupational therapists, and 2 speech therapists) who were not involved in the study reviewed the translation to determine if the Korean translation was readable and easy to understand. Based on their feedback, some minor changes in wording were made. The Korean translation was then translated back into English by an occupational therapist who was fluent in English and Korean and was not involved in this study. After minor revisions, CanChild approved the Korean MPOC-SP.
Statistics
Descriptive statistics were computed, then psychometric properties were assessed within the classical test theory (CTT) framework. Internal consistency for each of the four scales of the Korean MPOC-SP was assessed using the Cronbach’s alpha coefficient. An alpha-value between 0.70 and 0.80 is deemed acceptable, and that≥0.80 indicates a good internal consistency, showing that the items of the scale measure about the same aspect [31, 32]. Spearman coefficient (rs) was used to assess correlations between each item score and its own scale score from which the respective item was excluded (item-total correlations). Pearson coefficient (r) was used to assess correlations among the four scales. Test–retest reliability was assessed using the intraclass correlation coefficients (ICCs, two-way mixed-effects model, absolute agreement), with values between 0.5 and 0.75 indicating moderate reliability, values between 0.75 and 0.9 indicating good reliability, and values > 0.90 representing excellent reliability [33]. All statistical analyses were performed using the IBM SPSS statistics software version 25.
Results
Of the 170 recipients, 132 returned the package with responses (77.6% response rate). Twenty service providers completed the Korean MPOC-SP on two occasions. The demographic characteristics of the participants are outlined in Table 1. Most respondents were female (81.8%), and a majority (93%) were in their 20 s and 30 s. Disciplines that were most highly represented in this sample were physical therapy (50.8%), occupational therapy (33.3%), and speech-language pathology (12.9%). On average, service providers had 5 years of experience in pediatrics (range, 4 months–25 years).
Demographic Characteristics of Participants (N = 132)
Demographic Characteristics of Participants (N = 132)
Statistical analyses were performed with data of 132 respondents, but the number of respondents included in each analysis varied due to missing or invalid items. Table 2 demonstrates the descriptive statistics and internal consistency for each of the four scales in the Korean MPOC-SP. Although some of the mean scores tended towards the higher end of the range, the distributions were not severely skewed (–0.08 to –0.26). The TPR scale was rated the highest (Mean = 5.12, SD = 0.82), whereas the PGI scale received the lowest ratings (Mean = 3.83, SD = 1.27). The TPR, SIS, and PGI scales showed good internal consistency (alpha≥0.88), and the CSI scale showed moderate internal consistency (alpha = 0.67).
Descriptive statistics and internal consistency for MPOC-SP
MPOC-SP = measure of processes of care for service providers; Ni = number of items; Ns = number of respondents; max = maximum; min = minimum; SD = standard deviation; α= Cronbach’s coefficient alpha. Mean scale scores can range between 1.00 and 7.00.
The 27 items of the Korean MPOC-SP significantly correlated with their own scale scores with the respective item excluded, rs ranging from 0.40 to 0.83 (p < 0.01). All the scales correlated significantly with the other scales (p < 0.01) (Table 3). Three had moderate to good reliability, with ICCs ranging from 0.57 to 0.78, yet the PGI scale showed poor reliability (Table 4).
Correlation between the MPOC-SP scales using Pearson correlation coefficient (r)
*Correlation with p < 0.01 (2-tailed).
Test-retest reliability of MPOC-SP (N = 20)
ap < 0.01, bp > 0.05.
Service providers rated the item ‘answer parents’ questions completely’ the highest (mean score 5.4), which was followed by the item ‘trust parents as the “experts” on their child’ (mean score 5.3). Both items belong to the TPR scale. On the other hand, there were six items with a mean score of 4 or lower (indicating ‘moderate extent or less’), including all five items of the PGI scale and one item of the CSI scale, ‘provide parents with written information about their child’s condition, progress, or treatment’.
The results suggest that the Korean MPOC-SP is a valid instrument for group-level research purposes with adequate scale structure and moderate to good reliability overall. However, caution is warranted regarding the low test-retest reliability of the PGI scale. Descriptive statistics demonstrated that the patterns of higher and lower scale scores in this study are comparable with those of the Canadian, Dutch, and Japanese studies [17, 26]. The TPR scale ranked the highest, whereas the PGI scale ranked the lowest. Overall, the scale scores were lower than those reported in the Canadian and Dutch studies [17, 26]. The higher scores reported in the Canadian and Dutch studies may be in part due to the inclusion of professionals whose primary role is to guide and support their clients, such as social workers, case managers, and teachers. Another possible reason for the lower scale scores in the study is that the majority of participants were young and had a relatively short clinical experience of less than 5 years.
Korean service providers perceived that they do a better job of providing the interpersonal aspect compared to the informational aspect of care; this is consistent with Korean parents’ perception [29]. The service providers reported that they performed the activities included in the PGI scale only to a moderate extent or less. This corresponds with a low mean score on the PGI scale of MPOC-56 reported by parents participating in the same study. In addition, compared with the MPOC-56 results, the mean scores of MPOC-SP on all scales were lower, which may indicate service providers’ critical attitudes towards or low confidence in their own behavior.
The analyses of the scale structure indicated that the items in each scale were grouped together in a meaningful manner. The correlation coefficients suggest that individual items are best correlated to their own scale and that the four scales are related to each other. Additionally, the Cronbach alpha values indicate that the items within each scale measure the same aspects of care, at least for the SIS, PGI, and TPR scales.
When the Korean MPOC-SP was retested in 3 weeks, all scales except for the PGI scale showed moderate–good ICC values, which indicated that the Korean MPOC-SP has acceptable stability. The low stability of the PGI scale might be due to recent events, such as participation in this self-assessment process. During the first completion of the Korean MPOC-SP, service providers became aware of the ideal ‘family-centered’ behaviors, which presented an educational opportunity. After the first completion, service providers might have paid more attention to improving low-score behaviors (e.g., 5 items in the PGI scale) or reflected on an ideal behavior rather than their actual behavior during the second completion. Another explanation is that there is no clear agreement on who should provide general information. Unlike other scales, the PGI scale evaluates the extent to which a service provider or organization has displayed each behavior/situation (e.g., questions in that scale begin with “To what extent did you or your organization . . . ”). Therefore, any recent events by therapists, programs, teams, or organizations may affect the subsequent responses to these items; thus, some may think that providing general information is not their role. In fact, the frequency of ‘0’ (not applicable response) was higher in the PGI scale compared to the other scales.
Limitations and future research
This study has several limitations. First, the PGI scale demonstrated low test–retest reliability for unclear reasons. For test–retest reliability, the test conditions should be similar at both times of completion. However, respondents were not asked if there were any variations in the conditions between the two measurements, or reasoning behind their individual responses on each item. Similar to any other self-assessment tool, it is unclear whether service providers responded based on the reflections of their actual behavior or ideal behavior. In addition, the sample size for the test-retest reliability was small. Another limitation is that the composition of the study participants was different from that of the original study in which service providers from more diverse disciplines, including nursing, case management, and social work, were involved [17]. Since almost all participants (97%) in this study were in physical therapy, occupational therapy, and speech-language pathology, there is a limit to applying study findings to the entire health care service. Studies involving larger and diverse samples of healthcare providers, cross-discipline comparisons, and factors that influence service providers’ implementation of FCS will further validate the Korean MPOC-SP and help in better understanding the family-centeredness of healthcare services in Korea. So far, only the CTT approach has been applied to the MPOC-SP, therefore applying the item-response theory (i.e., Rasch analysis) would be beneficial to evaluate and possibly improve the instrument.
Conclusion
Although FCS has become the standard of care in pediatric health care, the implementation of this approach, particularly in the informational aspects, appears to be a challenge for service providers in Korea. The Korean MPOC-SP can be used by service providers and healthcare organizations to identify areas for improvement and subsequently improve the quality of care. Evidence-based educational and supportive strategies to facilitate family-centered behaviors will benefit both service providers and families who receive these services. Although the Korean MPOC-SP has reasonable psychometric properties in general, the test–retest reliability of the PGI scale is low. Therefore, it may be insufficient to use the Korean MPOC-SP alone for program evaluation and quality assurance activities. When evaluating changes on family-centered behaviors after educational or training programs on the group level, it is recommended using the Korean MPOC-SP in conjunction with other measures of family-centered service such as the MPOC-56.
Footnotes
Aknowledgments
We would like to thank the service providers who generously participated in this study. No funding was obtained for this research.
Conflict of interest
The authors have no conflict of interest to report.
