Abstract
Background:
Health in children is multifaceted. More than just the absence of disease, good child health also comprises psychological well-being and social functioning. In paediatrics, optimal health care is a partnership between caregivers and doctors. Value-based medicine strengthens this partnership. Monitoring and tracking these patient-reported outcomes using validated measurement tools in a life-course manner helps us to understand and ultimately improve overall child health better.
Objectives:
This study aimed to assess parental opinions in Singapore on recommended outcome measures and approach recommended by the International Consortium for Health Outcomes Measurements (ICHOM) Overall Pediatric Health (OPH) working group.
Methods:
An international working group comprising 25 experts in paediatric health was formed under the auspices of ICHOM. Through a series of literature reviews, patient focus groups and open review periods, a set of health outcomes and case-mix variables was proposed, with the aim of improving paediatric care globally. Parents and caregivers were invited to participate in this survey from around the world, including from the USA, UK, Colombia and Singapore. The survey covered domains that were included in the ICHOM OPH Standard Set. An international anonymised online patient validation survey tool online survey was initiated to evaluate caregiver opinions of these proposed ICHOM OPH outcomes, and this was conducted from November 2019 to January 2020. Singapore participated in this caregiver survey on overall paediatric health outcomes. Of the 270 respondents in this ICHOM OPH consumer international open review online survey, 160 were from Singapore. We report here a cross-sectional study with data from the Singapore group of respondents.
Results:
Of the 160 responses from Singapore, 73.8% had no more than two children, and 38.8% of caregivers had children between 0 and 5 years old. In general, 76.9% of caregivers were satisfied with the health care that they received, and 60% of caregivers had had between one and four health-care appointments in the last 12 months. Assessment was done on a scale of 1 to 9, with importance defined as a score of ⩾7. All outcome measures were ranked to be important, with an overall mean score of 7.9 on a scale ranging from 1 to 9. Development was assessed as being the most important (M=8.34±0.93), and sexual health was assessed as being the least important (M=7.09±1.71). Almost all respondents felt that the list of ICHOM-recommended outcomes proposed was complete. Parents and caregivers in Singapore placed development and mental health as the most important across all age groups surveyed. They felt that the framework provided by the ICHOM OPH working group was adequate in addressing their concerns. Most parents and caregivers were satisfied with the health care they had received.
Conclusion:
Child health is a continuum that requires longitudinal attention from both the family and the clinician, and should take a life-course approach, utilising validated measurement tools to monitor and track key parameters. Parents and caregivers in Singapore place high importance on their child’s development and mental functioning. While there has been much community effort on development and mental functioning in children, more can be done in these two areas and to increase the emphasis and awareness on sex education both within the home and in schools. A framework for evaluating children during follow-up clinic visits would be helpful in continuing longitudinal care of children to provide holistic care and to improve adult health in the long term, guided by the framework set down by the ICHOM OPH standard set. As we recognise and embrace the importance not just of health care but the health of the next generation, we will need to use validated measurement tools to monitor and track these importance parameters in the journey of our children, so that we can identify and improve these outcomes in a timely manner.
Introduction
The World Health Organization defines health as ‘a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity’. 1 Indeed, health is not just the absence of disease, but rather transcends physical dimensions and encompasses the psychosocial elements and more, incorporating the ability to self-manage and adapt.2,3
It is widely recognised that many precursors to adult health and disease are evident and present in childhood. Adverse childhood environmental exposures, poor health-related behaviour and high-risk states are detrimental towards adult health. 4 Early identification and optimal management of pre-disease states and mental well-being are important in reducing the burden of subsequent disease in adulthood. Paediatricians and health-care providers taking care of children are important constants in the longitudinal care of the child. Their role extends beyond managing active medical issues to providing pre-emptive care and anticipatory guidance at each touch point, with the core aim being to modify child, and subsequently, adult health favourably.
In Singapore, the bulk of child health visits occur in the primary-care setting, where there is the opportunity to offer longitudinal care and pre-emptive management. In today’s increasingly busy world, clinicians are finding it challenging to identify and tailor their approach towards addressing the unique needs of each patient. Disease-specific needs are sometimes placed above the holistic well-being of the child, resulting in shortfalls in identifying important disease precursors that would otherwise be modifiable.
The International Consortium for Health Outcomes Measurement (ICHOM) is a non-profit organisation that aims to unlock the potential of value-based health care by defining global standard sets of outcome measures that really matter to patients and the population by driving the adoption and reporting of these measures worldwide to enable learning, sharing and ultimately improving true value-based medicine (VBM). 5
The ICHOM standard sets are derived through Delphi surveys and consensus from international expert panels. Case-mix factors and variables including demographic data such as age, sex as well as socio-economic factors such as employment and educational levels are identified. Case-mix variables are important to provide a clearer context and to allow for risk adjustments for subsequent comparison and benchmarking between different cohorts and for better and more accurate analysis. ICHOM has to date set up 28 standard sets covering different conditions and for specific patient populations.
An international ICHOM working group comprising 25 experts in paediatric health was formed to look at overall paediatric health (OPH). Through a series of literature reviews, patient focus groups and open review periods from 2017 to 2020, the first international and interdisciplinary standard set of health outcome measurements with case-mix variables that focused on the overall bio-psychosocial health of any infant, child or adolescent between 0 and 24 years old during all developmental stages was developed. 6 The group also recommended established and validated measurement tools to measure these proposed health outcomes. This selection of outcome measures was validated via a caregiver-filled questionnaire.
This OPH standard set offers health-care providers a comprehensive framework for outcome assessment that is tailored by age and development to each child, bearing in mind that child health extends beyond physical lack of disease to include psychological and social well-being. A set of baseline demographics is recorded at first encounter. Outcome measures covering a broad range across mental, social and physical functioning (Table 1) are collected via validated caregiver-reported questionnaires, comprising mainly PROMIS questionnaires and scales (Table 2). Objective measurements such as growth and nutritional status as well as visual acuity are also recorded based on standard measures such as the World Health Organization growth charts and the Snellen chart, respectively. Selected measures are then repeated at various intervals, depending on the age of the child, resulting in enhanced continuity of personalised care, adopting a life-course approach to tracking and monitoring the overall health of the child.
Overview of outcomes assessed in the ICHOM framework.
Overview of tools used during monitoring and frequency of monitoring using ICHOM Framework.
Methods
As part of the overall process of developing the OPH standard set, the OPH working group engaged with patients and caregivers, including patient advocacy groups.
Caregivers were surveyed on the final proposal of health outcomes in the OPH standard set through its consumer open review period via an online anonymised survey. This anonymised online survey was initiated to evaluate caregiver opinions of these proposed ICHOM OPH outcomes and was conducted between November 2019 and January 2020. Caregivers worldwide were invited to participate in this survey. Respondents were from the USA, UK, Colombia and Singapore. The consumer open review planned to obtain about 200 responses from each participating country. KK Women’s and Children’s Hospital (KKH) Singapore, as one of the supporting organisations for this ICHOM OPH initiative, reached out to our local paediatric caregivers through this anonymised online survey as part of its patient and caregiver quality-improvement and quality-control process. A thousand brochures were disseminated for recruitment in wards and outpatient areas Caregivers of children attending outpatient clinics or admitted to KKH were invited to take part in this anonymised online survey (see Annex A and Annex B). This study was undertaken as a quality-improvement initiative, and no identifiable data were collected as part of the survey. Participation in the survey was voluntary, and avenues were provided for queries pertaining to the survey, although no queries were subsequently received.
Caregivers were asked to evaluate: general functioning outcomes, including development, functioning and health-related quality of life; social functioning outcomes such as communication and school attendance; physical functioning outcomes such as vision, hearing, growth, nutrition, eating, sleep, being pain free, mobility and fitness, sexual health and survival; and mental functioning outcomes such as mental health, mood, self-efficacy, self-esteem, coping and cognition. Assessment of these outcomes was done on a scale of 1 to 9, with 1 being ‘not important’ and 9 being ‘most important’, and importance being defined as a score of ⩾7.
The ICHOM OPH consumer open review online survey had a total of 270 respondents. Of these, 161 (59.6%) were from Singapore, with an initial projected aim of 200 responses. We report here the data from the Singapore group of respondents.
Results
There were 161 respondents in the Singapore group of respondents to the international anonymised online survey. We excluded one from our analysis due to contradictory responses and likely misinterpretation of the survey, whereby all outcomes were scored as ‘not important’, despite the overall impression that the list of outcomes provided was complete and adequate. A total of 160 responses were therefore analysed. Most respondents (73.8%) had no more than two children (Figure 1), and 38.8% of caregivers had children between 0 and 5 years old (Figure 2). In general, 76.9% of caregivers were satisfied with the health care that they received (Figure 3), and 60% of caregivers had had between one and four health-care appointments in the last 12 months (Figure 4).

Number of children cared for per caregiver.

Age ranges of children.

Satisfaction of caregivers with health care received.

Number of health-care appointments received over last 12 months.
Overall, all outcome measures were ranked to be important by the respondents, with an overall mean score of 7.9 on a scale ranging from 1 to 9 (Table 3). Development was assessed to be most important (M=8.34±0.93), while sexual health was assessed to be least important (M=7.09±1.71). Almost all respondents felt that the list of proposed outcomes by the ICHOM OPH working group was complete and comprehensive (95.6%; 153 respondents), and the rest (4.4%; 7 respondents) proposed that the following aspects should be included: identification of potential health problems, social well-being, evaluation of accessibility to health care, genetic factors and disease-specific outcomes. These proposed outcomes from the respondents were subsequently discussed by the ICHOM OPH working group.
Importance of outcomes.
Degree of importance for various outcomes reflected as score on scale from 1 to 9, with 1 being ‘not important’ and 9 being ‘most important’.
Percentage of parents who deem outcome as important. Importance is defined as a score of ⩾7.
We further stratified the responses according to the age groups of children cared for by our respondents (Figure 5). Development and mental health remained the most importance across all age groups, with scores ranging from 8.1 to 8.48 on a scale from 1 to 9. School and sexual health were deemed to be of lesser importance by parents and caregivers, even in the adolescent age group, with scores ranging from 6.69 to 7.39. There were no obvious differences in caregiver opinions on each outcome between the age groups, with scores remaining similar across the different groups.

Caregiver opinion on outcome importance. Importance is defined as achieving score of ⩾7 on a scale of 1–9, with 1 being ‘not important’ and 9 being ‘most important’.
Discussion
VBM is defined as ‘the practice of medicine incorporating the highest level of evidence-based data with the patient-perceived value conferred by health-care interventions for the resources expended’. Value should always be centred around the patient.7–10
A key to patient-perceived value is what matters and is of value to the patients and caregivers themselves. Having meaningful patient-reported outcomes and robust tools and questionnaires to measure these outcomes is crucial to practicing true VBM in paediatrics.
This survey shows that parents and caregivers in Singapore place most importance on the development of their child. This is true across all age groups. Development encompasses holistic development as a whole person, extending across physical, emotional and spiritual health. The tempo of physical development is largely predictable, with most children achieving normal developmental milestones without much issue. This framework proposes using validated measurement tools such as the CDC Milestone checklist from age 0–5 years for developmental surveillance. Beyond this age, physical, emotional and cognitive development is assessed via a combination of PROMIS Parent Proxy Scales, and beyond 12 years of age, patient-filled PROMIS questionnaires are used. In Singapore, there are guidelines in primary health-care centres to detect developmental delays in children, starting in infancy. Assessment is based on a combination of the Denver Developmental Screening test and clinician reviews from 0 to 4 years of age. In recent times, measures to improve these screening practices have been initiated and are currently underway. Children from vulnerable families are also identified as early as at birth and followed up within an integrated community health and social care support system via the KIDS 0–3 and KidSTART programme, which seeks to optimise their developmental potential, track behaviour problems and strengthen family function through regular home visits and centre-based activities. 11 Those with developmental concerns are then referred to a tertiary centre for diagnosis and intervention.
Beyond four years of age, formal developmental screening tools are not implemented, and any physical, emotional or cognitive developmental issues would be flagged up by schools. It is thus worthwhile integrating this framework into routine health-care visits or school health services in order to continue holistic developmental screening beyond the age of four years.
Caregivers also recognise the importance of mental functioning, even at a young age. Mental functioning encompasses not just mental health and well-being, but also cognition/intelligence, mood, coping skills, self-efficacy and self-esteem. This is especially important, given that the incidence of mental illness in Singapore has been increasing in recent years, especially in youth. 12 The number of children between five and nine years of age seeking help for mental-health issues has also increased by fivefold within three years. 13 There has been an increase in suicide rates in those aged 10–19-years. 14 One possible contributing factor to mental illness in children in recent years has been the increased exposure to social media. There have been associations shown between the use of social media and self-esteem and body-image issues, 15 and improper usage may also increase cyberbullying and social isolation in teens.
In Singapore, childhood mental-health services are provided within the community and tertiary health-care centres. The Response, Early Intervention and Assessment in Community mental Health (REACH) programme caters for children between the ages of 6 and 15 years, consisting of school-based multidisciplinary teams of doctors, psychologists, medical social workers, occupational therapists and nurses. It provides assessment and intervention for students referred by the school counsellor for mental-health concerns,16,17 and is a channel for referral to tertiary-care centres for support and intervention.
Students also receive talks at 13–14 years of age covering coping mechanisms and new transitions at 15–16 years of age. 18 Educators of children aged 13–18 years undergo workshops in how to pick up mental-health issues in students and how to provide support and knowledge regarding available referral services. Other measures are currently in place to increase mental-health education in school such as establishing peer-support structures, creating mental-health resources for students within the school curriculum and delivering cyber-wellness programmes. 19 Measures such as adopting a whole-school approach, involving parents, providing more health education materials and resilience programmes have been studied with some benefits. 20 Creating a positive environment at school has the potential to impact mental health positively and even to be protective against negative social factors. 20 This helps to reduce any stigma associated with mental illness, encouraging more children to seek help actively. 21 The efficacy of these interventions can be promoted and even increased whilst concurrently providing a protective environment within the home. Identifying at-risk homes and providing parents with education on mental-health issues and parenting resources to improve parent–child relationships is an important step.
Having a framework for patient-centric and patient-reported outcome assessment also allows for standardisation of care across health-care institutions, which would be helpful in interdepartmental or inter-institution quality-improvement studies. It also allows for identification of gaps in paediatric health provision and education.
According to our survey, sexual health was recognised as less important across all age groups, even among caregivers with adolescent children. This may be due to the fact that the large majority of our respondents were caregivers of pre-teen children (Figure 2). Nevertheless, sexual well-being is important. In Singapore, the burden of sexually transmitted diseases in teenagers has been increasing in recent times. 22 In addition, adolescents who were sexually abused are at increased risk for resultant psychiatric disorders such as anxiety, post-traumatic stress, major depressive disorders 23 and even suicide attempts. 24
Sex education extends beyond instruction in anatomy and physiology of reproduction to healthy interpersonal relationships, sexual development, consent issues and intimacy. 25 Formal sexual health education in Singapore occurs mainly in schools and is abstinence based via mass talks and classroom lessons. This commences at age 11–12 years through to 17–18 years old, encompassing human development, interpersonal relationships, sexual health and behaviour, and education on the impact of legal positions. These aim to help students make informed decisions regarding sexuality and develop healthy relationships based on love and respect. 26 Sex education interventions can prevent or reduce the risk of adolescent pregnancy and sexually transmitted diseases. 27 Increasing the emphasis of sex education in schools can be a first step in reducing its resultant morbidity, but as many children and adolescents may not be engaged sufficiently or be comfortable enough in school to integrate this knowledge meaningfully, education should also start at home with the family, in conjunction with the paediatrician. 28 Parents should be mindful to initiate early open discussion with their children about sexual practices that are in line with their attitudes and values. Here, we recognise that parental awareness and involvement can be increased in sex education in children.
Our study has several limitations. We could not ascertain a response rate due to the mode of dissemination and the nature of the survey. Exact patient demographics could not be determined due to how the survey questions were framed. Information on socio-economic status, parental education and co-morbidities of children were not collected, which limit its applicability to the general population. There is the potential for selection bias, as the population surveyed was linked to a tertiary centre. There was no information on whether the respondents were staff, parents of patients or otherwise. However, as this survey sought parental opinions on general paediatric health, this should not have much of a confounding influence on the results and perceptions of caregivers locally. Applicability could be enhanced if the survey was administered in a larger population, especially in the primary-care setting which sees a large proportion of well-child visits. This international anonymised online survey was conducted online with respondents mainly from developed countries, which limits the applicability of the results to developing countries. Responses may also be limited to the English-proficient population and those more familiar with Internet use due to the web-based nature of the survey. While a more holistic surveillance would be more useful from an overall perspective, from both developed and developing countries, it can be argued that the follow-on, execution and ultimate intent to monitor and track OPH of the population would be easier in developed countries with the necessary resources and infrastructure in place. As shared above, there was a lower proportion of families with children in the adolescent age group (18.1%), which may have skewed results towards the views of families with younger children, possibly contributing towards lower importance scores for sexual health. Respondents who cared for more than one child were not asked to give responses specific to each child, so their views may have been more generalised.
We also did not have access to data collected from the other international sites that participated in the anonymised online survey, and thus we had no opportunity to evaluate if there were any differences between our local cohort and the rest of the international respondents.
One of the strengths of our study is that the outcomes surveyed were holistic and included the psychosocial aspects of child health. These results show that the majority of parents and caregivers surveyed concurred with the recommendations from the international expert panel on the important parameters to monitor and track for OPH. While we cannot comment on whether they are representative of our population with the various limitations in this surveillance, we believe that this strong concurrence from our respondents does show that these parameters recommended by ICHOM in its OPH standard set are important parameters to be tracked and monitored, and will help us as health-care practitioners caring for children to understand and improve the health of our paediatric population better.
Child health is a continuum that requires longitudinal attention from both the family and the clinician and should take a life-course approach, utilising validated measurement tools to monitor and track key parameters. Parents and caregivers in Singapore place high importance on their child’s development and mental functioning. While there has been much community effort on development and mental functioning in children, more can be done in these two areas and to increase the emphasis and awareness on sex education both within the home and in schools.
A framework for evaluating children during follow-up clinic visits would be helpful in continuing the longitudinal care of children in order to provide holistic care and to improve adult health in the long term, guided by the framework set down by the ICHOM OPH standard set.
As we recognise and embrace the importance not just of health care but also of the health of our next generation, we will need to use validated measurement tools to monitor and track these important parameters in the journey of our children, so that we can identify and improve these outcomes in a timely manner.
Supplemental Material
sj-docx-1-psh-10.1177_2010105820979324 – Supplemental material for What matters to patients: Parental and caregiver opinions on International Consortium for Health Outcomes Measurements overall health outcomes for children in Singapore
Supplemental material, sj-docx-1-psh-10.1177_2010105820979324 for What matters to patients: Parental and caregiver opinions on International Consortium for Health Outcomes Measurements overall health outcomes for children in Singapore by Moira Suyin Chia, John Yeow Yang Wong, Irene Pheck Mui Chan and Kee Chong Ng in Proceedings of Singapore Healthcare
Footnotes
Acknowledgements
Special thanks to the team from the Office of Patient Experience, KK Hospital, for taking charge and facilitating this patient quality-improvement project.
Authors’ contributions
N.K.C. participated in formulation of survey used for data collection. J.W. and I.C. were involved in data collection. M.C. was involved in data analysis and wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Availability of data and materials
The data sets generated and/or analysed during the current study are available from M.C.
Conflict of interest
The authors declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: N.K.C. is a member of the International Consortium for Health Outcomes Measurement (ICHOM) ‘Overall Paediatric Health’ Workgroup 2018–2020.
Ethical approval
Not applicable.
Funding
The authors received no financial support for the research, authorship and/or publication of this article.
Informed consent
Survey participants were asked to give their consent prior to doing the survey (Annexe A).
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
