Abstract
Background
Almost a quarter of children and young people (CYP) in England have a long-term health condition (LTC), which increases the risk of developing mental health difficulties. There is a lack of understanding regarding the routine provision and efficacy of mental health interventions for CYP with LTCs within Children and Young People’s Mental Health Services (CYPMHS).
Methods
This study analysed national service-reported data in England from two secondary datasets. Data were submitted by services between 2011 and 2019. We evaluated data on the presence or absence of a serious physical health or neurological issue, and which interventions were offered.
Results
A total of 789 CYP had serious physical health issues and 635 had neurological issues. The most common interventions delivered to CYP in either group have some evidence in the literature. Most CYP showed improvements across a range of outcomes.
Conclusions
This study found that prevalence rates and psychological intervention and outcome data were widely under-reported across both datasets, posing questions about their utility for this population. Such data would benefit from triangulation with data from other sources to understand pathways of care for these young people and the extent to which clinical datasets underreport the number of CYP with LTCs.
Plain language summary
Almost a quarter of children and young people (CYP) in England have a long-term health condition (LTC), such as asthma, diabetes, or epilepsy. We know that these young people are at increased risk of developing mental health difficulties. It is important these young people are able to access safe and effective treatments for their mental health. Therefore, they are sometimes referred to Children and Young People’s Mental Health Services (CYPMHS) for appropriate treatment. However, at the moment, not much is known about the types of mental health support these services offer to children with co-existing physical health needs, or if this support is effective. The aim of this study was to try and find this out. We used data that had already been collected from mental health services across England. We looked at specific parts of this data that gave us information about the type of mental health treatments delivered to children with a long-term health condition. We separated long-term health conditions into two categories: physical health, such as diabetes or asthma; and neurological, such as epilepsy. In the sample we looked at, a range of mental health treatments were delivered to young people in both groups. Encouragingly, many of the young people’s mental health improved. However, a lot of information we would hope to find was not available in the datasets. Also, the number of children with a long-term health condition was much lower than we expected. This might have been for a number of reasons, which we recommend other future research tries to find out. Going forward, it is important to think about how to make sure that accurate information about these children is collected from mental health services. This will help ensure that the right decisions are made for the care of young people with long-term health conditions.
Keywords
Introduction
Up to 23% of children and young people (CYP) in England have a long-term health condition (LTC) (Brooks et al., 2015). CYP with LTCs are much more likely to develop mental health issues than the general population (Blackman et al., 2011; Pinquart & Shen, 2011a, 2011b, 2011c), with latest reports suggesting that at least one quarter of young people with a LTC in England also have a probable mental health condition (NHS Digital, 2021), compared with around 16% in the general population (NHS Digital, 2021). Co-morbid psychiatric and physical symptoms can negatively impact the course and management of the physical health condition (de Araujo Filho & Yacubian, 2013; Sildorf et al., 2018). There are also financial implications for service providers, with an estimated 12%–18% of NHS costs for long-term physical health conditions linked to poor mental health (Naylor et al., 2012).
The level of collaboration between paediatric and psychological services addressing mental health needs in CYP with LTCs exists on a spectrum (Schlesinger et al., 2023) from complete separation of services, where an external referral to CYPMHS is required to co-located yet distinct services (Platt et al., 2018), to fully integrated psychology and psychiatry provision within physical healthcare clinics (Burkhart et al., 2020). Of those referred to mental health services in England, up to three quarters of CYP with LTCs have unknown referral outcomes (Children’s Commissioner, 2016). For those who do access treatments, the provision of services can often be inconsistent (Welch et al., 2018). There are increasing calls for the full integration of physical and mental health treatment for CYP with LTCs (Cottrell, 2015; Fazel et al., 2021) and there is also a need to incorporate multiple data sources to properly understand pathways of care for these young people.
There are a range of evidence-based treatments to address common mental health conditions in CYP. For example, the National Institute for Health and Care Excellence (NICE) guidelines recommend Cognitive Behavioural Therapy (CBT) for children with depression (NICE, 2019) and anxiety (NICE, 2014; NICE, 2013a) and parenting interventions for children with conduct disorders (NICE, 2013b). At present, there is only emerging evidence for the efficacy of these standard mental health interventions in CYP with LTCs (Bennett et al., 2015), although some argue that there is no reason they would not also be effective in this population (Cottrell, 2015). Overall, the paucity of evidence regarding the efficacy of mental health interventions for this group may make it difficult for services to decide on a course of action.
Given the unique challenges of living with a long-term health condition, approaches to addressing mental health symptoms in CYP with additional physical needs can be complex. Categorical approaches to mental health treatments depend on a young person meeting stringent diagnostic criteria so that the corresponding treatment can be prescribed. However, the dynamic severity of physical health needs when living with a LTC means that mental health needs may be correspondingly fluid, thus potentially not meeting diagnostic criteria but nonetheless causing a significant impact on the person’s life. This can present additional challenges around providing appropriate treatment, especially given known unmet needs even when a diagnosis is present (Ott et al., 2003). Mental health symptoms may also be attributed solely to the physical health issue (diagnostic overshadowing) and opportunities for referral to appropriate psychological services may be missed.
Given many CYP with LTCs are referred to CYPMH services (Children’s Commissioner, 2016), it is vital to understand which interventions are being offered to this group while under mental health services and how effective they are. This will help decision-making around whether standard approaches are effective for this group, or whether such children require more specialist mental health interventions, in order to support service planning. As well as clear benefits to the wellbeing of children and families, there is a health economic argument to ensuring that CYP with LTCs access the most effective mental health treatments for them at the right time. Early intervention in this group may impact not only on mental healthcare outcomes (Bennett et al., 2015; Moore et al., 2019) but also on physical healthcare outcomes (de Araujo Filho & Yacubian, 2013) with resultant savings to mental and physical healthcare services (Layard & Clark, 2015). Indeed, it has been argued that such are the benefits to physical health, that treating mental health disorders ‘costs the NHS nothing’ (Layard & Clark, 2015). Conversely, rejected referrals and ineffective treatments may cause conditions to persist and thus remain costly to health services (Naylor et al., 2012).
The aim of this study was to use existing data from two national databases from Children and Young People’s Mental Health services in England, the Child Outcomes Research Consortium (CORC+) and the CYP Improving Access to Psychological Therapies (IAPT) datasets, to investigate: • What type of psychological interventions are delivered to CYP who also have LTCs within CYPMHS? • What are the mental health outcomes of these CYP following a psychological intervention?
Materials and Methods
Participants and Procedure
The Child Outcomes Research Consortium (CORC) is an organisation that collects evidence from health services across the UK to improve mental health and wellbeing support for CYP. For this study, we used routinely collected child mental health services data gathered by CORC within the CORC+ and CYP IAPT databases. Cases were submitted by services between 2011 and 2019 (although services could choose the timeframes for the data they submitted, so the start dates for some of the cases could be prior to 2011). The variables were the same across both databases.
CYP from these datasets were included in our analysis if they had data on the presence of an LTC, defined in the database as a ‘serious physical health’ or ‘neurological’ condition, and the psychological intervention(s) they accessed (see Measures section). We analysed the group of CYP with neurological conditions separately due to the known significant elevation of mental health disorders in this group.
Data provided by mental health services was analysed in the form of ‘cases’, which are defined as a child or young person’s period of contact with a service. Services could optionally record interventions used in each case using a pre-specified list of intervention types from the CORC + dataset specification (as reported in Results section). This research did not require ethical approval as it involved secondary analysis of anonymous administrative data (Health Research Authority, 2021).
Measures
Outcome data from a range of mental health measures were collected from the combined datasets for analysis. The range of measures recorded by services are listed below.
Current View tool
The presence or absence of a serious physical health or neurological issue was identified using the Current View tool, which is a clinician-rated form in use within CYPMHS. The form involves rating the number of presenting problems, complexity and contextual problems, and school, work or training difficulties that the young person is experiencing (Jones et al., 2013). The clinicians could use the form to record whether the CYP had one or both of: ‘Serious physical health issues, including chronic fatigue’, or ‘Neurological issues, e.g. Tics or Tourette’s’. If the Current View form was completed more than once during a case, the last rating was taken. The form did not allow for the role of the clinician completing the form to be recorded.
Strengths and Difficulties Questionnaire (SDQ)
The Strengths and Difficulties Questionnaire (SDQ) is a 25-item behavioural screening tool for children, which can be child-, parent-, or teacher-reported (Goodman, 1997). In this study, only child- and parent-reported SDQ scores are analysed. The questionnaire comprises 5 sub-scales (emotional symptoms, conduct problems, hyperactivity-inattention, peer relationship problems, and prosocial behaviours), with each of these scales consisting of 5 items. A supplement of the SDQ that is also used in this study is the SDQ Impact score, which looks at the duration of symptoms and the impact that a child’s behaviour has on their everyday life.
Revised Children’s Anxiety and Depression Scale (RCADS)
The Revised Children’s Anxiety and Depression Scale (RCADS) is a 47-item questionnaire assessing the levels of anxiety and depression in children and young people, which can be self- or parent-reported (Chorpita et al., 2000; Spence, 1997). It includes five subscales for specific anxiety disorders: separation anxiety disorder; social phobia; generalized anxiety disorder; panic disorder; and obsessive-compulsive disorder; as well as one for low mood (major depressive disorder). Two scores can be generated from the questionnaire: the ‘Total Anxiety’ score is a sum of the five anxiety subscales, and the ‘Total Internalising Scale’ is the sum of all six subscales.
Goal Based Outcomes (GBOs)
Goal Based Outcomes (GBOs) are used to evaluate the progress that a young person or their family/carer is making towards goals set at the beginning of an intervention. A numerical scale of 0–10 (0 = no progress made; 10 = goal has been fully reached) is used to assess the progress the individual has made. Clinicians help children to set these goals, which can comprise a wide range of aims for them and their families (Law & Jacob, 2013).
Children’s Global Assessment Scale (CGAS)
The Children’s Global Assessment Scale (CGAS) is a clinician-reported questionnaire rating children and young people’s functioning (Shaffer et al., 1983). Children are given a numerical score (0–100) based on the clinician’s assessment of different aspects of their functioning, such as social and psychological. This numerical score will place the young person in one of ten groups, ranging from ‘extremely impaired’ (1–10) to ‘doing very well’ (91–100).
Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA)
The Health of the Nation Outcome Scale for Children and Adolescents (HoNOSCA) is a clinician-reported outcome measure assessing CYP’s general health and social functioning (Bilenberg, 2003). It includes 15 items indicating the severity of a range of problems, each rated on a scale of zero (no problem) to 4 (severe to very severe problem). The HoNOSCA is divided into two sections: the first assesses different types of problem the child or young person may have, and the second addresses the individual’s knowledge of the difficulties they face and the services available to them.
Analytic Strategy
Data Availability for Interventions and Presence of Serious Physical Health and Neurological Issues
Cases with data on both the use of a psychological intervention and the presence/absence of an LTC were then divided into the following sub-groups: (1) serious physical health issues recorded as present; (2) serious physical health issues recorded as not present; (3) neurological issues recorded as present; and (4) neurological issues recorded as not present. Presence of physical health issues or neurological condition were not mutually exclusive options and so some CYP are likely to be present in both groups. Data for CYP without either condition type were only available for interventions, therefore comparisons of outcomes were only possible for sub-groups (1) and (3).
Range of Interventions
Services could record ‘Yes’, ‘No’ or ‘Missing’ against a range of intervention types for each event during a CYP’s period of contact, such as an appointment. An intervention type was counted if ‘Yes’ was recorded for one or more events associated with that child. e.g., if a service reported that a CYP had only one session of CBT, then that would count as having used CBT. The number of cases for which each intervention was reported was extracted and the percentage of cases this number represented was calculated. This was conducted for each of the four sub-groups listed above.
Changes in outcome scores
Paired data (i.e. same outcome measure completed at two or more time points) were required to include outcome data for a participant. We refer to the first time point when a measure was completed as ‘T1’ (Timepoint 1) and the last point as ‘T2’ (Timepoint 2). Mean changes in score were calculated by subtracting the score at T1 from the score at T2. No minimum length of time between T1 and T2 was necessary for a change to be calculated. Margins of error were calculated both for the average change in outcome measure score and the average T1 and T2 scores.
Results
Out of 229,216 recorded cases in the combined CYP IAPT and CORC + datasets, 35,110 (15%) CYP had data on the use of a mental health intervention. Of these, only 789 CYP were recorded as having serious physical health issues and 635 as having neurological issues. Only a very small number of the CYP in these sub-groups also had outcome data at two timepoints, meaning we were unable to match interventions to outcomes on an individual level because the sub-samples were too small for meaningful analysis. Unfortunately, outcome data for CYP without either condition were not available so comparisons of dropout rates in the ‘with’ and ‘without’ groups between T1 and T2 were also not possible. Figure 1 (Supplementary material) shows how the data were identified. Furthermore, a total of 14,159 and 14,059 CYP were reported as not having a physical health or neurological condition, respectively. However, there was a large crossover of participants between these two sub-samples, with 13,261 CYP included in both groups. Flow chart of interventions and outcome data availability in the area of chronic illness.
Demographic information of CYP across all four groups.
Range of Interventions in Use
CYP with and without Serious Physical Health Issues
Range of interventions for CYP with and without physical health issues.
CYP with and without Neurological Issues
Range of interventions for CYP with and without neurological issues.
Outcomes
CYP with Serious Physical Health Issues
Changes in score for CYP with physical health issues.
CYP with Neurological Issues
Changes in score for CYP with neurological issues.
Discussion
This study analysed two national datasets to examine the range of psychological interventions delivered to children and young people with serious physical and neurological issues within CYPMHS. It also aimed to evaluate the mental health outcomes of these young people to understand the effectiveness of standard evidence-based interventions for CYP with LTCs.
Out of 229,216 cases in the combined CYP IAPT and CORC + datasets, a total of 789 CYP were flagged as having serious physical health issues and 635 CYP as having neurological issues, although these groups were not mutually exclusive. This reflects a very small number of CYP who had data on the presence or absence of a LTC in relation to the total database sample size and the known prevalence of LTCs in children and young people of around 23% (Brooks et al., 2015). This further emphasises previous evidence of unknown referral outcomes in this group (Children’s Commissioner, 2016) and highlights the need to ensure that this information is captured accurately within clinical settings, particularly given the increased mental health needs of this group and the sparse extant evidence base. These difficulties with incomplete data pose questions about the utility of these datasets for this population. One of the aims of CORC+ and CYP IAPT is to standardise data reporting, however the level of incompleteness that we encountered when focusing on CYP with LTCs suggests improvements are still needed when it comes to this group. Future work may be needed to explore barriers to data completion in clinical settings. It may be that detailing the importance to clinicians of clinical data reporting for research and service development could improve reporting. Previous research has found widely under-reported outcome measures in UK mental health settings (Batty et al., 2013), which was seen in this study, with staff training a potential key facilitator to ensuring higher levels of completion (Sharples et al., 2017). However, this is specific to measures and does not necessarily account for the low rates of additional health needs reported in this study.
In addition to reflecting difficulties with data completion, these low numbers may also reflect low referral acceptance rates to CYPMHS, so further research and/or service audits are desperately needed to understand what happens to mental health referrals for this group of young people. As described earlier, approaches to addressing mental health symptoms in CYP with additional physical needs can be complex and children and young people may have mental health needs that would benefit from support, but which may not meet strict diagnostic criteria required for some services. From the data investigated in this study alone, it is not clear whether any of the following apply: children and young people have not had mental health needs identified, perhaps due to diagnostic overshadowing or other factors; the mental health needs were picked up but they were referred to or treated by other services, such as paediatric psychology services; the mental health needs were picked up and the child was referred to CYPMHS but not accepted; or the mental health needs were picked up, the child was referred to CYPMHS and the referral accepted, but the data on whether they had a LTC was not recorded. A number of further avenues of investigation would be required to begin to have a more complete understanding of these pathways, for example triangulating other datasets from non CYPMHS providers. Specifically, data from paediatric psychology providers could perhaps be collated via the Paediatric Psychology Network, who focus on advancing the field of Paediatric Psychology across multiple sectors in the UK, including in professional practice, clinical governance, policy, research, and training. At the same time, further intervention research should be conducted and data should be collected from all services about the outcomes of different types of interventions in order to improve our understanding of the effectiveness of psychological interventions for CYP with long-term health conditions – e.g. the extent to which standard evidence-based interventions for mental health difficulties may be effective in this group (see Catanzano et al., 2020; Moore et al., 2019).
For the data that were available, we found that the mental health services within the datasets delivered a wide range of mental health interventions across all four population groups. However, most interventions were recorded as ‘Other’ for all groups, making them difficult to categorise. This is of potential concern as it may suggest the use of interventions that are not evidence-based. Due to limitations in the data recording process, we were not able to obtain any further details on these interventions.
However, within the data available the most commonly specified intervention for CYP with serious physical health issues was Cognitive Behavioural Therapy (CBT), while Parent Training was the equivalent for those with neurological conditions. This is promising as CBT is the NICE recommended intervention for anxiety (NICE, 2014; NICE, 2013a) and one of the NICE recommended treatments for depression (NICE, 2019), which are two of the most common mental health disorders in CYP (NHS Digital, 2017). Importantly, though, CBT is evidence-based only when a minimum set of criteria are conformed to and the data collection process of this study meant that there was no way of guaranteeing that these quality standards were met, i.e., CYP were recorded as having had CBT even if only one session was completed. Previous research in other clinical populations has found that treatments labelled as CBT often lack the fundamental components required by the evidence base (Cowdrey & Waller, 2015; Stobie et al., 2007; Waller et al., 2012). Also of note was that those CYP with a physical health condition were less likely to be given CBT than those without, which is consistent with previous research and may reflect clinicians’ lack of confidence in using CBT with this group (Byrne et al., 2023; Carroll et al., 2021). Behavioural parenting interventions are the NICE recommended treatment for disruptive behaviour disorders (NICE, 2013b) and these are frequently seen in CYP with neurological disorders (Davies et al., 2003).
Most CYP showed improvements across all psychological outcome measures in both groups, suggesting a positive response to the treatments that are being delivered. There is some evidence in the wider literature that CBT is effective at treating mental health symptoms in CYP with chronic physical illness (Bennett et al., 2015) and evidence that points towards the effectiveness of Parent Training interventions on emotional and behavioural outcomes in children with neurological disorders (Chalfant et al., 2007). Although we could not link effectiveness data with specific interventions, the high proportion of evidence-based interventions adds to the literature suggesting standard evidence-based interventions are effective for those with LTCs, including CYP with neurological conditions.
However, it is important to note that the data also show a deterioration in mental health scores in a sizeable subsample across the range of measures, which bears further investigation. Comparatively, a recent meta-analysis showed a reliable deterioration rate of 6% in CYP without LTCs who accessed routine specialist mental health care (Bear et al., 2020). The deterioration seen in the current study may be due to the nature of living with a physical illness, given that symptom severity can fluctuate and in turn be associated with fluctuations in scores on mental health measures, despite mental health improving. These physical changes, which may be unrelated to current mental health symptom severity, could be captured by items such as ‘I have trouble sleeping’ or ‘I have problems with my appetite’ on the RCADS, or ‘I get a lot of headaches, stomach-aches or sickness’ on the SDQ. This is the reason the Hospital Anxiety and Depression Scale (HADS; Zigmond & Snaith, 1983) is used in physical healthcare services for adults, as it accounts for somatic symptoms on other measures. It is important for future research to understand why some children with LTCs deteriorate over the course of a mental health treatment, with particular focus on the suitability of measures and assessing harms as well as benefits of standard interventions; the former has been explored in the context of CYP without LTCs, though the evidence base is sparse (Batic & Hayes, 2020; Jonsson et al., 2016). In addition, many CYP did not have T2 measures, and it is possible that this represents attrition from the intervention; therefore, in addition to accurate recording of such attrition, further work may be needed to understand the acceptability of standard evidence-based interventions in this group.
Goal-Based Outcomes (GBOs), whereby CYP work collaboratively towards targets they have set for themselves, was the measure by which the highest percentage of young people showed improvement. This could be because goal-based outcomes may be more likely to capture change than other measures. Goal-setting is recommended for use alongside any intervention (Law & Jacob, 2015), which may explain why it was so common in our sample. Goal-setting can also improve retention and engagement within youth mental health services (Cairns et al., 2019), thus leading to potential improvements in further clinical outcomes.
There are several strengths to this study. First, our findings add to the overall evidence of both the routine mental health provision and outcomes for this vulnerable and otherwise underreported population. Second, this is one of the largest samples looking at interventions for CYP with LTCs receiving care under CYPMHS. Treatment pathways have previously been unclear for many of these young people after they are referred for treatment, but it seems that those who are getting into services have been provided with, what appear to be, evidence-based interventions – i.e., CBT and Parent training. This study is one of the first to investigate the outcomes of referral to CYPMHS and highlights key gaps in our current understanding and the need for further research and audits in the area. The NHS Long Term Plan highlights the need for integrated mental and physical healthcare (NHS, 2019) and there is an urgent need to understand this area in its entirety to ensure optimal outcomes for children, young people and their families as well as the health service.
Limitations
This study has a number of limitations. First, the incompleteness of the datasets for CYP with LTCs limited our findings. Notably, the recorded rates of physical and neurological issues were far below expected prevalence, mental health outcome data were very sparse, and the large number of interventions recorded as ‘Other’ made it difficult to accurately determine which intervention type was indeed the most common. Future research should explore triangulation of these datasets with others (e.g. via the UK Paediatric Psychology Network) to understand pathways of care for these CYP, in addition to barriers and facilitators to data reporting in clinical settings and look to close these gaps. Second, limitations in the data reporting process meant that some key details about intervention delivery were not available, making it difficult to assess treatment fidelity and to what extent the interventions were truly evidence-based. The length of time between first and last treatments was also unclear, which precluded any evaluation of short- or long-term effects. Third, the majority of the CYP across all groups were White, suggesting that the results may not be generalizable across more diverse sub-samples. Last, we were unable to pair interventions with outcomes on an individual level due to small sample sizes, thereby limiting any conclusions about the effectiveness of specific interventions. This means that it is unclear whether the young people receiving these ‘evidence-based’ interventions were the ones who saw positive changes in their mental health. Future work should focus on establishing which psychological interventions are effective in this population, possibly using data linkage at an individual level, including acceptability measures, e.g. dropouts.
Conclusion
Current practices for addressing mental health issues in children with serious physical and neurological conditions within mental health services are unclear. Clarifying this is an important step to ensuring that these CYP receive safe, effective, and evidence-based treatments. This study found considerable likely under-reporting of prevalence rates and mental health data across both datasets, which poses questions about their utility when examining this population. Future work should explore the extent to which this is true under-reporting, through triangulation with datasets from other services, including paediatric psychology services, and causes of and solutions to under-reporting in order to ensure maximal validity and utility of clinical data. However, the available data showed that, promisingly, evidence-based CBT and Parent Training were the two most commonly reported interventions, although many intervention types remained unclassified. Furthermore, our analysis of outcome data suggests that many young people receiving these and other interventions showed improvements in their mental health over the course of their treatment. This is important as, consistent with other emerging literature (Catanzano et al., 2020), it suggests that standard evidence-based interventions can be effective in this group. Overall, our findings suggest that these children should not be turned away from CYPMHS and should be offered these standard evidence-based interventions as a first port of call where appropriate. However, this should be supported by future studies using larger samples to more accurately map the range of interventions in use for CYP with LTCs across services, as well as evaluating the effectiveness of these approaches for this population using robust research methods.
Supplemental Material
Supplemental Material - Mental health interventions for children and young people with long-term health conditions in Children and Young People’s Mental Health Services in England
Supplemental Material for Mental health interventions for children and young people with long-term health conditions in Children and Young People’s Mental Health Services in England by Thomas King, Gladys CM Hui, Luke Muschialli, Roz Shafran, Benjamin Ritchie, Dougal S Hargreaves, Isobel Heyman, Helen Griffiths, and Sophie Bennett in Clinical Child Psychology and Psychiatry
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the All research at Great Ormond Street Hospital NHS Foundation Trust and UCL Great Ormond Street Institute of Child Health is made possible by the NIHR Great Ormond Street Hospital Biomedical Research Centre. TK (Primary Author) a PhD student funded by Great Ormond Street Child Health Research studentship. No specific funding was received for this study.
Supplemental Material
Supplemental material for this article is available online.
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
