Abstract
Keywords
For decades, breast cancer screening has been viewed in a mostly positive light, despite voices from the beginning raising issues related to its harms. Public communications about breast cancer screening have tended to use persuasive techniques aimed at maximizing uptake. 1 False positives were considered the main disadvantage, and benefits were believed to outweigh risks. The focus in the scientific literature on the harm of overdetection, also known as overdiagnosis, is more recent. An independent meta-analysis of 11 screening mammography trials found that the percentage of screened women to experience overdiagnosis was 20% 2 and that for every breast cancer death prevented by screening, 3 breast cancers were overdetected (a 1:3 ratio). In Australia, the ratio was found to be 1:6. 3 In the Netherlands, a reversed ratio of 2:1 was reported, based on a comparison of regions where screening was introduced at different time points, as well as on modeling studies. 4 Given the evidence of overdiagnosis, an individual may rationally choose not to screen, and informed choice has become a major stated goal of information provision in screening programs. Information on overdetection has been added to online brochures and/or websites in, for example, the United Kingdom, Canada, Australia, and the Netherlands. Further, decision aids have been developed that include information on overdetection, to help women make informed choices based on their values and preferences. 5 Despite the increase in information available on overdetection, awareness of this downside of screening remains limited.6–8 Particularly if one has attended screening before, a revised brochure that now includes overdetection information may be left unread. Qualitative studies have assessed women’s views on overdetection, showing that women are often surprised or even upset when they hear about it.9,10 Studies that assessed acceptance of overdiagnosis quantitatively have shown widely varying ratios, for example, from 50% accepting at maximum a 1:1 ratio when starting screening, to 60% of previous screeners accepting a 1:10 ratio. 11 Van den Bruel et al. 12 added a limited description of the consequences of overdetection in terms of treatments that patients would undergo following diagnosis. Wide variation was again seen, with some respondents accepting no overdetection and some accepting as many as 1000 overdetected cases per 1000 screened. These studies did not assess whether acceptance was sensitive to the consequences of overdetection in terms of treatment. They did not give participants sufficient information to judge the impact of treatment either, thus leaving overdetection as a rather abstract disadvantage of screening.
We set out to examine whether providing explicit information on the nature of invasive treatment, with descriptions of important associated side effects, would lead to women making a tradeoff of benefit and harm. For example, women may accept screening at lower ratios of overdetection for a more invasive treatment scenario. Further, apart from age and education, no correlates of attitudes toward overdetection have been assessed. It may be expected that other characteristics such as experience with screening (and false-positive results) or with breast cancer in family or friends, as well as risk perception and anticipated regret, will affect acceptance. Given the continuing controversy in the screening field, it is important to obtain more insight into women’s willingness to accept overdetection and the personal characteristics that may influence this.
The aim of our study was therefore to assess the acceptance of overdetection in breast cancer screening and to determine if acceptance is sensitive to the treatment expected to follow overdetection. We also assessed whether acceptance was associated with women’s understanding of overdetection, their screening history, and psychological characteristics.
Methods
Study Design
The study consisted of an online survey in which 4 treatment scenarios were shown to participants in randomized order. The survey was sent to an online panel by Kantar (formerly Lightspeed GMI), an international survey company with an actively managed research panel, recruited using various methodologies (e.g., co-registration, email, and social media campaigns) in November 2016. Participants received points for participation.
Sample
The survey company randomly sampled women from their panel from the general Dutch and Australian public, both eligible (50–75 years) and almost eligible (45–50 years) for breast screening in the national programs. We oversampled the younger category, who were less likely to have a screening history. We excluded women with a breast cancer diagnosis and those not able to read Dutch or English. Our sample size was based on a rule of thumb of 20 women per variable that we wished to correlate with overdetection acceptance, and we oversampled to be able to assess simple interactions as well. We therefore aimed for about 400 women per country (i.e., 120 in the 45- to 50-year-old category and 60 in each other 5-year age category).
Survey
The online survey and randomization scheme were created using Qualtrics survey software. The questionnaire began with sociodemographic questions (age, marital status, education, major daily activities, children, and if so, age of youngest child), followed by questions about acceptance of overdetection and psychosocial correlates of acceptance. We explained the benefits and harms of screening, including the concept of overdetection 5 (see Box; for entire explanation, see Supplementary Appendix 1).
Explanation of the Concept of Overdetection in the Survey
Treatment Scenarios
We chose the most prevalent breast cancer treatments in the screening population.2,13,14 To restrict the survey length, we included adjuvant therapy only for the lumpectomy scenario, not for that of mastectomy, assuming its impact would not differ between the 2 surgical options. We used the following 4 scenarios in the survey, presented in randomized order: 1) mastectomy without adjuvant therapy, 2) lumpectomy without adjuvant therapy, 3) lumpectomy followed by radiotherapy, and 4) lumpectomy followed by radiotherapy and hormone therapy. For each scenario, we described the treatment process and short- and long-term side effects (Supplementary Appendix 2). After presenting baseline information on breast screening (number of breast cancer deaths avoided by screening 1000 women biennially over 25 years, the concept of overdetection, and the fact that treatment is recommended because overdetected cases cannot be individually identified), the 4 treatment scenarios were presented. For each scenario we, presented 5 pairs of options to choose from, each with “No Screening” on the left, showing a fixed number of breast cancer deaths (19), and “Screening” on the right, depicting a fixed number of breast cancer deaths (14) and breast cancer deaths prevented (5). 3 The number of overdetected cancers varied for each pair, from 0 to 30 cancers overdetected. For each pair, we asked women to indicate their preference for no screening or screening. We used pictographs to indicate the number of breast cancer deaths prevented, overdetected, and cured out of 1000 women (see Figure 1 for an example). The number of cancers overdetected was varied in the same order for each treatment scenario. First, no cancers were overdetected, to capture participants’ general attitude toward screening; women choosing not to screen were directed to the questionnaire assessing psychosocial correlates. Second, 5 cancers were overdetected, a conservative estimate alongside the 5 deaths avoided. Third, 30 cancers were overdetected (Australian estimate of overdetection 3 ). Fourth, 15 cancers were overdetected (Independent UK Panel estimate 2 ). Lastly, 2 cancers were overdetected (Dutch estimate 4 ).
Pictographs used in the paired-choice questions to depict the overdetection ratio (example provided for situation with 1:1 overdetection ratio [5 deaths prevented: 5 cancers overdetected] and treatment involving mastectomy). In the survey, the pictures were in color, and these are available from the first author upon request.
Measures
After completing all 5 pairs for each of the 4 scenarios, women filled out a questionnaire on potential correlates of acceptance. We assessed screening experience (mammography experience inside/outside the program, whether ever called back for further testing [including biopsy]), experience with breast cancer in family/friends, 15 prior beliefs and attitudes toward screening 5 (6-item scale, Cronbach’s alpha = 0.93), subjective norm 5 (4 items on what one assumes that important others’ views are, Cronbach’s alpha = 0.80), perceived risk of developing breast cancer 5 (2-item scale, Cronbach’s alpha = 0.80), 1 item on worry, 5 2 items on anticipated regret of screening and of not screening, 5 and the perceived seriousness of breast cancer. 15 Intention to screen 5 was measured at baseline and after the scenarios. Women’s comprehension of overdetection was assessed by 1) asking which of 2 statements is correct (correct: screening finds a cancer that would never have caused trouble; incorrect, referring to false positive: screening finds an abnormality but extra tests show it is not cancer), with a “Don’t know” option, 5 and 2) a 7-item true/false scale. 5 Finally, we asked 3 health literacy 16 and 3 numeracy items. 17
Pilot Study
To assess clarity, understanding, and feasibility of the questionnaire, we first piloted the survey in Australia and the Netherlands in a heterogeneous convenience sample of 6 to 10 women per country. In a cognitive interview, women completed the online questionnaire while thinking aloud, in the presence of an interviewer, who at the end posed debriefing questions. The questionnaire was then shortened slightly to the version above, in light of the piloting process.
Data Analysis
We performed descriptive analyses of respondent characteristics and compared the Australian and Dutch samples using t tests, analyses of variance, or nonparametric tests, where relevant. For each scenario, we created a variable with the highest overdetection ratio at which the respondent would accept screening (and above which she would switch to no screening), with levels 0 = never screen, 1 = screen at 2 overdetected, 3 = screen at 5 overdetected, 4 = screen at 15 overdetected, 5 = screen at 30 overdetected. We assessed univariate associations between these acceptance scores for the 4 scenarios and sociodemographics, psychological variables, and comprehension. Next, we tested whether there was an order effect, as an indication of a learning effect during the task, by comparing acceptance scores by a variable indicating which scenario was shown first. Finally, we performed multivariate analyses of variance, using general linear models, in which we assessed the effect of scenario (within-subjects) and the variables that showed an association (P < 0.10) in the univariate analyses (between-subjects), as well as possible interactions.
Funding and Ethics Approval
The funding source had no role in the study. The study protocol was reviewed and approved by the medical ethics committee of Leiden University Medical Center (protocol number P16.030). The study was performed in accordance with the Declaration of Helsinki.
Results
Participant Characteristics
We received valid data from 853 respondents, 422 Australian and 431 Dutch women. We deleted those who did not answer the acceptance questions for all 4 scenarios, leaving 400 Australian and 403 Dutch women. Table 1 shows the sociodemographic characteristics, experience with screening and breast cancer, and psychological variables. Compared with Australian women, more Dutch women had either low education or a university degree (P < 0.001) and lower health literacy and numeracy (91 v. 69, and 60 v. 43, respectively, both P < 0.001). Dutch women more often had mammography outside the screening program (35% v. 29%, P = 0.007) and had more family or friends with breast cancer (62% v. 42%, P < 0.001). Because of a programming error, the proportion of women who had ever been recalled after a positive mammogram was captured for the Australian sample only (29%). Baseline intention to screen was higher in the Netherlands (4.5 v. 4.2 on a 1–5 scale, P < 0.001) but decreased significantly after the survey (P < 0.001), such that afterward, no significant differences between countries were seen (Netherlands 4.3, s = 1.0 v. Australia 4.1, s = 1.2, P = 0.85).
Characteristics of the Women in the 2 Samples
P < 0.01; **P < 0.001.
Subjective norm, worry, and perceived risk were all significantly higher in the Netherlands than in Australia.
Acceptance of Overdetection: Scores and Sociodemographic Correlates
Since the data were highly skewed toward screening, we regrouped the overdetection variables into 4 categories: 0 = always screen (our highest ratio 30:5), 1 = screen depending on level of overdetection (15:5, 5:5, and 2:5), 2 = screen only if no overdetection (0:5), and 3 = never screen. Figure 2 shows the proportion of women falling into each category for each of the treatment scenarios. We tested whether there was an order effect. Indeed, a trend was seen for acceptance at 30:5 to be highest for the scenario shown first (P value lowest for lumpectomy, P = 0.08). Using multivariate analysis of variance, we saw only a slight effect (P = 0.12) for treatment scenario, with women accepting screening at the highest levels of overdetection for lumpectomy.
Acceptance of screening by treatment scenarios.
We did not see consistent differences in acceptance between the Australian and Dutch samples. Australian women tended to accept screening slightly more at the 30:5 ratio but also more often chose “never screen,” whereas Dutch women more often selected the categories in between (i.e., depending on the ratio). No associations were seen between acceptance and age, education, marital status, major daily activities, health literacy, or numeracy. The only significant effect seen was a slight one for having children, for the mastectomy and lumpectomy + radiotherapy + hormones scenarios: women with children were more likely to always accept screening, whereas those without children more often chose never to screen.
Acceptance of Overdetection: Experience with Screening and Psychosocial Correlates
Experience with the screening program was related to acceptance: those who had previously attended had a higher acceptance of overdetection, for all scenarios (Mann-Whitney U test, all P < 0.001). However, the effect was seen only for the extreme categories: those with experience were more likely to always choose screening, those without to never choose screening (for mastectomy, e.g., 53% of experienced women chose always to screen v. 39% of inexperienced, and 8% of experienced women chose never to screen v. 21% of inexperienced). No effect was seen for having been recalled for further tests, experience with mammography outside of the screening program, a history of breast biopsy, or having a friend or relative with breast cancer.
Women had very positive attitudes toward screening (
Acceptance of Overdetection: Correlation with Understanding
No between-country differences were seen in the item assessing understanding of overdetection. In both countries, only 33% correctly answered this item, with two-thirds choosing the false-positive description. The 7 true-false items were summed and rescaled to a 0 to 100 scale. Women in the Netherlands had better comprehension than those in Australia (
A better comprehension of overdetection correlated with a more negative stance toward screening: specifically, we observed a positive correlation with anticipated regret of screening (ρ = 0.34) and negative correlations with anticipated regret of not screening (ρ = −0.28); subjective norm (ρ = −0.25), attitude toward screening (ρ = −.21), and baseline intention (ρ = −0.20).
Those who correctly understood overdetection showed a lower acceptance of overdetection: for the mastectomy scenario, 37% of women who correctly picked the overdetection description would always screen versus 52% of women who (incorrectly) picked the false-positive description (P = 0.001). For lumpectomy, these figures were 49% versus 58% (P = 0.11); for lumpectomy + radiotherapy, 38% versus 54% (P < 0.001); and for lumpectomy + radiotherapy + hormones, 37% versus 54% (P = 0.001). Spearman correlations between the comprehension scale and acceptance of overdetection similarly varied between −0.35 and −0.40 for the 4 scenarios.
Multivariate Analysis
We performed multivariate analyses of variance to assess the effect of scenario (within-subjects), controlling for the variables that showed an association (P < 0.10) in the univariate analyses (between-subjects). In the first analyses, we did not include attitude, subjective norm, or anticipated regret of not screening, because we felt these to be proxies for acceptance. In the first model, with scenario, country, children, and screening experience, and first scenario shown, acceptance was associated with scenario (P < 0.001) and screening experience (P < 0.001). Acceptance decreased slightly from lumpectomy to lumpectomy + radiotherapy, mastectomy, and lumpectomy + radiotherapy + hormones (estimated marginal means of 0.81, 0.92, 0.96, and 0.97, respectively, with lower scores indicating higher acceptance). When adding comprehension of overdetection to the model (P = 0.002), the effect of children reappeared (0.04), and experience remained (P < 0.001). We also saw an interaction between scenario and the first scenario that was shown, a proxy for an order effect (P < 0.001). We therefore repeated the analyses separately for each of the 4 scenarios when shown as the first one; an effect of scenario on acceptance remained (all P < 0.01, except for lumpectomy + radiotherapy as first scenario, P = 0.06). We next repeated these analyses including attitude, subjective norm, and anticipated regret of not screening. These are all variables that we felt to be proxies for acceptance, and we expected that these might therefore explain the variation in acceptance. Indeed, only these 3 variables, and comprehension, remained highly significant (all P < 0.001, except for subjective norm, P = 0.009).
Since the interaction of scenario with the first scenario shown was again highly significant (P < 0.001), we again repeated these analyses separately for each scenario shown first. Similar results were seen, and the effect of scenario remained nonsignificant.
Discussion
We performed a survey in 2 countries to determine whether providing explicit information about one of the downsides of overdetection, the need for invasive treatment with side effects, would lead to a more informed decision, based on a benefit-harm tradeoff. We found the highest acceptance for the lumpectomy scenario and the lowest for lumpectomy plus radiotherapy and hormonal therapy, but the differences between scenarios were only small. Furthermore, most women indicated they would always participate in screening, even with a 1:6 ratio of breast cancer deaths avoided to cancers overdetected. However, 2 out of 3 women did not seem to understand the concept of overdetection, despite the explanation. Regarding sociodemographic predictors, we could not corroborate the findings of Van den Bruel et al., 12 who found that people aged 50 years or older (a proxy for experience with screening) accepted significantly less overdetection, as did people with lower education. Contrary to their findings, but in line with our expectations, experience with the screening program was positively related to acceptance, for all scenarios. This finding seems to point to cognitive justification, or even cognitive dissonance reduction, in women with experience of mammography screening. That is, if one has attended screening, it may be difficult to now concede that it may not have been a good idea. 18 These results support the findings of Wegwarth and Gigerenzer, 11 who showed that knowledge of overdetection affects the starting of screening more strongly than the continuation.
Contrary to what we hypothesized, worry, perceived seriousness of the diagnosis, and perceived risk were not correlated with acceptance. The lack of sensitivity of the acceptance of overdetection to these variables, as well as the relatively low sensitivity to the seriousness of ensuing treatments, was unexpected. For a rational benefit-harm tradeoff, one would expect worry and risk perception to be positively associated with acceptance and seriousness of treatments to be negatively associated. One explanation for these findings is the prevailing social norm to attend screening, which tends to be high in Western countries.19,20 We indeed found an association between acceptance of overdetection and subjective norm, despite minimal variation in the latter due to very high scores. Another explanation of these findings could be the poor comprehension of overdetection. Studies examining the understanding and acceptance of overdetection have shown that women tend not to be familiar with the concepts of overdetection and overtreatment9,10 and that after explanation, many women still demonstrate a lack of understanding. In the current study, 2 out of 3 women chose the description of a false-positive finding when asked to identify a definition of overdetection. These women were more likely to accept screening when overdetection was present. The concept of overdetection is cognitively difficult to grasp, partly because it is undetectable on an individual level and thus understanding it is not intuitive, 1 but also because cancer has a strong connotation of being aggressive.
We anticipated a learning effect from our survey and indeed saw that acceptance of screening at the 30:5 ratio was highest for the first scenario shown, which seems to imply that if women think about it more, they become more critical toward screening. This finding and the strong correlation with comprehension seem to point to a shift toward a more negative attitude when women become more aware of overdetection.
Some interesting differences were seen between the 2 countries. The number of women reporting family members or friends with breast cancer was much higher in the Netherlands (62%) than in Australia (42%). Further, health literacy and numeracy were higher in Australia. These findings could be due to the way women in the survey panels decided to participate in these countries. It may be that in the Netherlands, women with a relative or friend with breast cancer were more motivated to participate in the survey, whereas in Australia, there was interest to participate regardless. The Netherlands also has very high internet coverage, so populations with lower health literacy may have more access to participating than similar populations in Australia.
In a US study by Pappadis et al., 10 almost 9 out of 10 women who were informed about overdetection stated that hearing about overdetection did not influence their screening decision. The authors concluded that informing women of overdetection may not be sufficient to change screening intentions. Similarly, in an Australian randomized trial by Hersch et al., 5 a decision aid explaining overdetection increased knowledge and enabled more women to make an informed choice, compared with a control decision aid omitting this information, but it made women feel only slightly less positive about breast screening and only somewhat reduced screening intentions. It may be that even with (objective) understanding of the concept of overdetection, women’s (subjective) opinions about or even rejection of it influenced their as well as our findings. Indeed, Scherer et al. 21 assessed acceptance of a hypothetical cancer screening test that did not save lives and found that of participants who accurately remembered what they were told (that the test did not save lives), about 30% disbelieved this. The idea that it may not be beneficial to detect cancer early is in contrast with all the messages that women have received over the years. Given the strong “pro screening” public messages in most countries, more is needed for women to make a clear tradeoff than just adding information on overdetection to a brochure. Further, with increased detection rates and the increasing incidence of breast cancer, we will never know if a woman who survives after treatment for a screen-detected cancer had an overdetected tumor or was indeed saved by screening. The general public will assume the latter, which has been explained as the “therapeutic illusion,” a label that Thomas first applied in 1978 to “the unjustified enthusiasm for treatment on the part of both patients and doctors.” 22 When the public believes that screening and treatment are more effective than they actually are, the results can be unnecessary, costly, and even harmful. “Efforts to promote more rational decision making will need to address this illusion directly.” 23 As long as screening organizations do not educate the population on the role of this therapeutic illusion, it will be very difficult for the target population to make a rational benefit-harm tradeoff regarding participation.
It may be argued that overdetected cancers are less likely to be treated with, for example, mastectomy or chemotherapy, but the impact of a diagnosis, followed by lumpectomy, radiotherapy, and possibly hormone therapy, will still have a large impact on women and their families. The diagnosis of breast cancer with its associated fears and the impact of treatment on quality of life are not to be underestimated. 24 A wider public debate is needed in order for women to make a well-considered informed choice. Our data provide a first indication that with more knowledge and understanding, women may move from an uncritical acceptance of screening to a more informed choice. These findings not only are of benefit to policy makers but also indicate that improved information is needed for women considering screening.
Our study has several strengths as well as limitations. We assessed in an in-depth way the impact of presenting the consequences of overdetection on acceptance of screening. We randomized the order of the scenarios, allowing a learning effect to emerge, which provides important information on how women tend to answer these types of surveys. Further, we performed the study in 2 countries and saw similar patterns. The study was done online, which may have reduced the validity of the answers compared with an interview setting but allowed for large samples. We also used hypothetical scenarios, which may not have captured the inherent uncertainty surrounding the decisions in real life, particularly that related to the type of treatment that will be recommended after diagnosis. This may have led to the finding of only a small impact of treatment scenario on acceptance. We did ask respondents to assume they knew, before screening, what treatment would involve, but still, the hypothetical nature may have elicited a different response than in real life. We also decided not to include a chemotherapy scenario, since the Independent UK Panel noted that screen-detected breast cancer is less likely to be treated with chemotherapy. 2 Further, we aimed to explore responses to a range of ratios reflecting prominent published estimates and did not think a higher than 6:1 overdetection-to-death-avoided ratio to be realistic. We therefore cannot conclude whether women would still accept screening at higher rates. Finally, the level of education and health literacy was high, despite an effort from the survey company to obtain a representative sample, which shows that our sample cannot be seen as representative for the general population of women in our countries.
In conclusion, we found a high acceptance of overdetection but also found that it decreased with the impact of treatment on quality of life. Also, it decreased after the first scenario, pointing to a learning effect, with women developing a more critical attitude upon further reflection. Still, 2 out of 3 women seemed not to understand the concept of overdetection, which highlights a challenge to properly informing women about screening. Acceptance was more strongly associated with attitudes toward screening and anticipated regret than with the treatment following overdetection. Our data provide a first indication that with more knowledge and understanding, women may move from uncritical acceptance of screening toward a more informed choice.
Supplemental Material
Stiggelbout_acceptance_of_overdiagnosis_survey_MDM_Revised_August_9_2019.changes_accepted.rjf2.Appendix1.online_supp – Supplemental material for Women’s Acceptance of Overdetection in Breast Cancer Screening: Can We Assess Harm-Benefit Tradeoffs?
Supplemental material, Stiggelbout_acceptance_of_overdiagnosis_survey_MDM_Revised_August_9_2019.changes_accepted.rjf2.Appendix1.online_supp for Women’s Acceptance of Overdetection in Breast Cancer Screening: Can We Assess Harm-Benefit Tradeoffs? by Anne Stiggelbout, Tessa Copp, Gemma Jacklyn, Jesse Jansen, Gerrit-Jan Liefers, Kirsten McCaffery and Jolyn Hersch in Medical Decision Making
Supplemental Material
Stiggelbout_acceptance_of_overdiagnosis_survey_MDM_Revised_August_9_2019.changes_accepted.rjf2.Appendix2.online_supp – Supplemental material for Women’s Acceptance of Overdetection in Breast Cancer Screening: Can We Assess Harm-Benefit Tradeoffs?
Supplemental material, Stiggelbout_acceptance_of_overdiagnosis_survey_MDM_Revised_August_9_2019.changes_accepted.rjf2.Appendix2.online_supp for Women’s Acceptance of Overdetection in Breast Cancer Screening: Can We Assess Harm-Benefit Tradeoffs? by Anne Stiggelbout, Tessa Copp, Gemma Jacklyn, Jesse Jansen, Gerrit-Jan Liefers, Kirsten McCaffery and Jolyn Hersch in Medical Decision Making
Footnotes
Acknowledgements
We thank all women who filled in the survey for their participation.
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: GJ is a member of the Cancer Council Australia Expert Management Group for the project “Optimising the Early Detection of Breast Cancer in Australia” and a senior consultant at Quantium. The other authors declare no conflict.
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: TC was supported by an Australian Government Research Training Program (RTP) Scholarship and a Sydney Medical School Foundation Scholarship, The University of Sydney, Australia.
JJ was supported by the National Health and Medical Research Council (NHMRC No. 1037028 ). JH is supported by NHMRC Early Career Fellowship 1112509. The funding agreements ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
Authors’ Note
The study was presented at the 6th Preventing Overdiagnosis Conference in Copenhagen, Denmark, August 2018.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
