Abstract
Introduction
The 2018 National Comprehensive Cancer Network guidelines for prostate cancer genetic testing expanded access to genetic services. Few studies have examined how this change has affected provider practice outside of large cancer centers.
Methods
We conducted a qualitative study of multi-disciplinary health care providers treating patients with prostate cancer at a safety-net hospital. Participants completed an interview that addressed knowledge, practices, and contextual factors related to providing genetic services to patients with prostate cancer. A thematic analysis using both inductive and deductive coding was undertaken.
Results
Seventeen providers completed interviews. Challenges in identifying eligible patients for genetic testing stemmed from a lack of a) systems that facilitate routine patient identification, and b) readily available family history data for eligibility determination. Providers identified non-medical patient characteristics that influenced their referral process, including health literacy, language, cultural beliefs, patient distress, and cost. Providers who see patients at different times along the cancer care continuum viewed benefits of testing differently.
Conclusion
The use of digital technologies that systematically identify those eligible for genetic testing referrals may mitigate some but not all challenges identified in this study. Further research should determine how individual provider perceptions influence referral practices and patient access to genetics both within and across cancer specialties.
Keywords
Introduction
Genetic care is becoming increasingly salient to achieving better cancer outcomes in the emerging field of precision medicine; defined as using personal characteristics such as genes to identify optimal treatment pathways. 1 The National Comprehensive Cancer Network (NCCN) recommends the use of germline genetic testing for patients with prostate cancer since 2018. 1 Germline testing is used to identify inherited pathogenic variants in DNA, which can guide screening, familial testing, and treatment decision-making. This is in contrast to somatic testing, which focuses on tumor-specific pathogenic variants that may identify an indication for further germline testing and can guide some therapeutic decisions. 2 Germline testing should be considered for patients with clinically low-to intermediate-localized disease with a family history of prostate cancer; or high-to very high-risk localized disease. For regional or distant metastatic disease, germline testing is recommended regardless of initial risk.
While most prostate cancers occur in patients without a family history or inherited pathogenic variant, 12-15% of prostate cancer patients carry an identifiable germline DNA damage repair defect.2,3 Prostate cancer patients with BRCA1/2 pathogenic variants are at higher risk for progression during local therapy, metastases, and have lower survival. 4 Personalizing treatment regimens based on somatic and germline genetic information is now possible (e.g. use of polyadenosine diphosphate–ribose polymerase (PARP) inhibitors), thus reducing the burden of lethal prostate cancer and improving quality of life.5,6 The extent to which genetic testing and precision treatment remains restricted to tertiary or comprehensive cancer centers dictates, in part, whether such services are available only to a small portion of resource- and access-privileged patients. 7 The promise of precision medicine cannot be realized without diffusion across the many settings in which patients receive risk counseling and treatment.
Oncology clinical providers, as key implementers of NCCN Guidelines, have been slow to adopt genetic testing in practice.8,9 A nationwide survey of oncologists practicing in Prostate Cancer Clinical Trials Consortium sites found considerable variation in provider recommendations for testing. The majority of oncologists surveyed (62%) reported considering all metastatic patients for germline genetic testing, while 27% would only test based on family history or for inclusion in clinical trials. Fewer reported testing in high-risk localized or non-metastatic cases. 8 A survey of over 600 radiation oncologists and urologists found that urologists were significantly more likely than radiation oncologists to view genetic testing as important (46% vs. 20%) and report regular use of genetic testing (26% vs. 4%, all comparisons P < .001). 10 Reasons for underutilization and variation in referral patterns may also include a lack of available genetics expertise.8,11,12 Insufficient access to genetic counselors pressures oncologists and urologists to perform aspects of genetic evaluations. These factors likely contribute to lower than expected rates of genetic testing based on the population of eligible patients diagnosed with prostate cancer. 13
Most existing literature has focused on provider knowledge, attitudes and facility using on genetic tests, with studies sampling providers with specialized expertise in treating prostate cancer.8,10,14,15 Given the known racial and ethnic disparities in use of germline genetic testing for prostate cancer,16,17 understanding factors such as how patient characteristics, organizational supports for testing, and inter-personal factors influence the implementation of genetic testing in diverse, community practices is needed. 18 In the breast cancer literature, one study found that clinicians believed that Black patients had higher mistrust and cost-related barriers to genetic testing, were less likely to follow through on genetic testing recommendations, and experienced more distress after testing relative to White patients, although there were no differences in self-reported referral practices by patient race. 14 Other research has found that clinicians are prone to affective forecasting bias and other heuristics in making decisions about recommending genetic counseling and testing,19,20 overestimating the negative effects of learning about an inherited pathogenic variant relative to its true impact.21-23 These findings suggest that providers’ delivery of genetic services may be influenced by factors other than knowledge, and these factors are not well-characterized across diverse populations. 18 This study seeks to characterize health care providers’ decision-making about the use of genetic services for patients with prostate cancer at an urban, safety-net hospital serving predominantly racial and ethnic minority patients.
Methods
This study used qualitative interviews to explore how multi-disciplinary health care providers treating prostate cancer patients considered referrals to genetic services for their patients. To answer this question, we constructed a qualitative interview guide based on and the Behavioral Model for Vulnerable Populations, 24 an adaptation of Andersen’s Behavioral Model for identifying differences in health care utilization and outcomes based on pre-disposing characteristics (e.g., demographics, literacy, attitudes towards health care) and enabling factors (e.g., insurance, perceived barriers to care, competing needs), perceived and evaluated need, and evaluated health. The application of the Behavioral Model for Vulnerable Populations related each of these concepts to the process of genetic testing. Predisposing characteristics were defined as in the model as features of the individual: demographics, health beliefs, social structures, literacy, living conditions and psychological resources (coping); while enabling factors were personal and family resources such as insurance and access to health resources, including genetic counseling services. Perceived need related to the subjective evaluation of the need for genetic testing by the provider and evaluated need was defined as clinical or evidence-based criteria guiding genetic services. The interview guide (Supplement 1) covered the following topics from the provider’s perspective: 1) Knowledge about genetic testing and guidelines (an enabling factor); 2) Organizational incentives for genetic testing and processes to support it (an enabling factor); 3) Discussions about genetic testing with patients (need and use of health services domains); 4) Experience with the referral process (satisfaction with care domain); and 5) Personal and contextual factors that contribute to decision-making about making referrals for genetic services (pre-disposing characteristics). Research assistants piloted the interview guide with practicing clinicians affiliated with the study, who provided input on factors that they viewed as important to local referral practices. The local Institutional Review Board determined the study activities to be exempt based on federal criteria.
The study team generated a list of all practicing clinicians (medical doctors, nurse practitioners, physician assistants, or genetic counselors) who treated patients with prostate cancer at an urban, safety-net hospital and were thus eligible to be recruited. The hospital in which the research was conducted is private, not-for-profit, and serves an urban population. Three-quarters of patients are considered to be medically-underserved and rely on government payors for insurance (e.g. Medicaid). A majority of patients are racial or ethnic minorities, and 27% of patients speak a primary language other than English. We purposively sampled at least two clinicians across five relevant medical specialties: general internal medicine, urology, radiation oncology, medical oncology, and genetics. With support from a clinical champion, we sent email invitations soliciting participation in a 45-minute one-on-one qualitative telephone interview.
Interviews were conducted by study team members not involved in the clinical process; either a research assistant who had received didactic and experiential training in qualitative methods (EL), or a doctoral-trained qualitative health services researcher (CG). The interviewer used a flexible approach to covering topics to allow for a conversational flow and to encourage the individual to share their experience in a manner that was most consistent with their thinking. Participants received a $40 debit card for their time at the conclusion of the interview.
Upon completion, the interviewer generated a summary to reflect contextual nuance not captured in the audio recording and initial impressions about most salient themes or topics. All audio recordings were securely uploaded and transcribed by a third party. To ensure fidelity and preserve anonymity, research assistants verified transcripts and removed identifiers (names, locations) prior to analysis. De-identified transcripts were uploaded into NVivo version 1.5 for coding and analysis.
Qualitative Analysis
A thematic analysis was conducted, 25 utilizing a mixture of emergent codes and those based on the Behavioral Model for Vulnerable Populations. 24 Two coders independently coded 4 transcripts, using provisional codes from the conceptual model and generating emergent codes based on individual responses. The study team reviewed the provisional and emergent codes to develop a codebook for the remaining interviews. Each transcript was coded by two coders, and discrepancies resolved in consensus meetings. After initial coding with open and conceptual codes, we grouped codes and specified themes, describing the properties and dimensions of the generated codes. 26 Because only 2-4 interviews were collected within each specialty and the research question did not seek to address similarities or differences between groups, we did not pursue cross-specialty comparisons.
Results
Summary of Identified Topics and Associated Themes.
Topic 1: Identifying Patients Eligible for Genetic Testing
A consistent topic discussed across all providers was challenges in identifying patients who were appropriate for genetic testing. These themes related to enabling factors at the health system level (e.g. availability of health services that support genetic testing) and tools to assess evaluated need for genetic services at the individual level. One theme identified an absence of systems and/or processes, an enabling factor, to assist clinical teams in detecting eligible patients for referrals to genetic services (Theme 1a): “There is no specific policy or procedure. It's just a provider. It's a provider decision tree… there is no reminder, there's no policy, there's nothing in the electronic record to alert you. So, certainly it's one of those things that if you're moving along your day quickly, it's something that might not get ordered unless after the fact.” (HCP 05) “Yeah, I think that there’s no clear consensus from [the] department of identifying these patients and then who’s meeting criteria. So, it’s pretty variable, so increases the variability.” (HCP 07)
Providers attributed variability in referral patterns and noted that some patients were expected to be ‘missed’ given the lack of systems to facilitate this process.
A second theme (Theme 1b) acknowledged barriers to accessing relevant data, such as family history, that would facilitate providers evaluating need for referrals to genetic counseling: “I think one of the biggest challenges is making sure we actually have that family history. If, for example, the patient had pancreatic cancer in his family, and he brought it up with me. I wish I could say that I screened for it, but he said, ‘Look, I had a brother and my father who had pancreatic cancer under the age of 50. How do I know that I’m not going to get pancreatic cancer?’ And I was like, ‘Oh, good question.‘" (HCP 03) “Genetic counseling is kind of interesting, if you don't dig for the story, you're never going to figure it out. Like, ‘tell me about your parents. Tell me about your siblings’… Yes, we know there's family history in the [medical record], but that's all just lip service… So, we're short sighted… we don't tell the story of the patient. And genetic counseling is completely about the story of the patient.” (HCP 06)
In sum, identification of patients who may benefit from genetic testing was described as a fundamental challenge to effectively implementing genetic care for patients with prostate cancer. Without systems and data to identify patients, providers noted variability in who received referrals, an outcome they described as undesirable.
Topic 2: Decision Inputs Contributing to Referrals
The second aspect of genetic testing that participants described was the referral process that was initiated after it was decided that an individual might benefit from genetic services. The first theme (Theme 2a) highlighted the routine practice of consulting the NCCN Guidelines, demonstrating good knowledge about the criteria for referrals among this sample of providers. Here, physician knowledge served as an enabling factor: “It’s just making sure we’re following evidence-based recommendations and practice. You know, it’s part of NCCN Guidelines to refer patients to genetics. So, that’s definitely something I just consider with all patients.” (HCP 02) “There are some [NCCN] Guidelines that are for men with high-risk prostate cancer based on their cancer parameters, and for men that might even be intermediate to high risk that genetic testing is recommended. I’m not doing it at that consistency. I’m doing it more on a case-by-case basis if I think there is a stronger predisposition to a germline genetic issue. But I do think that others are potentially ordering it more regularly based on the NCCN Guidelines.” (HCP 05)
Genetic services providers supported the idea that clinician knowledge about eligibility was generally good, as one genetic provider stated, “I think the provider knowledge, I think it’s really great that they know that this person qualifies, this person doesn’t, and being able to refer them over. With that being said, again, it’s mostly a few oncologists, who actually refer over to me. So, perhaps there are some that don’t know about testing or don’t know about the wide range of people who do qualify. But I will say, overall, the knowledge seems to be really good, and we do get quite a number of referrals.” (HCP 12)
In addition to the NCCN Guidelines, provider perceptions of patient-specific considerations, or pre-disposing factors, that contributed to the genetic counseling referral decision were described (Theme 2b). These ranged in scope from how the provider perceived: patient understanding of genetics information, patient emotional burden, patient ability to afford genetic testing, and patient-provider language barriers. Table 2 provides examples of how providers considered these perceived patient factors on their own referral behaviors. In particular, providers detailed that non-English patients are less likely to be referred because of barriers related to interpreters and their perceived ability to understand the information: “I will say that I think it's the service that generally patients get as they advocate for. And so, it creates a disparity if it's something that an educated English-speaking patient says, ‘Hey, I know this is my family history, and I know I want to get further testing.’ I'm much more likely to send it… I do think my limited English proficiency patients who come from a different setting, are much less likely to self-advocate for this.” (HCP 03) Examples of Theme 2b: The consideration of patient characteristics in provider referral practices.
In sum, when participants described their referral practices, they recognized NCCN Guidelines as important to identify relevant clinical factors. However, provider perceptions of patient characteristics also clearly influenced their decisions as to choosing when, how, and whether to refer patients for genetic testing.
Topic 3: Applicability of Genetic Test Results to Practice
Genetic testing can have several purposes, including informing treatment choice for the affected patient, as well as cascade testing for family members. Whether providers offered referrals was based in part on their perceived need of the results for their recommendations to the patient. Those participants who emphasized a future value gained from genetic testing described hesitations in referring patients: “Rarely will [genetic results] actually impact the patient treatment, but it could certainly have implications for the patient's family. In particular, regarding their own assessment of risk and the need for appropriate screening for prostate cancer or for other genetically linked cancers, depending on the results of the genetic study… It doesn't generally guide my treatment and recommendations or the actual treatment that I provide much at all.” (HCP 04) “Oftentimes, people will get told that you have a genetic predisposition, but we can't tell you what it means, which is kind of a struggle that I heard… where you end up saying, “We give you a lot of information, but we don't really know how to act upon this information in any meaningful way.” (HCP 06)
A second group of providers emphasized immediate benefits gained from genetic information, and this group described this as a reason they provided referrals: “In some cases, if they’re currently being treated, there might be treatment options that were not yet utilized based on the results of the testing. So, more specific care for the patient. And secondly, there may be other cancers related to that, to cancer predispositions syndrome, for which other screening and management would be recommended. And then thirdly, for cascade testing of relative.” (HCP 14)
These orientations toward perceived need or utility derived now versus in the future were noted by some to influence which specialties might be more likely to refer patients to genetic counseling and/or testing. Specifically, participants described testing as occurring later, more often by medical oncologists, rather than earlier, as germline genetic information had more immediate treatment implications for those undergoing systemic therapy: “Typically, more of the genetic testing in my patient population is ordered by the Medical Oncology team, because they seem to have a larger role in that part of it by the time, they're seeing it… It doesn't guide the care that I provide that much, because again, I mostly personally treat the earlier stages of prostate cancer, which is why I think you probably have the medical oncologist ordering more of the genetic testing. Because it doesn't have as much of an impact on surgical management, or you know radiation for localized management as it does for more systemic treatments… So, that's why it doesn't have a major impact on me per say. And that's probably why I think surgeons are ordering it maybe a little bit less in the grand scheme of things.” (HCP 05)
For some, there was a preference for referral and testing to happen earlier, and in greater coordination with primary care and specialists: “So, you probably have to have some in-service or some instruction to all of the members of the prostate community, not just the medical oncologists or the radiation oncologist, but maybe from the primary care doctors, the urologists. I feel like those people do less referring to genetic testing because they assume that there’s a high-risk prostate cancer is going to be identified, then there’s going to be a medical oncologist who can handle all of that. But it’d be nice if everyone on the team knew the recommendations and were able to discuss with patients earlier on.” (HCP 10)
In conclusion, provider perceptions of the utility of referring for genetic services depended on where in the care timeline they were treating prostate cancer. Some noted that the impetus for referral related to immediate changes in decision-making, while others who were hesitant to refer perceived benefits would be derived solely at a future time in the patient’s cancer treatment.
Discussion
In this qualitative study of multi-disciplinary health care providers, three distinct topics related to providing genetic services to prostate cancer patients were identified as important: identification of patients, decision inputs for providing a referral, and the utility of genetic results to practice. Challenges in identification stemmed from a lack of systems to support routine patient identification and the lack of readily available data to support eligibility determination (e.g., structured family history). Referrals were supported by good knowledge and use of NCCN Guidelines, an enabling factor that is known to contribute to health utilization and outcomes as posited by the Behavioral Model. 24 Providers did identify other non-medical patient pre-disposing factors that influenced their discussions and referral practices, such as non-English language, health literacy, cost, and perceived distress. These factors are consistent with those that the Behavioral model suggests give rise to health disparities. Finally, whether providers perceived benefits of testing would be derived immediately or in the future was related to whether they referred patients. This fits with Gelberg’s conceptualization of perceived need, although our findings emphasize a temporal element to this contributor to the use of genetic services. This meant that providers seeing patients later in their cancer treatment (i.e., medical oncologists) were perceived to be most likely to be responsible for referring to genetic care.
The absence of systems to identify patients who may benefit from referral to genetic services was universally noted as a barrier to providing equitable and high-quality care for prostate cancer patients. This phenomenon has been noted as a critical impediment to implementing precision medicine across disease sites.27-29 A recent systematic review documented of 32 studies to improve genetic testing for those with hereditary cancers, 15 (47%) used family history and/or referral tools, and 10 of these (67%) also integrated clinical decision support (CDS) tools. 30 Other similar primary care-based interventions are being tested to promote the routine screening of patients eligible for genetic testing. 31 The use of well-designed, standards-based CDS tools overcomes several challenges identified by providers in this study beyond systematic identification. It allows for adaptation in the context of evolving guidelines and science, 32 reducing reliance on individual provider knowledge. Systems such as these can reduce potential bias present in interpersonal elicitation of risks or bias that prevents referral despite risk elicitation.
Providers in this study indicated that NCCN Guidelines facilitated referrals for patients with prostate cancer. However, provider perceptions about patient characteristics introduced factors beyond clinical need into their referral behaviors. Decision-making literature has demonstrated that clinicians tend to overestimate the negative consequences of genetic testing for their patients (‘impact bias’). 19 In this study providers noted factors, such as perceived patient distress, likelihood of following through, health literacy, and ability to pay as considerations in their provision of referrals. These factors were similar to those identified in a recent national survey with breast oncologists. That study found that providers perceived different barriers to genetic counseling and testing, and held different attitudes towards African American compared to White patients, reflecting inherent biases that may influence access to care. 14 In our study, language was a commonly identified consideration, which is unsurprising given that approximately 27% of the hospital’s patients primarily speak a language other than English. Other research has noted linguistic barriers to receiving genetic counseling, and only 6% of genetic counselors in the United States practice in Spanish. 33 Ensuring appropriate linguistic access to high quality counseling, from the time of referral discussions through testing and results disclosure is needed, as research has highlighted the benefits of genetic counseling in one’s native language. 34
When perceptions about predisposing characteristics (health literacy, language, cost) influence care, differences in utilization between minority or marginalized versus majority, privileged populations can widen. 24 Future work is critically needed to understand and quantify the extent to which these perceptions and biases influence referral patterns across groups (e.g. non-English speakers). Minimizing clinician biases that influence referral practices is necessary to remove both perceived and actual barriers to genetic services for structurally marginalized patients. 19 Implementing interventions that address enabling factors may reduce the impact of pre-disposing factors on health outcomes, although the most impactful solutions will address both. Efforts to address provider biases could include standardized intake and family history elicitation tools 35 to reduce the influence of individual perceptions and focus on clinical and familial factors that should drive referral. Additionally, plain language and communication interventions 36 , 37 are needed to support providers in delivering high-quality genetic information and counseling. Web-based tools targeting providers that offer patient intake questions combined with education may address many issues driving the inconsistent delivery of evidence-based genetic referrals identified in our study. 35
Perceptions about the monetary cost of genetic testing and patients’ ability to pay was emphasized by many of the participants as salient to making referrals. Others have documented that cost is a barrier to prostate cancer treatment and genetic testing, 18 and one that impacts racial and ethnic minorities more than White patients. For example, one study determined that financial hardship during cancer treatment was 23% higher in Black versus White patients, and Black patients were 41% more likely to limit care due to cost. 38 In a national survey of oncologists, 1 in 4 oncologists rarely/never mentioned costs in discussing genetic testing, although respondents practicing in clinics with a higher proportion of patients who were uninsured or covered by Medicaid were more likely to discuss cost. 39 However, coverage rules and restrictions as well as cost-sharing for genetic testing varies widely across insurers, 40 and may be unknown by providers. Increasingly, there are sponsored programs that can mitigate or eliminate patient out-of-pocket costs for genetic services for eligible patients. 41 Promoting awareness of such programs such that cost is removed as a barrier is a key imperative. Further policy advocacy that limits cost-sharing and provides more uniform criteria for coverage associated with genetic care for cancer patients is also warranted.
Finally, the question of timing and utility of genetic information across the cancer care continuum was raised in this study of multi-specialty providers. Studies have shown that clinicians’ interpretation and application of genetic results varies by their role, confidence in understanding results, and practice setting. 15 Similarly, survey research has demonstrated that 40-70% of clinicians feel confident in using genetic tests for treatment decision-making. 10 Understanding how confidence in using genetic results affects perceptions of its utility or perceived need for clinical care within and across specialties is warranted. At the same time, the optimal timing of genetic testing in the continuum of prostate cancer treatment is an area for future investigation 42 and may evolve with advances in genomic science and targeted treatment applications.
This study has some limitations. Responses reflect local practices at a single urban, safety-net hospital and individual provider perceptions. As a qualitative study, it is intended to generate hypotheses and the topics that emerged as salient to these providers can inform future measurement of provider attitudes, beliefs, biases, and behaviors across activities relevant to providing cancer genetic services. We used the Behavioral Model to guide our conceptualization and analysis, and other factors may be unaccounted for in these results, especially those best addressed from the patient perspective. Due to the relatively small sample size within each provider type, we were unable to look at comparisons across specialties. Based on our findings and other research, this may be a fruitful area of investigation, especially given the questions raised about utility of genetic information across different phases of prostate cancer care.
Conclusions
In conclusion, this qualitative study adds to the literature on the provision of genetic services for patients with prostate cancer since the NCCN expanded eligibility for testing in 2018, sampling from a safety-net hospital serving a population of predominantly racial and ethnic minority patients. We documented variability in offering referrals to genetic services based on the ability of the health system resources to identify patients in need of genetic counseling (an enabling factor), provider perceptions about non-medical patient pre-disposing characteristics, and utility to clinical practice (perceived need). Future research must consider how providers’ role and heuristics influence how and when they offer genetic services and impacts equity in patients’ options for precision medicine, attending to the potential for treatment and outcome disparities across patient populations and clinical settings.
Supplemental Material
Supplemental Material - Delivering Genetic Testing for Patients with Prostate Cancer: Moving Beyond Provider Knowledge as a Barrier to Care
Supplemental Material for Delivering Genetic Testing for Patients with Prostate Cancer: Moving Beyond Provider Knowledge as a Barrier to Care by Christine M. Gunn, Emma X. Li, Gretchen A. Gignac, Magdalena Pankowska, Stephanie Loo, Kimberly Zayhowski, Catharine Wang in Cancer Control
Footnotes
Author Contributions
Conceptualization: C.G.; Data curation: C.G., E.L., M.P.; Formal analysis: C.G., E.L., M.P., C.W., K.Z., G.G.; Funding acquisition: C.G.; Methodology: C.G., C.W.; Project Administration: M.P., S.L.; Supervision: C.G., C.W.; Writing – original draft: C.G., E.L.; Writing – review & editing: K.Z.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Prostate Cancer Research Program under Award No. #W81XWH-20-1-0110. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the Department of Defense.
Ethical Approval
The Institutional Review Boards at the Boston University Medical Campus (H-40698 approved March 5, 2021) and Dartmouth College (STUDY00032381, approved October 21, 2021) reviewed this study and determined activities to be exempt based on federal criteria.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
