Sage Journals: Discover world-class research

Abstract

BACKGROUND:

The Delphi technique is a consensus method aiming to obtain statistical estimations from a qualitative approach, through an iterative process that leads to consensus within experts. The main characteristics of the technique include iteration, anonymity, feedback, and consensus reaching. When high-quality, quantitative evidence on a particular topic is insufficient, the Delphi technique can be used for making decisions in clinical scenarios. However, the quality of studies on breast cancer conducted with this technique, has not been assessed.

OBJECTIVE:

We aim to evaluate the quality of studies on breast cancer which used the Delphi technique as their method.

METHODS:

A quality assessment tool (Quali-D) was created through consensus among experts on the Delphi technique. Then, the tool was applied to studies on breast cancer which used the Delphi technique as their method.

RESULTS:

Studies conducted through the Delphi technique mainly assessed for quality indicators and expressed needs in patients with breast cancer. High-quality characteristics were reported in 63.89% of the studies. 98.61% used the Delphi technique due to lack of a more adequate method to solve their research question. 98.61% summarized and presented results in a clear way. In 91.67% of the studies, at least two rounds were conducted. 86.11% described the methods for expert selection in a complete manner. Only 54.17% of the studies reported an anonymous process and 4.17% of the studies disclosed conflicts of interest thoroughly.

CONCLUSIONS:

A variety of topics were assessed through the Delphi technique in cases where no other technique would have been more appropriate for assessing these issues. Significant limitations are present in terms of anonymity and full disclosure of conflicts of interest. We found that the quality of studies conducted with the Delphi technique regarding breast cancer is overall good. However, the limitations of each study must be considered when applying their results to clinical practice.

Keywords

Delphi technique breast neoplasms qualitative research

1. Introduction

Cancer incidence and mortality continue to rise worldwide. According to GLOBOCAN estimates, there were 19.3 million new cases and about 10 million deaths related to cancer in 2020. Breast cancer correspond to 11.7% of the incidence and 6.9% of the mortality for this same year [1].

While breast cancer is an area of growing research, it is also a heterogeneous and wide field. A study conducted by Eccles et al. found some critical research gaps in breast cancer; including the need for tailored screening approaches, the evaluation of optimal duration, sequencing and combinations of therapy, and molecular or genetic changes responsible for malignancy [2].

In instances where available evidence is insufficient, the Delphi technique may be employed for identifying problems, setting goals, or assessing priorities through expert consensus [3,4]. Regarding breast cancer, some gaps in knowledge may be addressed through this technique, for guiding decision-making processes and patient care [5]. The Delphi technique is a consensus method aiming to obtain statistical estimations from a qualitative approach, through an iterative process that leads to consensus within experts [6]. The main characteristics of the technique include iteration, anonymity, feedback, and consensus reaching. Iteration, the first characteristic, means that experts must emit their opinion more than once through various rounds. As for anonymity, this quality is important for assuring that none of the participants are aware of the answer given by another participant, thus decreasing the risk for bias. Feedback is a characteristic meant to take place after each round of the technique has been held. Feedback aims to position each participant in regards to the group. Highlighting significant contributions, both agreeing or discordant that may modify future answers. Finally, consensus must be reached, which is the main goal of using a Delphi technique [7,8].

A relevant issue to consider in the field of evidence-based medical care is the quality of scientific articles. As such, multiple tools have been designed for assessing the quality of different types of studies. For instance, the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) aims to assess diagnostic tools studies, the Risk of Bias 2 (RoB 2) gives a guide for considering bias in randomized trials, and the Equator Network provides tools for reporting different types of studies with quality and transparency [6,9–12]. Regarding the Delphi technique, the CREDES checklist (Guidance on Conducting and Reporting Delphi Studies) provides some criteria for authors of studies employing this method [11]. Our research group recently developed the Quali-D tool, meant to allow readers to assess for the quality of studies conducted with the Delphi technique (Fig. 1). This study aims to report on the quality of breast cancer studies using Delphi technique, assessed through the Quali-D tool.

Fig. 1.

Quali-D tool for the assessment of quality in studies using the Delphi technique.

Fig. 2.

Selection process and studies included for analysis.

2. Methods

2.1. Tool creation

The complete methods for the creation of the Quali-D tool, used in this study are in process of publication. We will therefore describe these methods briefly in this section (more information is available upon request). Initially, a literature search was carried out to assess the characteristics most commonly reported in studies using the Delphi technique. We searched the Medline, Embase, Clinical Key and Scielo databases, using the MeSH term “Delphi Technique”. Technical reports, reviews, and systematic reviews were included as filters, with no limits on publication date. We included articles in Spanish and English, with information regarding the key points in the formulation of a study with this technique, and the mechanisms to report the conclusions derived from it. The most relevant information was used to generate questions that would be useful as a starting point in a focus group.

Then, we contacted experts on qualitative methods, via email. The participants were selected for their experience in reporting and interpreting studies carried out with the Delphi technique. Once conflicts of interest were excluded, a web-based questionnaire with the aforementioned questions was completed by each participant. Then, an expert dialogue and debate meeting were held through a web-based format, to generate suggestions for quality assessment and reporting of studies carried out with this technique.

The web-based meeting consisted initially of a brief presentation of findings in the searched literature, as well as a general synthesis of the answers provided by the panel on the questionnaire filled-out before the meeting. Participants were allowed to contribute their opinions, definitions, and ideas regarding high-quality characteristics in studies conducted with the Delphi technique.

Finally, a consensus document was generated proposing the high-quality characteristics that should be reported in studies conducted with the Delphi technique. This document was sent to the participants, their feedback was collected and then incorporated into the tool that is finally presented in this document (Fig. 1).

3. Literature search and tool application

We performed a literature search on studies regarding breast cancer using the Delphi technique, using the following search strategies: (“Delphi Technique” [All Fields] AND “Breast neoplasms” [Majr]) for Medline; and ‘breast tumor’/exp/mj AND ‘delphi study’/exp for Embase. The only filter used, restricted our search to articles published in English, Spanish and French, with no limits for publication date. The searches were conducted on October 4th 2022.

The search yielded 149 results. All abstracts were reviewed and duplicate records were removed. After removing conference abstracts, unretrieved reports, studies not using the Delphi technique, protocols without final results and studies not reporting breast cancer exclusively, 72 studies were included for review (Fig. 2). The studies included for review were analyzed using the Quali-D tool (Fig. 1). This is a descriptive study, as such, results are expressed through frequencies and percentages.

4. Results

The studies about breast cancer using the Delphi technique included a wide array of topics. The most common were: quality indicators (9.73%), expressed needs (5.31%), screening (4.42%), event definitions (4.42%) and reconstructive surgery (3.54%), among others (Table 1). A full list of topics can be found in the supplementary material (Supplementary Table S1).

Table 1
Topics most commonly assessed through the Delphi technique in breast cancer field

Topic n = 113 n (%)

Quality indicators 11 (9.73)

Expressed needs 6 (5.31)

Screening strategies 5 (4.42)

Event definitions 5 (4.42)

Treatment in special situations 5 (4.42)

Reconstructive surgery techniques 4 (3.54)

Advanced breast cancer therapies 4 (3.54)

Mastectomy (prophylactic) 4 (3.54)

Breast imaging uses 3 (2.65)

Shared decision-making 3 (2.65)

Patient functionality 3 (2.65)

Patient-centered outcomes 3 (2.65)

Oncology surgery techniques 3 (2.65)

Survivors 3 (2.65)

Healthcare records 3 (2.65)

Prevention of lymphedema 2 (1.77)

Therapy for HER2 negative metastatic breast cancer 2 (1.77)

Breast conserving surgery 2 (1.77)

Fertility preservation 2 (1.77)

Radiotherapy 2 (1.77)

Adherence 2 (1.77)

Neoadjuvant therapy 2 (1.77)

Topic n = 113	n (%)
Quality indicators	11 (9.73)
Expressed needs	6 (5.31)
Screening strategies	5 (4.42)
Event definitions	5 (4.42)
Treatment in special situations	5 (4.42)
Reconstructive surgery techniques	4 (3.54)
Advanced breast cancer therapies	4 (3.54)
Mastectomy (prophylactic)	4 (3.54)
Breast imaging uses	3 (2.65)
Shared decision-making	3 (2.65)
Patient functionality	3 (2.65)
Patient-centered outcomes	3 (2.65)
Oncology surgery techniques	3 (2.65)
Survivors	3 (2.65)
Healthcare records	3 (2.65)
Prevention of lymphedema	2 (1.77)
Therapy for HER2 negative metastatic breast cancer	2 (1.77)
Breast conserving surgery	2 (1.77)
Fertility preservation	2 (1.77)
Radiotherapy	2 (1.77)
Adherence	2 (1.77)
Neoadjuvant therapy	2 (1.77)

98.61% of the studies chose the Delphi technique due to lack of a more adequate method to solve the research question, the same number of studies summarized and presented results in a clear way. In 91.67% of the assessed studies, at least two rounds were conducted, and 86.11% of the studies described the methods for expert selection in a complete manner. On the other hand, only 4.17% of the studies disclosed conflicts of interest for both experts and authors, and 54.17% of the studies reported anonymous participation of experts or anonymization of their answers to other participants (Table 2). From the total analyzed studies, 40 (55.5%) were reported as modified Delphi studies. Of these, only 24 studies reported on the ways the authors had modified the original Delphi technique.

Table 2

High quality characteristics in breast cancer studies using the Delphi technique

High quality characteristics, n (%)	n = 72
Delphi technique chosen due to lack of a more adequate method to solve the research question	71 (98.61)
Results were summarized and presented in a clear way	71 (98.61)
At least two rounds were conducted	66 (91.67)
Method used for expert selection described in a complete manner and the reason why experts were considered as such is reported within the manuscript	62 (86.11)
The degree of consensus achieved was reported or the reasons for not reaching consensus were described	59 (81.94)
If the study was reported to be a Modified Delphi, the modifications to the technique and reason for using a modified approach were described	55 (76.39)
Results from the first round were summarized and informed to participants before the next round	55 (76.39)
The participation of experts or at least their answers were anonymized to other experts	39 (54.17)
Conflicts of interest disclosed for both experts and authors	3 (4.17)

When analyzing the number of high-quality characteristics reported by studies, we found that 46 (63.89%) reported 7 or more characteristics, while 3 (4.17%) reported 3 or less high-quality characteristics (Table 3).

Table 3

Number of high-quality characteristics reported in breast cancer studies using the Delphi technique

Number of characteristics reported, n (%)	n = 72
9	1 (1.39)
8	21 (29.17)
7	24 (33.33)
6	17 (23.61)
5	4 (5.56)
4	2 (2.78)
3	1 (1.39)
2	2 (2.78)

Regarding expert panels, most of the studies (68.06%) included only healthcare experts, seven studies (9.72%) included both patient representatives and healthcare experts, and six (9.72%) included healthcare and epidemiology experts. Only two studies (2.78%) had a panel composed solely by patient representatives (Table 4). We subanalyzed studies not reporting conflicts of interest for both the expert panel and authors. Of these, 73.52% reported the conflicts of interest for authors, while 27.69% did not report conflicts of interest of either the authors or the experts.

Table 4

Panel conformation in breast cancer studies using Delphi technique

Type of expert, n (%)	n = 72
Only healthcare experts	48 (66.66)
Patient representatives and healthcare experts	8 (11.11)
Healthcare and epidemiology experts	6 (8.33)
Healthcare and legal experts, patient representatives	2 (2.78)
Patient representatives, epidemiology and healthcare experts	2 (2.78)
Only patient representatives	2 (2.78)
Healthcare experts, patient representatives and policymakers	1 (1.39)
Healthcare experts and policymakers	1 (1.39)

5. Discussion

Our study found that the quality of articles conducted with the Delphi technique, regarding breast cancer, is overall good. Characteristics considered of high quality were reported in more than half of the studies. However, the anonymization of answers and the disclosure of conflicts of interest were the least reported characteristics in the analyzed studies. The studies considered different types of experts for their panels, but the greater proportion (68.06%) used only healthcare experts as members of their panel. A variety of topics were assessed through the Delphi technique, with quality indicators, expressed needs and screening among the most frequent. The Delphi technique was also employed for subjects such as radiotherapy techniques and treatment in men.

The least reported characteristics in the included studies were anonymization and conflict of interest disclosure. As for anonymization, our tool proposed to report on whether the panel members were blind as for who other experts were, or, when not possible, at least for their answers to be anonymized (i.e. other experts would not be able to know who each answer belonged to). We recognize that it is possible for studies to have correctly anonymized answers, but for authors to not have disclosed such process within the manuscript. Nonetheless, we consider that this aspect is crucial to the integrity of the Delphi technique, as previous reports have previously stated [12].

Only 4.17% of the studies disclosed conflicts of interest for both experts and authors. Conflicts of interest could be as mild as wholeheartedly forgetting to report an accepted meal, or tragedies such as the Tuskegee Experiment [13–16]. Therefore, we believe that conflicts of interest should be completely and profoundly reported for all participants in studies using the Delphi technique to avoid damaging the integrity of the study.

More than half of the studies were reported to have modified the Delphi technique. Of these, only a third reported on the ways the authors had modified the original methods. This information is valuable, since mild changes to the technique may be required for specific research questions, but a significant modification may lead to the study to become qualitative in nature, but outside the scope of a Delphi technique. This issue has been reported to be addressed through assessment of quality of other aspects within the study, with tools such as the one we herein propose [12].

Finally, a significant proportion of the studies did not report on why experts were considered as such. Expert opinion is the backbone of the Delphi technique, which allows for research questions to be answered in the absence of an adequate alternative method [17]. However, very scarce information exists regarding what an expert should be. We advise for authors to thoroughly report the reasons behind choosing a specific expert panel, as for readers to understand their opinions better.

Our study is the first to assess quality of breast cancer studies conducted with the Delphi technique. We found that the quality of studies conducted with this technique, regarding breast cancer is overall good. However, the limitations of each study must be considered when applying their results to clinical practice.

Footnotes

Acknowledgements

No funding was received from institutional, private and/or corporate organizations.

Conflict of interest

The authors declare that they have no conflict of interest to report.

Author contributions

DPH participated in the conception, analysis of data, preparation of the manuscript and revision for intellectual contents; DGFA participated in the conception, interpretation of data, preparation of the manuscript and revision for intellectual contents, OMMV participated in the conception, interpretation of data, preparation of the manuscript and revision for intellectual contents; ISML participated in the conception, preparation of the manuscript and revision for intellectual contents.

References

Sung

, Ferlay

, Siegel

, Laversanne

, Soerjomataram

, Jemal

, Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries, Cancer J Clin , 71: 209–249, 2021. doi:10.3322/caac.21660.

Eccles

, Aboagye

, Ali

, Anderson

, Armes

, Berditchevski

, Critical research gaps and translational priorities for the successful prevention and treatment of breast cancer, Breast Cancer Res , 15: R92, 2013. doi:10.1186/bcr3493.

Moossdorff

, van Roozendaal

, Strobbe

LJA

, Aebi

, Cameron

, Dixon

, Maastricht Delphi consensus on event definitions for classification of recurrence in breast cancer research, J Natl Cancer Inst , 106: dju288, 2014. doi:10.1093/jnci/dju288.

Shaw

, Coggin

, Using a Delphi technique to determine the needs of African American breast cancer survivors, Health Promot Pract , 9: 34–44, 2008. doi:10.1177/1524839907307674.

Agha

, Pidgeon

, Borrelli

, Dowlut

, Orkar

T-EK

, Ahmed

, Validated outcomes in the grafting of autologous fat to the breast: the VOGUE study. Development of a core outcome set for research and audit, Plast Reconstr Surg , 141: 633e–638e, 2018. doi:10.1097/PRS.0000000000004273.

Mullen

, Delphi: myths and reality, J Health Organ Manag , 17: 37–52, 2003. doi:10.1108/14777260310469319.

Von der Gracht

, Consensus measurement in Delphi studies, Technol Forecast Soc Change , 79: 1525–1536, 2012. doi:10.1016/j.techfore.2012.04.013.

Fernandez-Ávila

, Rojas

, Rosselli

, Delphi method in rheumatology research: Are we doing well? Rev Colomb Reumatol , 27: 177–189, 2020. doi:10.1016/j.rcreu.2019.04.001.

EQUATOR Network | Enhancing the QUAlity and Transparency Of Health Research, (n.d.). https://www.equator-network.org/ (accessed December 11, 2022).

10.

U. of Bristol, QUADAS-2, (n.d.). https://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2/ (accessed December 11, 2022).

11.

Cochrane Bias, RoB 2: a revised cochrane risk-of-bias tool for randomized trials, (n.d.). https://methods.cochrane.org/bias/resources/rob-2-revised-cochrane-risk-bias-tool-randomized-trials (accessed December 11, 2022).

12.

Jünger

, Payne

, Brine

, Radbruch

, Brearley

, Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review, Palliat Med , 31: 684–706, 2017. doi:10.1177/0269216317690685.

13.

Nasa

, Jain

, Juneja

, Delphi methodology in healthcare research: how to decide its appropriateness, World J Methodol , 11: 116–129, 2021. doi:10.5662/wjm.v11.i4.116.

14.

Australian Institute of Administrative Law, Conflicts of interest (2010). https://www.integrity.qld.gov.au/assets/document/catalogue/speeches-articles/conflicts-of-interest-speech.pdf.

15.

Napierala

, Schäfer

, Schott

, Schurig

, Lempert

, Management of financial conflicts of interests in clinical practice guidelines in Germany: results from the public database GuidelineWatch, BMC Medical Ethics , 19: 65, 2018. doi:10.1186/s12910-018-0309-y.

16.

Nix

, Tuskegee experiment: the infamous syphilis study, History (n.d.). https://www.history.com/news/the-infamous-40-year-tuskegee-study (accessed December 11, 2022).

17.

Hohmann

, Brand

, Rossi

, Lubowitz

, Expert opinion is necessary: Delphi panel methodology facilitates a scientific approach to consensus, Arthroscopy , 34: 349–351, 2018. doi:10.1016/j.arthro.2017.11.022.