Sage Journals: Discover world-class research

Abstract

Background:

As part of the improvement plan of the Organisation of European Cancer Institutes at the Fondazione IRCCS Istituto Nazionale dei Tumori of Milan (INT) a Committee composed of patients, caregivers, and representatives of patient associations has been established to actively participate in the planning and organization of both hospital services and research initiatives. The potential impact of implementation of the Committee could ensure a patient-centered care environment.

Aim:

To determine whether there is consensus among INT staff on key aspects of the Committee.

Method:

Selected INT staff (experts) were invited to participate in a Delphi study. Each expert was asked to rate their agreement on 13 statements and give a brief explanation for their answers. Only the statements that failed to reach an 80% consensus were reintroduced in the second round, following the same procedures.

Results:

Ninety experts responded to Round I (55.21% response rate). Eight statements reached the 80% consensus threshold, while five did not meet the predefined threshold and were therefore re-proposed in the second round. These statements concerned the Committee’s role (advisory vs decision-making), the perceived value of the Committee in research (improvement vs no involvement), and participant compensation (voluntary vs paid). In Round II, 69 experts participated (79.31% response rate), but none of the reintroduced statements reached the required consensus threshold.

Conclusion:

Although a Committee is a topic of theoretical interest and aligns with respondents’ values, the establishment of a Committee in INT is currently difficult to implement. Training and awareness initiatives are necessary, as requested by respondents.

Keywords

Cancer consensus Delphi involvement patient public health qualitative research

Introduction

In recent years, there has been increasing recognition of the active role that patients can and should play in managing their own health. Literature highlights the importance of transforming patients from passive subjects to active partners in medical care. Increasing evidence demonstrates that greater patient involvement leads to better health outcomes, improved adherence to treatments, and higher satisfaction with the care received.^1,2

The concept of Public and Patient Involvement (PPI) is part of this cultural movement, and it refers to active inclusion of patients, caregivers and the public in health research, service planning and evaluation, and development of health policies.^3,4

This involvement aims to ensure that health research and services are more aligned with needs, experiences, and preferences of patients and the public. PPI emphasizes the importance of transparency, accountability, and co-creation of knowledge between healthcare professionals, patients, and the public, with the goal of improving effectiveness and equity of healthcare.^5-7

The literature recognizes that greater patient involvement in the healthcare system brings a range of benefits to the planning and organization processes of hospital services, as well as clinical and experimental research. Nevertheless, real-world experiences of PPI remain limited. A recent systematic review of the literature highlights both barriers and facilitators to PPI, taking into account the perspectives of patients, healthcare professionals, and the organizational aspect of the healthcare system.⁸ Among the factors that facilitate involvement are patients’ sense of security and personal trust, feeling comfortable and in tune with the healthcare staff, and having good interpersonal and communication skills. Healthcare professionals play a very important role, especially in encouraging patients to participate in the involvement process. The positive contribution of the institution/healthcare system appears to be its ability to support patient-centered care through effective communication, training programs on collaborative work, supporting patient feedback on safety and complaints, and giving patients the opportunity to report incidents. The barriers originate from traditionally held perspectives. Among the obstacles reported in the literature, aspects related to the lack of trust and the perception of patients as not adequately trained to contribute significantly to research and health policy are highlighted, which can lead to merely symbolic involvement. Institutional and structural barriers are also noted, specifically related to the lack of mechanisms to effectively integrate patient contributions into decision-making processes. This may include the absence of dedicated committees or financial resources to support PPI.⁹ Effective PPI requires that they are adequately trained in research processes and health topics so that they can contribute in an informed manner. However, the necessary training programs are not always provided. Similarly, healthcare personnel also need training on how to facilitate patient involvement, as they often lack the skills to promote productive collaboration.¹⁰

This reinforces the need for organizational and cultural change, a shift that is often difficult to initiate as it requires dismantling traditionally held perspectives, such as beliefs about the characteristics of different professional roles and the management of power dynamics between patients and doctors.¹¹

The Fondazione IRCCS Istituto Nazionale dei Tumori (IRCCS National Cancer Institute Foundation of Milan, hereafter referred to as INT) is a public hospital in northern Italy, exclusively dedicated to cancer treatment, both clinically and through research. Since its establishment in 1925, INT has been a national reference center, excelling in healthcare and research. The close collaboration and seamless integration between the clinical and research departments ensure that INT qualifies as a "Comprehensive Cancer Center," as defined by the Organisation of European Cancer Institutes (OECI).

As part of the accreditation process required by OECI,¹² INT was asked to promote PPI in planning and organizing hospital services and research. The establishment of a Committee composed of patients, caregivers, and patient association representatives, together with the formal definition of its roles and responsibilities, was identified as the initial requirement.

The objective of this study was to assess the level of consensus on key aspects of the Committee among INT staff, patients, and representatives of patient associations.

Method

Participants

To achieve the research objective, it was considered essential to involve stakeholders directly engaged in the patient care process (experts): 1) INT staff (Department Directors, Unit Heads, Ward Managers, Nursing Coordinators, Research Nurses and Researchers); 2) INT patients; 3) representatives of patient associations active within INT and 4) caregivers of INT patients. The sampling method was based on convenience. INT staff were more heavily represented due to their central role in designing, implementing, and evaluating care and research pathways, as well as their institutional capacity to influence organizational changes related to patient engagement.

At the same time, the involvement of patients, caregivers, and association representatives was considered crucial, as they are active participants in the hospital community and directly connected to the topic under investigation.

The literature does not provide a unanimous recommendation regarding the number of respondents in Delphi studies; however, some studies suggest a minimum of 80 respondents.¹³

Methodology

The Delphi method¹⁴ was employed to achieve the research aims. This method anonymously gathers feedback from experts to check for consensus on a specific topic. It works through an iterative process (rounds), where experts review and rate a series of topics or statements. Unlike methods like focus groups, the Delphi method does not involve direct interaction between experts. Instead, after each round, they are presented with the overall results and the reasoning behind others’ ratings. Experts then have the opportunity to adjust their responses in the next round or keep them the same, based on the newly provided data.

The Delphi method encourages the exchange of ideas and information, ensuring equal input from all participants, and prevents social desirability bias¹⁵ or group thinking,¹⁶ thanks to the anonymity of responses.

Following Okoli and Pawlowski guidelines,¹⁷ experts who agreed to participate were emailed instructions for completing the first-round questionnaire. The same process was followed for the second round.

Procedure

Although Delphi procedures have been modified over time to serve a variety of purposes,^17,18 the literature is unanimous in suggesting that a Delphi study should include at least a two-round process.

Round 1. The statements used in the Delphi method were selected after reviewing the literature on the subject, to identify relevant opinions, attitudes, intervention models, guidelines, and recommendations.

The following key topics were identified to structure the statements:

Why use PPI. This topic reflects the recognition of patient experience as a legitimate and essential form of knowledge to improve the quality of care and research. The statements value experiential knowledge (Statement 1) and the need to select and train patient representatives so that they can effectively fulfil their role as “experts by experience” (Statement 3).^19-22 This topic also reflects statements that highlight the expected benefits in terms of service planning, quality improvement, research relevance, and patient acceptance (Statements 5, 6, 8, 9, 10, 11). The underlying logic follows a patient-centered paradigm, in which active involvement leads to more effective and appropriate services;^8,23

Organizational and individual factors that facilitate/hinder PPI. This topic reflects economic and cultural barriers that may hinder sustainable participation (Statements 12). The lack of dedicated resources and failure to acknowledge the value of patient contributions are well-documented obstacles.^20,24,25 It also reflects statements addressing the need for formal participatory structures, ongoing training programs for staff, and institutional coordination units (Statements 2, 4, 7, 13). From these topics, 13 statements were created and presented to the experts. For each statement, they were asked to indicate their level of agreement on a 4-point Likert scale (from 1 = Strongly disagree to 4 = Strongly agree) and explain the reasons for their choice in an open-ended question. In addition, INT staff were asked eight specific items to share their views (multiple-choice) and indicate their usage and interest (yes/no) in the PPI topic.

For all experts, some socio-demographic information (age and educational level) was collected. For INT staff, we also collected the years of experience in INT.

Round 2. The statements that did not reach consensus in the first round were presented again, along with the reasons provided by participants. Participants were asked to rate their level of agreement on a 4-point Likert scale and explain their choices through an open-ended question.¹⁷

Analysis

The response steps for the 13 statements were grouped so that steps 1 and 2 indicated "disagreement" and steps 3 and 4 indicated "agreement" with the proposed statements. Percentages of "agreement" and "disagreement" were then calculated. Although the literature does not agree on a specific threshold to define consensus,²⁶ the most widely used threshold is 75%.²⁷ Given the research objectives and the potential for implementing PPI in the “real world”, this study set an 80% threshold consistent with previous studies.^17,28,29

For all proposed items, frequencies (n) and percentages (%) were calculated. For the statements that did not reach consensus, a conceptual analysis³⁰ of the open-ended responses was also conducted to better understand the reasons behind the assigned scores. Conceptual analysis helps researchers identify the essential properties of a concept, resolving any ambiguities in reasoning. It breaks down complex ideas into their constituent parts to better understand their nature and functioning.³⁰ Open-ended responses were grouped by meaning, and quantified. For example, answers such as “Does the Committee have the competence to decide where resources should be allocated?”, “Extensive preparation is needed,” and “There is a high risk of deviation toward poorly considered directions by stakeholders with heterogeneous training and preparation until the Committee’s independence/preparation is proven” were all conceptualized under the category “The Committee must be properly trained/informed.”

Ethical consideration

This study was approved by the institutional Ethics Committee of INT (Ref. INT 10/23) and complied with the Declaration of Helsinki.

Results

Round I

Participants

Seventy-eight experts from INT staff and 12 experts from patients, caregivers and representatives from patient associations responded to the first round (n = 90; 55.21% response rate). Ten experts from INT staff declined to participate in the study (it is not possible to know whether these individuals later provided consent or not). The characteristics of the participants are shown in Table 1.

Table 1.

Characteristics of participants in Round I.

	INT staff(n=78)		non-INT experts(n=12)		Total(n=90)
Age
Range	27-69		36-78		27-78
Mean	52.4		57.8		53.2
SD	11.2		12.8		11.5
Median	56		58		57
	n	%	n	%	n	%
Gender
Male	23	29.5	4	33.3	27	30.0
Female	55	70.5	8	66.7	63	70.0
Non-binary	0		0		0
Prefer not to answer	0		0		0
Educational qualification
Diploma	7	9.0	5	41.7	12	13.3
Bachelor’s degree	17	21.8	1	8.3	18	20.0
Master’s degree	14	17.9	4	33.3	18	20.0
Master’s/PhD/Specialization	40	51.3	2	16.7	42	46.7
Role in research
Case Manager	6	7.7
Nursing Coordinator	22	28.2
Technical Coordinator	7	9.0
Head of Complex Unit	12	15.4
Head of Simple Unit	16	20.5
Researcher	5	6.4
Other	10	12.8
Caregiver/Family Member			2	16.7
Expert Patient			7	58.3
Association Representative			3	25.0
Years of experience in INT
Less than 1 year	13	16.7
2 to 5 years	14	17.9
6 to 10 years	11	14.1
More than 10 years	40	51.3

Item ad hoc

Regarding the opinions of INT staff (multiple-choice response), about half believe that "PPI is morally/ethically the right thing to do" (56.4%) and that "PPI increases the chances of success in research studies" (48.7%). A smaller number of professionals, although present, expressed that they see no utility in PPI. Around a third of INT staff (34.6%) have used the help of patients, users, caregivers, or patient representatives in carrying out their activities, and a large portion of the staff is interested in further exploring the topic (74.4%) (Table 2).

Table 2.

Opinions of INT staff on PPI.

	Yes (multiple-choice response)
	n	%
PPI is morally / ethically the right thing to do	44	56.4
PPI increases the chances of success in research studies	38	48.7
The public – patients, service users, caregivers, or patient representatives – cannot significantly contribute to hospital service planning	12	15.4
PPI is not always necessary	8	10.3
I am not convinced of the benefits of PPI	4	5.1
The public – patients, service users, caregivers, or patient representatives – cannot significantly contribute to research	3	3.8
Have you ever involved patients, service users, caregivers, or patient representatives in your service design or research activities?	27	34.6
Would you be interested in attending a training course on PPI?	58	74.4

Delphi

Of the 13 statements proposed to the experts, eight reached the 80% consensus threshold: statements 1, 2, 3, 4, 5, 6, 10, and 11. Two statements (8 and 13) came close to the threshold but did not reach it, while three statements fell significantly short of the predefined threshold (statements 7, 9, and 12) (Table 3).

Table 3.

Results Round I of Delphi study.

Statement	INT staff (n=78)		non-INT experts(n=12)		Total(n=90)
Statement	n	% consensus	n	% consensus	n	% consensus
Item 1) The patient’s experience with care should be considered an integral part of the quality improvement system at the Fondazione IRCCS Istituto Nazionale dei Tumori of Milan (hereinafter INT)	77	98.7	11	91.7	88	97.8
Item 2) A Committee representing patients and everyone involved at various levels in the disease journey (patients, family members, caregivers, patient associations) should be established to promote their involvement in the planning and organization of care services and research at INT	69	88.5	11	91.7	80	88.9
Item 3) Committee members should be selected and trained to serve as experts in the lived experience of illness (personal or indirect)	66	84.6	11	91.7	77	85.6
Item 4) INT should implement structured and ongoing awareness and training programs for its medical, nursing, and administrative staff	78	100	12	100	90	100
Item 5) The Committee should act as a bridge between INT and its patients	74	94.9	12	100	86	95.6
Item 6) The Committee should provide guidance on service quality, risk management, and the relevance of research objectives at INT	62	79.5	11	91.7	73	81.1
Item 7) The Committee should be involved in the decision-making by INT’s main governing bodies (e.g., General Management, Scientific Directorate, Health Directorate)	50	64.1	10	83.3	60	66.7
Item 8) The Committee’s involvement in healthcare service planning, resource allocation, and prioritization at INT should better address public needs and expectations	60	76.9	11	83.3	71	78.9
Item 9) The Committee’s involvement should improve the relevance of INT’s research	50	64.1	12	100	62	68.9
Item 10) The Committee’s involvement should improve recruitment and retention in research projects by ensuring that communication methods and research approaches are responsive to the needs, habits, and circumstances of the community	60	76.9	12	100	72	80.0
Item 11) The Committee’s involvement in healthcare and research decisions at INT should increase patient-centeredness, as well as patient trust and acceptance of these decisions	70	89.7	12	100	82	91.1
Item 12) INT should allocate a budget to reimburse Committee members for expenses and support engagement activities	36	46.2	9	75.0	45	50.0
Item 13) INT should establish an institutional coordination center to promote and ensure engagement activities	61	78.2	9	75.0	70	77.8

Table 4 shows the results of the conceptual analysis quantification regarding the reasons provided by respondents for their choices (for those who offered explanations). It is important to note that much of the information provided was classified as "uninformative," as respondents either reiterated their Likert scale score (e.g., "I strongly agree," "Definitely") or expressed irrelevant or unclear concepts (e.g., "yes," "project purpose"), or indicated that they were unable to answer (“I don’t have to say it”/ “I can’t know”).

Table 4.

Results of the conceptual analysis quantification.

	Grouping of open-ended question responses	Round I	Round II
7) The Committee should be involved in the decision-making by INT’s main governing bodies (e.g., General Management, Scientific Directorate, Health Directorate).		n = 25	n = 35
	The Committee should have only an advisory role / serve to represent the patient perspective	36%	42.86%
	There is no need for full involvement / it’s enough for the Committee to be kept informed	12%	5.71%
	The Committee should be involved only in specific decisions / should only be kept informed	12%	17.14%
	The Committee is effective only if it is heard by the leadership of INT	12%	14.29%
	The Committee must be properly trained / informed	4%	14.29%
	Uninformative responses	24%	5.71%
8) The Committee’s involvement in healthcare service planning, resource allocation, and prioritization at INT should better address public needs and expectations.		n = 22	n = 36
	The Committee should have only an advisory role / serve to represent the patient perspective	22.72%	27.78%
	The Committee should be involved only in specific decisions	4.55%	11.11%
	The Committee must be properly trained / must have specific expertise	18.18%	25%
	It is not necessary	4.55%	2.78%
	It is difficult to implement	0%	5.56%
	The Committee can be a source of enrichment.	0%	13.89%
	Uninformative responses	50%	13.88%
9) The Committee’s involvement should improve the relevance of INT’s research. (Round I) / The Committee’s involvement should improve the selection of research goals by aligning them with patient needs (Round II).		n = 22	n = 33
	The Committee should have only an advisory role / serve to represent the patient perspective	4.54%	6.07%
	The Committee should be involved only in specific decisions	4.54%	0%
	The Committee must be properly trained / must have specific expertise	4.54%	12.12%
	Scientific research should be left to the researchers / Scientific Directorate	4.54%	18.18%
	The Committee must promote the dissemination of research	4.54%	3.03%
	The Committee can be a valuable resource	0	18.18%
	It’s not necessary / It may only be useful in certain situations	13.64%	12.12%
	Uninformative responses	50%	30.30%
12) INT should allocate a budget to reimburse Committee members for expenses and support engagement activities.		n = 17	n = 36
	Participation must be voluntary	17.64%	22.22%
	Only for specific purposes /activities	17.64%	5.55%
	Only in the case of ‘active collaboration’ from patients	5.89%	2.78%
	Participation must be compensated	11.76%	25%
	Uninformative responses	47.06%	22.22%
13) INT should establish an institutional coordination center to promote and ensure engagement activities. (Round I)/ INT should establish an institutional coordination center to promote and ensure the involvement of patients and others involved at various stages of the illness journey.		n = 16	n = 42
	Risk of excessive bureaucracy / complexity	25%	23.81%
	Structuring is useful for achieving the objectives	25%	9.52%
	The Committee must be independent / suprainstitutional	0	9.52%
	It is not necessary	12.5%	14.28%
	Perceived usefulness	0	19.05%
	Uninformative responses	37.5%	32.81%

Regarding statement 7 ("The Committee should be involved in the decision-making by INT’s main governing bodies"), respondents who disagreed with the statement and provided an informative response generally believed that the Committee should only have an "advisory" role, not a decision-making one (36%).

For statement 8 ("The Committee’s involvement in healthcare service planning, resource allocation, and prioritization at INT should better address public needs and expectations"), respondents who disagreed believed that: 1) the Committee should have an advisory role rather than a decision-making one (22.7%); and 2) the Committee may lack the expertise needed to fulfill this function (18.2%).

For statement 9 ("The Committee’s involvement should improve the relevance of INT’s research by focusing on patient needs"), responses were mixed. Notably, 13.64% of respondents stated, "It’s unnecessary / It may only be useful in certain situations". Some respondents, however, requested clarification on the meaning of the statement.

For statement 12 ("INT should allocate a budget to reimburse Committee members’ expenses and support engagement activities"), respondents who disagreed believed that the Committee: 1) should be volunteer-based (17.64%); and 2) expenses should be reimbursed "only for specific purposes / activities" (17.64%), although these were not elicited.

Regarding statement 13 ("INT should establish an institutional coordination center to promote and ensure engagement activities"), about 25% of respondents who disagreed with the statement expressed concerns about the risk of excessive bureaucracy or complexity (25%), while an equal percentage believed that creating such a structure would be useful in achieving the goals. Some respondents, however, requested clarification on the meaning of the statement.

Three of the five statements that did not reach the expected threshold (statements 7, 8, and 12) were presented again in Round II in their original wording. In response to requests for clarification on certain answers, two statements (9 and 13) were reworded (see Table 4 for details).

Round II

Participants

Respondents in Round II were those who had also taken part in Round I. Fifty-seven experts from INT staff and 12 experts from patients, caregivers and representatives of patients’ associations responded to the second round (n = 69; 79.31% response rate). The characteristics of the participants are shown in Table 5.

Table 5.

Characteristics of participants in Round II.

	INT staff(n=57)		non-INT experts(n=12)		Total(n=69)
Age
Range	27-67		36-78		27-78
Mean	53.37		57.83		54.16
SD	11.03		13.27		12.78
Median	57.5		58		57.5
	n	%	n	%	n	%
Gender
Male	16	28.1	4	36.3	20	29.0
Female	41	71.9	8	66.6	49	71.0
Non-binary	0				0
Prefer not to answer	0				0
Educational qualification
Diploma	1	1.8	5	41.7	6	8.7
Bachelor’s degree	14	24.6	1	8.3	15	21.7
Master’s degree	13	22.8	4	33.3	17	24.6
Master’s/PhD/Specialization	29	50.9	2	16.7	31	44.9
Role in research
Case Manager	4	7.0
Nursing Coordinator	14	24.6
Technical Coordinator	6	10.5
Head of Complex Unit	8	14.0
Head of Simple Unit	12	21.1
Researcher	2	3.5
Other	11	19.3
Caregiver/Family Member			2	16.7
Expert Patient			7	58.3
Association Representative			3	25.0

Delphi

Of the five statements presented to the experts, none reached the 80% consensus threshold. Statements 7 and 8 had the highest agreement, while statement 12 remained the most problematic (Table 6).

Table 6.

Results of Round II of Delphi study.

Statement	INT staff (n=57)		non-INT experts (n=12)		Total (n=69)
Statement	n	% consensus	n	% consensus	n	% consensus
Item 7) The Committee should be involved in the decision-making by INT’s main governing bodies (e.g., General Management, Scientific Directorate, Health Directorate)	40	70.2	9	75.0%	49	71.0
Item 8) The Committee’s involvement in healthcare service planning, resource allocation, and prioritization at INT should better address public needs and expectations	43	75.4	8	66.7	51	73.9
Item 9) The Committee’s involvement should improve the relevance of INT’s research by aligning them with patient needs (*)	34	59.6	9	75.0	43	62.3
Item 12) INT should allocate a budget to reimburse Committee members for expenses and support engagement activities	22	38.6	7	58.3	29	42.0
Item 13) INT should establish an institutional coordination center to promote and ensure the involvement of patients and others involved at various stages of the illness journey (*)	36	61.4	8	66.7	44	62.3

Note: * Items revised from Round I based on respondents’ feedback.

For statement 7 ("The Committee should be involved in decision-making by INT’s main governing bodies") and 12 ("INT should allocate a budget to reimburse Committee members and support engagement activities"), the same arguments from Round I resurfaced, with only minor percentage differences (Table 4).

For statement 13 ("INT should establish an institutional coordination center to promote and ensure engagement activities"), a new idea emerged, suggesting that the Committee should be independent or suprainstitutional, although this was mentioned by a small percentage (9.52%).

For statement 9 ("The Committee’s involvement should improve the relevance of INT’s research"), there is an increase in both those who believe research should be left to the researchers / Scientific Directorate and those who view the Committee as a valuable resource (both 18.2%).

General discussion

This study revealed that, although PPI is a topic of theoretical interest (about 74% of INT staff expressed interest in exploring it further), it aligns with respondents’ values (56.4%), and is seen as a potential success factor for research (49%), it is not yet a well-established practice at INT (only around 35% have made use of PPI). Additionally, the establishment of a Committee in the “real world” is currently difficult to implement. The Delphi results indicates that experts agree on the importance of patient experience in improving service quality, the need for a Committee to strengthen the link between patients and the hospital, and that the Committee should play an advisory role in improving the quality of services and research at INT. However, there is no agreement on the Committee’s role. This disagreement may reflect a broader challenge, already described in the literature,^19,20 related to the lack of clarity concerning the tasks, responsibilities, and powers of PPI in organizational processes. The key challenge lies in defining the Committee’s roles and responsibilities, which underscores the cultural and organizational barriers to effectively integrating the Committee into current practices. In particular, there is disagreement over whether the Committee should have an advisory or decision-making role, a key issue raised in several open-ended responses. This may involve uncertainties about how to effectively implement patient and public involvement so that it truly impacts decisions and policies.^8,31

There are also concerns about the training and qualifications of the Committee, fears that an additional body could become overly cumbersome or bureaucratic, and finally, the question of whether participation should be voluntary. This last point, in particular, may reflect a lack of unanimous recognition of the economic value, time, and energy invested by participants in PPI Committees, a phenomenon already noted in the literature.³² This may reflect a broader belief that the work of the Committee is "non-professional".

These results, as a whole, reflect the gap between the theoretical interest in PPI and its practical implementation, a phenomenon well documented in the literature.^7,8 Mackenzie et al highlight, for example, how meaningful patient and public involvement can be hindered by a lack of clarity in roles and responsibilities, and they suggest the need for a conceptual and methodological framework to guide the effective and systematic integration of patients and the public into decision-making processes related to health research, service design, and health policy formulation.^33,35

Liabo et al.²¹ propose using deliberative democracy as a framework to improve PPI quality. This approach emphasizes fair, structured participation with clearly defined ways for participants to contribute. The authors argue that having clear, well-understood roles can improve the validity and effectiveness of patient and public involvement.²¹ Greenhalgh et al. also propose a co-creation model for PPI, emphasizing that clearly defining roles and responsibilities within collaborative frameworks can lead to a tangible impact on community healthcare services.³³ The model highlights the importance of frameworks that promote fair and transparent collaboration among all stakeholders.

In conclusion, there appear to be significant barriers to implementing PPI in our hospital, primarily due to cultural factors. Believing that the Committee is not adequately trained / does not have specific expertise, that it should only have an advisory role and not a decision-making one, and that it should work voluntarily reflects a cultural mindset that does not value PPI. This underscores the need for genuine engagement and the development of strategies to promote structural and cultural changes within our hospital.²³ A cultural shift requires the adoption of a new mindset by all parties involved (patients, caregivers, public and doctors, researchers). Several international organizations, such as INVOLVE in UK,³⁶ Patient-Centered Outcomes Research Institute (PCORI)³⁷ in the US, Patient - Canadian Institutes for Health Research,³⁸ have been created to promote resources like guidelines and recommendations, as well as training and information programs, for those involved in this new approach to care and research.

To address these challenges, we propose some actionable strategies. 1) Formal recognition of the expert patient role within organizational policies and strategic plans, including the definition of competencies, responsibilities, and ethical standards; 2) Mandatory PPI training modules for healthcare professionals and researchers, integrated into institutional continuous education programs, aimed at developing communication, co-design, and relational skills; 3) Creation of a central coordination unit for PPI to support engagement processes, promote cross-departmental collaboration, and monitor progress; 4) Allocation of dedicated budgets to cover reimbursement for patient representatives, logistic support, and communication materials; 5) Finally, including PPI indicators in performance and quality assessment systems might be helpful, to promote meaningful engagement as part of institutional excellence frameworks.

This study has some limitations that should be considered. First, the response rate from INT staff in Round I of the Delphi was low: 55.21%. Such a low rate does not ensure the respondents are representative. However, this is also an interesting research finding, as it may indicate only partial interest in the topic among INT staff. Notably, researchers—who should be key players in promoting PPI—had especially low participation. Second, item 4 of the Delphi survey (“INT should implement structured and ongoing awareness and training programs for its medical, nursing, and administrative staff”) reached 100% consensus, likely due to poor wording. Respondents may have interpreted it in general terms, rather than in relation to PPI specifically.

Conclusions

The results of this study are valuable as they highlight that, in the “real world”, the cultural shift regarding the role that patients, patient associations, and caregivers can play in the planning and organization of care services and research, as widely advocated in the literature, has not yet occurred. Therefore, further training and awareness initiatives on these topics are necessary, drawing from models and guidelines developed by international organizations that can serve as resources to facilitate this change³⁶ and that encourage the active inclusion of patients in key stages of research, from formulating research questions to disseminating results, using a collaborative approach that ensures patient contributions have a tangible impact on the decision-making process.^37,38 These organizations offer useful models that can support the adoption of meaningful and systematic involvement practices, helping bridge the gap between theoretical interest and practical application of patient and public involvement (PPI) in research and healthcare settings.

Footnotes

Acknowledgements

We sincerely thank the patients, caregivers, association representatives and colleagues who kindly participated in the research.

Author contributions

All the authors contributed to the study conception and methodological design. L.G., V.A. and Cl.B. contributed to project ideation. L.G. and V.A. contributed to data acquisition. S.A. performed data analysis. All authors contributed to results interpretation. The first draft of the manuscript was written by S.A. and all authors commented on and contributed to previous versions of the manuscript. All authors read and approved the final manuscript.

Data availability

Data are available upon request to the corresponding author.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Sara Alfieri

Cinzia Brunelli

References

Entwistle

Watt

. Treating patients as persons: a capabilities approach to support delivery of person-centered care. Am J Bioethics 2013; 13: 29–39.

Bombard

Baker

Orlando

, et al. Engaging patients to improve quality of care: a systematic review. Implement Sci 2018; 13: 1–22.

Richards

Montori

Godlee

, et al. Let the patient revolution begin. BMJ 2013; 346: f2614.

Baumann

Brütt

. Public and patient involvement (PPI) in health policy decisionmaking on the health system-level: protocol for a systematic scoping review. BMJ Open 2021; 11: e043650.

Vinnicombe

Bianchim

Noyes

. A review of reviews exploring patient and public involvement in population health research and development of tools containing best practice guidance. BMC Pub Health 2023; 23: 1271.

Johnson

Abraham

Conway

, et al. Partnering with patients and families to design a patient-and family-centered health care system. Inst Patient Family Centered Care Inst Healthcare System. Bethesda: Institute of Family-Centered Care; 2008.

Arumugam

Phillips

Moore

, et al. Patient and public involvement in research: a review of practical resources for young investigators. BMC Rheumatol 2023; 7: 2.

Ocloo

Garfield

Franklin

, et al. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Syst 2021; 19: 1–21.

Hippalgaonkar

Nguyen

RHT

Cohn

, et al. Are we the problem? A call to action for addressing institutional challenges to engaging community partners in research. Int J Enviro Res Pub Health 2024; 21: 236.

10.

Baines

Regan

Bere

. Optimizing patient and public involvement (PPI): identifying its “essential” and “desirable” principles using a systematic review and modified Delphi methodology. Health Expectations 2018; 21: 327–335.

11.

Chuah

FLH

Srivastava

Singh

, et al. Community participation in general health initiatives in high and upper-middle income countries: a systematic review exploring the nature of participation, use of theories, contextual drivers and power relations in community participation. SocSciMed 2018; 213: 106–122.

12.

OECI. Accreditation and Designation User Manual Version 2.0, https://www.oeci.eu/Attachments/OECI_ACC_Manual_2_0.pdf

13.

Manyara

Purvis

Ciani

, et al. Sample size in multistakeholder Delphi surveys: at what minimum sample size do replicability of results stabilize? J Clin Epidemiol 2024; 174: 111485.

14.

Dalkey

Helmer

. An experimental application of the Delphi method to the use of experts. Manage Sci 1963; 9: 458–467.

15.

Snyder

. Self-monitoring of expressive behavior. J Pers Soc Psychol 1974; 30: 526.

16.

Janis

. Victims of groupthink: A psychological study of foreign-policy decisions and fiascoes. Boston: Houghton Mifflin; 1972.

17.

Okoli

Pawlowski

. The Delphi method as a research tool: an example, design considerations and applications. Info Manage 2004; 42: 15–29.

18.

Fletcher

Marchildon

. Using the Delphi method for qualitative, participatory action research in health leadership. Int J Qualitative Methods 2014;13(1):1–18.

19.

Boivin

L’espérance

Gauvin

, et al. Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expectations 2018; 21: 1075–1084.

20.

Greenhalgh

Hinton

Finlay

, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expectations 2019; 22: 785–801.

21.

Liabo

Boddy

Bortoli

, et al. Public involvement in health research: what does ‘good’ look like in practice? Res Involve Engage 2020; 6: 1–12.

22.

Manafo

Petermann

Mason-Lai

, et al. Patient engagement in Canada: a scoping review of the ‘how’ and ‘what’ of patient engagement in health research. Health Res Policy Syst 2018; 16: 1–11.

23.

Carman

Dardess

Maurer

, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff 2013; 32: 223–231.

24.

Staniszewska

Denegri

Matthews

, et al. Reviewing progress in public involvement in NIHR research: developing and implementing a new vision for the future. BMJ Open 2018; 8: e017124.

25.

Koch

Brust

Weigl

. Barriers and facilitators of patient engagement activities to improve patient safety in healthcare organizations: A Delphi-Based Expert Survey. Risk Management Healthcare Policy 2024; 3217–3226.

26.

Giannarou

Zervas

. Using Delphi technique to build consensus in practice. IJBSAM. 2014; 9: 65–82.

27.

Diamond

Grant

Feldman

, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014; 67: 401–409.

28.

Hsu

Sandford

. The Delphi technique: making sense of consensus. Pract Assess Res Eval 2007; 12: 10.

29.

Weigl

Cieza

Andersen

, et al. Identification of relevant ICF categories in patients with chronic health conditions: a Delphi exercise. J Rehabil Med 2004; 36: 12–21.

30.

Coombs

. Conceptual analysis. Forms Curr Inquiry 1991; 27.

31.

Shields

Brown

Wells

, et al. Utilising patient and public involvement in stated preference research in health: learning from the existing literature and a case study. Patient Centered Outcomes Res 2021; 14: 399–412.

32.

Hamilton

Hoens

Backman

, et al. An empirically based conceptual framework for fostering meaningful patient engagement in research. Health Expect 2018; 21: 396–406.

33.

Greenhalgh

Jackson

Shaw

, et al. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q 2016; 94: 392–429.

34.

Selman

Clement

Douglas

, et al. Patient and public involvement in randomised clinical trials: a mixed-methods study of a clinical trials unit to identify good practice, barriers and facilitators. Trials 2021; 22: 1–14.

35.

McKenzie

Hanley

(2014). Planning for consumer and community Participation in Health and Medical research: A practical guide for health and medical researchers. Telethon Kids Institute.

36.

National Institute for Health and Care Center. Resources and training for public involvement in research, https://www.learningforinvolvement.org.uk/ Accessed April 17, 2025.

37.

Patient Centered Outcomes Research Institute. Research Engagement Guide, https://www.pcori.org/sites/default/files/AltaMed-Research-Engagement-Guide.pdf Accessed April 25, 2025.

38.

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research, https://cihr-irsc.gc.ca/e/41204.html Accessed April 18, 2025.

Delphi Study on the creation of a Committee of patients,caregivers,and representatives of patient associations

Abstract

Background:

Aim:

Method:

Results:

Conclusion:

Keywords

Introduction

Method

Participants

Methodology

Procedure

Analysis

Ethical consideration

Results

Round I

Participants

Item ad hoc

Delphi

Round II

Participants

Delphi

General discussion

Conclusions

Footnotes

Acknowledgements

Author contributions

Data availability

Declaration of conflicting interests

Funding

ORCID iDs

References