Sage Journals: Discover world-class research

Abstract

Introduction:

The importance of research and development in all aspects of healthcare is well acknowledged. Within critical care, national guidelines provide a limited number of standards and series of recommendations on Research and Development (R&D) activity. The aim of this study was to create a broader set of standards in support of R&D activity in critical care departments.

Methods:

A modified Delphi study was undertaken across NHS Wales critical care units. Proposed standards were developed by a group of experts, which across three rounds, majority agreement was sought. Additional standards were added based on participant’s responses.

Results:

This study identified 49 standards for R&D activity within critical care units in NHS Wales. All these standards reached majority agreement, as determined by >70% of multi-disciplinary participants determine each standard as essential.

Conclusions:

The results of this study will be utilised within Wales to inform wider service specification with regard workforce requirements, responsibilities, reporting and collaboration.

Keywords

Research critical care intensive care unit standards Delphi

Introduction

The UK Guidelines for the Provision of Intensive Care Services (GPICS v2.1)¹ recognises the importance of participation in research and development (R&D) within critical care. GPICS includes a single standard (that which must be addressed in routine practice) and six recommendations (those which should represent routine practice). All individuals participating in R&D activity must have completed Good Clinical Practice (GCP) training for research and keep this up to date.¹ GPICS additionally recommends the need for critical care units to nominate a lead for R&D activities,² for units to participate in research networks³ and for all research studies to be registered on the UK National Institute for Health Research (NIHR) Clinical Research Network Portfolio whenever they fulfil eligibility criteria.⁴ Furthermore, GPICS recommends the provision of information to patients, relatives, and other surrogate decision-makers (SDM’s) about research participation, the development of clear procedures for approaching SDM’s in a timely and effective manner, and the adoption of co-enrolment policies based upon national guidance when multiple studies are undertaken.⁵

The National Institute for Health and Care Excellence (NICE) makes recommendations to implement and fund new treatments and interventions based on cost-effectiveness from the NHS perspective⁶; these factors make participation in Research and Development (R&D) activities of vital importance to critical care units and the patients they serve.¹ Furthermore, given its highly visible contribution to the delivery of urgent public health studies during the COVID-19 pandemic,⁷ there is an imperative for the critical care speciality to ensure that such momentum persists, and that clinical research becomes part of everyday practice for all healthcare professionals.⁸

At present there are no recommendations regarding workforce required for the design, conduct and delivery of research, or the overall responsibility for R&D activity beyond the naming of a recognised lead. Beyond GPICS, to the best of our knowledge there have been no previous attempts to define the wider standards or specifications for research within critical care, nor any other clinical speciality. The aim of this study was therefore to utilise a modified Delphi approach to create a set of standards in support of R&D activity in Welsh critical care departments. Our ultimate intention was for these standards to be adopted within a wider NHS Wales service specification for critical care.

Methodology

Design

A modified Delphi technique⁹ was used to obtain a majority agreement on the professional practice standards for R&D activity in critical care units in Wales. A Delphi technique allows many individuals across diverse locations to participate in free discussion of views in an anonymous manner and the combination of many opinions into a collective response prevents domination of the process by one or a few participants.^9
–11

Participants, therapists, centres and eligibility procedure

Potential participants were selected as experienced critical care stakeholders with an interest in the outcomes of the project.¹² We aimed to select critical care clinicians and researchers from across NHS Wales with experience in research design, conduct, delivery or oversight within a critical care setting. The target panel size required to enable majority agreement while remaining practically achievable was 25 and 40 participants. This was selected on the basis of being a relatively homogenous sample (limited to clinicians with experience of R&D within critical care) but to also appreciate the range of professional groups invited to participate and potentially allow sub-analysis of the agreement findings if deemed appropriate.¹³

Potential participants were identified via the research and clinical leads for each critical care unit within NHS Wales. Furthermore, the invitation to participate was extended to all relevant healthcare professions in all NHS Wales critical care units. The eligibility for inclusion criteria was:

Healthcare professional working within a critical care unit in NHS Wales

Involved in some form of critical care R&D activity including experience as chief or principal investigator and/or involvement in research design, set-up, conduct, delivery or oversight.

An eligibility questionnaire was utilised to allow this background data to be collected and to confirm eligibility.

Participants who wished to remain anonymous were unable to be included in this study as respondents needed to be able to receive personalised feedback to enable comparisons with other respondents.¹⁴ This was considered an essential part of the Delphi technique to enable consensus to be reached.¹⁵

Questionnaire development

An initial ‘expert’ focus group was held to establish the key themes to be captured within the Delphi and to formulate initial statements to be used in round 1 of the Delphi study.⁹ This focus group consisted of seven ‘experts’ with representatives of the Welsh Critical Care Research Group (a collaboration between Wales Critical Care and Trauma Network and Welsh Intensive Care Society), the Health and Care Research Wales (HCRW) speciality research lead for critical care, HCRW speciality research lead for Trauma and Emergency Medicine, critical care clinicians from multi-disciplinary backgrounds actively involved in R&D as chief or principal investigators and one member from an NHS Wales R&D department with extensive experience of supporting trials within critical care.

Procedure

Three rounds of questionnaires were administered to the selected participants with each round allowed up to 4 weeks to complete.¹⁴ Questionnaires were disseminated electronically via Online Surveys. A participant information sheet was included in the invitation email. Electronic reminders to submit were issued to non-responders at set time periods (2 and 3 weeks).

Participants were asked to indicate support for each statement, using a three-point scale to define its relevance as a standard for research and development activity. Items were ranked as Essential, Not Essential or Unsure. Open comment fields were possible for in each statement in all rounds, and participants were invited to submit additional statements in Rounds 1 and 2.

At the end of each round, each participant was sent a personalised report containing quantitative group results, qualitative feedback and the participant’s own responses for comparison purposes.

Analysis

Demographic data were collected in the eligibility screening survey and the Round 1 Delphi questionnaire. Data analysis was primarily descriptive and open comments collated and grouped narratively. Data entry and descriptive analysis was performed using Microsoft Excel™.

Majority agreement for inclusion or exclusion on a statement was determined to have been achieved a priori when agreement as the item being ‘Essential’ was documented by greater than 70% of respondents, or conversely where ‘Not Essential’ was documented by greater than 70% of respondents. Statements were excluded from subsequent rounds when less than 30% of respondents documented them as ‘Essential’ items. In the absence of a recommended criterion for inclusion within the literature,¹⁶ the 70% threshold was selected based on previous research conducted by members of the research team and other healthcare related Delphi studies.^17,18 Similarly, there is a paucity of literature currently available on the optimal number of response categories. Whilst reliability and factorial validity has been shown to improve with increased numbers of categories,¹⁹ this is heavily offset by potential for inconsistency in category interpretation and misleading results,²⁰ hence only three categories were utilised.

Items for which majority agreement was achieved were removed from each subsequent questionnaire round.²¹ Items for which majority agreement was not reached were refined by the authors (if required) and included in the subsequent round. Additional item suggestions from Rounds 1 and 2 were cross-referenced against existing survey items and were included in following rounds if they did not duplicate an item.

Ethics approval

This study was discussed with the local research and development department and Health Care Research Wales and deemed not to require full research permissions or ethical application. This decision was based on the absence of randomisation of participants, no changes to any treatment or patient care and understanding that the findings would not be generalised outside of the identified population without additional research. Furthermore, no participant identifiable data has been used within the analysis or manuscript.

Results

The eligibility questionnaire was completed by 44 critical care clinicians, of whom 43 were invited to participate in Round 1. Of these, 31 completed Round 1 (72.1%) and 25 participants completing all three rounds (see Figure 1).

Figure 1.

Participant involvement.

Of the 31 participants in Round 1, there were 10 critical care consultants, 2 senior medical trainees, 1 advanced critical care practitioner (ACCP), 5 critical care nurses, 5 research nurses (either directly employed by critical care or the R&D department), 5 allied health professionals and 3 pharmacists. All were regularly involved in R&D activity within critical care, including being chief or principal investigators and/or involved in research study set-up, delivery of research interventions, data collection and data analysis. Representation was achieved from all six NHS Wales Health Boards with a critical care unit in all rounds.

Round 1 consisted of 43 statements items, of which 36 were deemed as essential. Based on responses, an additional three questions were added to Round 2, during which a further 10 items were deemed as essential. Three additional statements were added for Round 3, of which all three were determined as essential. After three rounds, 49 statements reached consensus to be classified as essential, whereas 0 were determined as not essential or majority agreement agreements was not reached (see Figure 2).

Figure 2.

Inclusion and exclusion of statements.

Tables 1 –7 detail all items reaching majority agreement to be essential, not essential and no majority agreement respectively. Items have been themed. Note items for which there was >70% ‘essential’ majority agreement did not proceed to subsequent rounds.

Table 1.

Essential items relating to workforce.

Workforce
	R1	R2	R3
Wales Critical Care and Trauma Network must have a nominated lead for R&D activity (ordinarily the Health and Care Research Wales (HCRW) speciality lead).	96.8%
Critical Care departments must have a recognised departmental lead for R&D activities.	100%
Where the nominated departmental critical care lead for R&D activities is a consultant or SAS doctor, they must have a minimum of 2 sessions recognised for this activity.	64.5%	80.0%
Critical care departments must distribute and clearly specify 5% of the total Supporting Professional Activity (SPA) sessions available in the department for research	61.3%	84.0%
Where the recognised departmental lead is non-medical, they should have a minimum of 0.2 WTE of their job plan designated for this activity.	77.4%
Critical care departments must have a recognised nurse R&D lead.	93.5%
The nurse R&D lead must have a minimum 0.2 WTE protected time for R&D within their job role	87.1%
Critical care departments must have a recognised AHP R&D lead.	74.2%
The AHP R&D lead must have a minimum of 0.1 WTE protected time for R&D within their job role.	67.7%	84.0%*
Critical care departments must have a recognised pharmacy R&D lead.	77.4%
The pharmacy R&D lead must have a minimum of 0.1 WTE protected time for R&D within their job role.	67.7%	80.0%*
The 0.1 WTE protected time for the pharmacy lead must be separate from pharmacy clinical trials support	67.7%	72.0%
Critical care departments participating in observational, interventional, or commercially sponsored trials must have dedicated research staff (nurse/AHP/pharmacist) support at a ratio of 0.2 WTE per critical care bed, providing a service 7 days per week, to include screening, consenting, interventions as necessary, follow-up and data management	64.5%	76.0%*
Critical care departments participating in observational or interventional studies must have dedicated trial administrative support from R&D.	96.8%
All critical care departments must have a recognised link and/or named contact within the heath board R&D department to support critical care research	N/A	N/A	100%
The quantity of the trial administrative support from R&D must be appropriate for the complexity of the study.	93.5%

Statements reworded between rounds based on comments (see Appendix 1).

Table 2.

Essential items relating to education and training.

Education and training
	R1	R2	R3
All individuals participating in R&D activity must have completed proportionate good clinical practice (GCP) training for research and keep this up to date.	90.3%
All critical care departments must maintain a record of staff that have completed proportionate GCP training.	100%
All potential and current principal investigators must attend principal investigator training provided by Health and Care Research Wales or local health board	87.1%
All critical care departments must offer further training in research governance via their internal CPD programme or via their R&D Department.	80.6%
All critical care departments must offer study specific training to the whole MDT team as appropriate.	93.5%
Critical care departments must ensure all members of the MDT are able to access training relevant to R&D activity, including study-specific training, GCP and informed consent	N/A	100%
NHS Wales Critical Care and Trauma Network in collaboration with Welsh Intensive Care Society must hold at least yearly critical care research event/conference, and research webinars every 3 months	N/A	92.0%

Table 3.

Essential items relating to responsibilities.

Responsibilities
	R1 (%)	R2	R3
The critical care departmental lead for R&D must coordinate R&D activity and ensure it is carried out to UK Policy Framework for Research standards.	100
The critical care departmental lead for R&D must liaise with the Health and Care Research Wales (HCRW) speciality lead to inform on progress and potential future studies for adoption.	93.5
The critical care departmental lead for R&D must liaise with the local R&D department to inform on progress and potential future studies for adoption.	93.5
The critical care departmental lead for R&D, in conjunction with the AHP/Pharmacy/Nurse R&D leads, dedicated research staff and health board R&D representative must meet regularly to discuss all relevant aspects of current and future critical care R&D activity	96.8
The critical care departmental lead for R&D must distribute lessons learnt from research governance reports	93.5
The AHP R&D lead will have responsibility for facilitating R&D activity including the promotion of AHP led critical care research, involvement in wider critical care R&D activity and support the critical care departmental lead for R&D.	90.3
The pharmacy R&D lead will have responsibility for facilitating R&D activity including the promotion of pharmacy led critical care research, involvement in wider critical care R&D activity and support the critical care departmental lead for R&D.	93.5
The nurse R&D lead will have responsibility for facilitating R&D activity including the promotion of nurse led critical care research, involvement in wider critical care R&D activity and support the critical care departmental lead for R&D.	93.5
All critical care departments participating in research must provide information to patients, relatives, and surrogate decision makers (SDM’s) about ongoing research.	96.8
Critical care departments participating in research must have clear procedures for approaching patients, families and SDM’s in a manner that minimises stress, but provides adequate information in a timely manner.	100
Critical care departments participating in multiple research studies should have clear co-enrolment policies based on UK co-enrolment guideline.	96.8

Table 4.

Essential items relating to reporting processes.

Reporting processes
	R1	R2	R3
The critical care departmental lead for R&D must report at least 3-monthly to the local critical care directorate management team (may be verbal discussion which is recorded in the minutes or written summary included in minutes).	51.6%	88.0%*
Critical Care departments must hold R&D meetings accessible to the entire MDT to update on current and future R&D activity, and to feedback on outputs from research completed on the unit at a minimum quarterly	N/A	96.0%
Information regarding ongoing trials should be accessible online for all staff within the critical care department	N/A	N/A	92.0%

Statements reworded between rounds based on comments (see Appendix 1).

Table 5.

Essential items relating to collaboration.

Collaboration
	R1 (%)	R2	R3
The critical care departmental research lead, dedicated research staff, AHPs, pharmacists and PIs/Associate PIs should be Table 5: encouraged to attend local research network meetings	93.5
The critical care departmental research lead, dedicated research staff, AHPs, pharmacists and PIs/Associate PIs should be encouraged to attend the annual UK Critical Care Research Forum.	80.6
Critical care departments must collaborate with other clinical speciality research teams/departments for example, anaesthesia, emergency medicine and trauma.	80.6
Critical care departments must collaborate with higher education institutes (HEIs) for R&D activity.	71.0

Table 6.

Essential items relating to development of future research workforce.

Development of future research workforce
	R1 (%)	R2	R3
All critical care departments must support the entire MDT to be involved in named research studies with opportunity to shadow the CI/PI.	83.9
All critical care departments must offer the possibility of mentoring new prospective PIs via the NIHR Associate PI scheme.	87.1
All critical care departments must actively encourage the entire MDT to seek research funding available via Health and Care Research Wales for example, Research for Patient and Public Benefit (RfPPB)	77.4
All critical care departments must actively encourage the entire MDT to seek research professional development opportunities available via HCRW for example, time for research, HCRW fellowships.	80.6

Table 7.

Additional essential items not within existing themes.

Others
	R1	R2	R3 (%)
Each critical care department must ensure admin facilities for the completion of all aspects of R&D activity	N/A	N/A	88.0

During the three rounds 104 comments were made by the participants regarding the items in the questionnaire. Of these, 61 comments were made in Round 1; 38 in Round 2 and 5 in Round 3. Table 8 shows the highest ranked comments occurring in each round.

Table 8.

Five highest ranked themes across all three rounds.

Overall rank	Theme	Round 1	Round 2	Round 3	Total
1	Potential inability to appropriately job plan the required medical workforce including SPA time	27	14	2	43
2	Requirement for altered workforce requirements depending on department R&D activity*	13	9	3	25
3	Requirement for altered workforce requirements depending on size of critical care unit*	10	9	0	19
4	Recommended whole time equivalent for AHP lead role too high	6	3	0	9
5	Recommended whole time equivalent for pharmacy lead role too high	5	3	0	8

Refers to all aspects of workforce including lead roles and dedicated research staff.

As can be seen most comments related to proposed workforce models and the ability to appropriately job plan individuals to R&D activity (n = 43). Comments referring to whole time equivalent requirement for lead roles and medical workforce were reviewed by the expert group, with four statements being modified between rounds (see Appendix 1). All other statements remaining unchanged prior to being included in future rounds.

Discussion

In this three-round Delphi study, a multi-professional group determined that 49 statements represented essential standards for R&D activity within the Welsh critical care network. No statements were determined as non-essential, and no statements failed to reach majority agreement.

To the authors’ knowledge, this is the first study which has attempted to define R&D service standards for critical care and hence is likely to be of significant interest across the wider UK critical care community. Quantitative and qualitative benefits of R&D activity to healthcare service performance and outcomes have previously been described,^22
–25 and a number of national guidelines now promote this.^2
–4,6 Specifically for critical care, GPICS¹ states a limited number of standards and recommendations requirements for R&D activity. In comparison, the current Delphi has identified 49 standards reaching >70% majority agreement (including GPICS items).

For ease and to aid future projects, the standards produced in this Delphi were themed and considered to reflect key aspects of R&D activity including workforce development, responsibilities, reporting processes and collaboration. Notably, these standards highlighted the importance of protected research activity time to multi-professional leads (medical, nursing, AHP and pharmacy). The standards also recommended that critical care departments participating in observational, interventional, or commercially sponsored trials must have dedicated research staff (nurse/AHP/pharmacist) support at a ratio of 0.2 WTE per critical care bed, providing a service 7 days per week, to include screening, consenting, interventions as necessary, follow-up and data management. These standards may help to overcome a major barrier to R&D activity previously highlighted in the literature, particularly regarding the involvement of nursing and AHP members of the MDT.^26
–28

Strengths of this study were that a high proportion of participants (81%) completed all three rounds, with representation in all three rounds from across all health boards with a critical care unit in Wales.¹⁵ Furthermore, participants represented a range of multi-disciplinary professions and grades, and included staff involved with all aspects of local R&D activity. We acknowledge some limitations of our study. Although conducted in line with expectations of Delphi technique, participant numbers were considered to represent a low proportion of all of those working involving with the delivery of critical care research in Wales, and a fraction of those involved with R&D activity in critical care across the UK, which may limit generalisability to other regions and networks. It is clearly possible that views of this relatively small expert group may have biased the themes and the specifics of the statements created, especially true where statements were made around the need for job planning and allocation of sessional/whole time equivalent time resource. Given that these statements all reached majority agreement within the Delphi study (subject to minor wording alterations), this would suggest that the statements created aligned with the wider views of those involved. It is also acknowledged that there are differences in the overall co-ordination and strategic oversight of critical care R&D between Wales and England. Most notably, Wales is considered as one region when participating in the wider NIHR regional speciality networks. The lack of granularity and inter-regional differences might introduce bias towards the more research active sites, which tend to be the larger critical care units. Efforts have been made within NHS Wales to rectify this through the development of the Welsh critical care research group, but it is acknowledged that the set-ups, funding streams and governance processes are different. This may impact on the generalisability of the findings to critical care departments within England, although the exact impact is currently unknown.

As with other Delphi studies, it is acknowledged that the length of the questionnaire (particularly Round 1) may have resulted in response fatigue and lower completion rates (72% of those originally deemed to be eligible and expressing interest in involvement). While participants were informed at the start of the study that it would comprise of three rounds, there may have been some overall questionnaire fatigue resulting in non-completion or completion of later rounds with less deliberation. The questionnaires had long windows for completion (4 weeks each) but it is appreciated these were completed over the winter period when there were significant distractions for critical care services and the staff.

It is also appreciated that whilst this study achieved its proposed sample size, the 31 health care professionals recruited is only a very small reflection of the entire critical care workforce within NHS Wales and was unlikely to have captured all of those involved in R&D activity within critical care.

However, this study should be considered a starting point for exploring the standards for research in NHS Wales critical care units. Future research is required to explore the if the standards provided are agreed by a much wider population of health care professionals working within critical care within NHS Wales and beyond. This would need to include both those involved in R&D activity and those not, as well as all clinical directors and clinical leads with a vested interest in the potential findings and recommendations.

After this wider consensus approach and based on an appreciation that these proposed recommendations exceed the existing standards across critical care, a gap analysis would be required. The gap analysis would determine the existing compliance with the suggested recommended standards and the requirements to be fully implemented across all health boards, considering existing set-ups in critical care departments, size of units and desire to undertake higher levels of R&D activity. This would include the potential costs involved to achieve the required workforce, education, collaboration, and reporting processes. Through this process it would also be relevant to capture the potential improvements and benefits of achieving these standards in terms of increased R&D activity (including potential financial benefits), improved R&D processes and staff feedback.

Conclusion

The results of this study will be utilised within Wales to inform wider service specification with regard workforce requirements, responsibilities, reporting and collaboration. Further research should explore whether the standards identified are deemed similarly essential in other regions and networks and would support the further development of UK-wide R&D standards for critical care. Our challenge is now to build on this initial work, to effect change at local, network and national level to ensure the resources and processes necessary for critical care research to become part of daily practice for all healthcare professionals.

Footnotes

Appendix

Appendix 1.

Modified statements.

Round 1 statement	Round 2 statement
The AHP R&D lead must have a minimum of 0.2 WTE protected time for R&D within their job role.	The AHP R&D lead must have a minimum of 0.1 WTE protected time for R&D within their job role.
The pharmacy R&D lead must have a minimum of 0.2 WTE protected time for R&D within their job role.	The pharmacy R&D lead must have a minimum of 0.1 WTE protected time for R&D within their job role.
Critical care departments participating in interventional or commercially sponsored trials must have dedicated research staff (nurse/AHP/pharmacist) support at a ratio of 2 WTE to every six beds, providing a service 7 days per week.	Critical care departments participating in observational, interventional, or commercially sponsored trials must have dedicated research staff (nurse/AHP/pharmacist) support at a ratio of 0.2 WTE per critical care bed, providing a service 7 days per week, to include screening, consenting, interventions as necessary, follow-up and data management.
The critical care departmental lead for R&D must report at least monthly to the local critical care directorate management team.	The critical care departmental lead for R&D must report at least 3-monthly to the local critical care directorate management team (may be verbal discussion which is recorded in the minutes or written summary included in minutes).

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Professor Tams Szakmany is an Associate Editor for the Journal of the Intensive Care Society. No other authors have any disclaimers or conflicts of interest.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethics approval

The study was discussed with Joint Cardiff and Vale UHB and Cardiff University Research and Development department and deemed not to require R&D approval or national ethics approval.

ORCID iDs

Paul Twose

Tamas Szakmany

Data availability

Raw data available by direct request to the corresponding author.

References

FICM. Guidelines for the Provision of Intensive Care Services - Version 2.1, https://ficm.ac.uk/sites/ficm/files/documents/2022-07/GPICS%20V2.1%20%282%29.pdf (2022, accessed 8 February 2023).

HRA. UK Policy Framework for Health and Social Care Research, https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/uk-policy-framework-health-social-care-research/ (2020, accessed 26 March 2023).

NIHR. Critical Care, https://www.nihr.ac.uk/explore-nihr/specialties/critical-care.htm (2023, accessed 11 March 2023).

NIHR. CRN Portfolio, https://www.nihr.ac.uk/researchers/collaborations-services-and-support-for-your-research/run-your-study/crn-portfolio.htm (2023, accessed 11 March 2023).

Felton

Pattison

Fletcher

, et al. Co-enrolment to UK Critical Care Studies – A 2019 update. J Intensive Care Soc 2022; 23: 53–57.

NICE. The guidelines manual, https://www.nice.org.uk/process/pmg6/resources/the-guidelines-manual-pdf-2007970804933 (2012, 16 January 2023).

NIHR. Critical Care Research and COVID-19, https://www.nihr.ac.uk/documents/critical-care-research-and-covid-19/27447 (2021, accessed 11 March 2023).

NIHR. RCP and NIHR call for research to be a core part of clinical care, https://www.nihr.ac.uk/news/rcp-and-nihr-call-for-research-to-be-a-core-part-of-clinical-care/31691 (2022, accessed 8 February 2022).

Beiderbeck

Frevel

von der Gracht

, et al. Preparing, conducting, and analyzing Delphi surveys: cross-disciplinary practices, new directions, and advancements. MethodsX 2021; 8: 101401.

10.

Powell

The Delphi technique: myths and realities. J Adv Nurs 2003; 41: 376–382.

11.

Vernon

The Delphi technique: a review. Int J Ther Rehabil 2009; 16: 69–76.

12.

Boulkedid

Abdoul

Loustau

, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011; 6: e20476.

13.

Trevelyan

Robinson

PN.

Delphi methodology in health research: how to do it?

Eur J Integr Med 2015; 7: 423–428.

14.

Duffield

The Delphi technique: a comparison of results obtained using two expert panels. Int J Nurs Stud 1993; 30: 227–237.

15.

Hsu

C-C

Sandford

BA.

The Delphi technique: making sense of consensus. Pract Assess Res Eval 2007; 12: 10.

16.

Diamond

Grant

Feldman

, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014; 67: 401–409.

17.

Twose

Jones

Cornell

Minimum standards of clinical practice for physiotherapists working in critical care settings in the United Kingdom: a modified Delphi technique. J Intensive Care Soc 2019; 20: 118–131.

18.

Skinner

Thomas

Reeve

, et al. Minimum standards of clinical practice for physiotherapists working in critical care settings in Australia and New Zealand: a modified Delphi technique. Physiother Theory Pract 2016; 32: 468–482.

19.

Lozano

García-Cueto

Muñiz

Effect of the number of response categories on the reliability and validity of rating scales. Methodology 2008; 4: 73–79.

20.

Jones

Loe

SA.

Optimal number of questionnaire response categories: more may not be better. Sage Open 2013; 3: 2158244013489691.

21.

Skulmoski

Hartman

Krahn

The Delphi method for graduate research. Inf Technol Educ 2007; 6: 001–021.

22.

Ozdemir

Karthikesalingam

Sinha

, et al. Research activity and the association with mortality. PLoS One 2015; 10: e0118253.

23.

Jonker

Fisher

Dagnan

Patients admitted to more research-active hospitals have more confidence in staff and are better informed about their condition and medication: results from a retrospective cross-sectional study. J Eval Clin Pract 2020; 26: 203–208.

24.

Jonker

Fisher

SJ.

The correlation between National Health Service trusts’ clinical trial activity and both mortality rates and care quality commission ratings: a retrospective cross-sectional study. Public Health 2018; 157: 1–6.

25.

Boaz

Hanney

Jones

, et al. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open 2015; 5: e009415.

26.

Battle

Lynch

Discroll

, et al. Nurse and allied health professional experiences and attitudes toward critical care research: a national questionnaire study. J Intensive Care Soc 2022; 23: 2–10.

27.

Twose

Jones

Bharal

, et al. Exploration of therapists’ views of practice within critical care. BMJ Open Respir Res 2021; 8: e001086.

28.

Twose

Terblanche

Jones

, et al. Protected therapy services for critical care: a subanalysis of the UK-wide workforce survey. Aust Crit Care. Epub ahead of print 4 January 2023. DOI: 10.1016/j.aucc.2022.11.011.

Standards for research in NHS Wales critical care units: A modified Delphi study

Abstract

Introduction:

Methods:

Results:

Conclusions:

Keywords

Introduction

Methodology

Design

Participants, therapists, centres and eligibility procedure

Questionnaire development

Procedure

Analysis

Ethics approval

Results

Discussion

Conclusion

Footnotes

Appendix

Declaration of conflicting interests

Funding

Ethics approval

ORCID iDs

Data availability

References