Enhancing quality of scoping reviews in paramedicine research – guidance for authors

Abstract

Scoping reviews are an increasingly common method for conducting evidence synthesis in paramedicine. Over the past decade clear methodological guidance has emerged, adding rigour and credibility to this review approach. Paramedicine receives many scoping review submissions, and views these as valid evidence synthesis capable of helping the journal achieve its strategic vision and mission. However the Editorial Board has noted that submissions frequently fail to adhere to essential elements of scoping methodology and reporting standards. This editorial aims to provide guidance to authors regarding Paramedicine's expectations and requirements for scoping review submissions, with the hope of contributing to a greater understanding of scoping review science and enhancement of quality.

Since their emergence almost two decades ago, scoping reviews have become an increasingly popular approach to evidence synthesis in healthcare. ¹ Originally viewed with some scepticism as a review design lacking rigour and structure, their acceptance as a credible and valid evidence synthesis tool grew following publication of the first methodological guidance by Arksey and O’Malley in 2005. ² A period of proliferation ensued, quantified in 2014 by Pham et al. who identified 355 scoping reviews published between 1999 and 2012. ³ Subsequently, the original framework by Arksey and O’Malley has been advanced as the conduct of scoping reviews became more defined and recognised. Levac et al. ⁴ progressed that original work with an enhanced framework in 2010; this was further developed and expanded in updated contemporary methodological guidance from the JBI Scoping Review Methodology Group in 2020 and again in 2022.^5,6 Concurrent to the methodological evolution, guidance for reporting of scoping reviews emerged promoting transparency and quality, most notably the PRISMA extension for scoping reviews (PRISMA-ScR). ⁷

Among paramedicine researchers, scoping reviews are an increasingly popular approach. ⁸ Consistent with this growth, scoping reviews are amongst the most common manuscripts submitted to Paramedicine, the official journal of the Australasian College of Paramedicine and the Paramedic Chiefs of Canada. However, the Editorial Board has identified considerable variation in methodological and reporting quality, a problem not unique to Paramedicine but also seen and documented in other healthcare disciplines.^1,8 The aim of this editorial is to highlight recurrent areas of concern relating to methodology and reporting of scoping review manuscripts as identified by the Editorial Board and provide guidance to paramedicine researchers specific to the requirements and expectations of Paramedicine.

One frequent issue appears to be a fundamental misunderstanding as to what a scoping review is and, given a particular review question, when it should be employed as the appropriate review methodology. In 2020, the JBI Scoping Review Methodology Group released a definition of a scoping review derived from an international consensus project:

Scoping reviews are a type of evidence synthesis that aims to systematically identify and map the breadth of evidence available on a particular topic, field, concept, or issue, often irrespective of source (ie, primary research, reviews, non-empirical evidence) within or across particular contexts. Scoping reviews can clarify key concepts/definitions in the literature and identify key characteristics or factors related to a concept, including those related to methodological research. ^{9 (p. 950)}

Comprehensive elaboration of the key elements within the definition are beyond the scope of the editorial, rather the Editorial Board wishes to make prospective authors aware of several resources available to help determine the suitability of a scoping review for a proposed project. The definition itself is unpacked in detail by Munn et al., providing clear guidance on the purpose of a scoping review and its potential scope. ⁹ Authors should demonstrate that the stated aim of their work is consistent with what can be achieved from a scoping review. Most noticeable is confusion between systematic reviews and scoping reviews. Clarity in determining which synthesis approach to select based on the proposed aims of a project is provided in another paper by Munn et al. ¹⁰ That being said we caution against paramedicine researchers adopting a dichotomous mindset when selecting a review methodology; systematic and scoping reviews are but two of a larger family consisting of at least 14 types of review. For an overview of review typology and respective indications, the Editorial Board suggests engaging with the papers by Grant and Booth, ¹¹ Tricco et al., ¹² and Sutton et al. ¹³ It is recommended that in-depth discussion and conceptualisation of the aim of an intended review needs to occur amongst the review team when developing the protocol.

Paramedicine acknowledges that no single methodological guidance enjoys exclusivity over the conduct of scoping reviews. However, as a journal that values high-quality research dissemination and discourse, Paramedicine is committed to engaging with contemporary guidance and thus chooses to endorse JBI methodological guidance in its submission guidelines. The JBI Evidence Synthesis Manual provides clear and detailed step-by-step guidance for conducting a scoping review. ¹⁴ A common problem observed by the Editorial Board is poor awareness or a lack of recognition of developments in scoping review methodology since the first framework by Arksey and O’Malley in 2005. ² Authors should engage with contemporary, enhanced methodology, whilst acknowledging the seminal nature of the original framework. Furthermore, authors should not only explicitly identify the methodological guidance used, but more importantly demonstrate that the guidance was actually followed and not merely given a cursory reference. It appears common for a methodology to be stated, but on reading the manuscript non-adherence with core elements of the methodological guidance becomes evident. Paramedicine editors and reviewers look beyond the statement referring to the use of guidance and scrutinise the description of applied methods during their assessment of rigour and guideline adherence. Common methodological flaws noted in submissions to Paramedicine include unclear or no articulation of the Population, Concept, and Context, non-adherence to the three-stage approach to data searching and identification recommended by JBI, and incongruent approaches to data synthesis and presentation. ¹⁴

Data synthesis and presentation of results have been noted as an area in need of quality enhancement; this has been recognised as a particularly challenging aspect of the scoping review process. ¹⁵ Paramedicine acknowledges the flexibility that exists in how data can be presented and synthesised in the results section of a scoping review and embraces the richness that results from it. However, many scoping reviews submitted to Paramedicine do not adhere to recommended principles of synthesis and presentation, and in such cases tend to have results incongruent with their stated aims. A common example is the generation of “themes” presented in the results, or analysis of effect sizes. The synthesis of themes implies that thematic analysis was conducted, and analysis of effect sizes signals that authors examined the relationship between variables or the difference between groups, both of which constitute a form of data synthesis overreaching the scope of scoping reviews. Useful guidance exists within the JBI Evidence Synthesis Manual providing contemporary recommendations to support authors. ¹⁵

Whilst adherence to a methodological framework is important in the conduct of a review, adherence to a reporting framework is equally critical when reporting a review. In 2018, the PRISMA-ScR became available to enhance the quality and transparency of reporting of scoping reviews. ⁷ Acknowledging that PRISMA-ScR does not have exclusivity on the reporting of scoping reviews, and that various reporting guidelines exist, ¹⁶ Paramedicine requires scoping reviews to explicitly state adherence to PRISMA-ScR and, most importantly, demonstrate adherence to it. In addition, authors must submit a completed PRISMA-ScR checklist as a supplementary appendix.

Whilst debate exists between methodologists about the necessity of registering a prospective review, there is a strong argument that registration of protocols contributes to transparency and accountability, reduction in risk of bias, and increased efficiency through minimising unnecessary duplication of research efforts. ⁶ As stated in the submission guidelines, Paramedicine requires scoping reviews to have been preceded by registration of an a-priori protocol in a publicly accessible platform such as Open Science Framework (https://osf.io/) or Figshare (https://figshare.com/) or by publication in a peer-reviewed journal, and that the relevant citation or link is included in the submitted manuscript. It is worth noting though that Paramedicine does not publish review protocols.

Finally, Paramedicine strongly encourages paramedicine researchers to pursue formal training in scoping review methodology. Too often it seems that the developmental and methodological considerations for scoping reviews receive far less attention compared to core primary research designs. We urge paramedicine researchers, paramedicine research students, and those providing research supervision to embrace formalised research training in scoping review methodology to ensure the substantial effort that goes into completing a review does not go unrewarded and to ensure the review achieves maximum impact.

In summary, Paramedicine encourages submission of scoping reviews that conform to the Journal's submission guidelines; when conducted and reported well, the Editorial Board views them as a credible and robust approach to evidence synthesis that contributes to Paramedicine's strategic mission and vision. However, there is a clear need for enhancement in the quality of conduct and reporting of these in submissions to Paramedicine. The Editorial Board stresses the importance of appropriate selection of scoping review methodology aligning with the aims of the research, engagement with and adherence to methodological and reporting guidance, preregistration of an a-priori protocol, and pursuit of formalised methodological training.