Abstract
Policymakers increasingly seek to use systematic reviews to inform policy decisions in a wide variety of policy areas. The use of rigorous methods for assessing the available evidence makes systematic reviews a key tool in this context. However, the emphasis on rigorous methods can also pose challenges for policymaking, as this can constrain systematic reviews from adapting to complex and localised policy needs. Subsequently, commentators have argued that systematic reviews need to be more accommodating of policymakers’ interests. In this paper, we aim to show how researchers and policymakers can work together to produce systematic reviews which are useful for policy purposes, using hermeneutics as a theoretical framework. Specifically, we describe the central processes of a systematic review with reference to hermeneutics, with a view to developing an understanding of what is happening when researchers and policymakers work together. This draws attention to the importance of shared interpretive decisions that shape the development and conduct of systematic reviews, but which are typically unacknowledged in conventional methodological accounts. Furthermore, we argue that hermeneutics can provide reassurance that the inevitable interpretive decisions which take place in this context are not deviations from best practice, but in fact strengthen a systematic review for policy purposes. The paper sets out this account in several stages which align with key systematic review tasks relating to planning, carrying out and reporting a systematic review.
Background
Systematic reviews aim to answer research questions by assessing the available evidence using rigorous methods which are pre-specified in a protocol and carried out with minimal deviation (The Campbell Collaboration, 2020; Centre for Reviews and Dissemination, 2008; Higgins et al., 2019). In this way, systematic reviews seek to eliminate bias introduced through a partial or expert opinion-based assessment of the available evidence (Higgins et al., 2019). To this end, guidance sets out recommended approaches for carrying out and reporting a systematic review (The Campbell Collaboration, 2020; Centre for Reviews and Dissemination, 2008; Higgins et al., 2019; Page et al., 2021; Petticrew and Roberts, 2008). The robustness of systematic reviews for making evidence-based assessments makes systematic reviews a key tool for decision makers such as policymakers (Whitty, 2015). First used in their current form to inform medical decision making, systematic reviews have subsequently been used in many diverse fields including education, environmental science, crime and justice, social welfare and international development (Haddaway and Pullin, 2014). The main tasks which are undertaken to plan and carry out a systematic review, as described in systematic review methods guidance, are detailed in Table 1 (The Campbell Collaboration, 2020; Centre for Reviews and Dissemination, 2008; Higgins et al., 2019).
Main tasks for planning and conducting a systematic review.
There are, however, many challenges associated with making systematic reviews relevant to policymakers. For example, systematic reviews typically seek to answer narrowly defined questions whereas policymaking requires a broader assessment of evidence (Laupacis and Straus, 2007), including contextual factors which are often excluded from systematic reviews (Burford et al., 2013). Furthermore, there is often limited discussion on how the findings of systematic reviews can be implemented in real-world situations (Ahmad et al., 2010; Gruen et al., 2005; Kite et al., 2015), and systematic review reports are often long and detailed which can make them hard for people unfamiliar with systematic reviews to understand (Lavis, 2009). Greenhalgh and Malterud (2017) characterise these challenges as arising from the meeting of two profoundly different cultures: science, which values truth, and policy, which values the ‘possible, acceptable, and reasonable, in a particular set of circumstances’.
In our experience, a contributing factor to this dilemma is the assumption that systematic reviews should rely on pre-specified methods which are set out in a protocol and followed without deviation. This can give the impression that systematic reviews involve a series of mechanistically applied processes which are insensitive to challenges that arise as a review unfolds (Greenhalgh et al., 2018). However, evidence suggests that this view of systematic reviews is an idealised and potentially unhelpful account of how systematic reviews are carried out, which does not reflect actual practice. Qualitative studies which explore how systematic reviewers undertake systematic review tasks show that – even without the input of policymakers – systematic reviews involve interpretive judgements which are not captured by pre-defined rules, and which rely on the expertise of the research team and stakeholders. For example, Briscoe et al. (2022) draw attention to the hidden practical knowledge which librarians and information specialists rely on when searching for studies for systematic reviews (Briscoe et al., 2022). Moreira’s ethnographic study of a systematic review team describes how data in primary studies is re-interpreted through deliberation amongst systematic review colleagues (Moreira, 2007). Lorenc et al.’s qualitative interview study of how systematic reviewers work with complexity found that the need for expert judgement is intensified in reviews which synthesise heterogeneous data (2016), and Schniedermann shows that the use of systematic review reporting guidance, such as PRISMA (Page et al., 2021), is not a mechanistic process but rather an interpretive process (Schniedermann, 2024).
The potential for systematic reviews to involve interpretation in this way can be represented as a hermeneutic process, in which understanding is gained during the planning and conduct of a review which causes the research team to refine their sense of the whole which they started with, that is a hermeneutic circle (see Figure 1). Boell and Cecez-Kecmanovic (2014) describe a hermeneutic process for conducting literature reviews, in which two interlinked hermeneutic circles (‘search and acquisition’ and ‘wider analysis and interpretation’) are used as a methodological framework. Although they make occasional references to systematic approaches, overall Boell and Cecez-Kecmanovic’s (2014) description of the review process is more akin to a non-systematic review, in which, for example, the review question emerges as part of the literature review process rather than pre-specified in advance. The argument presented herein is that hermeneutics is also apparent in how systematic reviews are carried out which, contrastingly, seek to apply pre-defined inclusion criteria and methods. This builds on Greenhalgh et al.’s (2018) argument that hermeneutic reviews can provide insight where systematic reviews are unable to, which we extend here to argue that systematic reviews themselves can be understood as a hermeneutic process, and that explicitly framing systematic reviews in this way can strengthen their relevance as tools for policymakers.
Hermeneutic circle.
Thus, the aim of this paper is to show how researchers and policymakers can work together to produce systematic reviews which are useful for policy purposes, using hermeneutics as a theoretical framework. Specifically, we describe the central processes of a systematic review with reference to hermeneutics with a view to developing an understanding of what is happening when researchers and policymakers work together. To this end, the paper does not provide step-by-step guidance, but rather sets out an illustrative account of how hermeneutics can be applied to the conduct and reporting of systematic reviews for policymakers. Nor do we attempt to introduce novelty into systematic review practices, but rather show how hermeneutics can be used as a framework for existing, albeit typically undisclosed, practices. Furthermore, we argue that hermeneutics can provide reassurance that the inevitable interpretive decisions which take place in this context are not deviations from best practice, but in fact strengthen a systematic review for policy purposes. We use the term systematic reviews herein broadly to refer to all systematic approaches to evidence synthesis, drawing out differences between types of systematic review where these are significant from a hermeneutic perspective.
The paper proceeds by a brief overview of hermeneutics. We then describe how hermeneutics can be applied to systematic reviews commissioned by policymakers, with reference to three case studies form our own work with policymakers as illustrations. Finally, we contextualise our approach in the wider literature and discuss opportunities and challenges.
Hermeneutics
The term hermeneutics derives from the Greek word hermēneuō, which means to interpret (Palmer, 1969). Hermeneutics was developed into a theoretical account of how interpretation is undertaken by Friedrich Schleiermacher (1768–1834), with a specific focus on the interpretation of texts (Moran, 2000). In particular, Schleiermacher developed the concept of the hermeneutic circle, which describes how interpretation is a circular process of relating the whole of a text to parts of a text and back to the whole (Moran, 2000). (Technical terms in this section are italicised when they first appear and summarised in Table 2.) Specifically, any interpretation of the parts of a text must be prefigured by a preliminary understanding of the whole text, and understanding gained from interpreting the parts is fed back into an increased understanding of the text as a whole (Debesay et al., 2008). As this process is repeated, understanding is gradually deepened. For Schleiermacher, this involved uncovering an author’s intended meaning of a text (Palmer, 1969).
Key concepts in hermeneutics.
Wilhelm Dilthey (1833–1911) extended Schleiermacher’s account of hermeneutics through its application to the social sciences (Palmer, 1969). Thus, Dilthey extended the role of hermeneutics from interpreting texts to a wider set of cultural and social phenomena. In particular, Dilthey argued that interpretation involves bringing together our experiences of cultural or social phenomena into a unitary whole or lived experience from which meaning is derived (Palmer, 1969). Lived experience can be unique to an individual or shared across groups and societies, and typically develops overtime in response to changing historical circumstances (Palmer, 1969). In contrast to Schleiermacher’s view that meaning is inextricably linked to an author’s intention, meaning for Dilthey was historically bounded and constantly changing (Palmer, 1969).
The significance of lived experience to hermeneutics is drawn out further by Martin Heidegger (1889–1976). Heidegger integrated hermeneutics with phenomenology to conceptualise interpretation as foregrounded in the lifeworld (Lebenswelt) of a person or community, which inescapably shapes their understanding (Moran, 2000). Within this all-encompassing framework, all interpretation is historically situated, extending to the natural sciences in addition to the social sciences (Moran, 2000; Palmer, 1969). Furthermore, for Heidegger, understanding gained through interpretation is not merely a cognitive phenomenon situated in the mind, but is an embodied phenomenon through which persons come to act competently to achieve practical goals without consciously using cognitive thought-processes (Dreyfus, 1988). Importantly, this practical understanding is developed and utilised in such a way that resists formulation in terms of rules or formulae: practical understanding is only attained through repeated exposure to situations which form pre-reflexive thoughts and habits, such as the skill of observation amongst scientists which is developed overtime and taken for granted as a background condition for scientific understanding (Dreyfus, 1980). To describe this more practically oriented understanding, Heidegger uses Aristotle’s term, phronesis, in contrast to theoretical understanding or techne (Dreyfus, 2006).
At this juncture, it is important to note that the application of hermeneutics to all understanding through its integration with phenomenology has not eclipsed hermeneutics’ initial focus on textual interpretation. The historical development of hermeneutics has led to two separate but overlapping foci: first, an approach to interpretation of phenomena in general and, secondly, an approach to textual interpretation (Palmer, 1969). This dual focus is significant in the context of systematic reviews commissioned by policymakers, which are focussed on textual interpretation (of journal articles, etc.) in the wider context using systematic review methods to meet a policymaker’s needs.
Hans-Georg Gadamer (1900–2002) made developments to Heidegger’s phenomenological account of hermeneutics which are significant for textual interpretation in particular (Moran, 2000). Gadamer accepted Heidegger’s account of practical understanding as a type of background knowledge for theoretical understanding, and introduced the concept of prejudice to describe the effect that this has on interpretation (Gadamer, 1979). In contrast to prejudice commonly defined, for Gadamer, prejudice is both good and bad, inasmuch prejudice is a type of pre-understanding which does not necessarily lead away from true understanding (Palmer, 1969). In the context of textual interpretation, prejudice is a condition for understanding anything at all, without which the content of a text would be meaningless; however, for the attentive reader, understanding is developed as the text is read. This can be extended to prejudice in practical situations, in which pre-understanding is refined through practice overtime. In this process, the hermeneutic circle is evident, as understanding of the whole is refined with understanding from each part. Gadamer also drew attention to the way in which interpretation thus conceived involves a dialectical process between the reader and the text, and also describes this dialectical process as part of conversation between individuals. The goal of this dialectical process is to move to a shared understanding, in which multiple viewpoints or horizons are drawn together through a process of dialogue towards a fusion of horizons (Gadamer, 1979).
In summary, hermeneutics characterises interpretation through the device of the hermeneutic circle. This includes both specifically textual interpretation, and a more general account of interpretation, including interpretation of practical tasks and interpersonal dialogue. We approach a practical situation, dialogue or text with a degree of pre-understanding, which Gadamer conceptualises as a type of prejudice. As a text or situation is interpreted, we develop our understanding through applying our whole understanding to parts, and back to the whole. In this way, pre-understanding must always adapt to a new text or situation, but can be developed and honed overtime through repeated exposure into practical knowledge or phronesis. This type of knowledge is embodied in our lived experiences and is not amenable to development in rules or formulae.
Hermeneutics as a framework for systematic reviews commissioned by policymakers
In this section we present the process of completing a systematic review within a hermeneutic framework using headers which describe generally agreed systematic review tasks as set out in commonly used systematic review methods guidance (The Campbell Collaboration, 2020; Centre for Reviews and Dissemination, 2008; Higgins et al., 2019; Petticrew and Roberts, 2008). These are divided into two parts: first, planning a systematic review, secondly, conduct and reporting of a systematic review, each of which follows the stages set out in Table 1. We note at the outset that the application of hermeneutics to systematic reviews stratifies both text-based interpretation and general interpretation; the former including interpretation of guidance, journal articles and other evidence sources, the latter including interpretation of practical tasks involved in undertaking a systematic review, and interpersonal dialogue between colleagues and policymakers. We also emphasise that the account below is not attempting to introduce novelty into systematic review practices, but rather to show how hermeneutics can be used as a framework for existing practices.
Planning a systematic review for policymakers
Scoping
Scoping is the process of finding out about a potential review topic (Centre for Reviews and Dissemination, 2008). For any review topic, a review team will have initial ideas of how to approach it based on their pre-understanding of different types of systematic review and of the topic area for review (Maxwell et al., 2020). Pre-understanding of systematic review methods is shaped by guidance and – perhaps most significantly – via first-hand experience of carrying out systematic reviews, including training and mentorship (Lorenc et al., 2016; Shepherd, 2013). Thus, textual interpretation of systematic review guidance is supplemented by the interpretation of practical tasks which are learnt through repeated exposure to planning and conducting systematic reviews. A highly experienced review team will have a level of expertise which may be characterised as phronesis, which may inform a shared pre-understanding which is more insightful than a less experienced review team. Pre-understandings of a topic area are similarly shaped by wider reading, and the lived experiences of health and social care in the review team (Maxwell et al., 2020). However, a review team’s pre-understanding of a topic area is typically not that of an expert: as described by Gadamer, pre-understandings may be characterised as prejudice, some of which will be good, and some of which may need revising through attentiveness to relevant literature, and dialogue with more knowledgeable collaborators. The latter can include patient and public involvement and engagement (PPIE) representatives and topic experts, in addition to the policymaker (Gadamer, 1979).
In our experience, a review team’s first encounter with a proposed research question from a policymaker comes in the form of a topic brief, which also sets out the background to the problem to be addressed, and the policymaking decision which the review is intended to inform. Reading the topic brief is supplemented by literature which is cited in the topic brief for background understanding and context, and initial searches to identify the extent of the available evidence (Anderson et al., 2021). Discussion with the policymaker and others with real-world situational understanding can help to interpret the topic brief and the wider literature. In this process, the hermeneutic circle is evident, as reviewers apply their pre-understanding of the topic as a whole to an understanding of parts, which informs their understanding as a whole, and so on. Importantly, this is a shared experience with the policymaker, which starts to build shared understanding which informs the development of the review question and protocol (Anderson et al., 2021).
Protocol development
The agreed approach to a systematic review must be set out in a protocol (Centre for Reviews and Dissemination, 2008). Protocol development is a continuation of the interpretative processes which begin with scoping. Between the review team and the policymaker, the aim is to reach a fusion of horizons which will ensure relevance (the purview of the policymaker) as much as methodological rigour (the purview of the review team; Lorenc et al., 2016). As in the scoping stage, this will entail interpreting relevant texts with input from the policymaker, alongside practical understanding of systematic review methodology. Perhaps most critically, the review team and policymaker must agree on the question and inclusion criteria, which sets the focus and boundaries for the review. The question largely determines the type of review which is carried out, whether this seeks to answer an effectiveness or a qualitative question, or to gather and summarise the extent of evidence (Grant and Booth, 2009). Setting appropriate inclusion criteria requires understanding from the policymaker of the characteristics of the population of interest, of the interventions or phenomena which are most relevant to consider, and their relevant outcomes and contexts. These are then integrated into the inclusion criteria by the review team, who need to refine each criterion according to its operationalisability; considering, for example, the extent of the available literature and the feasibility of including all the desired criteria.
Whilst researchers typically want to focus on narrow questions, policymakers are typically interested in broader questions which consider the context and real-world implications for implementation (Greenhalgh and Malterud, 2017; Laupacis and Straus, 2007). This can include data relating to the setting of interventions (Ahmad et al., 2010; Burford et al., 2013), or data relating to how intervention effectiveness is stratified by health opportunities and outcomes, that is health equity data (Welch et al., 2022; see Case Study 1 in Box 1).
Women’s health review (case study 1).
Setting appropriate inclusion criteria also requires the review team to become familiar with the extent of the relevant literature. During this process, the bibliographic database search strategy takes shape as combinations of search terms are tested and refined. Herein, the development of the inclusion criteria and search strategy is itself a hermeneutic circle of successive development of understanding of granular detail about what is possible to achieve, in view of the relevant literature and the time and resources available for the review, which feeds into an overall understanding of the approach which is used (Briscoe et al., 2022). The policymaker can engage with this process through assisting in the selection of relevant search terms and sources to search, particularly websites and specialist databases and repositories.
The hermeneutics of planning a systematic review for policymakers discussed in this section are summarised in Figure 2.
Hermeneutics of planning a systematic review for policymakers.
Conduct and reporting of a systematic review for policymakers
Searching for studies
Once the protocol is finalised, the first stage of a systematic review is the search for studies. In the majority of systematic reviews, bibliographic databases are the main way in which potentially relevant studies are identified (Bramer et al., 2017). As noted above, the bibliographic database search for at least one database may be developed as part of the protocol. However, there is additional work to translate this search for other bibliographic databases, potentially using a similar iterative process of testing and refining search terms. During this process there is a continued integration of textual interpretation with the interpretation of practical tasks, as decisions are made about which papers are relevant to retrieve alongside the practical decisions relating to searching for studies which are required to achieve this. Supplementary searches, such as checking reference lists and web searching, enhance the recall of a search where bibliographic databases are less effective (Lefebvre et al., 2019a, 2019b). There is more flexibility for supplementary searches to be used in an exploratory fashion – using techniques such as snowballing to identify links between studies which may not have been planned in advance (Briscoe et al., 2022). These more flexible approaches have been identified as reliant on phronesis rather than rule-following, and can only take shape once the review is underway in response to studies which are initially identified (Briscoe et al., 2022). Thus, supplementary searches in particular reflect the hermeneutic circle of increasing an understanding of the whole through understanding of parts.
Selecting studies
In a protocol, study selection follows sequentially from the search for studies. However, this does not capture the entirety of how study selection takes place. Relevant studies for inclusion in a systematic review are often identified at the scoping and protocol development stages via background reading and search strategy development. Thus, although any such studies may also be retrieved by search methods which are set out a priori in the protocol, they are already identified prior to this formal identification process. Study selection thereafter uses the application of pre-defined inclusion and exclusion criteria set out in the protocol by two independent reviewers to identify relevant studies in a two-stage process: title and abstract screening, and full-text screening (Centre for Reviews and Dissemination, 2008). A review team may choose to initially screen a proportion of the total records to calibrate their interpretation of the inclusion criteria. This entails meeting to review the level of agreement, and to share experiences of how the inclusion and exclusion were applied in cases of disagreement. Through this dialogical process the reviewers aim to reach a shared understanding of the application of inclusion criteria, particularly in cases where the application of pre-defined rules could not be operationalised without making judgements which are not explicit in the rules. Sometimes this may require the inclusion and exclusion criteria to be refined or clarified. Once again, a hermeneutic process of understanding of the whole, which is developed by understanding parts (specifically, granular detail about the inclusion criteria), which informs the whole, is evident.
It is important that a shared understanding between the reviewers and the policymaker is reached, which integrates the rules of study selection with the particular needs of the policymaker. Reviewers may find it useful to liaise with the policymaker to check that the interpretation of inclusion criteria meets their understanding of the aims and objectives for the review (see case study 2 in Box 2).
Rare diseases review (case study 2).
Data-extraction
During data-extraction the interpretive process moves to a relatively micro level, in which the minutiae of a study are inspected to identify and extract relevant data. The process typically involves inspecting the methods and findings of a study to identify various data items in a pre-designed data-extraction form, into which the relevant isolated data are inserted. The aim is to copy the data accurately, but there is also a need to ensure that the data are captured in a similar form across the different studies to aid the subsequent synthesis process (Petrova, 2014). This can involve calculations to ascertain a specific datum which is not explicitly reported but can be extrapolated (Moreira, 2007). It can also involve categorisation of data into categories which are not explicitly reported in a study. Thus, the studies are broken down into relevant components, as the data are disentangled and re-represented for the purpose of synthesis (Moreira, 2007). Furthermore, the identification of relevant data items relies partly on that which is laid out in the protocol, but also inductively on the type of data which are identified in the relevant studies. These interpretive processes take shape over the course of the data-extraction process. As the review team begins to see a fuller picture of the data across the different studies, they may return iteratively to earlier studies to re-categorise or re-calculate qualitative or quantitative data respectively.
Engaging with the policymaker at this stage can help to ensure that the data extracted are appropriate for their purposes. As noted above, policymakers may be particularly interested to capture contextual data, for example, relating to the setting of interventions (Ahmad et al., 2010; Burford et al., 2013), or health equity data (Welch et al., 2022). Hermeneutics at this stage is apparent in ongoing interpretation of the data in preparation for something which is meaningful for the end user. This continues to use a combination of textual and general interpretation, as texts are interpreted through the shared understandings of the review team and the policymaker attained through interpersonal dialogue.
Quality appraisal
Quality appraisal is an interpretive process that typically makes use of a pre-designed tool. Depending on which tool is used, the ratings may be different. Elements of quality appraisal may be hard to determine, and differences in rating may be apparent between different reviewers. The appropriate use of a tool may become more apparent after the review team have used the tool on several studies; through discussion, breaking down each raters decisions regarding a particular score, and coming to a shared understanding of how criteria are applied. It is important that the tool draws out features of the evidence that are relevant to the policymaker.
Data-analysis
The aim of data analysis is to draw together the identified studies in a way that answers the review question which is meaningful and practically useful for the policymaker. A wide variety of approaches to analysis are undertaken for systematic reviews, encompassing quantitative and qualitative analysis, mixed methods analysis and summative analysis (Grant and Booth, 2009). Some elements of data analysis are common across different types of systematic review, and some are unique to specific types of systematic review. Regarding shared elements, tabulation of data is commonly carried out for different types of review, and can be a useful first step to seeing all of the identified data in one place (Higgins et al., 2019). This can help with interpretation of data, including the subsequent categorisation of data into components for the presentation of the findings. Tabulated data can also be a useful way for a policymaker to understand what the review has initially found, and can be a basis for discussion between the review team and the policymaker for how the data are presented. This may also involve resolving uncertainty, such as addressing ambiguity or reconciling conflicting evidence, through contextualisation of data in discussion with the policymaker.
Thereafter, a wide variety of approaches to analysis are undertaken (Grant and Booth, 2009). Each of these has its own logic of interpretation with respect to the data involved. For some types of analysis, a hermeneutic process is explicitly part of the analysis. This is particularly the case for qualitative analyses which take an iterative approach to working with data to identify and refine themes in the included studies. As each new theme is identified in a study, the previously inspected studies are re-inspected for the emergent theme (Braun and Clarke, 2006). In this way, the review team’s understanding of each individual paper is informed by a growing sense of the papers as a whole. For the analysis of quantitative data, the significance of interpretation is not so obvious. However, the interpretation of quantitative studies has already begun through screening and data-extraction, and now continues as identified isolated relevant data are interpreted to answer the review question (Melendez-Torres et al., 2017). To this end, data analysis requires the categorisation of data into components which collectively make sense of the data as a whole. Making sense of and presenting this data in a way which is useful to the policymaker may require ongoing discussion with the policymaker (see case study 3 in Box 3). Summative reviews, including mapping and scoping reviews, are themselves interpretive processes in which understanding is gained of the extent of research in a topic area. Here hermeneutics is evident in analysis as an emergent understanding of a body of evidence is gradually placed into a framework which aids understanding (Arksey and O’Malley, 2005).
NetZero review (case study 3).
Reporting
Standard approaches to writing up a systematic review focus on reporting procedures transparently, in such a way that this is reproducible (Centre for Reviews and Dissemination, 2008). Deviations from the protocol should be documented (Centre for Reviews and Dissemination, 2008). Whilst we agree with the importance of transparency and reproducibility, systematic reviewers who are sensitive to hermeneutics may find ways to take account of post-protocol decisions which are not limited to listing as deviations. Hermeneutics shows that decisions we make are sometimes imperceptible, particularly as these relate to phronesis, which is not amenable to formulation in rules or formulae, or descriptive accounts generally (Briscoe et al., 2022). But, it may be possible to capture decisions which were made throughout a systematic review in the methods which provides a rationale that is in keeping with the aims and objectives, whilst not adhering to all the detail in the protocol. Thus, rather than insisting that deviations from the protocol are framed negatively (due to increasing risk of bias, etc.), decision making can be explained in the context of the review and its policy need and show how decisions make the review useful whilst still being scientifically robust.
The review team can liaise with the policymaker as the writing up process takes shape. Discussion with the policymaker to ensure that the findings are presented relevantly and accessibly may also lead to additional outputs which are tailored for specific end users, either the policymaker themselves or other potential users.
The hermeneutics of conducting and reporting a systematic review for policymakers discussed in this section are summarised in Figure 3.
Hermeneutics of conducting and reporting a systematic review for policymakers.
Discussion
In this paper we have set out an account of how systematic reviews commissioned by policymakers can be framed as a hermeneutic process. In setting out this theoretical account, we hope we have provided a lens for understanding the inevitable interpretative decisions which are made in this context, not as deviations from best practice, but as integral to successful conduct of a systematic review for policymakers. In this section, we contextualise our approach in the wider literature, with particular reference to Boell and Cecez-Kecmanovic’s (2014) hermeneutic account of literature reviews, and set out some practical implications and challenges.
Boell and Cecez-Kecmanovic’s (2014) hermeneutic account of the literature review process contains many similarities with our account. As noted above, Boell and Cecez-Kecmanovic’s (2014) account is based around two interlinked hermeneutic circles: ‘search and acquisition’ and ‘wider analysis and interpretation’. The researcher enters these overlapping hermeneutic circles with an initial idea for a research question which is explored through the searching and acquisition circle. This involves a process of searching, sorting, selecting, acquiring and reading, through which potentially relevant texts for a literature review are identified. At this point the researcher has pre-understandings which are challenged and developed by immersion in the texts. The wider analysis and interpretation circle involves mapping and classifying the identified texts, critical assessment, the development of an argument and further refinement of the research question which then further informs the development of additional searching. This leads to more analysis and interpretation, which ultimately leads to a point where the review is complete. This is conceptualised by Boell and Cecez-Kecmanovic (2014) as a fusion of horizons between the researchers and ‘numerous texts’, in which a new perspective on the literature is reached.
We see in this short summary that both our account and that of Boell and Cecez-Kecmanovic (2014) conceptualise the review team as beginning the review process with pre-understandings which are developed through immersion in the literature, during which there is an ongoing process of refinement of the focus of a review. Furthermore, in both accounts, the end point is a fusion of horizons which draws together different texts into a whole. However, Boell and Cecez-Kecmanovic’s (2014) account also contains significant differences with ours. Most prominently, the protocol stage is omitted by Boell and Cecez-Kecmanovic (2014): the only mention of a protocol is to demarcate the difference between a ‘protocol based, formal approach’ from their hermeneutic approach. Furthermore, once the searching and screening processes have yielded a body of potentially relevant literature, Boell and Cecez-Kecmanovic (2014) write that the next phase of the review process, analysis and interpretation, starts with ‘more or less clear ideas about a research problem or a topic…’. This relatively vague conceptualisation of a research question, compared to the pre-defined question set out in a protocol for a systematic review, owes to their framing the literature review process as one in which the hermeneutic circles of searching and acquisition, and analysis and interpretation, move around several cycles before the final question is established (Boell and Cecez-Kecmanovic, 2014). Thus, whereas the interpretative decisions which we describe in this paper are still constrained by the protocol, for Boell and Cecez-Kecmanovic (2014) the final research question is developed towards the end of the review process. Thus, our account makes significant new ground in developing a hermeneutics of protocol-driven systematic reviews.
Despite the fact our account challenges a common rule-following understanding of systematic reviews, there is evidence that systematic review processes do in fact involve interpretation very similar to that which we outline. As summarised above, several studies have sought to draw out the interpretive decisions which are made during the process of conducting a systematic review (Briscoe et al., 2022; Lorenc et al., 2016; Moreira, 2007; Schniedermann, 2024). Importantly, neither we nor these accounts recommend that a rule-following account of how systematic reviews are carried out should be completely discarded. It is still important that there are clearly defined processes, and, particularly for novice systematic reviewers, it is important that standard approaches are clearly described in systematic review guidance and that there is agreement about key stages (Shepherd, 2013). As such, we are not recommending a ‘radical hermeneutics’ account of systematic reviews, in which the readings of key texts and execution of practices differ wildly to pre-existing agreement (Caputo, 1988). However, studies such as those above show that there is also an important role for interpretation which goes beyond the standard guidance, and that this inevitable feature of systematic reviews is partly what makes them relevant and useful.
More broadly, framing systematic reviews for policymaking as a hermeneutic process is relevant to debate about how to make systematic reviews more relevant in this context. In their scoping review of barriers and facilitators to uptake of systematic reviews by policymakers, Tricco et al. (2016) found that creating collaborations between policymakers and researchers was a factor in determining the usefulness of systematic reviews for policymakers. We suggest that hermeneutics can provide a framework for what is happening when policymakers and systematic reviewers work together in this way. Specifically, this can help to think through how to address the issues outlined above, including setting appropriate questions (Laupacis and Straus, 2007), taking account of contextual factors (Burford et al., 2013), addressing the significance of findings for real-world situations (Ahmad et al., 2010; Gruen et al., 2005; Kite et al., 2015), and ensuring final reports (or other tailored outputs) are understood by the intended users (Lavis, 2009). It will not necessarily be possible to address all of these issues in every systematic review, but as a minimum hermeneutics can help researchers and policymakers reach a shared understanding of what a review is aiming to achieve and manage expectations. Some key practical issues to consider are presented in Box 4.
Practical considerations derived from hermeneutics for undertaking systematic reviews for policymakers.
There are also challenges inherent in this approach. Knowing what the limits are of interpretive decisions is difficult to judge. (As a general guide, we have suggested that this must be within the aims and objectives of the review, with a view to making the review more practically useful for the end-user whilst remaining scientifically rigorous.) Furthermore, interpretive decisions can be difficult to report transparently. Indeed, hermeneutics shows that many of our interpretive decisions, particularly as these relate to phronesis, are imperceptible and resistant to expression in rules or formulae. This inherently challenges how feasible it is to transparently report these decisions. But where the decisions can be reported, this retains transparency and can be defended within the rationale of the review. We do not anticipate that systematic review authors explicitly label systematic reviews as using a hermeneutic process, rather, we hope to have provided a theoretical basis and greater appreciation for the inevitable interpretation which takes place throughout a systematic review. Further qualitative research, and potentially guidance, on how systematic reviewers make interpretative decisions which retain the scientific rigour of systematic reviews could help to understand and develop this underacknowledged dimension of evidence synthesis.
Gadamer’s (1979) magnum opus on hermeneutics was titled Truth and Method, but commentators have noted that there is no guarantee that working through the method of the hermeneutic circle, in collaboration with others towards a fusion of horizons, will finally reach the truth (Moran, 2000). Prejudice may remain in the more negative sense of a ‘consensus which is ideologically distorted’ (Moran, 2000). From a hermeneutic perspective, this may be reason to include others in the systematic review process, such as topic experts and PPIE representatives, alongside systematic reviewers and policymakers, to try to take account of other viewpoints. A less sympathetic suggestion might be that we should remove people from the interpretative process of a systematic review to focus more narrowly on methodological rigour (Gøtzsche and Ioannidis, 2012). But, once the inevitability of interpretation is admitted, we must acknowledge that this more narrow focus has its own limitations, and may fatally lead to systematic reviews which are not useful for policymaking.
Conclusion
Hermeneutics can provide a framework for how systematic reviews commissioned by policymakers are carried out. This provides a theoretical understanding for undertaking systematic reviews in this context, and can furthermore provide reassurance that the interpretative processes involved are not deviations from best practice but are in fact central to the process of carrying out a systematic review. If systematic reviewers can incorporate hermeneutics in such a way that the decision-making is openly reported, this may improve the transparency and rigor of systematic reviews where these decisions are sometimes hidden from view.
Footnotes
Acknowledgements
The authors are grateful for the opportunity to present and receive feedback on this paper at the HSR UK conference 2024 at the University of Oxford.
Correction (October 2025):
The article has been updated to include the sentence “This draws attention to ...... in conventional methodological accounts.” in the abstract.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical approval and informed consent statements
Not applicable.
Data availability statement
Not applicable.