Abstract
Easy access to information, expert opinion, and research results, along with the proliferation of systematic reviews, meta-reviews, consensus statements, clinical practice guidelines, and other scientific statements, can create confusion for the practitioner when searching for evidence-based recommendations. As a result, they are left wondering “What is the best approach to identify and select high-quality research evidence to guide my clinical practice?” This may be akin to finding a top athletic prospect without a standardized way to evaluate and rank available players. An elite coach would not select an athlete without knowledge of player characteristics, such as the athlete’s development history and his/her strengths and weaknesses. Clinicians should take a similar approach when selecting clinical research evidence to inform clinical decision making.
Evidence-based practice (EBP) is a framework for clinical decision making that has been around since the early 1990s.7,12 As such, many clinicians may be “familiar” with the term but lack understanding with respect to what it is or how it should be used. The term EBP implies that clinical research evidence should be used when making decisions about patients. However, the best available clinical research evidence should be used in conjunction with clinician expertise and patient values and expectations to provide the highest quality care and to improve patient outcomes. 11 Clinical research is specific to human patients – not ex vivo data. Basic steps in the EBP process are: ask a clinical question, search for the best clinical research evidence, critically evaluate and appraise the clinical research evidence, integrate the evidence with clinical expertise and patient preferences, evaluate the outcomes of the decision, and disseminate findings. 11 Importantly, EBP should be an iterative process in which clinical practice is altered based on how the population responds to the intervention or treatment. The steps of acquiring, critically evaluating, and appraising the clinical research evidence are accomplished with a systematic review of the literature and serve as the backbone in the EBP process. 8 The use of a systematic review and evidence grading is the primary way in which scientific statements (eg, Clinical Practice Guidelines, Position Statements, Expert Consensus Statements, etc) vary - and is one of the most important developmental “milestones” that distinguishes a “5-star recruit” from other lower “ranked” statements. Keep in mind that systematic reviews and meta-analyses are only as good as the primary research included in the review and a “5-star recruit” for a systematic review and meta-analysis is a Level 1 clinical study. 10
When searching for research evidence to guide your practice, you will inevitably find an overwhelming number and type of results from which to choose. For example, searching https://pubmed.ncbi.nlm.nih.gov/ for “treatment of knee osteoarthritis” generated >3000 results that ranged from randomized clinical trials, narrative reviews, systematic reviews, network meta-analyses, expert consensus statements, and clinical practice guidelines. Adding the term “evidence based” before “treatment of knee osteoarthritis” dramatically reduced the number of results but the diversity in results remained (ie, systematic reviews, network meta-analyses, guidelines, clinical practice guidelines, etc). It has been such a source of confusion that several have called for a clearer distinction between terms such as “guidelines,” “consensus,” and “position.”1,3,6 Thus, it is important to understand the ways in which statements differ in their development, strength of evidence, recommendations generated, and appropriateness of their use in different situations.
Clinical Practice Guidelines (CPGs) are the “5-star recruits” and, most often, are developed using an EBP methodology. That is, they are based on specific questions to be answered (incorporating the PICO(T) [patient, intervention, comparison, outcome, (time)] framework), a systematic search of the available research literature, evaluation of individual studies, and grading of the overall body of work included. 5 An evidence triangle (Figure 1) illustrates the perceived quality of distinct types of clinical research studies and summaries. 9 And, depending on the question to be answered, different types of studies and summaries may be included in the systematic review. CPGs generally provide summary statements to answer the research questions and GRADES to indicate the quality and strength of the underlying research. 5 Finally, CPGs go through a public comment period so that people have an opportunity to provide feedback on the near final draft before publication/release. CPGs are developed largely by scientific organizations and many are centrally housed in repositories (e.g., GuidelineCentral [https://www.guidelinecentral.com/guidelines/] and the National Guideline Clearinghouse [http://www.ahrq.gov/gam/index.html]). Thus, CPGs are the “all stars” and should be acquired and utilized when possible.
Quality of evidence triangle. Hierarchy of strength of evidence from primary and secondary research sources. Reprinted with permission from Hoffman et al. 9
The rest of the “recruiting class” are a mixed bag. Other types of scientific statements (eg, consensus statements, position papers, expert opinion statements, roundtable summaries, scientific advisories, etc) vary in their development protocols and the types of literature utilized. Most often, they are a blend of research evidence and expert opinion and do not go through a public comment period before release. They are sometimes developed when there is little high-quality literature, when the topic is new or developing and little research evidence (of any quality) exits, or when an organization wants to briefly highlight new developments in a field, rather than carry out an exhaustive summary. 5 As such, it is especially important to read the methodology of those papers carefully to understand developmental steps, such as what types of research studies were included and how expert consensus was gained.3,8 Because these statements are varied and do not use the stringent development process of CPGs, the results of such papers should be viewed cautiously. In addition, the reader must realize that recommendations based heavily on expert opinion and/or less well-controlled studies may change rapidly as the evidence base develops. 5 That is not to say they should never be used. Perhaps you locate a CPG in your topical area, but the summary statements do not address your specific questions of interest. A consensus statement might provide you with an initial place to start, clarify issues, point to gaps in the literature, etc. So, while the 5-star recruit has all the characteristics you are looking for, lower ranked recruits can make significant contributions to your team in a more limited or more specialized way. The same holds for scientific statements that “rank” below CPGs.
An example from sports medicine can provide further insight into how these numerous documents can be used in clinical decision making. Let’s suppose that a team physician is interested in understanding the efficacy and safety of orthobiologics for the treatment of osteoarthritis of the knee. In particular, the physician works with elite adult athletes and is curious about platelet-rich plasma (PRP) and cell-based therapies (stem cells). A great place to start is to search for a recent CPG on this topic. Fortunately, it is found in this 2021 Clinical Practice Guideline by the American Academy of Orthopaedic Surgeons covering the management of knee osteoarthritis. 2 Further, the physician is in luck because this CPG attempts to answer a PICO question focused in the area of interest: “In adult (>17) patients with symptomatic osteoarthritis of the knee, what locally invasive treatments are most effective for improving patient outcomes?” The evidence summary for PRP is concise and straightforward, saving time and facilitating knowledge of the current literature, recommendations, and strength of evidence for the use and safety of PRP. However, the CPG does not mention stem cell therapy. Stem cell therapy is newer and may not have a robust literature to review. In addition, we are curious as to which (if any) studies have been conducted using PRP or stem cells in adult elite athletes (ie, basketball players). The patients also ask whether orthobiologics work better or worse in athletes. Athletes generally have better overall health and conditioning, but also have higher volume and intensity of physical training. Digging further, the search is refined by searching for a consensus statement specific to orthobiologics, adult elite athletes, and PRP. Next the 2020 NBA Orthobiologics Consensus Statement is found. 4 This statement, while not utilizing a systematic review of the literature, summarizes the literature from several databases and covers PRP, as well as the use of stem cells, for common musculoskeletal injuries in professional basketball players. With respect to the effectiveness of these treatments for knee osteoarthritis, a summary of the relevant evidence and recommendations is observed. Reading this paper carefully, we see that the authors encourage physicians to pay attention to emerging literature and stay abreast of orthobiologic preparation protocols. Further, the authors state that this type of consensus statement needs updating regularly to remain current and accurate. Understanding how these documents were developed, and their strengths and weaknesses, we are confident in the best use of each when discussing treatment options with the athletes.
In summary, the scouting report indicates that not all athletes, or scientific statements, are created equal. All statements will not be a good fit for your evidence needs. What are common traits of “top prospects” as you pursue your recruits (ie, evidence)?
Transparent and reproducible development protocols;
findings and recommendations based on a systematic review of the clinical literature;
assessment of included studies (ie, each individual study) for quality and risk of bias;
grading of the overall body of evidence included;
for consensus statements, a transparent rubric is used to evaluate consensus;
if lower-level evidence is included in the statement, recommendations are cautious and do not overstretch the results.
By using a systematic approach, starting with the best clinical evidence, and carefully considering available scientific statements, you will be able to make educated choices for and with your patients, share data-driven recommendations with your colleagues, and stay well-informed of research gaps that require additional investigation.
For additional tools, resources, and tips related to EBP and clinical decision making, visit the Center for Evidence Based Medicine (https://www.cebm.net/) at Oxford University.