Abstract
Background: The importance of evidence-based practice is being recognized across a broad range of healthcare disciplines as a means for improving patient outcomes and also efficiently managing healthcare resources.
Objective: The objective of this work was to obtain information from clinicians about the underlying barriers and facilitators relating to evidence-based practice in prosthetics and orthotics.
Study design: Cross sectional survey.
Methods: An internet survey was developed and distributed to 300 prosthetists and orthotists currently practicing in Canada.
Results: A principal component factor analysis of the survey results revealed ten primary factors affecting evidence-based practice. These include time constraints, workload and system demands, limited relevant evidence from research, and gaps in skills and knowledge required to perform evidence-based practice.
Conclusions: Clinicians value research as a means of improving clinical practice, but they are faced with a number of practical barriers in performing evidence-based practice.
This study provides empirical data about the underlying barriers and facilitators relating to evidence-based practice in prosthetics and orthotics. Such data are essential in order to inform those involved in improving existing clinical practices, including educators, professional organizations, and governing bodies.
Keywords
Introduction
Evidence-based practice (EBP) refers to the process of ‘integrating individual clinical expertise with the best available external clinical evidence from systematic research’ 1 so as to provide the best clinical care possible. The importance of EBP is being recognized across a broad range of healthcare disciplines as a means to improve patient outcomes while efficiently managing healthcare resources. 1 Moreover, steps are being taken to develop an understanding of the inhibitors and facilitators of EBP in the various healthcare areas, so that measures can be taken to ensure its effective utilization. 2–9
In the field of prosthetics and orthotics (P&O), EBP is similarly receiving increasing attention. In the early 1990s, a number of publications began to elucidate some of the underlying components of EBP in P&O, including aspects relating to researching published information, 10 making inferences from research, 11,12 training and education related to EBP, 13,14 and participation of clinicians in research. 15,16 Subsequently, a review by Ramstrand and Brodtkorb 17 has highlighted the importance of EBP in the development and growth of the P&O industry, cautioning that inaction in this respect may cause P&O practice standards to fall behind those being implemented in allied healthcare areas such as physical therapy. 17 To facilitate EPB, the authors stress that a cultural shift in the P&O profession is vital in the way that clinicians perceive and utilize clinical research. And while they describe the factors that are likely to play a role in the adoption of EPB, such as lack of expertise, time and motivation, there is currently limited empirical data specific to the P&O field that can further aid our understanding of these underlying factors.
Allied healthcare areas have taken initiatives to address these gaps in knowledge about EBP in their respective areas. In physiotherapy, for example, Jette et al. 2 and more recently Salbach et al. 4,5 used survey questionnaires to examine the EBP-related barriers, beliefs, attitudes, knowledge and behaviours of practising physical therapists. To gain similar insights, comparable approaches have been utilized in nursing. 7,9 For example, using a survey instrument, Nagy et al. 7 established six factors contributing to evidence-based nursing, which include (1) the availability of support to develop EBP, (2) belief in the value of EBP, (3) skills in locating and evaluating research reports, (4) clinical usefulness of research, (5) knowledge of research language and (6) time constraints. 7
Similarly, in order to develop an understanding of the factors that facilitate or inhibit EBP in P&O, empirical data are needed about existing practices as well as clinicians’ attitudes and beliefs relating to EBP. The purpose of this work was to gain information from clinicians about the various aspects of EBP, including current practices, their beliefs about the value and usefulness of EBP in P&O, and the limitations associated with training, time and facility support that are needed to perform EBP effectively.
Methods
Study overview
A cross-sectional email/internet survey study was conducted of certified prosthetists and orthotists providing services in Canada. A web-based questionnaire was prepared based on an initial review of literature and pilot testing, and the consideration of protocols outlined by Dillman. 18 The survey was administered via an email invitation, with two follow-up emails and one phone call at two to three week intervals to encourage response. Prior to its commencement, the study was approved by the Holland Bloorview Kids Rehabilitation Hospital Human Research Ethics Committee. Consent was considered implied by clinicians who submitted the questionnaire.
Participants
Certified prosthetists and orthotists were considered eligible as long as they were currently practising in Canada. The database of clinicians in Canada was obtained from the Canadian Board for Certification of Prosthetists and Orthotists (CBCPO), a publicly available website that includes email addresses and phone numbers (http://www.cbcpo.ca/). The original list of 324 names was reviewed and duplicate listings were eliminated. The exclusion of invalid email addresses as well as seven clinicians originally involved in pilot testing resulted in a final list of precisely 300 names.
Instrument
A questionnaire was designed to identify clinician and institutional factors influencing EBP. The items in the surveys were based on EBP studies performed in related fields, including nursing, 7 physical therapy 5 and medicine, 19 and were adapted to ensure relevance to the P&O field. Items in the survey were broadly categorized into three sections, including (1) demographics, (2) information sources and (3) barriers and beliefs. More specifically, the ‘demographics’ section consisted predominantly of respondents’ demographics and information about work settings (11 items). The 21 items in the ‘information sources’ section aimed to identify the sources of information clinicians have access to, and which of these sources they are most likely to use. The ‘barriers and beliefs’ section consisted of 34 items aimed to identify factors relating to self-efficacy to perform EBP (skills, knowledge and training), peer and organizational support, time constraints, access to information sources, perceived value of EBP in clinical and patient care, system and financial constraints, and general perceptions about the quality of the evidence currently available. Moreover, two questions were included to assess the respondents’ understanding of the hierarchy of evidence in published literature. All questions in these latter sections were in the form of statements to which respondents expressed their level of agreement on a 5-point Likert scale, with the options (1) strongly disagree, (2) disagree, (3) neutral, (4) agree and (5) strongly agree. Statements were phrased positively and negatively so as not to reflect preferences toward EBP. For the analysis, the scores for the negatively phrased items were reversed so that scores could be aggregated. Moreover, the term evidence-based practice (or EBP) was not used except when explicitly defined within the item, as not all clinicians may be familiar with it.
Summarized questions and response options for the demographics and information sources part of the questionnaire.
Note: these choices are provided individually for each source.
Respondent characteristics from ‘demographics’ section of the questionnaire (n = 114).
Questions, responses and analysis results from the ‘barriers and beliefs’ section of the questionnaire. Factors predicted by varimax rotation factor analysis and given a descriptive label. In this table only, responses were collapsed similar to Nagy et al. 20017 so that ‘Agree’ includes both ‘Strongly agree’ and ‘Agree’ (scores 4 and 5) and ‘Disagree’ includes ‘Strongly disagree’ and ‘Disagree’ (scores 1 and 2). The ‘Neutral’ category includes scores of 3.
Note: Questions with terms in CAPITAL letters have been rephrased from negative to positive statements.
Analysis
To facilitate the grouping of related items in the ‘barriers and beliefs’ section, an exploratory factor analysis with variax rotation was performed. 20 The criterion for the existence of a factor was an eigenvalue of greater than 1.0. The common theme of each resulting factor was assessed and a descriptive label attached to each factor based on a consensus of the authors. The internal consistency of factors was subsequently assessed using Cronbach’s alpha coefficient. Statistical analyses were performed in SAS statistical software.
Results
Demographics
The survey was emailed to 300 addresses and after follow-up contact, 114 surveys were returned. Fifteen of these surveys were partially completed, but included complete demographic information and full information in either the ‘information sources’ section or the ‘barriers and beliefs’ section. This represents a 38% response rate. The actual number of respondents for each item can be found in tables 2 and 3. The respondents’ characteristics are summarized in table 2. Male respondents outnumbered females by two to one, which is likely representative of a profession that has traditionally been male dominated. Only two respondents were under the age of 30 years, while five were over 60 years old. The majority of respondents (93.8%) were between 30 and 60 years old with a relatively even distribution across those ranges. The majority (59.7%) had been practicing for 10 to 30 years.
A slight majority (55.3%) of respondents worked in private facilities, while the remainder were employed in public institutions. Most (41.2%) worked in small clinics of one to two clinicians and about 36.0% of respondents worked in clinics with five or more clinicians. The highest representation of respondents came from the province of Ontario (46.5%), which appears reasonable given that 39% of Canada’s total population resides in Ontario. 21
Nearly half (49.1%) of all respondents reported having a bachelor’s degree as their highest degree, and 38.6% a college diploma. Overall, 80.7% of all respondents had a college diploma, which was expected since a college program is mandatory for certification in most Canadian provinces. Only seven respondents (6.1%) had earned a master’s degree and none a doctoral degree.
In terms of credentials, about half of the respondents (53.6%) were orthotists, one third (33.0%) prosthetists and the remainder were certified as both. Nearly one half (45.6%) of all respondents claimed to have held an executive position in an organization within a provincial, national or international prosthetics and orthotics organization. Finally, less than one third (28.1%) of all respondents reported having authored or co-authored a peer-reviewed journal article.
Information sources
A dominant majority of clinicians had access to peer-reviewed journal articles (93.9%), industry conferences (98.2%), industry courses and seminars (97.4%), and, as expected, industry publications (96.5%). Of the minority who did not have access, most expressed that they would like to have access.
To answer a practice-related question, respondents indicated that they were most likely to inquire with their colleague (76.6%) or to perform an internet search (64.0%). Respondents were less likely to access message boards or listserves (24.3%), peer-reviewed articles (38.7%), conferences (47.7%) or industry documentation (41.4%).
When asked to rate their frequency of use of each of these sources of information, 28.8% of respondents had accessed a message board, 61.3% a peer-reviewed journal and 73.0% industry documentation at least once in the past month. Most common methods for accessing information were by internet searches and asking colleagues, with 84.7% and 91.0% of respondents, respectively, indicating that they had used these sources at least once in the past month. Furthermore, 87.4% of respondents had attended at least one conference over the past year.
Barriers and beliefs
The exploratory factor analysis produced 10 factors as outlined in table 3. Questions that were rephrased from negative to positive are identified in table 3 with the modified term(s) capitalized. Table 3 presents the computed ‘agreement scores’, which are simply the proportions of participants who ‘strongly agree’ and ‘agree’ on a particular item. Furthermore, the summary of factor scores (average score of all items within a factor) is presented in figure 1. Higher scores indicate greater positive agreement (i.e. a score of 5 indicates strong agreement and 1 indicates strong disagreement).
Mean factor scores with error bars showing ± 1 standard deviation. Higher scores indicate greater positive agreement (i.e. a score of 5 indicates strong agreement and 1 indicates strong disagreement). Refer to table 3 for the elaborated descriptions of the factors.
Referring to factor 1, which assessed the value of research in improving clinical practice, there was a high level of agreement (66.7–92.9%), indicating that clinicians felt research was valuable in clinical practice. In contrast, a lower level of agreement was found in relation to having adequate time to devote to EBP (factor 5) and system limitations (factor 2), with levels of agreement ranging from 35.4% to 45.4% and 25.0% to 55.6%, respectively. That is, respondents felt that EBP would place additional demands on their overloaded clinical practices, and that it was too difficult and time consuming to implement. Moreover, many felt that research studies presented too many contradictory results and that these were out of date. Respondents felt moderately confident about their skills to apply EBP (factor 3), with agreement scores ranging from 45.4% to 73.1% and their knowledge and understanding of aspects of EBP, with scores ranging from 59.6% to 69.4% (factor 6). Access to research information was rated highly (63.0–73.1%) (factor 8). Respondents also generally felt that facility attitudes about EBP were positive (factor 4) with agreement scores of 67.6% to 87.9% for three of the four items, but clinicians felt that there was limited encouragement to undertake research activities (item 3 under factor 4) with a score of 38.9%. Factor 7 reflects the relevance of EBP to clinical practice, showing relatively low agreement scores when compared to the other factors (37.4–50.5%). However, it should be noted that internal consistency of this factor was quite low (table 3). Factor 10, which consists of a single item, shows that there is typically a high level of financial support for clinicians to attend conferences, with agreement scores of 79.6%. Finally, factor 9 demonstrates moderate agreement that the presentation format of research evidence is adequate (54.5%).
Discussion
This study represents one of the first attempts to evaluate the multi-faceted nature of EBP as it pertains to P&O using a mass-distributed questionnaire survey. First and foremost, the results indicate that while clinicians value research as a means of improving clinical practice, they are faced with a number of practical barriers in performing EBP, including time constraints, but also other system limitations that hinder them from being able to translate research results into clinical practice (table 3 and figure 1). Clinicians were moderately positive about possessing the knowledge and skills needed to perform EBP. They felt very confident in some aspects of their skills, such as searching and evaluating articles, but many also felt that certain skills could be improved. Based on the last item under factor 3 in table 3, less than half of the surveyed clinicians felt they possessed the necessary skills to evaluate and implement research findings in order to change their current practice.
The resulting factors relating to EBP in P&O were similar to those identified in other healthcare disciplines. Specifically, a survey evaluating evidence-based nursing revealed factors relating to value and clinical usefulness, skills and knowledge, time constrains and facility support. 7 Nagy et al. 7 identified major obstacles associated with EBP in nursing, including the perception among nurses that research was not relevant to practice, and that they possessed inadequate skills and had insufficient time to perform EBP. 7 Similarly, Jette et al. 2 found that for physical therapists, the primary barrier to implementing EBP was a lack of time. Furthermore, time demands were identified as a major barrier to EBP in P&O in a recent review by Ramstrand and Brodtkorb. 17
The present survey revealed similar obstacles relating to time constraints, as well as the questionability of relevance of existing research to clinical practice, a topic that has been of recent focus in P&O. 22,23 In addition, Ramstrand and Brodtkorb 17 have suggested that clinicians lack the expertise to perform research, and this is supported here by the fact that only a few respondents held a master’s degree and none a doctoral degree. To combat this barrier, Ramstrand and Brodtkorb 17 propose that ‘evidence-based practice needs to be prioritized within the profession and that a cultural change needs to be initiated which supports the need for prosthetists/orthotists to become more active in generating research rather than relying on other professional groups to contribute to their professional body of knowledge’. While facilities generally appear to promote EBP and encourage clinicians to pursue new ideas, as well as providing financial support for conferences, there does appear to be a lack of encouragement for clinicians to pursue research. This may not be surprising, given that the majority of facilities are small private organizations (table 2), likely without affiliation to a research resource, and with limited resources (time and money) to pursue research activities on their own.
One factor that did not appear to restrict EBP was access to research information. The majority (>90%) of clinicians indicated having access to sources of information including peer-reviewed journal articles, based on the results from the ‘information sources’ part of the questionnaire. This is further supported by responses to the factor relating to access. However, despite potentially adequate accessibility, fewer than 40% of all clinicians surveyed indicated that they would be very likely to use a peer-reviewed publication to answer a practice-related question. About 40% indicated that they had not used a peer-reviewed publication in the past month, and, next to message boards, peer-reviewed articles were reported as the least accessed source of information. Accessing and reviewing peer-reviewed articles is one of the more demanding methods of information acquisition, and given the findings relating to time restrictions, it is not surprising that clinicians prefer to access most of their information via less time-consuming methods, such as internet searches or asking their colleagues. The presentation method of articles may be an additional limiting factor, as evidenced in the lower score for factor 9. This reliance on non-peer-reviewed information sources raises an important concern about the quality of information potentially being used as ‘best evidence’ in clinical practice.
A number of limitations of the study are noted here. We sampled all clinicians who were registered with the CBCPO in Canada. However, our response rate was less than 40%. Non-respondents may have beliefs, views and experiences that are systematically different than the respondents. Secondly, our sample size was somewhat below that recommended for an exploratory factor analysis. However, we took the additional step of estimating the internal consistencies of the factors to demonstrate that, in general, the items were measuring similar constructs. An exception may be the results associated with factors 7 and 8, which should be interpreted with some caution since internal consistencies fell below the typically acceptable level of 0.7. Similarly, factors 9 and 10 comprise single items for which internal consistency could not be verified. Furthermore, additional validation of this instrument, including the need for evaluating test-retest reliability and validity to confirm the factor structure is suggested. This includes re-examining the wording of certain questions to ensure that they are not eliciting biased responses. Moreover, greater insights may be gained by examining the responses for various demographics strata, including education, age, authorship and professional involvement. Finally, since the survey was conducted in Canada, the generalizability of the findings to other parts of the world should be examined. Thus, the authors suggest that similar initiatives be implemented in other countries.
Conclusion
EBP refers to the utilization of research-based evidence to inform and improve clinical practice. The ‘informing’ part of the process requires that research data are available. Similarly, data about existing EBP are essential in order to inform those involved in improving existing clinical practices, including educators, professional organizations and governing bodies. This study represents one of the initial steps in acquiring empirical data to gain a better understanding of the underlying barriers and facilitators relating to EBP in P&O. The findings suggest that while clinicians perceive EBP to be highly valuable in clinical practice, barriers exist that restrict them from adequately performing EBP. These include time constraints, workload demands, limited relevant evidence from research, as well as gaps in skills and knowledge required to perform EBP.
Footnotes
Acknowledgements
We would like to thank Bryan Steinnagel, Shane Glasford and Kristen Matthews for their valuable inputs during the development of the survey.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
