Abstract
Background: An important way of improving healthcare services is through the implementation of evidence-based practice; but this requires an understanding of the extent to which it is occurring and the factors that are driving its implementation.
Objective: To examine the associations among the demographics of clinicians, the factors involved in the implementation of evidence-based practice, and the access of clinicians to various sources of information.
Study Design: Cross-sectional survey.
Methods: An online survey that was distributed to 300 Canadian prosthetic and orthotic clinicians. Associations of selected survey items were determined.
Results: Four primary associations were found and a further 18 were considered to be indicative of potential trends. Two of the primary associations were related to authorship and the utilization of scientific literature. Specifically, those clinicians who had previously authored or co-authored a peer-reviewed journal article were more likely to utilize scientific literature to guide their clinical practice.
Conclusions: This study has highlighted important demographics which can be targeted for greater implementation of evidence-based practice. Above all, facilitating engagement of clinicians in research and its dissemination may promote a higher consumption of research evidence leading to improved evidence-based practice.
This study provides information about the underlying facilitators and inhibitors of evidence-based practice in prosthetics and orthotics. The findings aim to inform those involved in improving existing clinical practices, including educators, professional organizations and governing bodies.
Keywords
Introduction
The purpose of health-related research, be it directly or indirectly, is ultimately to benefit the patient. One important way this can be effectively accomplished is through evidence-based practice (EBP). Defined as ‘the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients’, 1 EBP has been extensively researched across many healthcare fields, including nursing, physiotherapy, medicine and surgery. An overarching concern appears to be the extent to which EBP is occurring and what can be done to facilitate its implementation. Specific topics that have been studied include the strategic implementation of EBP,2-4 evaluation of EBP educational programs,5-8 quality of research and its presentation9-11 and assessment of the attitudes, perceptions and abilities of healthcare professionals in implementing EBP.12-18
In an effort to promote the strategic implementation of EBP on a large scale, Mohide and Coker 2 and Caldwell et al. 3 documented attempts to implement EBP within large allied healthcare institutions, while Kitson proposed a strategic framework for examining ‘the interplay of the three core elements’ in implementing EBP – ‘the level and nature of the evidence, the context or environment into which the research is to be placed, and the method or way in which the process is facilitated’ 4 . Research on the evaluation of educational programs has included pre- and post-intervention surveys that measure the self-assessed skills of the participants7,8 and the validation of a tool used to evaluate the effectiveness of EBP teaching programs. 6 Finally, Newhouse et al. described a pilot project designed to train nurses through the application of EBP-related skills. 5 The importance of being able to adequately evaluate research was outlined by Fairman and Frederic 11 while two different approaches to improve the clinical significance and usability of research were suggested by improving the clinician–researcher relationship 9 and making the research more appealing to the reader through presentation and content changes. 10 Each of the studies that examined the attitudes, perceptions and abilities of healthcare professionals in implementing EBP used a self-report style survey to collect the data.12-18
Unfortunately, in the field of prosthetics and orthotics (P&O), EBP and its implementation has not been researched to the same extent, and this is likely to have important implications when it comes to providing quality patient care. At the very least, as suggested by Ramstrand, 19
…professional colleagues including medical doctors, occupational therapists and physiotherapists are all working to increase their own use of EBP stimulated by their own professional groups and certifying bodies… It is subsequently reasonable to assume that our peers and fellow members of the clinical team will come to expect and demand that prosthetists/orthotists practice from an evidence based platform.
We recently presented the results of a survey which identified various factors that may either promote or inhibit the implementation of EBP among certified prosthetists (CPs) and orthotists (COs) in Canada. 20 The study included an exploratory factor analysis that was similar to work previously completed by Nagy et al. directed at Australian nurses. 14 From this initial work, we established 10 factors which were ranked in terms of how they promoted or inhibited the implementation of EBP (Appendix 1).
A demographic directed section was also included in the survey. An examination of the data from this section revealed a diverse population concerning characteristics such as the level of formal education or training, the type of practice (public or private), clinic size (number of clinicians) and publication experience of the clinician. 20 This extent of the diversity that is evident within the P&O profession suggests that a more comprehensive understanding of EBP in the P&O field could be gained by assessing EBP practices not for the population sample as a whole, but rather by examining strata within demographics and other meaningful characteristics of the sample. Hence, stemming from our previous work, we hypothesized that there exist informative associations among the demographics of the clinicians, the access clinicians have to various sources of information, and the factors involved in the implementation of EBP. 20 The objective of this study was to examine these associations as they may elucidate the factors or characteristics that play a role in implementing EBP and this may suggest ways in which EBP can be more widely and effectively put into practice in the P&O field.
Methods
This study involved the creation of a survey and its distribution via email to certified prosthetic and orthotic clinicians across Canada. The questionnaire was generated by completing a thorough review of surveys focusing on EBP across allied healthcare industries. A bank of questions was created using the items in these surveys. These questions were reviewed by a focus group which included prosthetic and orthotic clinicians as well as a researcher. Questions were eliminated if they were duplicates (as determined by group consensus) or if they were specific to the healthcare sector targeted by the survey (i.e. unrelated to P&O). Some unique questions were also generated to investigate the demographics and other characteristics that are specific to the P&O industry. The initial email invitation was followed by two reminder emails and finally a phone call to promote participation in the survey. The questionnaire was completed using online survey software. The study was approved by the institution’s Research Ethics Committee. Consent was considered implied by clinicians who completed the questionnaire.
Participants
Three hundred CPs and COs who were in good standing with the Canadian Board for Certification of Prosthetists and Orthotists (CBCPO) were invited by email to participate, of which 114 responded by filling out the survey (response rate of 38%). The database of emails was generated from a publicly available website (www.cbcpo.ca).
Instrument
The questionnaire was divided into three sections (1) demographics (11 items), (2) information sources (20 items) and (3) barriers and beliefs (36 items). The information sources section contained questions relating to what sources the clinicians have access to and what sources they are most likely to use. The barriers and beliefs section was used to gather the opinions of the clinicians regarding any potential reasons they may have for not integrating the results from research into clinical practice as well as their opinions as to whether or not the implementation of an evidence-based practice is valuable to their clinic and their patients. All questions in this section were posed in the form of statements and the clinician was asked on a five-point Likert scale to agree fully, somewhat agree, neither agree nor disagree, somewhat disagree or fully disagree. Statements were written both positively and negatively so that the questions did not reflect a positive or negative disposition toward EBP. Also the term evidence-based practice was not used except when explicitly defined as not all clinicians may be familiar with the term. The questionnaire was originally pilot tested and a French version was developed as outlined previously. 20
Analysis
The Mann-Whitney U test and Kruskal Wallis test were used to examine the relations between selected items in each of the three sections of the questionnaire. The groups of data that were analyzed included (1) the set of 10 factors (time, system limitations, relevance, presentation, knowledge, skills, access, facility support, value and financial) shown to be either facilitative or prohibitive to the implementation of EBP via a previously conducted factor analysis, 20 (2) eight categories of demographic data and (3) 12 questions examining the clinicians’ access to various information sources (Appendix 1). To limit the number of comparisons, a number of items were excluded based on the assumption that they would not provide meaningful or relevant information or that they were highly correlated with another item. Examples include practice location, which would not produce reliable results given the very large differences among locations in terms of the numbers of respondents, and the number of years practicing which is highly correlated to age of the clinician. In total, 296 comparisons were made between the items in the three groups of data.
Statistical analyses were performed using SPSS software. An alpha value of 0.05 was used to determine statistical significance, however this was adjusted using the Bonferroni correction in order to minimize the effect of type I error. Given the number of comparisons, the alpha value was set at 0.00016 (0.05/296). This, however, may be somewhat of a conservative approach given that all of the items are likely not entirely independent of each other. Hence, results were presented for all data with p-values of 0.01 or less.
Results
Demographic characteristics of the respondents are shown in Table 1 while a more in-depth look at the demographics can be found in an earlier publication. 20 The association results are displayed in Table 2, and include four primary associations that were found to be statistically significant at p < 0.00016 and another 18 secondary associations representing potential trends at p < 0.01.
Characteristics of respondents including age, number of years practicing, prosthetic and orthotic credentials, highest degree received and facility domain.
Statistically significant associations in order of statistical significance. An asterisk (*) indicates a primary result. § indicates no clear trend shown in the data. Factors (bolded) and items from the ‘information sources’ portion of the questionnaire (italicized) are described in more detail in Appendix 1.
CO, certified orthotists; CP, certified prosthetists; CPO, Certification of Prosthetists and Orthotists; P&O, prosthetics and orthotics.
Two of the four primary associations were related to authorship and utilization of peer-reviewed journals. Specifically, those clinicians who had previously authored or co-authored a peer-reviewed article used journals more in the past month in their clinical practice than those who had not previously been an author or co-author of a peer-reviewed journal article (result 1). They also rated the access factor as less prohibitive (result 3) indicating that they perceive themselves to have greater (1) access to the journal articles that they would like to read and (2) access to an online database to search for journal articles, as these are the two items that make up the access factor. 20
Furthermore, as part of the primary results an increase in the number of clinicians in a facility was associated with an increase in the number of times clinicians consulted with a colleague on a clinical problem (result 2). And finally, those that worked in a public clinic reported that the financial factor was more prohibitive than those that worked in private clinics (result 4).
The additional 18 results (5 through 22) with p < 0.01 can be summarized within the following themes based on the demographic categories and/or the inhibitive and facilitative factors (Table 2). These are clinic size, the domain of the clinic (public or private), the highest level of education of the clinician, clinician’s age, clinician P&O credentials, whether or not the clinicians have held an executive position, authorship, the facility support factor, the skills factor, the access factor and the time factor.
Discussion
This work builds on a previous study that examined the factors that were prohibitive and facilitative in the implementation of EBP. 20 It, however, also shares certain study limitations that were identified as part of this previous work, including the potential of a sampling bias, low or undetermined internal consistencies of several of the factors, potential specificity of findings to P&O in Canada, and the need for further validation and reliability testing of the survey instrument. 20 In addition, the initial work revealed that the population that was being studied was demographically diverse in categories such as the highest level of education, clinic type and size, and publication experience, highlighting the need for further examination of these demographic strata. As such, the focus of the current work has been to analyze the relationships and associations between the previously established EBP factors, the clinicians’ access to various sources of information, and the demographic characteristics of the clinicians. Through this effort it is our intent to highlight ways in which EBP in P&O can be enhanced.
Those surveyed were asked to indicate how many clinicians (including him or herself) worked at their clinic in an attempt to gauge its size. It was found that respondents from clinics with higher numbers of clinicians reported consulting with their colleagues more frequently (result 2). This is not of great surprise as there are many clinics with only one clinician, in which case, it would be impossible to consult with a colleague. This result is important, however, as it shows that with convenient access to fellow colleagues, collaboration among fellow clinicians on a single clinical problem is more likely. This suggests that for those at clinics with only one clinician, alternate means of collaboration and sharing of ideas over long distance may be of benefit. An example of this is the online O&P Edge Listserve (www.oandp.com/oandp-l/). However, only 28% of clinicians had previously stated that they had used online message boards at least once in the past month. 20 Clinic size was also found to be associated with the financial factor (result 15). That is, those that worked in smaller clinics felt that they were better financially supported in attending conferences. This may be due to the effort of smaller clinics to make available a variety of sources of information in order to compensate for decreased colleague support.
Type of facility – whether the clinician was from a public or private clinic – was found to be significant in terms of the financial support factor for conference attendance (result 3) and the number of conferences attended in the past year (result 5). These results appear to go hand in hand because with less prohibitive financial support one would expect employees in the private sector to be more likely to attend conferences. This causality was not examined and therefore cannot be confirmed but at the very least a statement can be made about the variation in employee conference attendance between the private and public sectors. Clearly, if clinicians in the public sector are expected to deliver similarly evidence-based care as those in the private sector then they should be given equal opportunity to do so. Budgets of public P&O clinics may need to be reassessed as to how much is allocated for clinician access to conferences. Alternately, it may simply be an issue of communication, and public clinics may need to better communicate to their clinicians about the financial support that is available to pursue this type of professional development. In either case what we have found is that those in the public sector believe that financial support for conference attendance is more prohibitive than those in the private sector.
One of the most important findings of this work is the association between involvement in authorship of journal articles, and the access to, and incorporation of, research findings into clinical practice. Specifically, those that have authored or co-authored a peer-reviewed article are more likely to access and have access to scientific literature (results 1, 4 and 21). To better understand this association, it is useful to examine in greater depth the potential factors or facilitators of authorship.
First, publishing in a peer-reviewed journal requires certain skills, which can be obtained through training in appropriate scientific and research methods. From our survey, those that responded more positively toward the skills factor claimed that they would be more likely to search for evidence in journals than those who reported that the skills factor was more prohibitive (result 20). This was also the case among those that responded positively towards the time factor (result 22). In addition, the time factor was shown to be less prohibitive to those with university degrees compared to those without (result 18). These results complement each other in that, presumably, at the university level individuals would acquire more exposure to peer-reviewed journal articles and would develop a greater comfort level and increased aptitude with respect to accessing, interpreting and utilizing scientific literature. This is consistent with other studies looking at the effectiveness of EBP tutorials as a means of improving skills and knowledge.7,8
While undergraduate studies at the university level may provide an opportunity to develop the necessary skills to critically assess scientific literature, ultimately, conducting research that is publishable in quality peer-reviewed journals requires more extensive training provided as part of higher academic degrees, including Masters and PhD level graduate studies. Unfortunately, as stated by Ramstrand 19 ‘the research culture within prosthetics and orthotics has not evolved to the extent that it has for other allied health professions and there are currently fewer persons qualified to pursue higher degrees and lead research from within the profession’. And, in fact, in this survey not one clinician had a PhD degree, only 6.1% had a Masters degree, and 49.1% had a Bachelors degree. It is only very recently that in Canada, P&O programs have taken steps to improve on this standard, both by raising the admission requirements into P&O programs to include a Bachelors degree, and by offering more opportunities for formal academic training in the form of graduate-level P&O programs.
The second factor in publishing is providing opportunities for meaningful involvement in research activities. This allows clinicians to gain an understanding of the research process and subsequently to develop the skills to critically assess research findings, design appropriate research studies, and disseminate scientifically founded matter in scientific papers. 21 In allied healthcare fields, such as medicine, research training is considered to be an essential part of the formal education/training process and is appropriated during residency through research fellowships. However, these educational/training practices are typically not mirrored in P&O. Residencies in P&O are directed primarily on improving clinical skills and developing clinical experience, with a relatively minor focus on research.
Where opportunities do exist that facilitate clinicians in getting appropriate research-related education and training, as the third factor, engaging and promoting clinicians to continue to have a research focus requires further and ongoing organizational support. For example, in nursing, Mohide and Coker outlined an effort to use an evidence-based nursing committee to create an organizational culture shift toward EBP. 2 They found that efforts would need to be made on individual, group and organizational levels and highlighted 13 necessary principles which they followed. The very first principle stated that ‘organizational endorsement is a critical prerequisite in the cultural shift toward EBP and best practices’ (p. 373). 2
Similarly, individuals surveyed in this study who responded more positively toward the facility factor used journals more in the past month than those who found the facility factor more prohibitive (result 16). This result suggests that greater support and encouragement from the institution and its management may promote more frequent usage of peer-reviewed journals by subordinates when solving clinical problems. To further examine the importance of this association the individual items which make up the facility factor can be used. The statements comprising the facility factor include: (1) ‘clinicians in my facility have shown that it is possible to change practice on the basis of research findings’, (2) ‘my facility has a climate which encourages staff to pursue new ideas’, (3) ‘my facility encourages me to carry out research’, (4) ‘prosthetic and orthotic research is of interest to clinicians’. 20 To reiterate, those that answered more positively toward the above statements used journals more so when addressing clinical problems than those that were less positive. Again causation was not explicitly examined and it could be interpreted that either, a positive atmosphere toward research and the integration of its findings into practice encourages clinicians to use journals, or that journal usage leads to a more positive disposition toward the adaption of EBP within the clinic.
As a final thought, these results raise the important question of how much support should be provided by facilities or institutions. Taking example from allied healthcare professions, the ultimate level of institutional support would allow highly qualified clinician researchers who are interested in pursuing research in P&O to adopt the roles of ‘clinical investigator’ or ‘clinician scientist’. Support would be provided to ensure access to human resources (research assistants, students, administrative support, etc.), research facilities (lab space, equipment, etc.,), and adequate clinical relief time. From a practical standpoint, this extent of support would be expected more from public institutions, such as university-affiliated rehabilitation centers, rather than private clinics. Furthermore, clinicians employed by private facilities with presumably fewer resources dedicated to research should be encouraged and supported in their efforts to collect standardized outcome measures as part of their daily practice. Not only would this strengthen the level of evidence within their own clinic but if multiple clinics were to combine their results it could potentially result in a pool of empirical clinical data, which may be disseminated to provide important new insights into aspects of clinical practice. Further research should look at these two avenues, the creation of researcher–clinician positions in the public facility, and a more systematic application of patient outcome measures in both private and public facilities, as important avenues for improving evidence-based practice in the field of prosthetics and orthotics.
Conclusions
The production and consumption of scientific literature, which is an important factor driving EBP, appears to be a self-perpetuating one in that those clinicians that have been involved in research and who have published on it are more likely to utilize it in their clinical practice. Therefore, at an institutional level, it is important that clinicians are given opportunities to not only develop their research skills, but to utilize them as well.
Footnotes
Appendix
The 10 factors with descriptions as well as an example of one of the statements from within the factor which those surveyed were asked to agree or disagree with on a five-point Likert scale (top). Also shown is the wording of the information sources questionnaire items (bottom).
| Factor | Description of factor and example of item within that factor |
|---|---|
| Time | Time to devote to evidence-based practice (EBP) E.g. I believe that I can search for the best evidence to answer clinical questions in a time-efficient way |
| Limitations | System limitations and constraints in applying EBP E.g. The process of adapting research results into practice does not place other demands on already overloaded clinicians |
| Relevance | Relevance of research to clinical practice E.g. Most prosthetic and orthotic research is relevant to clinical practice |
| Presentation | Research present format E.g. I find the typical journal/research presentation method (introduction, methods, results, discussion) adequate |
| Knowledge | Knowledge and understanding of EBP E.g. I understand the procedure necessary in implementing evidence based practice |
| Skills | Skills and abilities to perform EBP E.g. I do feel I have the skills to properly use an online database to search for articles |
| Access | Access to research information E.g. I have access to an online database to search for journal articles |
| Facility | Facility support and attitudes about EBP E.g. Clinicians in my facility have shown that it is possible to change practice on the basis of research findings |
| Value | Value of research in improving clinical practice E.g. I am sure that using results from research in clinical practice will improve the care that I deliver to my patients |
| Financial | Financial support for conferences E.g. Financial support to attend conferences is available if I apply to my facility |
| Questionnaire item | Related question |
| Past month colleagues consulted | In the past month please indicate approximately how many times you have used each source of information |
| Past month journal usage | |
| Past month industry documentation usage | |
| Likelihood of journal usage | Please indicate the likelihood of using each source of information to seek out an answer to a specific question you may have during practice |
| Likelihood of industry documentation usage | |
| Likelihood of conferences attendance | |
| Past year conference attendance | In the past year approximately how many days have you spent attending conferences |
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.