Abstract
A replication survey of physicians’ expectations and experience with speech recognition technology was conducted before and after its implementation. The expectations survey was administered to emergency medicine physicians prior to training with the speech recognition system. The experience survey consisting of similar items was administered after physicians gained speech recognition technology experience. In this study, 82 percent of the physicians were initially optimistic that the use of speech recognition technology with the electronic medical record was a good idea. After using the technology for 6 months, 87 percent of the physicians agreed that speech recognition technology was a good idea. In addition, 72 percent of the physicians in this study had an expectation that the use of speech recognition technology would save time. After use in the clinical environment, 51 percent of the participants reported time savings. The increased acceptance of speech recognition technology by physicians in this study was attributed to improvements in the technology and the electronic medical record.
Introduction
Speech recognition technology (SRT) has become part of our daily lives. iPhone owners routinely query Siri to find their favorite restaurant or to get the directions to a friend’s house. It is now commonplace to talk to our automobiles using everyday language and to expect an appropriate response. Health care professionals routinely use SRT. Hospital providers use SRT to reduce report turnaround times and to decrease the time it takes to get those reports to health care providers. Radiologists have used SRT for the past 12 years, 1 while pathologists and technicians have enhanced their productivity through applied speech recognition. 2
The aim of this replication survey was to (a) identify how attitudes and perceptions among physicians affected the acceptance and success of the speech recognition system and the associated work procedures and (b) investigate whether speech technology was more widely accepted 8 years after an earlier study conducted in Denmark. 3 The questions used to guide this research were as follows:
Can SRT assist emergency room (ER) physicians with the input of electronic medical record (EMR) information?
Will there be a reduction in the turnaround time for patient records?
Will SRT improve the completeness of patient records?
The ER medical staff accepted the implementation of the EMR during the year prior to this study, and it was felt that medical record turnaround time and the quality of the records could be improved while minimizing the cost of using additional staff (i.e. scribes) to assist the physicians in providing accurate medical records. After discussing the research with the hospital and ER staff representatives, the research team was encouraged to proceed with the research study.
According to Nuance Communications, 4 the four different modes that physicians use to enter data into an EMR are (a) manually driven EMR, (b) traditional transcription, (c) speech-assisted transcription, and (d) speech-driven or speech-assisted EMR. Physicians are interested in maximizing the quality of medical records while minimizing data entry record turnaround time and the total system costs.
The acceptance of SRT may be dependent on the acceptance of the electronic health record. El-Kareh et al. 5 surveyed 86 primary care clinicians using SRT between December 2006 and January 2008 and reported that communication among clinicians increased over a 12-month period (72%–93%; p < .001), and improvements in the overall quality and efficiency of the system were also noted. The authors concluded that primary care clinicians reported increasingly positive perceptions of a new electronic health record within 1 year of implementation across a spectrum of domains of care.
According to Koivikko et al. 6 and Kang et al., 2 SRT has improved productivity in health care settings. Research conducted in the HUS Helsinki Medical Imaging Center from 2005 through 2007 concluded that SRT speeds up processing time from the completion of imaging and archiving to when the report was stored and then made available to clinicians online. 6 Liu et al. 7 researched the characteristics of effective structured reporting and its integration with SRT. According to their findings, the process of dictation followed by transcription does not take maximum advantage of the digital advances in radiology and does not fully exploit present-day and future software capabilities.
SRT has been touted for providing a positive return on investment in clinical settings. At the University of Pittsburgh, a study of SRT for the dictation of biopsy reports and reporting final diagnoses showed a marked decrease in turnaround time and decreased error rates. The researchers concluded that SRT was a good investment, despite start-up costs. 2 Physicians, particularly those just coming out of medical school and entering the field, are seeking up-to-date work environments where the level of care is consistently developing, maturing, and improving.
SRT can play an important part in patient care. When the patient leaves the examination room, (a) the patient note is completed, (b) follow-ups are scheduled, (c) prescriptions are written and sent to the pharmacy, and (d) billing information is submitted. 4 If the focus remains at the individual patient’s level, the service and care will be better than if staff were jumping from one task to the next without closing the first. Physicians using SRT software reported a greater ability to portray the uniqueness of each patient encounter better within the note. Physicians relying on transcription often do not review their work for a week and may have difficulty in remembering what was said or to which patient the information was referring. 4
Methodology
Voice recognition research design
A study of the acceptance of SRT by physicians was conducted at a hospital in Denmark during 2005–2006. Their investigation provided important results, and the authors of this study sought and received permission to replicate those parts of the Denmark study by Alapetite et al. 3 that related to this study’s research questions.
Participants in the Danish survey consisted of 186 anonymous physicians who worked at the Vejle and Give Hospital. The departments involved were medicine, neurology, oncology, organ surgery, orthopedic surgery, and otology. The age and gender distributions of the participants were not detailed in the Denmark study. The Danish research team developed and deployed a survey consisting of 16 items in two phases. The prospective phase surveyed the physicians’ expectations and attitudes toward SRT, and the retrospective phase surveyed the physicians’ experiences with the technology.
Description of Emergency Medical Service department and hospital practice
In this study, the Emergency Medical Service department was staffed by contract with a separate physician firm. This firm provided the services of physicians for all four hospital ERs. The firm also provided scribes who worked with the doctors and entered the majority of the patient information into each patient’s electronic health record. Although the use of scribes was very expensive, the senior management of the physician firm felt the expense was necessary in order to provide high-quality medical records and to enable the physicians to keep up with the demand for physician services during very busy times.
The technology has improved significantly since 2006 when the original Denmark research was done. Nuance Dragon Software was used in this research, and it is the accepted standard throughout the United States. The software used in the Denmark study was Philips Speech Magic. Nuance has also worked with Epic representatives, the EMR software, and integrated their Dragon software with Epic, so it is easier for the physicians to use and has fewer errors as a result. US citizens use SRT technology in many parts of their lives including on their cell phones, in their cars, and also in response to phone systems.
Training was provided by two Epic Credentialed Trainers with a background as Dragon Train the Trainer (provided by Physician Technology Partners) in a 3-h training session. Providers were given their choice to attend one of multiple scheduled sessions. During the session, a standardized lesson plan was followed. The providers set up their Dragon profile and learned the four levels of correction, navigation within Dragon, additions to vocabulary, and use and creation of commands. Training materials were provided to the end users.
Originally, go-live with the electronic health record (Epic) did not have Dragon use. Mercy Health had decided to implement the use of Dragon throughout 30 emergency departments (EDs). The decision to utilize Dragon in the EDs was based upon the need to improve the ED provider’s efficiency in creating notes due to the volume of patients.
ED providers were very dissatisfied with use of Epic alone. They were bogged down and unable to keep up with patient volumes. Smart tools within Epic were not allowing providers to document a complete, accurate note in an efficient manner. Many of the providers had resorted to using scribes in an effort to allow them time for medical decision making. Across the Mercy Health Network of hospitals, Dragon with Epic has been offered in go-lives. Mercy Health Youngstown’s (MHY) seven EDs and Urgent Cares experienced a time lag between Epic go-live and Dragon go-live as infrastructure was built and the use spread.
Workflow did not change in seeing patients and creating notes, placing orders, reassessing results/patients’ responses, medical decision making, and disposition of the patient.
Once the availability of SRT software was identified, the senior management of the physician firm sought to determine whether the patients’ electronic record quality would be the same or better if scribe services were not used and the physicians used voice recognition software in place of the scribes. This led us to propose the hypotheses to be tested:
Survey development and implementation
The first task of this study was to develop a survey to measure attitudes and resistance to change. Results of the earlier Danish study were presented to the hospital staff representatives, and their opinions were solicited as to the viability of replicating that study. Upon their acceptance, the initial Danish study survey was modified, so it was more suitable to this study’s participants (i.e. ER physicians). The initial (pre-SRT implementation) survey assessed the physicians’ expectations of SRT prior to its implementation. The physicians were encouraged, but not required, to use SRT for producing medical records. The follow-up (post-SRT experience) survey assessed their perceptions after they acquired more than 6 months’ experience with SRT. In this study, 10 items were included in both the pre-implementation and post-experience surveys. The survey items are provided in Table 1. Some questionnaire items are used in the analysis and are designated with the letter A. Several items included in both the expectations and the experience surveys are designated below with a “
Expectations and experience surveys.
Results and discussion
Physicians’ SRT expectations and experiences
In contrast to the Denmark study in which several medical departments were involved, the participants in this study were all ED physicians. Of the 61 participants, 67 percent were male and 33 percent were female. The majority of the participants identified themselves as White (84%), followed by other (7%) and Asian (5%), and the remaining 4 percent were equally distributed among Black, Hispanic, and Unspecified. Participants ranged in age from 24 to 66 years with an average age of 42 years.
A total of 61 participants (nP = 61) completed the expectations survey, and of those, 47 completed the experience survey (ne = 47). SRT use varied among the participants. The largest proportion of those completing the experience survey (43%) indicated that on average, they used SRT between 81 and 100 percent of the time for medical record documentation. This was followed by those using it 61–80 percent of the time (19%), 1–20 percent of the time (15%), 21–40 percent of the time (6%), and 41–60 percent of the time (4%), and 13 percent of the participants indicated no use of SRT (Figure 1).
SRT use reported by participants.
There were 10 items included in both the expectation and experience surveys. The percentage of positive, neutral, and negative responses for these items is shown in Figure 2 with descriptive statistics in Table 2. Less than half of the participants expected that technology use during their clinical work would raise their level of stress (45%). After having the opportunity to use SRT for 6 months, fewer than one in four (23%) felt that using the technology would increase their stress levels. Physicians’ perception of the impact that technology would have on their level of stress was significantly related to their perception of whether it was good idea to implement SRT (see Table 4). The questions that received more positive responses toward using SRT suggested that the majority of physicians felt that (a) it was a good idea to introduce SRT for medical record keeping; (b) SRT was easy to use; and (c) SRT would optimize the process of keeping the medical record, save time, and decrease the time spent on medical records and (d) result in records being more complete. Most of the physician’s (74%) expected SRT would improve completeness of the EMR, and after using SRT, the majority of physicians (67%) indicated that they perceived medical records were more complete (see Figure 2).
Speech recognition—expectation/experience comparison.
Descriptive statistics.
SD: standard deviation.
Three items showed a significant difference between physicians’ expectations and experience after using SRT for 6 months (Wilcoxon test; see Figure 2). These items suggested that (a) SRT would be easy to use (95% agreed decreased to 75%), (b) much effort would be needed to use SRT (decreased from 54% to 20%), and (c) the time required producing the medical record would be shorter (83% decreased to 53%).
A statistically significant relationship was identified between the overall assessment item that implementing SRT was a good idea and the physicians’ expectations and experience regarding (a) completeness of the medical record (rho = .32 and .35), (b) optimization of the record keeping process (rho = .50 and .43), (c) SRT ease of use (rho = .55 and .53), and (d) time required (rho = .35 and .29). The strength of these relationships was moderate for all but the physicians’ perception of time required after SRT use which was weak. The relationship between the overall assessment item and the perception that service provided by a scribe could not be matched by SRT showed a decrease in the strength of the positive relationship as it went from a significant relationship (rho = .67) when expectations were assessed to a weaker non-significant relationship (rho = .30) after the physicians experienced SRT. See Table 3 for additional information.
Relationship of items concerning performance expectancy, effort expectancy, and facilitating conditions with overall assessment of speech recognition system (N = 61; unless otherwise specified).
Single items from the expectations and experiences surveys.
Correlations (Spearman’s rho) *p < .05; **p < .01.
A significant positive relationship between physicians’ expectations regarding the implementation of SRT and physicians’ technology attitudes was identified. The results from the expectations survey indicated that a positive, moderate relationship existed between the SRT implementation item and items assessing the degree to which participants enjoyed trying new technology (rho = .50), that technology use will increase their level of stress (rho = .36), improve staff efficiency (rho = .43), benefit patients (rho = .39), and that others often seek their advice about technology systems (rho = .30). See Table 4 for additional information.
Relationship of items concerning computer attitudes with overall assessment of speech recognition system (N = 61).
Single items from the expectations survey.
Correlations (Spearman’s rho) *p < .5; **p < .1.
In the Danish study, physicians were approximately evenly divided among those (a) who thought it was a good idea to introduce speech recognition (33%), (b) who thought it was not a good idea (31%), and (c) who were neutral on the subject (36%). Workflow improvements and the possibility to access the records immediately after dictation were almost unanimously appreciated. The Danish physicians tended to report a more negative view of the system after having used it for several months. They felt that (a) they spent much more time producing medical records than before, (b) they used more time correcting speech recognition, and (c) the overall quality of the records declined. Physicians’ experiences were more negative than their expectations with respect to the quality of medical records and the time spent producing them.
Physicians’ experience with SRT in this study is perceived more positively than that reported in the earlier Denmark study. Experiences of the physicians in this study revealed that a strong majority felt that introducing SRT for medical record keeping was a good idea (87%). The majority of physicians in this study felt that (a) less time would be spent producing medical records than before (53%), (b) the record keeping process had been optimized (65%), and (c) the records were more complete (67%).
Strengths and limitations
The strengths of this study are that (a) nearly 91 percent of the ER physicians completed the pre-implementation (i.e. expectations) survey, (b) a current SRT system was utilized, (c) ER physicians were given 6 months to use the SRT system before the post-implementation (i.e. experiences) survey was implemented, and (d) key items of the survey by Alapetite et al. 3 were adapted and used in this investigation.
The limitations of this study that may reduce its accuracy include that (a) the subjects and the reviewed medical records were not randomly selected, (b) the physicians were not required to utilize the SRT in producing their medical records, (c) the accuracy of the medical records was not assessed, and (d) not all of the participants completed the expectations survey (77%), which may have resulted in a response bias.
The overall assessment is that the ER doctors were happy with the use of SRT coupled with the EMR since 82 percent of the ER doctors thought that SRT was a good idea prior to implementation, and after implementation 87 percent thought so. This is a consistent finding before and after the introduction of and the 6-month use of SRT. The finding that 75 percent of the participants perceived SRT implementation as positive after using it is similar to findings noted in an earlier study at a military hospital and outlying clinics where 69 percent of the physicians continued the use of voice recognition software after an initial period of use. 8
After 6 months of SRT implementation, 23 percent of the ER doctors thought that stress was increased while initially 45 percent of the doctors thought that stress would be increased. Therefore, fewer physicians than expected felt that stress was increased. Prior to implementation, 95 percent expected SRT to be easy. After experiencing the use of SRT technology, three-fourths of the participants (75%) thought it was easy. Initially, 54 percent of the ER physicians thought that SRT would require too much effort, but after implementation of the system, only 20 percent stated that SRT required too much effort. In addition, 82 percent of the doctors felt that SRT would optimize record keeping, and after 6 months of use of SRT 65 percent still felt that way.
Prior to implementation of SRT, 72 percent of the participants expected time savings, and after 6 months of use, 51 percent agreed that time was saved. Initially, 83 percent of the physicians felt the time to produce medical records would be shorter with SRT. After 6 months of SRT implementation, 53 percent perceived that record production time was shorter.
Summary and on-going research
The results of this study indicated that using SRT along with the EMR is not too stressful. Even after 6 months of using SRT, ER doctors still felt it was a good idea. Most of the physicians in this study thought that it was easy, and few thought it required too much effort. Of the physicians, 65 percent thought that SRT would optimize record keeping and 51 percent reported that it would save time benefiting patient care. Therefore, the implementation of SRT contributed to ease of use.
Workflow did not change in seeing patients and creating notes, placing orders, reassessing results/patients’ responses, medical decision making, and disposition of the patient. Although the use of Epic’s Smart Tools was modified to maximize efficiency. MHY providers began using a combination with the Power Mic II (Nuance Dragon Microphone) in one hand and a mouse in the other. Complete, accurate notes could be made in a little over 1 min. Use of scribes has been almost eliminated. The quality and timeliness of notes have been demonstrated. Other researchers have recommended the use of SRT to support the production of medical records but not to fully automate the process. 9 Future research could also explore the use of SRT in this support role. The research team and hospital administrators explored future opportunities to conduct research studies as the use of SRT is implemented in other departments throughout the hospital and the corporation.
Future plans to spread the use of Dragon to inpatient providers across all of Mercy Health are being evaluated at a MHY hospital (St Joseph Warren Hospital) through a pilot study. Training began on 2 February 2015 with the go-live on 5 February 2015. An impact study is in progress to evaluate return on investment (ROI).
Footnotes
Acknowledgements
We would like to thank A. Alapetite, H.B. Andersen and M. Hertzum for their willingness to share their research materials with our research team. This included their research design and surveys.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.