Assessing the Utility of an Electronic Research Database to Capture Whole Systems Practice at Two Naturopathic Outpatient Teaching Clinics

Abstract

This observation study assessed the utility of an electronic Naturopathic Patient Database at 2 naturopathic teaching clinics as a research and educational tool. Focusing on the whole systems practice of naturopathic medicine using individualized treatment protocols, patients from the Canadian College of Naturopathic Medicine outpatient teaching clinic and the Boucher Institute of Naturopathic Medicine outpatient teaching clinic were assessed. Data were collected on patient demographics, laboratory values, physical measures, treatment protocols, and validated questionnaires. Results showed the Naturopathic Patient Database was able to capture both standardized and individualized measures of care for research purposes. A number of substudies were generated by student interns using the database. Limitations identified included limited resources for training faculty, interns, and administrators on program use and data entry; also, data quality and completeness were problematic. However, the Naturopathic Patient Database captured data on the individualized nature of naturopathic care and had the ability to collect and compare large amounts of data.

Introduction

A challenge to nonconventional health care systems such as naturopathic medicine has been the inability to measure its treatment protocols as an intact whole system. ¹ Naturopathic medicine describes itself as a distinct system of medicine with terms and definitions for diagnosis and application of multiple therapies in its treatment protocol. To date, only a few published studies have examined the effectiveness of naturopathic medicine as a whole medical system. ^{2 –5} There is a paucity of literature examining the daily clinical reality of patients seen at naturopathic clinics. Such information is essential to the understanding of patient-identified needs and treatment outcomes (eg, change to quality of life, increase in pain, and/or discomfort), which are essential components of overall care. ⁶

A more comprehensive understanding of the clinical environment and the effectiveness of prescribed treatment protocols requires the collection of a high volume of data that are reflective of clinical practice. In the past, studies on nonconventional medicine have used clinic audits for data collection and patient evaluation. ⁷ This form of clinical audit is valuable to describe the “real-world” setting of naturopathic practice but requires a large sample size (ie, a large volume of data) to evaluate and validate patient care and treatment outcome. ⁷ Building on the concept of a “living laboratory,” ⁸ a practice-based unit would allow for practice/research evaluation on an ongoing basis. Furthermore, the data gathered would allow the formulation of relevant and practical research questions, building of practice-based evidence, and provide an avenue for knowledge uptake and transfer.

For naturopathic medicine, the scarcity of research on actual clinic practice and outcomes is due in part to the fact that no tool has been designed to collect the information needed in a real-life clinical setting. A relatively new software called the Naturopathic Patient Database developed at the Canadian College of Naturopathic Medicine provided us with a starting point to collect the broad base of information needed to adequately study these issues.

The Naturopathic Patient Database was designed as a data collection system to be implemented within a clinic setting with the goal of obtaining information on patient intake, treatment protocols, and outcome measures. It would serve as a valuable resource for instructors to search for real naturopathic cases relating to their area of instruction. The Naturopathic Patient Database could also be used to evaluate whether the naturopathic training program was effective in preparing students to be practice-ready as primary care providers. This could feed back into curriculum development, refining the delivery of core competencies at naturopathic educational institutions. Ultimately, the Naturopathic Patient Database would allow faculty members to conduct their own research initiatives such as case reports and case series.

On discussion among educators and researchers, an idea developed to see if the Naturopathic Patient Database could be explored and expanded as a tool to collect and develop measurable outcomes. As a research tool, the Naturopathic Patient Database could potentially help naturopathic doctors to increase their capacity for research. The goal of this project was to determine the utility of the Naturopathic Patient Database as both a research tool and clinical teaching aid. Specific goals were to modify and improve the Naturopathic Patient Database to

Monitor the pattern(s) of treatments prescribed by naturopathic student clinicians

Compare practice patterns or patterns for diagnosis and assessment

Inform patient perceived benefit from treatment through validated questionnaires

Enable the conduct of high-quality and in-depth case series, observational studies, and cross-sectional reports on population data

In a joint effort between the Boucher Institute of Naturopathic Medicine and Canadian College of Naturopathic Medicine, this project refined the Naturopathic Patient Database as an education and research tool for clinic practice, assessed its implementation at both schools, and examined the merits and challenges of trying to develop a single tool to meet research and clinical education needs.

The purpose of this study was to refine the Naturopathic Patient Database and determine whether it was suitable as a research and clinical database tool. The objectives of this formative article are the following:

To describe the implementation of the upgraded Naturopathic Patient Database at the Canadian College of Naturopathic Medicine and Boucher Institute of Naturopathic Medicine

To assess the utility of the Naturopathic Patient Database program based on feedbacks from faculty and student clinicians

To determine the feasibility of using the Naturopathic Patient Database in a clinical setting for educational and research purposes.

Materials and Methods

This observational study explored the application of the Naturopathic Patient Database in the teaching outpatient clinics at the Boucher Institute of Naturopathic Medicine and Canadian College of Naturopathic Medicine. This section describes the multiple steps involved in the process of Naturopathic Patient Database development and implementation.

Audit of the Existing Version of the Naturopathic Patient Database at Canadian College of Naturopathic Medicine

The Naturopathic Patient Database was first developed at the Canadian College of Naturopathic Medicine in 2007 (see the appendix for background on the software program). To assess the utility of the Naturopathic Patient Database, a number of stakeholders within both academic and clinical departments at the naturopathic colleges were consulted, using focus groups, surveys, and comparisons with patient charts. Issues discussed were the following:

Accessibility of the Naturopathic Patient Database from a user’s perspective: time required to enter data, ease of use of data entry fields

Searchability of the database: categorization of items/terms, the ability to define and extract specified data sets, and the ability to export variables

Internal consistency of data entry: completeness of data compared to actual patient charts

Actual and potential contributions of the Naturopathic Patient Database to research, academic, and clinical objectives

As the Naturopathic Patient Database was to serve research, clinical, and academic needs at 2 separate institutions, a series of strategic meetings between representatives of Naturopathic Patient Database user groups were held to identify concerns of disparate stakeholders. The investigative team met monthly to review the feedback and survey findings and to provide directions for the modification of the software.

Implementation of Recommendations

Upgrades to software were conducted based on the final list of recommendations created through the process described above. A Faculty Working Group was established at the Canadian College of Naturopathic Medicine to provide suggestions from a faculty perspective on the proposed changes to the Naturopathic Patient Database. The Group continued to meet after the implementation of the revised version of the Naturopathic Patient Database. Comments from this group were fed back to clinic and academic deans, as well as to the Naturopathic Patient Database management team. Focus groups with students at both schools provided further feedback on the upgraded version, resulting in additional changes to the Naturopathic Patient Database. The final version of the Naturopathic Patient Database was released to the Canadian College of Naturopathic Medicine in May 2010 and implemented at the Boucher Institute of Naturopathic Medicine in September 2010.

To test the research utility of the Naturopathic Patient Database, 2 prospective substudies were conducted with the following groups: (a) cancer patients at the Adjunctive Cancer Care shift at the Robert Schad Naturopathic Clinic of the Canadian College of Naturopathic Medicine and (b) diabetes patients at the Boucher Outpatient Clinic at the Boucher Institute of Naturopathic Medicine. Detailed description and analysis of each substudy exceeds the scope of this article and are to be published separately.

Results

Audit

Comprehensive engagement of a wide range of Naturopathic Patient Database stakeholders revealed a number of challenges with the practicality and functionality of the Naturopathic Patient Database for students, supervisors, administrators (of the clinical training program), and researchers. Student concerns included the length of time for data entry, repetition of some fields, level of details, and navigating the Naturopathic Patient Database. For supervisors, concerns included accuracy and completeness of the data and the ability to link details in the Naturopathic Patient Database to clinical care. For researchers, concerns included the functionality of Naturopathic Patient Database search features, ease of data extraction, and inadequate data input in practice. Concerns for clinic and academic administrators included the relevance of the Naturopathic Patient Database to inform learning objectives for the Naturopathic Doctor training program.

Collating of Feedback

An initial list of concerns was drafted based on the audits. This “wish list” for upgrading the Naturopathic Patient Database included recommendations to streamline data entry, addition of drop-down menus, and removal of nonessential fields.

Consensus Building

At the Canadian College of Naturopathic Medicine, where the Naturopathic Patient Database had been in use already for 4 years, attempts to reach consensus over changes to the tool resulted in discussions about the overall role of the Naturopathic Patient Database in the teaching curriculum. Concerns were raised repeatedly over the amount of time needed for data entry and the relevance of this activity to naturopathic education. Through a series of consensus-building meetings and through ongoing feedback from the Faculty Working Group, this wish list was revised repeatedly to meet the needs of all users, for academic and research purposes.

Implementation

The upgraded version of the Naturopathic Patient Database, nicknamed the “NPD Lite,” was implemented in May 2010 at the Canadian College of Naturopathic Medicine and in September 2010 at the Boucher Institute of Naturopathic Medicine. Developers of the Naturopathic Patient Database provided tutorials to train faculty and student interns at both the Canadian College of Naturopathic Medicine and Boucher Institute of Naturopathic Medicine.

Evaluation

The Faculty Working Group continued to meet following the implementation of the Naturopathic Patient Database–Lite version. Ongoing evaluation of the new version found the following:

Persisting concerns regarding the amount of time required for data entry and redundancy with paper charts

Continued instability of Naturopathic Patient Database software resulting in “crashing” of system and necessitating adjustment of end-of-term academic deadlines

Ongoing user issues: incomplete drop-down menus, absence of basic user functions (eg, ability to cut and paste), unreliable online access

Continued complaints from students regarding the applicability of Naturopathic Patient Database use to their learning

Lack of benefit to faculty and students resulting from absence of accessible searching interface

Continued student frustration with the Naturopathic Patient Database requirements and utility

Research Capability of the Naturopathic Patient Database

To test the research capability of the Naturopathic Patient Database, a number of small projects were conducted. These inquiries served as the basis for examining specific cohorts within the Robert Schad Naturopathic Clinic patient population and provided research oriented exercises for clinical interns. They include the following:

Profile of cancer care patients from a naturopathic teaching clinic database: demographics, treatments and outcomes. Presented at the American Association of Naturopathic Physicians conference, August 2010.

Naturopathic treatments for Functional Dyspepsia: A best case series using the Naturopathic Patient Database at the Canadian College of Naturopathic Medicine. Presented at the Natural Health Products Research Society Conference, March 2008.

An analysis of patient outcomes by supervisor years of experience at the Robert Schad Naturopathic Clinic. Unpublished internal report.

Following implementation of the Naturopathic Patient Database–Lite version, other studies were conducted as a result of the improved search functionality of the Naturopathic Patient Database. They include the following:

• A cross-sectional study of patients of the Robert Schad Naturopathic Clinic using the Naturopathic Patient Database. Posters displayed at the 2011 American Association of Naturopathic Physicians and 2011 Canadian Association of Naturopathic Doctors conference.

• Treating type 2 diabetes: A cross-sectional audit of naturopathic care within a naturopathic college teaching clinic using the Naturopathic Patient Database. Presented at the 2011 American Association of Naturopathic Physicians conference; poster displayed at the 2011 Canadian Association of Naturopathic Doctors conference.

To assess the research capability of the Naturopathic Patient Database–Lite version in collecting prospective data, an observational study of adjunctive cancer care was conducted at the Canadian College of Naturopathic Medicine, and a study on the use of naturopathic medicine for type 2 diabetes was conducted at the Boucher Institute of Naturopathic Medicine. In each case, the Naturopathic Patient Database was found to lack the capability to comprehensively capture the necessary data. A major problem was missing data due to lack of consistent data entry in specific fields and over time. For example, clinic requirement had student clinicians entering data on 3 visits, whereas research needed data entry for every visit. In the case of the Canadian College of Naturopathic Medicine study, patient charts were used rather than the electronic records contained within the Naturopathic Patient Database; results from this study were presented at the 2011 American Association of Naturopathic Physicians conference and the 2011 Society for Integrative Oncology conference, and a poster was displayed at the 2011 Canadian Association of Naturopathic Doctors conference.

Discussion

The Naturopathic Patient Database was developed as an electronic database to capture real-life patient care (eg, chief concerns, individualized treatment protocol, and outcome) in a naturopathic clinic setting. The goal was to use the Naturopathic Patient Database to enable training of naturopathic students and increase clinical research capacity. However, multiple demands on the Naturopathic Patient Database proved that it was unable to meet multiple and diverse needs of the stakeholders.

Merits

Implementation of the Naturopathic Patient Database at the Robert Schad Naturopathic Clinic has provided students with the opportunity to have some first-hand experience of naturopathic research. Through entry fields prompting interns to collect objective data from their patients, it may also have encouraged all interns in the collection of laboratory and imaging results, as well as results of quality of life tools such as the Measure Yourself Medical Outcomes Profile, the European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire, and the Diabetes Treatment Satisfaction Questionnaire.

Through comparisons of data housed in the Naturopathic Patient Database and that contained in patient charts, it is clear that the Naturopathic Patient Database–based data are predictive of trends seen in patient charts. The diabetes audit ⁹ reports comparable scoring based on the Naturopathic Patient Database and on patient charts, whereas the profile of Adjunctive Cancer Care patients demonstrates parity between prescription trends in the Naturopathic Patient Database and in patient charts.

The evolution of the Naturopathic Patient Database within the context of the learning institutions has also had some interesting yet unanticipated effects. Missing data in the Naturopathic Patient Database have accelerated conversations at the Canadian College of Naturopathic Medicine and Boucher Institute of Naturopathic Medicine regarding the adoption of electronic medical records in place of the paper charting that is currently in use. Awareness of the importance of comprehensive and accurate charting has also been raised by inconsistencies in Naturopathic Patient Database data. Data access policies have been reviewed, and finally, some intake forms at the Canadian College of Naturopathic Medicine have been revised to be more inclusive based on the nature of data in the Naturopathic Patient Database. Finally, audits of patient files have raised institutional awareness about the need for ongoing evaluation and feedback of clinic practices to all stakeholders.

The database has also yielded a number of studies and continues to be a repository for assessing specific cohorts seen at the Robert Schad Naturopathic Clinic. In addition, the development of this tool toward the study of specific conditions has led to the systematic adoption of validated questionnaires for people with cancer at the Adjunctive Cancer Care at the Canadian College of Naturopathic Medicine (unpublished) as well as to a more effective audit of care for patients with diabetes at the Canadian College of Naturopathic Medicine. ⁹ Numerous applications for data mining are available for this tool, and to date, the potential yield of this database has been minimally exploited. Even when incomplete, files within the Naturopathic Patient Database can flag specific patient charts for researchers to review further in their investigations.

Challenges

The Naturopathic Patient Database was hampered by significant challenges throughout its evolution. As a tool designed to serve the needs of many user groups, the conflicting needs of these stakeholders made the development of an ideal tool impossible. As a result, efforts to build consensus and buy-in among Naturopathic Patient Database users were not unanimously successful.

Even after the streamlining process occurred, data entry into the Naturopathic Patient Database remained a time-intensive and somewhat redundant undertaking with the retention of paper charting. The questionable relevance of moderate-volume data entry to the training of future naturopathic doctors was a central reason for the termination of this initiative, in addition to the negative effect that Naturopathic Patient Database–related requirements had on student morale. Significant requirements for other resources such as additional faculty and intern training, on-site technological support, and portable data tablets made this initiative unsustainable over the long term.

Competing Factors

The utility of the Naturopathic Patient Database was further questioned by the emergence of 2 other data collection tools at the Robert Schad Naturopathic Clinic. The use of the clinic’s Point of Sale software program (Microsoft Dynamics Retail Management System Store Operations) began to expand alongside the redevelopment of the Naturopathic Patient Database. This system currently collects visit-by-visit information about prescription categories and International Classification of Diseases–10th Revision codes associated with each patient, thus negating the usefulness of the Naturopathic Patient Database. It should be noted, however, that the rationale for adoption of the Point of Sales and the information captured by it were partly informed by the learning experience that came out of the Naturopathic Patient Database.

An additional competing factor is the desire to move toward the use of electronic medical records at both the Canadian College of Naturopathic Medicine and Boucher Institute of Naturopathic Medicine. At the Canadian College of Naturopathic Medicine, the Point of Sales serves as a bridging technology between paper charts and a fully functional electronic medical record system. Continued investment of resources in the Naturopathic Patient Database would yield a system that would duplicate much of the data collected by either the Point of Sales or electronic medical records. As the Naturopathic Patient Database contains many details that are already recorded in patient files, data entry was also a redundant process. The practicality of the Naturopathic Patient Database was questioned given that it served as a database but not electronic medical records, thus requiring duplicate data entry and capturing incomplete data compared to the patient charts. This project found that upgrading the Naturopathic Patient Database did not increase its utility in a clinic setting. Ultimately, the cost was deemed too high for the value of the data yielded. Institutional support for this initiative diminished and all data entry into the Naturopathic Patient Database ceased at the Canadian College of Naturopathic Medicine and Boucher Institute of Naturopathic Medicine after the completion of this project.

Conclusion

This study provided insights into the merits and challenges of developing the electronic Naturopathic Patient Database as a research and clinical educational tool. Despite numerous challenges, the Naturopathic Patient Database generated data that captured numerous elements reflective of naturopathic practice, including naturopathic diagnoses and individualized treatment protocols. Furthermore, the data were used for small projects by student interns and research fellows in their research training. However, because of competing demands of research and clinical education, the Naturopathic Patient Database did not meet those needs sufficiently. The value of conducting clinical audit to describe the “real-world” setting of naturopathic practice and generate research outputs is well recognized by the naturopathic profession. The need for a tool to collect large volume of data on naturopathic care has been identified as a priority by national naturopathic associations. Thus, we hope the lessons from this project will help future development of software to capture clinical data for research purposes and build on research evidence for naturopathic medicine.