Sage Journals: Discover world-class research

Abstract

This study aims to develop and evaluate a new Health-oriented Electronic Oral Health Record that implements the health-oriented status and intervention index. The index takes the principles of holistic oral healthcare and applies them to the design and implementation of the Health-oriented Electronic Oral Health Record. We designed an experiment using focus groups and a consensus (Delphi process) method to develop a new health-oriented status and intervention index and graphical user interface. A comparative intervention study with qualitative and quantitative methods was used to compare an existing Electronic Oral Health Record to the Health-oriented Electronic Oral Health Record, focusing on dentist satisfaction, accuracy, and completeness of oral health status recording. The study was conducted by the dental staff of the Inter-country Center for Oral Health collaborative hospitals in Thailand. Overall, the user satisfaction questionnaire had a positive response to the Health-oriented Electronic Oral Health Record. The dentists found it easy to use and were generally satisfied with the impact on their work, oral health services, and surveillance. The dentists were significantly satisfied with the Health-oriented Electronic Oral Health Record compared to the existing Electronic Oral Health Record (p < 0.001). The accuracy and completeness values of the oral health information recorded using the Health-oriented Electronic Oral Health Record were 97.15 and 93.74 percent, respectively. This research concludes that the Health-oriented Electronic Oral Health Record satisfied many dentists, provided benefits to holistic oral healthcare, and facilitated the planning, managing, and evaluation of the healthcare delivery system.

Keywords

Accuracy and completeness electronic oral health records health index user satisfaction

Introduction

The provision of adequate oral health services is a major problem in developing countries. The magnitude of this problem can be estimated from the scope and coverage of services, the numbers of oral health personnel in each country, and the proportion of the national health services budget devoted to these services. The biological factors that affect the size and character of the oral health service are oral diseases. These influence the service because of their prevalence and severity.¹ Changes in disease prevalence, changes in demographics resulting from the aging and urbanizing of populations, and changes in oral care technology have further increased the need for a revised framework for oral healthcare. This framework should place the appropriate emphasis on primary healthcare, starting from self-care, health education, and health promotion, and be complemented by restorative care. Oral health delivery services should also be organized so they can respond adequately to changing needs.

The health information system is among the building blocks of the World Health Organization (WHO) health system framework, which ensures the production, analysis, dissemination, and use of reliable and timely information on health determinants, health system performance, and health status.² At present, Electronic Health Records (EHRs) are used to improve healthcare systems. The concept of EHRs comprises a wide range of information systems, from files compiled in single departments to longitudinal collections of patient data.³ The implementation of EHRs can help lessen patient suffering due to medical errors and improve the ability of analysts to assess quality.⁴ EHR systems can help reduce medical errors by providing healthcare workers with decision-making support such as fast access to medical literature and information on current best practices in healthcare.⁵ EHRs increase user and patient satisfaction, which might lead to significant improvements in healthcare practices.^6,7 The advantages of EHRs also include reduction in cost, improvement in the quality of care, promotion of evidence-based practice, and improvement in record keeping. By integrating with Clinical Decision Support Systems (CDSSs), the clinician would be able to analyze patient data and make decisions regarding diagnosis, prevention, and treatment of health problems.⁸

In today’s age of information, with its extraordinary technological potential, precision and versatility terms are required. Converting these terms to numeric values as the core terminology for specifying oral health status, anatomy, prescriptions, interventions, skills, and settings is proposed for healthcare providers and institutions that educate and train healthcare personnel. Currently, the data on oral health status and oral health indices for monitoring disease are disease-oriented, focusing on some aspect of oral diseases, for example, the Simplified Oral Hygiene Index (OHI-S),⁹ the Community Periodontal Index (CPI),¹⁰ Decayed Missing and Filled Teeth (DMFT) index,¹¹ and the International Caries Detection and Assessment System (ICDAS).¹²

Considering the scope of the concern in oral health services enhanced by EHRs, significant problems include the absence of a useful model for health-oriented oral care that links the concept of health and the goals of healthcare with all of the details about health problems. This study first aimed to develop a new Health-oriented Electronic Oral Health Record (Health-EOHR) that integrated the health-oriented status and intervention (SI) index to facilitate planning, managing, and evaluating the healthcare delivery system. Second, a comparative intervention study with qualitative and quantitative methods was used to compare the existing EOHRs to the Health-EOHR and focused on dentist satisfaction, accuracy, and the completeness of the record.

Methodology

As a research process, it began with the initial conceptualization of the SI Model and oral health status graphical user interface and was assessed by a consensus Delphi process; the results from the first phase were continually applied throughout the implementation of the Health-EOHR phase, and finally, the system was assessed by dentist satisfaction, accuracy, and completeness of the recording in system evaluation phase (Figure 1).

Figure 1.

Three phases of the research process in development and evaluation of the Health-EOHR: (1) development of the health-oriented status and intervention index, (2) system implementation, and (3) system evaluation.

Development of the health-oriented SI index

Intervention index

Since 1978, much emphasis has been placed on achieving “Health for all by the year 2000” and establishing indicators to measure the progress toward this goal in different sectors. Much less attention has been paid, however, to formulating appropriate strategies for the various programs involved. An economically feasible care model that is relevant to all aspects of health and provides a structure within which primary healthcare can be effective is badly needed. With the assistance of the WHO, the Inter-country Center for Oral Health (ICOH, Chiang Mai, Thailand) has pioneered an alternative community oral healthcare model based on the primary healthcare concept.¹³ The ICOH’s primary aims are to recognize the importance of oral health and support and disseminate technology and knowledge about oral health among developing countries.

Two districts in the Chiang Mai and Lamphun Provinces have been selected as sites for testing the model. The district in the Chiang Mai Province is called Sa-merng and has a population of approximately 19,000. The terrain of Sa-merng is mountainous, and household clusters are widely separated from each other. The socioeconomic level is very low, with an average annual per capita income of only US$60. The Pa-sang district in the Lamphun Province is a low-lying and highly cultivated area, with a population of approximately 84,000. The average annual per capita income is US$500. The project provided health-promoting comprehensive oral healthcare to the community in accordance with its needs and, at the same time, evaluated the acceptability, effectiveness, and economic feasibility of the model, the service system, the associated training programs, and the recording and information system.

We adopted the idea from the ICOH’s WHO project and developed the intervention index, which is a numerical classification of procedures in oral health. It is used to identify care needs and classify the tasks, instruments, and personnel needed to provide the care required for an individual and community. Its advantages include better understanding with minimal dependence on the memorization of terms specific to a given field of information. Numbers (Arabic numerals) are universally used (i.e. as a global language). Their use as the core language of specification minimizes the language barrier, bias, and subjectivity inherent in mother tongues. Thus, numeric terms are a logical choice for widespread and systematic use in the health information system.

The orientation, planning, and implementation of the community-care model are geared toward the concept of “health” or the absence of the need for care (“concept zero”) (Table 1). In addition to identifying the goals of healthcare, “concept zero” provides the basis for a logical numeric scale for use in classification and recording and in relating the goal and the scope of concern in a given health area. For this project, the goal for community oral health is “0” (i.e. the absence of the need for oral healthcare); the goal is also “0” for the health of the oral cavity and the surrounding tissues. Using the numerals 0 through 9, the meanings of which are defined in the SI index, care specifications, settings, and activities can be related to the goal of oral health. The “concept zero” approach means that the need for any activity, setting component, or intervention must be justified in the context of an approach to the goal. Thus, before implementing any activity, the fundamental question, “Is this likely to contribute to an overall improvement in oral health in the long term?” should be answered.

Table 1.

The status and intervention (SI) index for classifying health status and procedure.

Concept 0	Health and healthcare										Concept −1
	−.0	−.1	−.2	−.3	−.4	−.5	−.6	−.7	−.8	−.9
Health	Collection of information	Self-care	Maintenance of surface, support, or structure	Position anomaly dimension	Loss of support, structure, function				Replacement		Dependence handicap
	Collection of information	Self-care	Maintenance of surface, support, or structure	Position anomaly dimension		Partial		Full	Fixed	Removable	Dependence handicap

	Oral healthcare needed
	0	1	2	3	4	5	6	7	8	9
Absence of need for care	Recording	Self-care mouth	Maintenance	Maintenance-restoration				Surgery	Replacement
	Recording	Self-care mouth	Maintenance	Gingival-tooth		Tooth
	Status	Monitoring explanation exercise	Surface, by performer	Position	Support	Dentin–enamel	Pulp	Access removal	Fixed	Removable

Oral health status graphic user interface design

The graphic user interface of each oral health status item has been designed to follow steps in status recording, from chief complaint to hygiene and periodontal condition, defect, and restoration. Table 2 shows an example of the graphical user interface design for the status as well as the translation of the intervention into International Classification of Diseases (ICD) codes.

Table 2.

Part of status and intervention (SI) index (the whole model contains 80 items that cover all aspects of oral health SI.).

Status		Intervention
Status description	Graphic	Index	Intervention description	ICD-9-CM
Calculus –Z– = <2 mm		1	Self-care	96.54
Calculus –Z– 2–3 mm		1	Self-care	96.54
Calculus –Z– = >3 mm		2 (23)	Scaling	96.54
Horizontal pocket		4 (44)	Root planning	24.31
Minor defect		2 (25)	Protective/disease arrest	96.54
Dentin defect		5 (53)	Filling	23.2
Pulp threat		5 (54)	Filling	23.2
Roots/enamel		5 (57)	Replacement (laboratory processed)	23.41
Pulpitis/necrosis		6 (66)	Root canal treatment	23.71
Supernumerary tooth		7 (75)	Tooth removal	23.09
Missing		8 (84)	Replacement (tooth)	23.42

ICD-9-CM: International Classification of Diseases, Ninth Revision, Clinical Modification.

SI index

The SI index and decision support have been developed to provide the basis for a completely numerical recording system that can cover all of the data on oral status treatment needs, records of planned and completed procedures, clinic organization, and scheduling of patients. As for decision support, the appropriate intervention, care provider, time, setting, and cost have been provided for a given oral status, which can be altered according to the dentist and patient preferences (Table 3). The SI index can be used for detailed identification of the treatment needed and the conditions that should be referred to the community or province hospitals for treatment. In addition, the SI score is a holistic indicator of individual or community health status and of types of intervention. The SI score can be used for identifying the needs of the population to specify the tasks to be accomplished in oral healthcare and to plan the workplace and community for sufficient resources for effective performance without unnecessary or excess facilities. The system also enables the epidemiological evaluation of community status and the quantity, quality, and effectiveness of care provided; the data can be rapidly and economically summarized, either by hand or by computer.

Table 3.

An example of the decision support interface of the Health-EOHR program after input oral health status data.

No.	Tooth	Status description	Intervention code	Operator	Time (min)	Instrument set	Cost (Baht)
1	12	Vertical pocket	4	Dentist	15	Periodontal	340
2	11, 21, 22, 31, 32, 41, 42	Horizontal pocket	4	Dentist	45	Periodontal	1360
3	16, 25	Dentin defect	5	Dental nurse	20	Restoration	600
4	11, 12, 15, 16, 18, 21, 22, 23, 31, 41, 42, 43	Recession	2	Dental nurse	30	Scaler	280
5	14, 17, 24, 26, 27, 32, 35, 36, 38, 44, 45, 46, 47, 48	Missing	8	Dentist	10	Impression	5400
6	11, 13, 21, 22, 23, 25, 31, 32, 33, 34, 35, 41, 42	Gingivitis –Z– 2–3 mm	1	Self-care	—	—	—
7	34	Root filling +post +crown	—	—	—	—	—
8	12	Root filling +Crown	—	—	—	—	—
9	37	Crown	—	—	—	—	—
10	11, 15, 33, 41, 42, 43	Tooth-colored filling	—	—	—	—	—
11	16, 18, 25, 28	Metal, amalgam filling	—	—	—	—	—
12	13	Long axis rotation	—	—	—	—	—
13	41	Distal tilt	—	—	—	—	—

EOHR: Electronic Oral Health Record.

Delphi assessment

Five ICOH experts participated in the evaluation of the proposed SI index. The evaluation consisted of questionnaires, focus groups, and a Delphi assessment. The experts were asked to grade their agreement with 80 items in the SI model on a 5-point Likert scale; both positive and negative statements were included to avoid bias. All experts took part in focus groups. The experts were invited to discuss positive and negative aspects of the SI index and to give suggestions for its development. To provide a robust evaluation of specific components of the SI model, a consensus method was used, which consisted of a two-panel, three-round adapted Delphi technique. A well-executed Delphi technique provides an effective method of group communication.¹⁴ Using an iterative process with controlled feedback enables consensus about a given issue to be measured and maintains interparticipant anonymity.¹⁵

System implementation

Our idea is to develop an open-source Health-EOHR that can plug into the existing Hospital Information System. The Health-EOHR operates in client–server architecture that connects infrastructures and networks of community and healthcare centers. The data are recorded in a database server. Both types of client–server applications are available to support a multitier operation. An example of an oral health status user interface is shown in Figure 2. The diagram shows how the Health-EOHR program works from the time users enter data to the time when they get the decision support message.

Figure 2.

An example of an oral health status and decision support user interface design after patient’s oral health status is recorded in the Health-EOHR.

System evaluation

A comparative intervention study approach involving qualitative and quantitative research aspects was used. Based on the questionnaires, interviews, and oral health status recording reports, the dentist satisfaction, accuracy, and completeness of the recording were assessed to complete the oral health status recording reports, while the existing EOHRs were used for compiling reports in June 2012. The Health-EOHR was implemented in July 2012 and was assessed in the same approach in August 2012. The study was not designed to investigate the existing EOHRs and the Health-EOHR in the same period of time. We made the following hypotheses: (1) the dentists who use the Health-EOHR will be more satisfied than those who use the existing EOHRs and (2) the accuracy and completeness of the oral health status recording reports in the Health-EOHR will be higher than the existing EOHRs.

Interviews and questionnaires were conducted to assess dentist satisfaction with the existing EOHRs and the Health-EOHRs. The criteria for the dentists who participated in this interview included having at least 1 year of experience using EOHRs in the hospital. Therefore, 26 dentists from 11 ICOH-collaborating hospitals were recruited. The participants’ experience with EOHRs was quite wide ranging from 1 to 19 years. Most participants considered their skill in using EOHR systems as average (65.4%) or novice (11.5%).

The questionnaire was developed by modifying questionnaires from previous studies.^16–19 The questionnaire comprised questions that covered the level of satisfaction or dissatisfaction with each of the following issues: current use, function, alerts, and prompts. We used the scale Very Satisfied, Satisfied, Neutral, Dissatisfied, and Very Dissatisfied. Researchers also asked participants to indicate their level of agreement or disagreement with each of the following issues: ease of use and flexibility, impact on work, impact on patient care, and overall satisfaction. We used the scales Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree. The questions were open-ended and had suggestions for system improvement.

To evaluate the accuracy and completeness of the oral health information recording using Health-EOHR, we randomly selected two cases that had oral examinations for routine clinical care by the second author (S.S.); the two cases had complete information, intraoral pictures, and x-ray films. All participants were briefly instructed on the use of the Health-EOHR and the requirements of the oral health status recording. The participants received a verbal explanation about the use of the system from the investigators, and familiarized themselves with the system interface for 15 min; however, they did not familiarize themselves with the task. During this familiarization or warm-up period, each participant was allowed to ask questions and receive further verbal explanation and suggestions from the investigators. After the familiarization, the participant recorded oral health status of the two cases again. The outcome measures were the accuracy and completeness of the recording. The accuracy and completeness evaluation of the oral health information recording using an existing EOHR could not be performed because the oral status recording and data entry were not provided. The accuracy and completeness of the recording were assessed by the degree of closeness of measurements of the oral health status recording reports and data entry field of oral health status information degree to the oral health status’ actual value, respectively. The high accuracy and completeness of the recording were tested by correct information and sufficiently complete information for experts’ needs.

Statistical analysis

The Wilcoxon test was used to detect any differences in user satisfaction between the existing EOHRs and the Health-EOHR. Statistical significance was defined as a p value < 0.001. The accuracy and completeness of the oral health status recording were presented. All analyses were performed using SPSS version 13.0 (SPSS Inc., Chicago, IL, USA).

Results

User satisfaction

For the survey results, we ranked the percentage of respondents for ‘the existing EOHRs’ and ‘the Health-EOHR’ to indicate their level of satisfaction and agreement. The results are shown in Figure 3. To help clarify the array of numbers, the highest four rankings for each statement are tinted green, and the lowest four are tinted orange. The systems are listed by the sum of their ranks. To better visualize the full range of responses, we turn to charts such as the “Response spectrum” below. Each bar in a response spectrum chart represents 100 percent of responses for a level of satisfaction and agreement, so all bars on the chart have the same overall height. The bars are divided into sections representing, from top to bottom, that the function was installed but not used (or no opinion), and the answers of very dissatisfied (or strongly disagree), dissatisfied (or disagree), neutral, satisfied (or agree), and very satisfied (or strongly agree).

Figure 3.

The response spectrums are based on the percentage of respondents for each system to indicate their level of satisfaction: (a) current use, (b) function, and (c) alerts and prompts.

To interpret the chart, the existing EOHRs had 16.3 percent Satisfied responses for current use, 12.8 percent Satisfied responses for function, 26.9 percent Satisfied responses for alert and prompts (30.8% Function installed but not used), 25 percent Agree responses for ease of use and flexibility, 25 percent Agree responses for impact on work, 9.6 percent Agree responses for impact on patient care, and 12.3 percent Agree responses for overall satisfaction. Approximately, 20 percent of dentists had Strongly Disagree responses for impact on patient care and overall satisfaction in the existing EOHRs. According to the questionnaires, the existing EOHRs did not provide benefits to improve the quality of dental care. The features that had Strongly Disagree responses included “make patient care less expensive,” “decrease patient waiting time,” “useful tools for disease management and customized features,” “easy and intuitive to use,” and “effective documenting dental care.”

In this survey, the Health-EOHR had positive responses: 70.2 percent Satisfied responses for current use, 76 percent Satisfied responses for function, 69.4 percent Satisfied responses for alert and prompts, 76.9 percent Agree responses for ease of use and flexibility, 78.9 percent Agree responses for impact on work, 65.4 percent Agree responses for impact on patient care, and 67.9 percent Agree responses for overall satisfaction (Figures 3 and 4).

Figure 4.

The response spectrums are based on the percentage of respondents for each system to indicate their level of agreement: (a) ease of use and flexibility, (b) impact on work, (c) impact on patient care, and (d) overall satisfaction.

A Wilcoxon test was conducted to evaluate dentist satisfaction in the existing EOHRs and the Health-EOHRs. The results of user satisfaction with current use, function, alerts and prompts, ease of use and flexibility, impact on work, impact on patient care, and overall satisfaction indicated a significant difference (p < 0.001) (Table 4).

Table 4.

User satisfaction in the existing EOHRs and the Health-EOHRs.

User satisfaction	Existing EOHRs (mean rank)	Health-EOHR (mean rank)
1. Current use	8.00	12.18
2. Function	5.50	14.17
3. Alerts and prompts	12.83	13.02
4. Ease of use and flexibility	11.00	13.08
5. Impact on work	4.50	11.33
6. Impact on patient care	8.50	13.39
7. Overall satisfaction	2.50	14.42

EOHR: Electronic Oral Health Record; Health-EOHR: Health-oriented EOHR.

Accuracy and completeness of the recording

Five views of the oral health status digital photos and x-ray films of two cases were used for oral health status recordings in the existing EOHR and the Health-EOHR to review the accuracy and completeness of the oral health information recorded. In the Health-EOHR, the accuracy of the oral health information was 96.76 percent and 97.54 percent in cases I and II, respectively. The completeness of the oral health information was 92.0 percent and 95.48 percent in cases I and II, respectively. The overall accuracy of the new Health-EOHR was 97.15 percent and its completeness was 93.74%.

Discussion

Over the past 20 years, researchers have worked to define and document the structure and content of the EOHR. Work on the EOHR is a particular aspect of Dental Informatics. Schleyer²⁰ discussed this term and its relationship to related terms such as information science and technology. Just as the interrelated biological concepts comprising healthcare are intrinsically complex, so is the relationship among various data items in the EOHR. Usually, existing EOHRs allow the practitioner to electronically document patient care, allow claims transactions to be more quickly and reliably processed, and are able to communicate with records from other healthcare disciplines. The Health-EOHR developed in this study has additional functions. The graphic user interface of each oral health status item has been designed to consider the concept of cognitive ergonomics. Cognitive ergonomics,²¹ as defined by the International Ergonomics Association, is concerned with mental processes, such as perception, memory, reasoning, and motor response, as these processes affect the interactions among humans and other elements in a system. The concept has been widely used in several domains, including human–computer interaction.²² Human–computer interaction involves the study, planning, and design of the interaction between people (users) and computers. It is often regarded as the intersection of computer science, behavioral sciences, design, and several other fields of study. The user interface in the Health-EOHR allows the user to follow steps in status recording, such as from chief complaint to hygiene and periodontal condition, defect, and restoration. Graphics for each status item have no ambiguity and are easy to remember.

For over 30 years, computerized applications have been developed to aid the clinician in the medical decision-making process. These applications have been collectively called a number of different terms. One of these terms is clinical decision support,²³ which has been defined as any computer program designed to help health professionals make clinical decisions, address medical data about patients, or provide the knowledge of medicine necessary to interpret such data. All clinical decision support systems have some type of execution paradigm, which is a method of organizing or processing input information to produce some type of output or result. The sequence in which data are requested and the algorithm or method for processing data depend on an underlying model of the decision problem. Currently, the data on oral health status and oral health indices for monitoring disease include only some aspects of oral diseases.^9–12 The important factors that make these information systems less useful are inability to provide an integrated view of patient and community data, lack of integration of dental work, and a limited understanding of dental information needs.²⁴ The Health-EOHR used production rules that had the format “IF condition THEN action,” where the condition was a Boolean logical expression with certain factors associated with the terms. When given a particular criterion, the appropriate treatment plan was automatically provided as an output of the program. In addition, it can be converted into the information needed for system management, such as the intervention along with the care provider, place and duration of time for the treatment, and cost. The Health-EOHR interprets health-oriented SI, providing tangible benefits to holistic oral healthcare and helps plan, manage, and evaluate the healthcare delivery system.

The assessment of user satisfaction and general observations by the researchers revealed that Dentists mainly appreciated the ability to record the oral health status and patient information, precision, current use, function, alerts and prompts, ease of use and flexibility, impact on work, and impact on patient care. The accuracy and completeness of the oral health recording yielded very high rates. The dentists understand the potential benefits of using EOHRs in their practices not only for patient care but also for outcome measurements (when linked with other health and social care datasets), quality improvement, public health surveillance, and research.^25,26 The major barriers to implementation from this study were similar to the barriers described by Anderson²⁷ and Masic.²⁴ These barriers include the cost of investing in information technology, the workload implications, and any changes in work processes. Furthermore, rapid data entry and retrieval are necessary, now that health services are becoming more accountable through clinical governance.²⁸ Similar to the findings of Elizabeth et al.,²⁹ a limitation of this study is the lack of blinding. The participants may have been biased by the novel nature of the study and the impression that the researchers wanted results favoring the Health-EOHR.

Conclusions and recommendations for future work

We introduced a new Health-EOHR that integrates a health-oriented SI index. This research concludes that the health-oriented SI index implemented in the Health-EOHR improves dentist satisfaction, provides benefits to holistic oral healthcare, and helps plan, manage, and evaluate the healthcare delivery system. For future work, it is recommended that the classification used in the SI index be further tested and considered as a basis for training, instrumentation, monitoring, and evaluation. There is a need for an improved flow of information to promote flexibility and provisions for change.

Footnotes

Acknowledgements

The authors acknowledge the contributions and assistance of the staff of the Inter-country Center for Oral Health, Chiang Mai, Thailand. In particular, the authors would like to thank Mr Panu Pakdeesan and Mr Supoj Chamnanprai. We thank all of the dentists for their participation in the system evaluation.

Declaration of conflicting interests

The authors declare that there are no conflicts of interest.

Funding

This work was supported by Department of Health, Ministry of Public Health, Thailand.

References

Aubrey

. Oral health, general health and quality of life. Bull World Health Organ 2005; 83: 644–645.

World Health Organization. Everybody’s business: strengthening health systems to improve health outcomes: WHO’s framework for action. Geneva: World Health Organization, 2007.

Kristiina

Kaija

Pirkko

. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 2008; 77: 291–304.

Richard

Bigelow

Bower

. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff 2005; 24: 1103–1117.

Tom

Dean

Tod

. Electronic health records (EHR). Am J Health Sci 2012; 3: 201–210.

Cyrille

Lise

Judith

. A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? Int J Qual Health Care 2004; 16: 407–416.

Jihad

Jennifer

Ruta

. The use of electronic health records in the exam room and patient satisfaction: a systematic review. J Am Board Fam Med 2009; 22: 553–562.

Eneida

. Clinical decision support systems: perspectives in dentistry. J Dent Educ 2004; 68: 589–597.

Heena

Rushik

. Oral hygiene status in Central Gujarat, 2010—an epidemiological study. J Dent Sci 2010; 2: 51–53.

10.

World Health Organization. Oral health surveys: basic methods. 4th ed. Geneva: World Health Organization, 1997, pp. 36–38.

11.

Larmas

. Has dental caries prevalence some connection with caries index values in adults? Caries Res 2010; 44: 81–84.

12.

Ismail

Sohn

Tellez

. The International Caries Detection and Assessment System (ICDAS): an integrated system for measuring dental caries. Community Dent Oral Epidemiol 2007; 35: 170–178.

13.

WHO Expert Committee. Alternative systems of oral care delivery. World Health Organ Tech Rep Ser 1987; 750: 24–26.

14.

Harold

Murray

. The Delphi method: techniques and applications. 2002. Available at: http://is.njit.edu/pubs/delphibook/

15.

Jeremy

Duncan

. Consensus methods for medical and health services research. BMJ 1995; 311: 376–380.

16.

Antonios

Donald

Daniel

. Physician and nurse satisfaction with an electronic medical record system. J Emerg Med 2004; 27: 419–424.

17.

Kenneth

Robert

. Electronic health records: the 2007 FPM user-satisfaction survey. Fam Pract Manag 2007; 14: 27–30.

18.

Simon

Kerstin

Thomas

. Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda): a qualitative and quantitative comparative study. Methods Inf Med 2008; 47: 489–498.

19.

Kenneth

Robert

. The 2011 FPM survey of user satisfaction with EHR systems. Fam Pract Manag 2011; 18: 15–17.

20.

Schleyer

. Dental informatics: a work in progress. Adv Dent Res 2003; 17: 9–15.

21.

Hoc

. Cognitive ergonomics: a multidisciplinary venture. Ergonomics 2008; 51: 71–75.

22.

Koch

Wagner

Schneider

. Effective and quality-controlled use of digital radiography in dental practice. Int J Comput Dent 2000; 3: 107–118.

23.

Shortliffe

. The science of biomedical computing. Med Inform 1984; 9: 185–193.

24.

Masic

. Information systems in dentistry. Acta Inform Med 2012; 20: 47–55.

25.

Majeed

. Sources, uses, strengths and limitations of data collected in primary care in England. Health Stat Q 2004; 21: 5–14.

26.

Holt

Stables

Hippisley-Cox

. Identifying undiagnosed diabetes: cross-sectional survey of 3.6 million patients’ electronic records. Br J Gen Pract 2008; 58: 192–196.

27.

Anderson

. Social, ethical and legal barriers to e-health. Int J Med Inform 2007; 76: 480–483.

28.

Thiru

Hassey

Sullivan

. Systematic review of scope and quality of electronic patient record data in primary care. BMJ 2003; 326: 1070.

29.

Elizabeth

David

William

. A comparison of paper with electronic patient-completed questionnaires in a preoperative clinic. Anesth Analg 2005; 101: 1075–1080.

Health-oriented electronic oral health record: Development and evaluation