Abstract
Objective:
Evaluate healthcare providers’ and staffs’ knowledge, self-reported abilities, activities, and barriers to providing preventive oral health services (POHS) at primary care medical visits before and after participation in the Rocky Mountain Network of Oral Health (RoMoNOH) project.
Methods:
The RoMoNOH project integrated POHS into primary care medical visits of young children at 22 community health centers (CHCs) in Arizona, Colorado, Montana, and Wyoming by medical team members and/or by embedded dental hygienists (DHs). Twelve CHCs embedded DHs onto their teams. In an observational pre/post evaluation, a convenience sample of healthcare providers’ characteristics were surveyed at baseline and 3 years across 4 oral health domains: knowledge, self-reported abilities, behaviors, and barriers. Each domain was scored from 0% to 100%, with 100% being optimal. Differences between pre- and post-project domain scores were assessed using chi-square, t-tests, and linear and logistic regression adjusting for providers’ age.
Results:
Embedding DHs into CHCs and staff turnover impacted pre/post survey participants. The final analytic cohort included 213 (pre-survey response rate: 71%) and 165 (post-survey response rate: 52%) healthcare providers who worked with children < age 3. Participants were female (pre: 81%, post: 81%) and aged >35 years (pre: 39%, post: 41%). Unadjusted mean differences across surveys improved across all oral health domains (pre/post): knowledge: 65%/81%, P < .001; self-reported ability: 52%/71%, P < .001; activities: 32%/57%, P < .001; barriers: 27%/21%, P = .011. After adjustment for age, these improvements remained significant (all P ≤ .011).
Conclusions:
Healthcare providers’ oral health practices improved over a multi-year oral health integration project aimed at increasing delivery of POHS at medical visits.
Introduction
Dental caries is the most common chronic disease in the United States, affecting over half of young children and most adults, and rates are higher among low-income and minority populations. 1 Despite being largely preventable, untreated cavities can lead to pain, infection, and problems with eating, speaking, and learning. 2 Disparities in oral health and access to dental care exist, disproportionately affecting marginalized and underserved populations. 3 Limited access to preventive and restorative dental services exacerbates these disparities, with factors such as income, insurance coverage, and geographical location significantly influencing an individual’s ability to obtain timely and adequate care.1,4 As a result, certain groups, including young children, experience higher rates of dental caries and related complications.1,5 Integrating preventive oral health services into primary care medical visits—medical-dental integration (MDI)—offers a promising approach to address these disparities. In the first years of life, children have substantially more medical visits than dental visits. According to data from the Medical Expenditure Panel Survey (2007-2016), 89% of young children have at least one medical visit annually, compared to only 4% having a general dental visit. 3 MDI builds on existing medical infrastructure to expand access to care, yet evidence on effective implementation of MDI models remains limited.6 -8
The Rocky Mountain Network for Oral Health (RoMoNOH) is 1 of 3 Networks for Oral Health Integration funded by the U.S. Department of Health and Human Services Maternal and Child Health Bureau, with the overarching goal of improving the oral health of young children aged 0 to 3 years through MDI. The RoMoNOH developed, implemented, and evaluated MDI models aimed at providing preventive oral health services (POHS), including caries risk assessment, oral health education, fluoride varnish application (FVA), dental referrals, and parental oral health engagement, to children during primary care medical visits at CHCs in Arizona, Colorado, Montana, and Wyoming. MDI models included POHS delivered by the medical team, DHs embedded in the medical team, or a combination of these 2. A 4-state learning collaborative led the RoMoNOH project and was comprised of a project director, project manager, data manager, subject matter experts, and state primary care association (PCA) practice facilitators (PFs) who were primarily responsible for facilitating activities in participating CHCs.
Our objective was to assess change in the self-reported oral health knowledge, self-reported abilities, activities, and barriers to providing POHS of healthcare providers and staff working in the participating CHCs prior to and 3 years into the project.
Methods
We conducted an observational pre/post evaluation of the delivery of POHS in CHCs participating in the RoMoNOH project.
CHC Participants: In 2019, PCA PFs enrolled a convenience sample of 22 CHCs to participate in the project (Arizona [8 CHCs], Colorado [8 CHCs], Montana [3 CHCs], Wyoming [3 CHCs]). Each CHC completed a baseline assessment to determine their community’s oral health needs, which assessed the CHCs’ unique annual patients (count), patients’ age distribution, medical and dental provider full-time equivalents (FTEs), co-located dental clinics, and current delivery of POHS at medical visits.
Implementation
Provider Education: PFs directed healthcare medical and dental team members to an on-demand digital training (henceforth eLearning) on the delivery of POHS in primary care including: Introduction to Oral Health, Clinical Skills and Integrating Oral Health, Oral Health Communication and Education, Interprofessional Collaboration, and Perinatal Oral Health. Users earned 1 continuing education credit per completed module.
Practice Facilitation: The RoMoNOH PFs utilized a quality improvement (QI) approach, guided by an MDI change package and its driver-associated activities. The MDI change package was developed from the evaluation of previous MDI efforts.7,9,10 The change package drivers included: data-driven improvement, engaged leaders, engaged providers and staff, devoted time to drive practice change, and team-based care strategies. PFs coached their state’s CHCs from September 2020 to October 2023 implementing MDI change package activities.
The PFs met with each CHC roughly monthly to review metric QI feedback reports and CHC’s progress toward reaching project objectives and change package drivers. PFs reviewed staff training and workflows on provision and documentation of POHS and extraction of discrete data from the electronic health record for reporting. PFs coached CHCs on identifying opportunities for POHS-delivery and documentation improvement using a variety of pragmatic, non-scripted QI methods including Plan-Do-Study-Act, Lean, and 6-Sigma.11 -13 The PFs completed monthly online coaching assessments, which quantified and summarized their coaching activities; PFs provided 1079.5 h of coaching to CHCs over project years.
Evaluation
Objectives: The primary objective of this evaluation was to evaluate healthcare providers’ knowledge, self-reported abilities, activities, and barriers relating to providing POHS before and after their participation in the RoMoNOH project.
Population: Participants included a convenience sample of all healthcare team members who completed the eLearning and were providing primary care to young children aged 0 to 3 years in participating CHCs including dental providers embedded in the medical clinics. PFs maintained updated rosters of CHC team members to track participants.
Survey Administration: The RoMoNOH evaluation team first piloted the survey with 22 healthcare providers working in a non-RoMoNOH CHC prior to its use. After minor edits, the evaluation team sent an email invitation with the final survey link to all roster members before project launch (Fall, 2020) and conclusion (Spring, 2024). Participants were given 6 weeks to complete the 28-item survey and were sent up to 3 reminders. Participants were incentivized to complete the follow-up survey with a $40 gift card.
Sociodemographic: Participant age was assessed by asking their birth year. To include only participants who provided clinical care to RoMoNOH’s target population, participants were asked, “Are you part of a healthcare team that takes care of children ages 0-3” (yes/no); those responding “yes” were included in the final analytic cohort. Participants were asked their role in the clinic, and behavior health providers, managers/administrators, midwifes, other, or unknown were excluded.
Oral Health Domains: Participants’ oral health characteristics were assessed using items from the American Academy of Pediatrics (AAP) Periodic Survey of Fellows across 4 oral health domains: knowledge, self-reported abilities, activities, and barriers. 14 The AAP survey’s validity is supported by its design by experts and use of repeated measures over survey years across a large, nationally representative sample of pediatricians.
Four survey items assessed knowledge. One item asked, “By what age do you believe healthy children should have their first dental visit?” Three items asked participants if their CHC participates in water fluoridation, has an established referral relationship with a dental home, or has someone who is primarily responsible for dental referrals and care coordination; “Yes” and “No” responses were collapsed (score = 1) and “Unsure” was scored as 0.
Twelve survey items assessed self-reported abilities. Three items asked participants to rank how prepared they feel to counsel patients and their families on the following fluoride topics: parental concerns about community water fluoridation, explaining topical and systemic fluoride mechanisms, and instructions for home use of fluoride. Response options were on a 4-point Likert scale, from “not prepared” to “very prepared” and dichotomized, combining very/moderately prepared and slightly/not prepared. Nine items assessed self-reported abilities by asking, “How would you rate your ability to perform the following. . .?” Responses were on a 5-point Likert scale, from “poor” to “excellent” and dichotomized (excellent/very good/good [score = 1] vs fair/poor [score = 0]).
Thirteen survey items assessed oral health activities. Ten items asked, “During health supervision visits with patients, birth to 3 years of age, what proportion do you perform the following oral health assessments/tasks at least once?” Response options were on a 6-point Likert scale (0% of patients, 1%-25%, 26%-50%, 51%-75%, 76%-99%, and 100%) and dichotomized, combining 76% to 99% and 100% (score = 1) and 0 to 75% (score = 0). Three items asked if participants’ clinic/medical practice: applies fluoride varnish to young children, contacts dental office directly with a dental referral, and completes referrals to dental providers through the EHR. Responses were dichotomized (“Yes” [score = 1]) vs (“No” or “Unsure” [score = 0]).
Eleven survey items assessed barriers to providing POHS. Six items asked, “During health supervision visits with patients, birth to 3 years, how much of a barrier to providing oral health assessments/counseling are the following. . .?” Four items assessed barriers to making dental referrals by asking, “To what extent are the following barriers to making dental referrals for patients?” Response options were on a 4-point Likert scale, ranging from “significant barrier” to “not a barrier” to and dichotomized combining significant/moderate (score = 1) and somewhat/not (score = 0). Lastly, 1 item asked “Medical/Dental teams are responsible for addressing oral health” on a 5-point Likert scale from “not at all” to “very much” (somewhat/not at all [score = 1] vs very/mostly [score = 0]).
The primary outcomes of this evaluation are the 4 oral health domain scores. Scores were calculated by summing dichotomized responses for items within each oral health domain, dividing the sum by the total number of domain-level item responses, and multiplying by 100 to provide a domain-level score from 0% to 100% with 100% representing optimal improvement. Descriptive statistics (frequency distributions, proportions, means, standard deviations) chi-square, and unpaired t-tests were used initially to examine differences in individual items and overall scores for each domain between pre/post survey years. Linear and logistic regression, adjusted for providers’ age (aged <35 years vs ≥35 years) as a potential confounder, were performed to determine whether there was significant change from pre- to post-survey responses. After applying the Bonferroni correction to account for comparisons of the primary outcomes, a P-value of <.0125 was considered statistically significant. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC).
These methods comply with the STROBE protocol. The Colorado Multiple Institutional Review Board exempted this evaluation from human subjects research.
Results
CHC Baseline Characteristics: The CHCs were in urban 12 (55%), rural 8 (36%), and frontier 2 (9%) locations. At enrollment, the mean number of annual unique patients receiving care in the CHCs was 8938 (range 1702-40,180). The average patient visits/day/CHC was 69 (range 9-294). Twelve (54%) had co-located medical and dental clinics, 7 (32%) had affiliated but not co-located dental clinics, and 3 (14%) had no affiliated dental clinic. POHS were being provided in 8 (41%) of the CHCs, which wanted to improve their POHS-delivery.
MDI Model: CHCs implemented an embedded dental hygiene model (12 CHCs), a combined model (7 CHCs) where some patients received POHS from the medical team and some by the embedded hygienist, or a combined model in which medical providers completed some of the POHS (e.g., CRA) and then referred the patient to the embedded hygienist who then provided additional POHS (2 CHCs). At 1 CHC, the medical team coordinated with dental residents for POHS, rather than a hygienist. In 10 CHCs, medical providers provided the POHS.
Provider Oral Health Education: A total of 287 healthcare team members completed at least one eLearning module. In total, 1068 modules were completed. On a scale of 0 to 6 (poor to excellent), users rated the eLearning module’s quality an average of 5.17.
Survey Results: At baseline and follow up, 213/300 (71% response rate) and 165/317 (52% response rate) participants completed the survey, responded they were part of a healthcare team that cared for children ages 0 to 3 years, and were included in the final analytic cohort. Embedding dental providers into CHCs as part of the project, and staff turnover accounted for a difference in the participants completing the pre/post surveys. In both surveyed cohorts, participants had a mean age of 40 years and were female (81%). Most respondents were physicians (pre: 34%, post: 20%, P = .002) or medical assistants (pre: 28%, post: 29%, P = 0.84) in both survey years. As anticipated, more dental providers responded to the follow-up survey than baseline (pre: 2%, post: 24%, P < .001) (Table 1).
Baseline and Follow-Up Characteristics of Healthcare Providers Participating in Rocky Mountain Network for Oral Health Integration Project (2020-2024).
There was significant improvement across all 4 oral health domains of knowledge, self-reported abilities, activities, and barriers (Table 2). Unadjusted mean differences from baseline to follow up include knowledge: 65% to 81% (confidence interval, CI: 0.16 [0.11, 0.21], p < .001); self-reported ability: 52% to 71% (CI: 0.19 [0.13, 0.25], p < .001); activities: 32% to 57% (CI: 0.25 [0.20, 0.31], p < .001); barriers: 27% to 21% (−0.06 [−0.11, 0.02], p = .01). After adjusting for age, there were improvements in all oral health domains (CIs): knowledge: 0.15 (0.11, 0.20), P < .001; self-reported ability: 0.19 (0.13, 0.25), P < .001; activities: 0.25 (0.20, 0.31), P < .001; barriers: −0.06 (−0.11, −0.02), P = .011. Further analysis of individual domain items also indicated improvement across most survey items except for participants’ knowledge regarding recommended age of first dental visit (adjusted CI: 1.59 [0.98, 2.6], P = .06), which was already favorable at baseline; self-reported ability to explain topical and systemic fluoride mechanism explain topical or systemic fluoride mechanism (adjusted CI: 1.5 [0.99, 2.27], P = .05), inform parents on oral health effects of putting a child to bed with a bottle (adjusted CI: 1.8 [1.05, 3.12], P = .03) and of sugary food and drinks (adjusted CI: 1.77 [0.98, 3.19], P = .057) (Table 3).
Unadjusted and Adjusted Linear Regression Analysis of Health Care Provider Oral Health Domains.
t-test.
Odds ratio, adjusted for age (<35 years vs. ≥35 years).
Unadjusted and Adjusted Logistic Regression Analysis of Health Care Provider Oral Health Domain Items.
Chi-square.
Odds ratio, adjusted for age (<35 years vs. ≥35 years).
Discussion
The overarching aim of this pragmatic MDI project was to increase delivery of POHS to young children aged 0 to 3 years at medical visits in CHCs. Our findings describe a new approach to incorporating POHS into medical visits across 4 states, however, the interpretation of our results is limited by the observational pre/post design of this evaluation. The oral health knowledge, abilities, and activities of healthcare providers and staff working in participating CHCs improved over the course of the project. They also reported fewer barriers to the delivery of POHS 3 years into the project than prior to their participation.
Our results suggest the external validity of our MDI approach in CHCs, but external secular trends may account for stated changes. Interpreting our results within context of other MDI work is warranted. Our results may not be directly transferable to non-CHC settings. There is a paucity of evidence describing the change in healthcare provider oral health practices related to an MDI-intervention at medical visits. Multiple studies describe healthcare providers’ oral health practices at a point in time, 15 but few assess a change in practices related to participation in an MDI-intervention. In 1 large clinical trial with randomization of 18 Ohio primary care practices, medical healthcare providers received didactic education and skills training around the provision of POHS at medical visits. This trial’s primary outcomes were children’s untreated caries and dental referral attendance. Healthcare provider practices were measured through documentation in the electronic medical record. In our intervention, we also measured provider practices through enhancing the electronic medical record, 16 but we additionally surveyed the healthcare providers on their practices and skills to enhance our understanding of their experiences. In a 2020 scoping review by Dickson-Swift et al 17 of 42 studies across 19 countries, they found that pediatricians’ oral health knowledge in general was poor with gaps in areas including initial clinical signs of dental caries, recommended age for first dental visit, dental caries etiology, and recommended use of fluorides. They note barriers to the provision of POHS for pediatricians include inadequate education and training, time constraints in practice, and lack of referral pathways. Amongst U.S. pediatricians, the proportion responding to the AAP 2008, 2012, 2018 Periodic Surveys reported an overall improvement across survey years in oral health practices and a reduction in barriers related to oral health, for example, these AAP member surveys report an increase in pediatricians offering fluoride varnish application from 4% in 2009 to 19% in 2018. 14 While this is an increasing trend overall, most pediatricians reported that they did not provide this evidence-based care. The effectiveness of fluoride varnish application by medical providers for caries prevention in children aged 0 to 5 years is supported by the U.S. Preventive Services Task Force’s Grade B recommendation for this practice in 2014, reaffirmed in 2021. 18 Over the past 15 years, there has been a gradual yet steady increase in the proportion of medical providers reporting the provision of POHS, but there is room for improvement. Our approach suggests that CHCs can incorporate POHS into primary care medical visits with support of an on-demand eLearning supported by coaching. At baseline in the RoMoNOH project, the oral health knowledge, abilities, and activities were generally higher than noted in previous studies and increased over the 3 years into the project.
In a small evaluation of an oral health digital training module focused on adult oral health, Snogren et al 19 found minimal change in healthcare professional oral health attitudes or knowledge. They did find a difference in improvement of healthcare provider practices related to career stage with providers with more than 19 years of professional experience having more difficulties with providing POHS than their counterparts with fewer than 19 years. In our regression analysis, after adjusting for healthcare provider age, improvement in all oral health domains remained. Our findings suggest that medical providers benefit from a structured education and PF when asked to incorporate POHS into their care. In a systemic review of 23 studies (1398 participating practices) and meta-analysis of PF in primary care settings, authors conclude that there is a moderately robust effect size of 0.56 (95% CI, 0.43-0.68) favoring PF when implementing evidence-based interventions over no PF. 20 Our findings were consistent with this review and support incorporating PF into interventions targeting healthcare provider behaviors.
Historically, barriers to medical providers delivering POHS include difficulties integrating dental procedures into clinical workflows, resistance among staff and colleagues, and challenges in making dental referrals.21,22 However, these barriers have not been recently re-evaluated, representing a knowledge gap warranting further investigation. This evaluation of healthcare providers’ change across 4 oral health domains, including barriers, contributing to findings on implementing MDI models. Understanding these barriers could inform future oral health promotion strategies targeting medical providers, including the use of silver diamine fluoride as an additional caries management tool.
Our work has strengths and limitations. Strengths include its pragmatic approach to implementing best-practices across CHCs in 4 states; however, lack of control makes our findings only suggestive of the project’s impact on providers’ practices. Another strength is its 3-year duration allowing us to estimate both the adoption and maintenance of providing POHS at medical visits. A limitation is that surveys included self-reported practices, which can result in reporting biases and over- or under-reporting. An additional evaluation of the RoMoNOH project, however, the delivery of POHS was measured via the electronic health record over time and demonstrated a sustained improvement in the provision of POHS. 16 Also, we did not link individual participants’ pre/post survey responses, which has potential to introduce non-response or attrition bias; however, our findings suggest that change over time was at a CHC level and not just at an individual level.
Conclusions
In conclusion, in this observational pre/post evaluation of a MDI project aimed to increase provision of POHS at medical visits of young children in CHCs across 4 states, we report an improvement in healthcare providers’ self-reported oral health knowledge, abilities, activities and a reduction in barriers to providing POHS over the project’s 3 years. Further investigation and evaluation regarding the impact interventions on reducing oral health disparities are needed, as well as support of policies and payment that encouraged provision of POHS at medical visits.
Footnotes
Ethical Considerations
The Colorado Multiple Institutional Review Board exempted this evaluation from human subjects research.
Author Contributions
Patricia A. Braun led the RoMoNOH project, secured funding, developed the project concept and design, supervised the evaluation team, and was primary author of the manuscript. Kimberly Wiggins processed and cleaned data, conducted the analyses and assisted in the interpretation of results. Cherith Flowerday was the project manager and coordinated evaluation activities, ensured adherence to timelines and protocols, managed communication among team members, and supported manuscript preparation logistics. Andrew Bienstock oversaw data collection systems, ensured data integrity and quality. Miriam Dickinson helped design the statistical analysis plan, verified the accuracy of results, and contributed to the methods and results sections of the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) as part of an award totaling $6.4 million with zero percent financed with non-governmental sources. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by HRSA, HHS, or the U.S. Government. The REDCap© Survey tool used in this project was supported by NIH/NCATS Colorado CTSA Grant Number UL1 TR002535.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Authors will share data in a relevant public repository upon request.
Clinical Trial Registration
Not Applicable
Article Summary
Healthcare providers’ oral health knowledge, abilities, behaviors, and perceived barriers significantly improved after participating in a multi-year oral health integration project across community health centers.