Abstract
Most evidence implicating periodontitis as a causal agent in systemic disease comes from observational research, where confounding is ubiquitous. There are substantial methodological differences among observational studies, of which an important one is the choice of covariates used to adjust for confounding. The present study assesses the impact of covariate selection on the association between periodontitis and systemic disease, using cardiovascular disease and cognitive function as examples. Data were taken from the National Health and Nutrition Examination Survey, where periodontal status was assessed by full-mouth examination. Nineteen covariates were available for selection, including sociodemographic factors, health behaviors, and physiologic measures. Multiverse analysis and specification curves were used to visualize how the distribution of estimated associations between periodontitis and systemic disease vary across 393,216 models, depending on the selection of covariates. The findings show that estimates of the association between periodontitis and systemic disease are sensitive to the choice of covariates included within models, in some cases spanning positive and negative directions (i.e., Janus effect). Depending on the covariates included in the adjustment set, the odds ratio for the association of severe periodontitis with cardiovascular disease ranged from 0.81 to 1.51, while the linear regression coefficient for the association with cognitive function ranged from −1.56 to 0.50. Our study makes 2 important contributions. First, we introduce the idea of multiverse analysis to periodontal research and show that it can be a valuable tool for understanding challenges around model selection and misspecification. Second, we use rigorous quantitative methods to highlight the importance of careful covariate selection in determining our understanding of the relationship between periodontitis and systemic disease.
Keywords
Introduction
A substantial portion of oral health research is focused on the relationship between periodontal and systemic health, with most evidence coming from observational studies. Based on this, it has been claimed that periodontitis is an important determinant in the risk of developing systemic diseases (Preshaw et al. 2012; Sanz et al. 2020; Chapple et al. 2024).
A key challenge in this field is substantial methodological differences among studies. There has been considerable work on measurement error due to the use of partial- versus full-mouth examination protocols (Akinkugbe et al. 2015; Alshihayb et al. 2020) and differences in case definitions of periodontitis (Manau et al. 2008; Alhassani 2023). The present work focuses on the selection of covariates, which are used to adjust estimates for confounding. The concern is that groups of people with and without periodontitis may systematically differ from one another in a way that is also connected to the systemic disease of interest. Many primary studies do not justify the choice of covariates used to control for confounding, resulting in studies of similar systemic health outcomes adjusting for markedly different covariate sets (Chen et al. 2017; Choi et al. 2019; Nascimento et al. 2023). This is an issue that then permeates systematic reviews, as many published reviews do not state the basic set of covariates that primary studies included or their suitability for pooling in a meta-analysis. Concerns about covariate selection, residual confounding, and model variability have been discussed (Beck and Offenbacher 2002; Baelum and Lopez 2003; Demmer and Desvarieux 2006). In effect, bias related to the selection of covariates may play a substantial role in the interpretation of studies on the periodontal-systemic disease relationship, but the extent of this problem is currently unclear. What is missing in the literature is a systematic, quantitative, and reproducible framework to visualize instability in estimates, thereby strengthening the empirical foundation for long-standing theoretical concerns.
Investigators working in meta-research have formally probed the issue of covariate selection. Depending on the field, the terminology differs, including multiverse analysis, specification curve analysis, garden of forking paths, and the vibration of effects (Steegen et al. 2016; Lonsdorf et al. 2019; Simonsohn et al. 2020; Vinatier et al. 2025). The premise is simple: anytime that investigators choose to adjust or not adjust for a covariate in their model, the number of models that they could have fit grows exponentially. Of the possible models, usually only 1 will be fit, and these are the results that are reported. If the estimates across all possible models would not have differed greatly from one another, we say that the results are stable. Stability of results instills some confidence that model misspecification is not inducing substantial bias.
The multiverse analysis approach is to fit all possible models and then compare the results. If the estimates are widely dispersed, one should be more cautious in the interpretation of significant findings, as these can be more easily attributed to the investigator having simply picked a combination of covariates that yielded a statistically significant result. Such P hacking undermines epidemiologic research. Also concerning is when the instability is so substantial that it is possible to obtain estimates that point in opposite directions, depending on how the model is specified. The term Janus effect has been used in the literature to describe the scenario when the estimated associations can be positive or negative, depending on which variables are adjusted for (Vinatier et al. 2025). This is concerning because an investigator could reach the conclusion that periodontitis either increases or decreases the risk of developing a systemic disease, depending on which covariates were included in the model.
The aim of this article is to investigate the stability of periodontal-systemic disease associations with respect to covariate selection, by application of a multiverse analysis framework.
Materials and Methods
Data Source
We extracted and combined National Health and Nutrition Examination Survey demographic, examination, laboratory, and questionnaire data for the 2011–2012 and 2013–2014 cycles. These multistage cross-sectional surveys are designed to take a representative sample of the noninstitutionalized US population (Chen et al. 2020). Participants are interviewed at home and then invited to a mobile examination center for further interviews, tests, and examinations. These examinations include an oral examination conducted by state-licensed dental practitioners.
Measures
We assessed 2 systemic health outcomes: cardiovascular disease (CVD) and cognitive function. Individuals were classified as having CVD if they reported a previous medical diagnosis of coronary heart disease, heart attack or myocardial infarction, or stroke. Cognitive function was measured in individuals aged ≥60 y by the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) test (Fillenbaum et al. 2008). The CERAD test comprises 3 learning tests and 1 delayed recall test, each of which has a maximum score of 10 points. The final CERAD score is the sum of the 4 test scores, ranging from 0 to 40. Higher CERAD scores indicate better cognitive function. All analyses were restricted to those individuals who had either reported a CVD status or took the CERAD test.
We classified periodontal status for participants using case definitions per the Centers for Disease Control and Prevention and the American Academy of Periodontology for population-based surveillance of periodontitis. Dentate individuals were categorized as having no periodontitis (reference) or mild, moderate, or severe periodontitis (Eke et al. 2012). No periodontitis is defined as the absence of mild, moderate, or severe periodontitis. Mild periodontitis is defined as ≥2 interproximal sites with clinical attachment loss (CAL) ≥3 mm and ≥2 interproximal sites with probing pocket depth (PPD) ≥4 mm (not on same tooth) or 1 site with PPD ≥5 mm. Moderate periodontitis is defined as ≥2 interproximal sites with CAL ≥4 mm (not on same tooth) or ≥2 interproximal sites with PPD ≥5 mm (not on same tooth). Severe periodontitis is defined as ≥2 interproximal sites with CAL ≥6 mm (not on same tooth) and ≥1 interproximal site with PPD ≥5 mm. Edentulous individuals were excluded.
We considered 19 covariates as our set of possible adjustments. There is no consensus on which variables should be adjusted for when assessing periodontitis as an exposure, so we selected a broad range of plausible variables for this evaluation. Since age and sex are well-known risk factors that are universally adjusted for in epidemiologic analyses, we kept these in all models as baseline variables, treating age as a continuous variable (in years and top coded at 80) and sex as a categorical variable (men, women). This left the remaining 17 variables to be used as our set of varying adjustments. Categorical variables were as follows: race and ethnicity (non-Hispanic White, non-Hispanic Black, Mexican American, other), educational attainment (below, at, and above high school), marital status (married or living as married, separated, and never married), smoking status (never, former, and current smoker), health insurance (insured, uninsured), and self-reported history of diabetes (whether one had received a diagnosis of diabetes from a medical professional). Continuous variables were as follows: poverty-income ratio (defined as the ratio between household income and the federal poverty threshold; i.e., a value <1.0 indicates that a family is living in poverty), alcohol consumption (drinks per day; a drink was defined as a 12-oz beer, a 4-oz glass of wine, or 1.5 oz of liquor), physical activity (metabolic equivalent hours per week), body mass index, systolic and diastolic blood pressure, glycated hemoglobin (percentage HbA1c), total and high-density lipoprotein cholesterol, vitamin D (25-OH D2 + 25-OH D3), and systemic immune-inflammation index. Systolic and diastolic blood pressure were averaged across 3 separate measures. The systemic immune-inflammation index is computed as P × N/L, where P, N, and L are the peripheral platelet, neutrophil, and lymphocyte counts, respectively; it has been used as a marker of systemic inflammation in the study of chronic disease (Hu et al. 2014; Ma and Li 2023). When cognitive function was assessed as an outcome, history of CVD (as previously defined) was also included in the list of covariates, bringing the total to 18.
Statistical Analysis
The total number of possible covariate adjustment sets is 131,072 (217) for CVD and 262,144 (218) for cognitive function. We extracted the estimated regression coefficient for the periodontitis exposure, with its associated 95% CI and P value. All analyses were complete-case analyses with respect to covariate combination, which adjusted for the complex sample survey design through use of the probability sampling weights, pseudo-primary sampling units, and pseudo-strata. For CVD, the estimated association was the log(odds ratio [OR]) from logistic regression. For cognitive function, the estimated association was the regression coefficient from linear regression.
We visualized the results from multiverse analysis by creating volcano plots of estimated associations against the –log10(P value). We declared the presence of a Janus effect if the 1st and 99th percentiles of the estimated associations took opposite signs (Vinatier et al. 2025). To evaluate how specific covariates affected the analyses, we plotted specification curves (Simonsohn et al. 2020). Specification curves show the distribution of estimated associations, as well as a binary inclusion matrix highlighting the combination of covariates that were adjusted for to produce a given estimate. They present the estimated associations for every possible combination of covariates. Beneath each estimate is a matrix that indicates whether a given covariate was included in the model to obtain that estimate. This allows for assessment of which covariates may be most influential in determining the stability of the models.
We carried out sensitivity analyses where we replicated the main analyses but instead assessed periodontitis as a continuous exposure in the form of mean CAL across all inspected sites (mean CAL).
Results
A full description of the included participants is presented in Appendix Tables 1 and 2. A flowchart showing the procedure for selecting participants is presented in Appendix Figure 1.
Multiverse Analysis
Cardiovascular Disease
The results of the multiverse analyses are presented in Figure 1 (edentulism and mean CAL results are presented in Appendix Fig. 2).

Volcano plots where each point represents the estimated regression coefficient and –log10(P value) from 1 model, colored by its density (red = high, blue = low). Vertical and horizontal dashed red lines represent the 1st and 99th percentiles of the estimated regression coefficients and –log10(P value), respectively. Vertical and horizontal solid gray lines represent the null effect and nominal threshold of significance (P = 0.05). No periodontitis is the reference category. Left column: x-axes in cardiovascular disease volcano plots are measuring the log(odds ratio). Right column: x-axes in cognitive function volcano plots are measuring the linear regression coefficient. All models are adjusted for age and sex.
For CVD, a Janus effect is observed in the distribution of estimated associations for all severities of periodontitis. The proportion of negative estimated associations (OR <1) were 50.3% for mild periodontitis, 5.6% for moderate, and 29.3% for severe. The estimates ranged from 0.77 to 1.27 for mild periodontitis, 0.93 to 1.55 for moderate, and 0.81 to 1.51 for severe. In all, none of the estimates for mild periodontitis, 2.1% for moderate, and none for severe had a corresponding P < 0.05.
As an example of the Janus effect, adjusting for the following variables results in a positive estimated association of moderate periodontitis with CVD (OR, 1.35, 95% CI, 1.02 to 1.78): age, sex, race and ethnicity, educational attainment, physical activity, health insurance status, HbA1c, total and high-density lipoprotein cholesterol, and vitamin D. Yet, adjusting for age, sex, race and ethnicity, educational attainment, poverty-income ratio, smoking status, alcohol consumption, body mass index, diastolic blood pressure, and HbA1c results in a negative estimated association (OR, 0.96; 95% CI, 0.66 to 1.40).
Cognitive Function
For cognitive function, the distribution of estimated associations differs depending on the severity of periodontitis. Mild and moderate periodontitis showed a consistently negative association, while a Janus effect was observed with severe periodontitis. The proportion of negative estimated associations (β < 0) were 100% for mild and moderate periodontitis and 60.5% for severe periodontitis. The estimates ranged from −4.19 to −2.93 for mild periodontitis, −1.40 to −0.42 for moderate, and −1.56 to 0.50 for severe. In all, 79.3% of the estimates for mild periodontitis, 13.1% for moderate, and <0.1% for severe had a corresponding P < 0.05.
As an example of the Janus effect, adjusting for the following variables results in a positive estimated association of severe periodontitis with cognitive function (β = 0.11; 95% CI, –1.32 to 1.54): age, sex, race and ethnicity, educational attainment, marital status, smoking status, physical activity, body mass index, and HbA1c. In contrast, adjusting for age, sex, race and ethnicity, smoking status, alcohol consumption, physical activity, health insurance status, systolic blood pressure, and total cholesterol results in a negative estimated association (β = –0.36; 95% CI, −1.78 to 1.05).
Specification Curves
Cardiovascular Disease
Specification curves for CVD are presented in Figure 2 (edentulism and mean CAL results are presented in Appendix Fig. 3). For mild periodontitis, the association with CVD was typically positive when adjusting for income, smoking, and total cholesterol, but it became negative without these adjustments. For moderate periodontitis, the association was typically positive when not adjusting for smoking and alcohol consumption, but it became negative after these adjustments. For severe periodontitis, the association was typically positive when not adjusting for smoking and educational attainment, but it became negative after these adjustments.

Specification curves for CVD as the outcome. Within each plot, the upper panel shows the distribution of estimated log(odds ratios) from logistic regression with CVD as the outcome and periodontitis as the exposure, and the lower panel shows the binary inclusion matrix. Gray lines in the inclusion matrix indicate that a covariate was adjusted for in the model. White lines indicate that a covariate was not adjusted for in the model. All models are adjusted for age and sex. For categorical variables, inclusion of the variable (gray line) indicates that all levels of the variable were adjusted for, and exclusion (white line) indicates that none of the levels were adjusted for. Interpretation: positive coefficients indicate that periodontitis is associated with increased odds of having CVD. CVD, cardiovascular disease.
Cognitive Function
Specification curves for cognitive function are presented in Figure 3 (edentulism and mean CAL results are presented in Appendix Fig. 4). For mild periodontitis, the association with cognitive function was always negative, strengthening with adjustment for race and ethnicity and alcohol consumption but weakening with adjustment for educational attainment. For moderate periodontitis, the association was always negative but weakened with adjustment for race and ethnicity, educational attainment, and income. For severe periodontitis, the association was typically negative when not adjusting for race and ethnicity, educational attainment, and income but became positive after these adjustments.

Specification curves for cognitive function as the outcome. Within each plot, the upper panel shows the distribution of estimated regression coefficients from linear regression with CERAD score as the outcome and periodontitis as the exposure, and the lower panel shows the binary inclusion matrix. Gray lines in the inclusion matrix indicate that a covariate was adjusted for in the model. White lines indicate that a covariate was not adjusted for in the model. All models are adjusted for age and sex. For categorical variables, inclusion of the variable (gray line) indicates that all levels of the variable were adjusted for, and exclusion (white line) indicates that none of the levels were adjusted for. Interpretation: negative coefficients indicate that periodontitis is associated with worse cognitive function. CERAD, Consortium to Establish a Registry for Alzheimer’s Disease.
Discussion
We evaluated the robustness of observational associations between periodontitis and systemic disease, using CVD and cognitive function as applied examples, by applying a multiverse analysis framework. Our findings highlight instability in these associations, suggesting that covariate selection plays an important role in determining the magnitude and direction of estimated associations. Multiverse analysis provides a transparent, reproducible method to examine how analytic decisions affect the results of a study as well as to visualize the instability underlying single-model reporting (Simonsohn et al. 2020; Vinatier et al. 2025). This framework can thus improve transparency in periodontal research. However, it addresses only model selection and misspecification, not measurement error, selection bias, or residual confounding. It also treats all model choices as equally plausible without using causal theory to prioritize specifications.
The presence of Janus effects means that observational associations may be highly prone to publication bias, as it is possible that the model specifications that produce associations in a positive or significant direction are preferentially published. This can distort the literature and mislead future investigators in that, even if they assess the entirety of the published evidence, they may arrive at the wrong conclusion regarding the association of periodontitis with systemic disease.
Recommendations for Future Research
Some recommendations can be made for future researchers regarding best practices in selecting covariates for studies investigating the oral-systemic disease relationship (Table). An important first step would be for investigators to justify their selection of covariates. Ideally, covariates adjusted for should be based on a mechanistic understanding of the common causes of periodontitis and systemic diseases. If there are covariates that are more contentious (i.e., it could be argued that inclusion and exclusion of the variable are both valid), then some explanation of the investigators’ thought process that led to its inclusion may be helpful. We strongly suggest the use of directed acyclic graphs to make explicit the investigators’ modeling assumptions, which should then drive the use of a prespecified adjustment strategy.
Key Considerations in Covariate Selection.
If researchers have complete knowledge of a causal diagram for common causes, they can apply the backdoor path criterion to identify a set of covariates along backdoor paths from the exposure to the outcome, which would be enough to adjust for confounding. However, comprehensive knowledge is often unavailable, but there is at least some understanding of which preexposure covariates cause the exposure and/or the outcome. Here, the disjunctive cause criterion can be used to select an appropriate set of covariates (VanderWeele and Shpitser 2011; VanderWeele 2019). This postulates that sufficient control for confounding can be achieved by 1) controlling for each covariate that is a cause of the exposure, outcome, or both; 2) excluding from this set any variable known to be an instrumental variable for the exposure; and 3) including as a covariate any proxy for an unmeasured variable that is a common cause of exposure and outcome (Lash et al. 2021). These ideas are presented in Figure 4, with additional discussion in Appendix Figure 5. In most cases, purely data-driven methods for covariate selection, such as stepwise regression or the change in regression coefficient, will not be appropriate if one wishes to draw causal inferences on the relationship between periodontitis and systemic health (Hernan and Robins 2025).

Directed acyclic graphs show the disjunctive cause criterion: (1) confounding, (2) instrumental variables, and (3) proxies for unmeasured confounding. Solid arrows indicate that one variable causally influences another, with the direction of the arrow representing the assumed causal relationship. Dashed lines indicate statistical association (i.e., correlation), without implying a direct causal effect. Orange indicates that a variable should be adjusted for, and blue indicates that a variable should not be adjusted for.
Investigators should consider study designs that are more appropriate for evaluating hypothesized causal relationships. A preference for longitudinal studies with repeated measures is important to elucidate the temporal association between periodontitis and systemic disease. Of course, such studies are often expensive and logistically challenging, but the insights that they can provide are invaluable. From an analytic perspective, several techniques have been described in the causal inference literature, such as the use of instrumental variables and negative control exposures (Lipsitch et al. 2010; Baiocchi et al. 2014). These methods complement traditional covariate adjustment by leveraging external sources of variation or exploiting assumptions that can help detect or adjust for unmeasured confounding. Replicating findings with diverse study methodologies and analytic approaches can enhance their credibility and build a more robust body of evidence (Lawlor et al. 2016).
Finally, investigators carrying out systematic reviews must be transparent regarding the set of covariates that they consider essential to adjust for confounding. The comparability domain in the widely used Newcastle-Ottawa scale suggests assigning a maximum of 2 stars to a study if it controls for the first and second most important factors (Stang 2010). In practice, this means that any periodontal-systemic disease study adjusting for sex and smoking may be considered at low risk of bias when other covariates, such as age, diabetes, and socioeconomic conditions, are theoretically also important to adjust for. Systematic reviewers should clarify whether pooled estimates correspond to crude, partially, or fully adjusted associations and whether these associations use comparable adjustment sets.
Study Limitations
First, we focus here on bias due to confounding, but there are other sources, such as biases that arise from participant selection, measurement error, and residual confounding. Second, we focus on only 2 outcomes, but we expect that similar concerns will be present in other systemic diseases. Third, we did not include interactions between covariates, some of which might be plausible. Fourth, we consider all model choices as equally plausible, but using domain knowledge can help to cut out some of the less plausible models. Fifth, there may be important dimensions of information missing in the covariates, such as a lack of information on duration and intensity of cigarette smoking and diabetes. This leads to our final point: we have addressed here the problem of covariate selection, but there are many other choices that must be made when analyzing observational data. These include how a covariate is defined and whether it is treated as continuous or categorical, as well as the methods for addressing collinear covariates and the different definitions for exposure and outcome. This just emphasizes our key point: researchers have a huge number of decisions when analyzing observational data, and these should be made in a principled manner.
Conclusion
This study demonstrates that estimates of association between periodontitis and systemic disease can be sensitive to model specification, particularly with respect to selection of covariates. Some recommendations are presented for future research.
Author Contributions
N.Z. Bashir, contributed to conception, design, data acquisition, analysis, and interpretation, drafted and critically revised the manuscript; B.A.R. Woolf, S. Burgess, contributed to data interpretation, drafted and critically revised the manuscript; E. Bernabé, contributed to conception, design, data interpretation, drafted and critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work.
Supplemental Material
sj-docx-1-jdr-10.1177_00220345251356469 – Supplemental material for Periodontitis and Systemic Disease: The Impact of Covariate Selection
Supplemental material, sj-docx-1-jdr-10.1177_00220345251356469 for Periodontitis and Systemic Disease: The Impact of Covariate Selection by N.Z. Bashir, B.A.R. Woolf, S. Burgess and E. Bernabé in Journal of Dental Research
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: N.Z.B. is supported by the Wellcome Trust (322777/Z/24/Z). S.B. is supported by the Wellcome Trust (9225790/Z/22/Z). N.Z.B., B.A.R.W., and S.B. are members of the Medical Research Council Biostatistics Unit (MC_UU_00040/01). B.A.R.W. is a member of the Medical Research Council Integrative Epidemiology Unit (MC_UU_00032/07).
Data Accessibility
A supplemental appendix to this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
