Abstract
Keywords
Introduction
Quality of preventive care in rural areas is a significant issue and will likely become even more critical in the coming few years. 1 In the United States, the Patient Protection and Affordable Care Act of 2010 promised to expand rural Medicaid and Medicare coverage to hitherto uninsured and vulnerable populations. However, not all delivery of care approaches could cope with the anticipated expansion of coverage. Consequently, identifying correlates that impede or enhance quality is of importance. This study makes a contribution by identifying organizational and environmental correlates to rural health clinics’ (RHCs) preventive quality of care.
Rural health clinics provide services for residents in areas with a shortage of health professionals and/or medically underserved areas. The RHC Services Act of 1977 defines an RHC as a primary care facility where a midlevel practitioner such as a physician assistant or nurse practitioner is available 50% or more of the time the clinic is open. 2 In other words, RHCs are primarily designed to be nonphysician facilities with periodic physician oversight. 3
For purposes of this study, we defined quality of preventive care as having lower preventable hospitalization rates as measured by Agency for Healthcare Research and Quality (AHRQ) prevention quality indicators (PQIs). 4 Prevention quality indicators assess conditions for which good outpatient care can potentially prevent the need for hospitalization, or for which early intervention can prevent complications or more severe disease. 4 Prevention quality indicators measure the outcomes of preventive care for both acute illness and chronic conditions, reflecting 2 important components of the quality of preventive care—effectiveness and timeliness. 4
Several studies have explored the effectiveness or quality of care in RHCs.5-7 Zhang et al and Probst et al found positive relationships between the presence of RHCs in counties and county-level ambulatory care sensitive conditions (ACSC) rates.5,6 Both studies have several limitations. First, Zhang et al did not use risk adjustment. Probst et al used population-level risk adjustments. Without individual-level risk adjustments, it is hard to ascertain whether the actual provisions of outpatient care by RHCs contributed to reduced ACSC rates. Second, given the ecological focus of both studies, the impact of organizational variables on ACSC rates was not examined. Third, both studies have limited generalizeability. Zhang et al studied RHCs in Nebraska; Probst et al studied RHCs from 8 states.
Ortiz et al conducted a national survey of 402 RHCs focusing on efficiency and effectiveness. 7 Using prevention of diabetes (DIAB) as a measure of effectiveness, provider-based RHCs and use of technology were found to be positively related with effectiveness. Use of disease management programs, use of interdisciplinary teams, process efficiency, participation in integrated health systems, poverty rates, percentage of elderly, and geographic location were not related to effectiveness. 7
The findings of our study built on previous works as follows. First, the results of our study validated a measurement model of quality of care based on multiple PQIs. A second improvement relates to risk adjustments of PQI rates for sex, age, race, severity, and comorbidity of patients. A third improvement stems from the use of Donabedian’s Structure-Process-Outcome (SPO) quality of care framework. 8 The framework incorporates organizational characteristics that could affect quality of care. The aims of this study are to identify organizational and environmental correlates to United States (US) rural health clinics’ preventive quality of care.
Methods
Study Design
The study design is a nonexperimental and correlational research design that employed structural equation modeling as a statistical procedure. Specifically, a retrospective observational cohort study design was used.
Study Period and Sampling
The study period is from January 1, 2007 to December 31, 2007. To obtain stable PQI rates, we excluded RHCs that did not have at least 1 expected or 2 observed preventable hospitalizations for 1 or more of the PQIs. Therefore, our study included 25.4% (N = 960) of all RHCs in 2007 (N = 3781 RHCs).
Data Sources
The data came from 7 secondary data sources that were assembled for a recently completed national study. 9 Environmental variables concerning the location of RHCs were obtained from the Area Resource File (ARF). 10 The ARF, assembled by the US Bureau of Health Professions, is a publicly available data source that provides environmental data related to health care. Cause-specific mortality rates, physicians per 1000 county population, and percentage of Medicare-eligible county population were based on US Census 2000 data. Area median income per county was based on 2004 data. Labor and patient visit data for provider-based RHCs were obtained from the Center for Medicare and Medicaid Services (CMS) Medicare cost reports for 2007. Center for Medicare and Medicaid Services Medicare cost reports are publicly available data sources that provide aggregate data on provider level use and cost measures. Labor and patient visit data for free-standing RHCs were obtained from 4 fiscal intermediaries contracted by CMS. The 4 fiscal intermediaries provide proprietary data that is similar in content to Medicare cost reports. Clinical data for RHCs were obtained from the CMS’s National Chronic Care Condition Data Warehouse (CCW) for 2007. 11 As of 2005, CCW provides outpatient and inpatient data on 100% of Medicare beneficiaries (Chronic Condition Data Warehouse, version 1.4, Buccaneer Computer Systems and Services, Silver Spring, MD, 2008). The CCW is a proprietary data source with patient-level information. All analyses were conducted on de-identified patient-level data. Therefore, there are no known ethical disclosure issues.
Sample Size and Power
Using the guideline of Wan and Ullman for 15 observations for each free parameter,12,13 the proposed study model has 22 free parameters (14 structure coefficients, 4 measurement error covariances, 1 prediction error variance, and 3 covariances). Therefore, the minimum sample size needed is 22 × 15, which yielded 330 clinics. The study had 960 RHCs. Therefore, there is sufficient statistical power to conduct the study.
Statistical Analyses
Structural equation modeling (SEM) followed the theoretical and methodological guidelines elucidated by Wan and Ullman.12,13 Structural equation modeling combines a number of factor analyses with a set of multiple regression analyses. When the phenomena under study are complex and multidimensional, SEM is the analysis tool that allows complete and simultaneous tests of all hypothesized relationships. 13 In addition, the measurement of quality of care will need to address both systematic and random measurement errors. Regression equations require the absence of systematic measurement errors. That assumption is often untenable. Structural equation modeling is the only procedure capable of accounting for both random and systematic measurement errors. 12
Structural equation modeling was conducted using 2 key steps. First, confirmatory factor analysis (CFA) was performed on the measurement model of quality of care. The CFA establishes the construct validity of the model through goodness-of-fit indices. Next, SEM was fitted by introducing organizational and environmental variables to the CFA model. Maximum likelihood estimation was used. AMOS (version 18, IBM SPSS Inc, July 30, 2009) was used to conduct SEM. SAS (version 9.1, SAS Institute Inc, Cary, NC, USA. February 10, 2006) was used for data management and statistical analyses.
The fit of SEM models was assessed mainly through goodness-of-fit indices. 13 For the power aspect of models, root mean square error of approximation (RMSEA) is taken as the best alternative. Values ≤ 0.05 are excellent. For comparison of nested models, comparative fit index (CFI) is the more desirable index. An excellent fit is indicated by values ≥ 0.95. For a focus on the variance explained by models, goodness of fit index (GFI) and adjusted goodness of fit index (AGFI) are recommended. Adjusted goodness of fit index and GFI are considered analogous to coefficient of determination (R2) in multiple regressions. 13 Adjusted goodness of fit index and GFI ≥ 0.90 indicate excellent fit.
Quality of Care Measures
The unadjusted PQI rates for DIAB were computed as a ratio of RHC patients with DIAB who later received inpatient services with admission diagnosis related to diabetic sentinel conditions. Sentinel admissions to any US hospital were considered. Prevention quality indicators rates for congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), and bacterial pneumonia (PNEU) were computed similarly. The ICD9 diagnoses codes for sentinel conditions were obtained from PQI technical specifications. 14
The facility-level risk-adjusted PQI rates for DIAB were computed as the difference between the total number of observed sentinel diabetic admissions and total number of expected sentinel diabetic admissions divided by the total number of expected sentinel diabetic admissions. Risk-adjusted PQI rates for CHF, COPD, and PNEU were computed in a similar manner. Multiple admissions of the same patient for the same condition were counted only once.
The risk-adjusted PQIs took into account age, sex, race, severity, and comorbidity of RHC outpatients. Severity levels were computed through software offered by AHRQ (Version 4.1 Inpatient Quality Indicators Download 3M APR-DRG Limited License Grouper, Agency for Healthcare Research and Quality December 2009). Comorbidity scores were computed through the Healthcare Cost and Utilization Project (HCUP) comorbidity software (HCUP Comorbidity Software, Version 3.6. November 9, 2010). Logistic regression was used to implement the risk adjustment.
Findings
Descriptive Analysis Results
Table 1 presents a summary of RHC characteristics. Nearly 54% of the clinics do not have physicians. Nearly 44% of RHCs were provider based (PROV). Provider-based RHCs are owned by hospitals, skilling nursing facilities, or home health agencies. PUDIAB and PUCHF were the prevention use level for DIAB and CHF, respectively. The codes for preventive services were obtained from the Healthcare Effectiveness Data and Information Set (HEDIS) reference guide, which is a publication of the National Committee for Quality Assurance. 15 Attempts were made to compute similar variables for PNEU and COPD. However, a large majority of RHCs appeared to have not provided preventive services related to PNEU and COPD.
Rural Health Clinic Characteristics (N = 960)
Abbreviations: CHF, congestive heart failure; FTE, full time equivalent; RHC, rural health clinic.
Risk-adjusted Prevention Quality Indicators.
Sum of the top 4 cause-specific mortality rates at county level divided by the US average.
The Rural Urban Commuting Area (RUCA) zip code approximation of rural classifications was used to group location of RHCs (RURAL). 16 Nearly 71% of RHCs were located in counties that are closer to urban-focused areas or large rural towns. About 29% of the clinics were located in counties that are closer to small rural towns or isolated rural areas.
Cause-specific mortality rate (CMR) was computed as the sum of mortality rates for the 4 leading causes of death in each county divided by the national average mortality rate for the 4 leading causes of death. Centers for Disease Control and Prevention (CDC) WISQARS online database reported that heart diseases, malignant neoplasms, cerebrovascular disease, and chronic low respiratory disease were the top 4 cause-specific mortalities for all races, all ages, and both sexes from 1999-2007. 17 Cause-specific mortality rates serve as a proxy measure for health risk differences at population level. 18
The average risk-adjusted PQI rates for CHF, COPD, DIAB, and PNEU were all positive values. The presence of more observed than expected preventable admissions is a proxy sign of lower quality of care. In other words, negative values of risk-adjusted rates indicate better quality of care, whereas the reverse is true for positive values.
Confirmatory Factor Analysis Results
A confirmatory factor analysis was conducted on the hypothesized measurement model of quality of care and support for it was found (RMSEA = 0.00, CFI = 1.00, GFI = 0.999, AGFI = 0.997).The model took 2 iterations to fit. Post-hoc modifications were not needed. All four risk-adjusted PQIs were significant. The model had an excellent fit, which indicates construct validity.
Structural Equation Modeling Results
Figure 1 presents the path diagram depicting the original (proposed) SPO model for RHC’s quality of care. Structure measures [Non Physicians (NONPHY), Providers (PROV), and Physicians (PHY)] and process measures (Total Utilization Visits (TOTUTN), Preventive Utilization for Diabetes (PUDIAB), Preventive Utilization for Congestive Heart Failure (PUCHF)]) were hypothesized to have direct effects on the quality of care measurement model. Environmental variables [Area Median Income (AMI), Medicare Population (MEDICARE), Physicians per 1000 population (PHYPOP), Cause-Specific Mortality Rates (CMR), and Rural Classification (RURAL)] served as control variables on RHC’s quality of care.

Original Structure-Process-Outcome model for rural health clinics’ quality of care.

Revised Structure-Process-Outcome model for rural health clinics’ quality of care.
The original or proposed model as shown in Figure 1 was fitted to raw data. All nonsignificant relationships were removed 1 step at a time. Post hoc model modifications were performed in an attempt to develop a better fitting and a more parsimonious model. On the bases of modification indices and theoretical relevance, 1 new path was introduced. RURAL and PROV were allowed to correlate. The correlation was sensible, since providers like hospitals are least likely to open RHCs in isolated areas or frontier places. The revised model attained good fit (RMSEA = 0.04, CFI = 0.98, GFI = 0.99, AGFI = 0.97).
Since the revised model was nested within the original model, a chi-square test of difference was conducted. The revised model was a substantial improvement over the original model (P <.001). Table 2 depicts further results for the revised model.
Parameter Estimates for Revised Model of Rural Health Clinics’ Quality of Care
Abbreviations:
proportion of nonphysicians: (PA and/or NP) FTEs divided by total clinical FTEs.
absence of physician FTEs.
provider affiliation or provider-based status.
sum of top 4 cause-specific mortality rates at county level divided by US average.
RHCs located in isolated rural areas or near small rural towns.
Since having less observed than expected preventable hospitalization is a proxy sign of higher quality of care, variables that have a negative association with the quality of care measurement model have a positive relation with quality of care. Provider-based status was negatively related to quality of care (P < .03). Absence of physicians in RHCs was negatively related to quality of care (P < .007). Rural health clinics with a higher proportion of nonphysicians were negatively related to quality of care (P < .007). Provider-based status was negatively related with rural location (P < .001). Lower cause-specific mortality rate at county level as compared to US average was positively related with RHCs’ quality of care (P < .05). Rural health clinics located in isolated rural areas or small rural towns were positively related with quality of care (P < .001). The significant variables in the revised model accounted for 11% of variation in quality of care.
Discussion
All 3 structure measures in the study were negatively related to RHCs’ quality of care. The absence of physicians in RHCs was detrimental to quality of care. This result was also corroborated by the negative relationship between higher proportion of nonphysicians and quality of care. Generally, provider-based RHCs, by virtue of their placement in larger systems, were assumed to access more resources that could enable them to attain quality of care in ways not possible for independent (free-standing) RHCs. 19 However, the empirical results in our study were to the contrary. Knott and Travers’s evaluation of quality assessment and performance improvement programs in RHCs indicated that provider-based RHCs lagged behind freestanding RHCs in assessment of clinical effectiveness. 20 Therefore, the negative association between provider-based RHCs and quality of preventive care should not be surprising.
None of the 3 process measures in our study was related to quality of care. Since a large majority of RHCs appeared to have not provided preventive services related to pneumonia and COPD, we were not able to include prevention use variables for these conditions. A focus group discussion with RHC practitioners revealed that a large majority of preventive vaccinations for pneumonia were coded as nurse practitioner (NP) visits rather than using the relevant HEIDS codes. Many RHCs do not have the practice of reporting preventive services to COPD. The equipments needed for COPD preventive tests may not be readily available for a majority of RHCs. In any case, the secondary data sources in use in this study lend only to use aspects of process measures. Adequate measures of process require not only detailed data sources like chart reviews, but also substantial qualitative assessment of patient-provider interactions. Reflecting on the volume of SPO-related literature since the 1960s, Donabedian concedes that it is still difficult to establish the relationship between process and outcome measures. 8
Environmental factors were also included in the SPO model. Population-level risk differences, as measured by cause-specific mortality rates, were positively related to RHC’s quality of care. In other words, areas with cause-specific mortality rates lower than the US average tended to have RHCs with higher quality of care (lower rates of preventable hospitalizations). The apparent positive relationship between rural location and quality of care probably reflects more on the limited access to hospitals in remote areas.
There are several key limitations in our study. First, quality of care is not the only dimension of care measured by PQIs. The dual nature of PQIs, as measures of access to care and quality of care, was indicated by the positive relationship between rural location and quality of care. Residents in remote regions have limited access to hospitals, which could lead to lower number of sentinel admissions. Lower number of sentinel admissions could then be misconstrued as “better” quality of care.
Second, a correlational research design could not rule out alternative explanations. For example, hospitalization rates for sentinel events could depend on the prevalence of the underlying chronic illnesses and the socioeconomic status of patients. In addition, case-mix adjustments of PQIs do not account for socioeconomic, genetic, and other differences among outpatients.
Third, quality of care is related to additional measures of structure and process. The use of technology (eg, electronic medical records) could have a positive impact on quality of care. In addition, organizational strategies (eg, disease management programs) could have an impact on quality of care in RHCs. The limited access to data precluded the inclusion of such variables.
Last but not least, the genearalizeability of the study will be limited. Although, the CCW data warehouse provides researchers with “100%” Medicare beneficiaries linked across the continuum of care, the claims and assessment data are primarily limited to the fee-for-service population. In addition, the CCW and other data sources used in the study face anomalies associated with large, national, administrative datasets. In particular, we were not able to generate stable PQI rates for all conditions listed by the AHRQ. For instance, hypertension was listed as one of the most common chronic conditions among rural Medicare beneficiaries. 21 However, most RHCs were associated with no preventable hospitalizations related to hypertension. Therefore, this study included only 7 of the 13 PQIs related to adult populations. In order to have stable rates of DIAB, this study summed up the 4 separate AHRQ PQIs for DIAB into a single PQI for DIAB.
Future research could improve on the following areas. This study was not able to include 6 more PQIs related to hypertension, dehydration, perforated appendix, angina, urinary tract infection, and asthma. The majority of RHCs in this study had less than 1 observed preventable hospitalization for these PQIs. Zero-inflated Poisson latent variable modeling could be used to accommodate PQIs with high prevalence of zero observed preventable hospitalizations.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: HealthCore, the company Dr Agiro works for, was not involved in the research, authorship, and publication of this article. The research was conducted during the tenure of Dr Agiro as a presidential doctoral fellow at the University of Central Florida.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a target grant from Health Resources and Services Administration (HRSA) [R04-RH10661-01-00].
