Abstract
Home health performance gained visibility with the publication of Home Health Compare and the Home Health Value-Based Payment demonstration. Both provide incentives for home health agencies (HHA) to invest in quality improvements. The objective of this study is to identify the association between quality initiatives adopted by HHAs and improved performance. A 2018 national survey of 7459 HHAs, yielding a sample of 1192 eligible HHAs, provided information about 23 quality initiatives, which was linked to 5 composite Super Quality Measures (SQMs): ADL/pain, self-treatment, timely care, hospitalizations, and patient experience. Exclusions for missing data and outliers yielded a final analytical sample of 903 HHAs. Regression models estimated associations between quality initiatives and SQMs. The relationships between sixteen of the SQM/quality initiative pairs were positively associated with improvement and 7 were negatively associated. Web-based technologies for staff and care-givers improved performance but deteriorated patient experience. Web support-groups for staff and review of HHC rankings reduced hospitalization rates. While this study offers insights for quality improvement, a limitation may be a lack of sensitivity to the nuances of quality improvement implementation. Therefore, this study should be viewed as hypothesis-generating concerning initiatives likely to have the greatest potential meriting further investigation.
Background
Over the last several decades, home health has grown as a viable alternative to institutional post-acute care settings, particularly for patients 65 years of age and older. In 2016, over 12 000 Centers for Medicare and Medicaid Services (CMS)-certified home health agencies (HHAs) made more than 110 million home visits to over 3.5 million Medicare beneficiaries, with a slight decline in the number of agencies and volume of services reported in 2017. 1 For-profit agencies, banned from Medicare before 1980, now account for over 75% of HHAs in the United States. 1
Given their increasing role in the provision of post-acute care, the performance of HHAs, particularly concerning quality of care, has emerged at the forefront of public policy concerns. In 2003, CMS mandated that Medicare-certified HHAs provide performance data for a variety of quality indicators using the Home Health Outcome and Assessment Information Set (OASIS). These data enabled the publication of the Home Health Compare (HHC) report card. 2 Emerging evidence suggests that several measures of home health quality improved for some, but not all, types of HHAs with public reporting, 3 suggesting that further incentive programs might be appropriate. In 2016, CMS implemented the Home Health Value-Based Purchasing (HHVBP) demonstration program in 9 states. Payment to all Medicare-certified HHAs in these states was adjusted based on their Quality Measures (QMs) calculated from the OASIS and hospital claims data. The payment adjustment started at 3% in 2018 and will increase annually, up to 8% by 2022, 4 with national implementation to be considered at that time. HHAs faced with HHVBP financial incentives will need to decide whether they want to avail themselves of the incentive by improving their quality of care related to one or more of the QMs. This strategic decision to be made by HHAs takes into consideration both which QMs to target and the best way to achieve improvement goals.
The environmental changes vis a vis quality, due to both publication of the report cards experienced by HHAs over the last 2 decades, and the expectation of HHVBP, may lead HHAs to innovate by undertaking quality initiatives with the potential to improve performance. Such organizational innovations, that is, the adoption of an idea or behavior that is new to the organization, the organization’s industry, the market, or the general environment, are a typical response to a gap in performance caused by changes in the external environment facing the organization. 5 For the HHAs, closing a perceived performance gap concerning quality was, and still is, a particularly salient goal as they are facing a potential threat to financial viability on 2 fronts. As has been noted, under HHVBP, quality is factored into the payment methodology, thus affecting revenues directly. Additionally, the HHA performance in comparison to its competitors in the HHC report card can potentially influence the demand for its services, and hence profitability. Thus, in addition to the intrinsic desire to improve patient outcomes, HHAs are likely to be financially motivated to innovate to improve their performance.
A few empirical studies have investigated the effects of innovation on performance in banking, computer and manufacturing industries, as well as some health care organizations, 6 and several have reported positive effects. There are a few anecdotal studies relating quality initiatives in-home health care to outcomes such as hospitalization, 7 as well as evaluations of quality improvement demonstration projects that are not targeted to specific initiatives.8,9 However, although it is a topic of considerable managerial interest and current policy relevance, to the best of our knowledge, none have considered the association between innovation, in this case in the form of specific quality improvement initiatives, and HHA quality performance. This study is designed to address this question. We have collected information about new quality initiatives (NQIs) adopted by HHAs via a survey of a large national sample and linked it to information about their performance as reported in HHC. We analyzed the data statistically to identify NQIs that are significantly associated with changes in quality performance.
Methods
Data
The study focused primarily on NQIs adopted during the 5-year period 2013 to 2018. To create the survey sample we used the CMS 2013 and 2018 Medicare cost reports 10 and included the 8388 HHAs that were large enough to be required by CMS to submit full reports. They represented 70% of the approximately 12 000 HHAs nationally. The data for these HHAs were linked to HHC data for 2013 and 2018 using the CMS Certification Number (CCN). Agencies without CCNs were excluded for a final sample of 7941 HHAs (95% of those eligible) which were mailed surveys about quality initiatives undertaken, as discussed further below.
Of the 7941 surveys mailed in 2018, 482 were returned undelivered for a surveyed sample of 7459. We received a total of 1314 surveys resulting in a response rate of 17.6%. This is a typical response rate obtained in surveys of business managers. 11 Of those, 122 surveys were ineligible because of the following: the agency’s CCN could not be determined; the respondent no longer worked for the agency; the respondent did not meet our qualification criteria; the respondent did not complete at least one NQI question. Therefore, the effective sample was 1192 and the effective response rate was 16.1%.
Of the 1192 eligible HHAs, 289 were excluded from the analyses either because they were missing variables (280; 23%) or because they were identified as influential observations by Cook’s Distance (9; .07%). The final sample included 903 HHAs. However, some quality measures (QMs) data were also missing randomly, excluding some HHAs from some of the estimated models, resulting in varying sample sizes across models. Sensitivity analysis limiting the sample to HHAs with complete data for all outcomes (N = 700) led to fewer significant findings (due to reduced statistical power) but substantively similar conclusions.
The survey was piloted through interviews with directors of HHAs varying by ownership types, size, and location, and the final survey was refined based on the feedback received. The survey was then mailed to HHAs in 3 waves. Eligible respondents were required to be responsible for managing the day-to-day operations of their agency and be knowledgeable about quality improvement initiatives. The survey was endorsed by the National Association for Home Care and Hospice (NAHC) and respondents were offered participation in a sweepstake with rewards ranging from $50 to $595 and a summary report comparing their agency’s data to the average of all other HHAs who participated in the survey.
To compare survey respondents and non-respondents, we obtained information on the number of unique patients from the Medicare Cost Reports for HHAs and Hospitals with HHAs for 2017. For those agencies whose fiscal year did not coincide with the calendar year, the number of patients was calculated as a weighted average of data reported in 2 consecutive Cost Reports with the weights corresponding to the proportion of the year covered by each report.
Measures
Dependent variables
We obtained QMs and patient experience measures data for 2013 and 2018 from the HHC website data download section. 12 The hospitalization QM was available only for 2017. We grouped the QMs into 5 Super Quality Measures (SQMs) that represent varying aspects of patient care: (1) patient improvement in ADLs and pain, (2) patient improvement in self-treatment, (3) patient care initiated in a timely manner, (4) acute care hospitalizations, and (5) patient experience of care. Prior studies 13 and factor analyses using current HHC data guided creation of the SQMs. Table 1 lists the 5 SQMs, their definitions, components and factor loadings for those SQMs based on multiple items (Cronbach’s alpha coefficients).
Super Quality Measures (SQMs).
SQMs for each year were calculated from the individual QMs reported in HHC. The individual QM for each year was standardized for that year’s sample. To create the outcome measures (SQMs) each individual quality measure was standardized by subtracting the mean and dividing by the standard deviation. In the case of our 3 composite outcome measures, we averaged the standardized QMs for the measure to create the outcome. Finally, we multiplied each outcome by 10. So, all outcomes are measured using the same units. For the ADL/Pain, Self-Treatment, and Patient Experience SQMs, the standardized individual QMs were averaged to create the SQM. The ADL/Pain SQM was based on an average of 3 non-missing component QMs. Self-Treatment and Patient Experience were averages of all their components. Finally, the 5 SQMs were re-scaled by multiplying each by 10. The hospitalization SQM, the only one that increased with poor quality, was also reversed, such that all SQMs had the lowest values indicating poor quality and the highest values indicating high quality.
Independent variables
We surveyed HHAs about the new quality initiatives (NQIs) they undertook to improve their quality in the past decade. The survey, administered in 2018, included a list of clinical and administrative NQIs that we identified through review of the literature and focus groups conducted with HHA directors. We constructed the list of new quality initiative candidates by first consulting the literature on information and mobile technology.14,15 We then conducted focus groups and interviews with Home Health Agency managers and others with field expertise (eg, trade group representatives, journalists) to supplement, update and refine the list. For example, 3 of the quality initiatives web-based support for staff, web-based support for patient, and web-based support for caregivers, emerged from our focus groups with HHA providers. In all 3 cases, support group refers to a site on the web where they can meet on zoom or some other application with a facilitator or a chat room with or without a facilitator. The resulting set of initiatives was grouped into 4 categories: (1) office NQIs, (2) clinical NQIs, (3) practices NQIs, and (4) regular monitoring of quality data published on HHC. (A complete list of the NQIs by group can be found in Table 2b in the Results section).
Respondents were given a choice of 4 options concerning NQI implementation: (1) never considered, (2) considered but never adopted, (3) adopted more than 5 years ago, and (4) adopted within the past 5 years. The survey also included questions about the respondent’s perception regarding competition for patients and competition for labor in their markets and about the ownership of the agency.
Analyses strategy
We performed a series of analyses to determine the association between adoption of NQIs and SQMs. The main analyses estimated 5 separate ordinary least square models for each of the 5 SQMs. The units of analysis were the HHAs. The dependent variables were the SQMs’ values in 2018 (2017 for hospitalization). The independent variables were 23 dichotomous variables for each of the 23 NQIs, obtaining the value 1 if the NQI was adopted at any time, 0 otherwise. The 5 actions listed in Table 2b under Regular Review of HHC Quality Measures and 5-star rating were included in the models as a single dichotomous indicator equal to 1 if any of the 5 actions were taken and 0 if none were taken. They were not included in the models separately, because they were highly correlated and also frequently implemented in unison (87% of HHAs reported taking all 5 HHC review actions). Initial analyses found that there was no distinction between adoption within the last 5 years or times prior. Several control variables that prior studies suggested could potentially influence quality performance in other post-acute contexts were also included: the baseline SQM value in 2013, the logged number of unique patients treated by the agency, ownership, competition for patients and competition for staff.16,17 The model included state fixed effects and inference was based on robust standard errors.
HHAs may have a propensity to adopt quality improvement technologies in bundles, in which case the presence of some NQIs might be highly correlated and the regression model will be unable to identify separate significant effects for some NQIs. We, therefore, tested all models for multicollinearity using the Variance Inflation Factor (VIF). The VIF for all models had low values, in the range of 1 to 3, well below the critical value of 10, which indicates presence of multicollinearity. 18
Results
Characteristics of Respondents and Non-Respondents
Table 2a and b present descriptive statistics for survey HHA respondents included in the analyses. There was no significant difference at the 5% significance level in average number of patients between responding HHAs, caring for 1019 patients, and non-responding agencies, which averaged 1067 patients. There were also no significant differences in the SQM values in either 2013 or 2018 between respondents and non-respondents except for patient experience (1.14 for respondents and 0.33 for non-respondents) and timely care (1.21 for respondents and 0.59 for non-respondents), both in 2018.
Descriptive statistics.
Data for hospitalization QM was available only for 2017.
Descriptive statistics.
There was also no significant difference between HHAs included in the analysis and those not included in the analysis in the number of patients. No SQMs differed at the 5% significance level except for timely care in 2018, with 1.57 for included HHAs and 0.14 for excluded HHAs. The 2 groups did not differ significantly in terms of ownership or perceived environmental competition. Sample HHAs were also significantly less likely to adopt web-based billing to patients. They were significantly more likely to adopt electronic medical records, improve/develop new treatment protocols, and to review HHC quality measures.
Association between NQIs and SQMs
Respondents were asked if they adopted any one of 23 specific quality initiatives. We tested the association of these 23 NQIs with 5 SQMs, leading to 115 comparisons. Thus, at the 0.05 significance level, we would expect 6 (115 × 0.05) of these tests to be significant by chance alone. We find a much larger number of significant associations, 22, indicating that these associations are not random.
Table 3 presents the estimated associations between the 23 NQIs and the 5 SQMs based on results from the 5 estimated models. The full models, including baseline SQMs, number of patients, ownership and competition for labor and patients are available upon request from the authors.
Associations between quality initiatives and SQMs: Ordinary least square models estimated separately for each SQM.
Models included number of unique patients, ownership type, competition for staff and competition for patients. Full model estimates are available upon request.
Office NQIs
Seven of the NQIs related to office technologies were observed to be significantly associated with SQMs (at the 5% level). Three NQIs had a positive effect. The largest was the implementation of electronic medical records which improved the ADL/pain SQM. The second largest was web-based staff scheduling, which improved patient experience and the third was web-based referrals to the agency, which improved both the ADL/Pain and the self-treatment SQMs. Two NQIs, web-based patient scheduling and web-based patient billing lowered the quality of patient experience. The hospitalization SQM was the only SQM unaffected by any NQI in this category.
Clinical NQIs
Six of the NQIs related to clinical technologies were observed to have significant associations with SQMs. The strongest and most pervasive effect was negative. Web-based support groups for patients lowered significantly (at the 5% level) the SQM scores for patient experience, hospitalizations, timely care, and self-treatment, and while reaching only marginal significance (P = .060), also negatively impacted the ADL/pain SQM. All other effects improved quality. Web-based support groups for staff improved the hospitalization SQM. This was followed by telehealth, which improved both the ADL/Pain and the self-treatment SQMs. Timely care was improved by educational modules on the web directed at patients and families, and web-based support group for caregivers improved the ADL/Pain SQMs. The one NQI in this category that had no effect at all was use of internet-based communications with clients about care instructions or consultations.
Practice NQIs
Four of the NQIs related to practice innovations were found to be significantly associated with SQMs. The largest effect was from improved or new treatment protocols which actually lowered the hospitalization SQM score. Improved care plans, on the other hand, increased quality for both the ADL/Pain and the patient experience SQMs. Similarly, improved admission or intake process improved the self-treatment SQM and timely care improved with marginal significance (P = .068). Lastly, increasing types of services offered to specific types of patients significantly lowered the self-treatment SQM.
Monitoring of publicly reported HHC quality measures
This action affected significantly and positively the ADL/Pain, self-treatment and hospitalization SQMs. The timely care SQM was also positively affected by the monitoring of publicly reported HHC quality measures but the effect was only marginally significant (P = .100).
Discussion
The purpose of this study was to determine the impact of specific new quality initiatives on HHA quality performance. Several NQIs related to office technologies, clinical technologies and practice innovations were found to be significantly associated with SQMs, providing preliminary evidence that these quality initiatives are related to significant changes, both positive and negative, in home health care at the agency-level.
A potential limitation of our study is that the information about implementation of the NQIs was obtained from a survey, and as a consequence is subject to recall bias. We asked respondents to distinguish between initiatives adopted within the prior 5 years and the period preceding the last 5 years. Our preliminary analyses indicated that this distinction was immaterial and did not lead to different results, and we have, therefore, assumed that all NQIs impacted quality between 2013 and 2018. This assumption may not be accurate if the NQI has been adopted by the majority of respondents before 2013 and all of the impact on quality has occurred by 2013. We do not believe that this is likely, but it might explain why some NQIs have not shown any change effects on any SQMs.
Another potential limitation is related to the sensitivity of our measurement to both the nuances of quality initiatives undertaken and the fidelity of quality improvement activities. We noted above that there were 115 combinations of NQIs and SQMs tested over all our models. Only 22 of those, or 19.1%, were found to be significant at the 5% level. This might be due to the low sensitivity of our measures due to a potential lack of precision. For example, the survey asked respondents if they adopted telehealth. Telehealth is a very wide area, encompassing anything from video chats to wearable monitors automatically transmitting vital signs using Bluetooth technology. Similarly, our quality measures were composites of several quality measures, thus possibly missing some, more specific quality improvements. However, this high level of aggregation is necessary when conducting a national study of this type.
Our findings have a number of policy and managerial implications that merit consideration. First, the initiatives undertaken vary in their impact on quality as measured by the number and type of SQMs positively affected. Sixteen NQIs had a positive impact indicating an improvement and 7 had a negative impact, indicating that adopting these NQIs is associated with a decline in quality. Thus, decisions to pursue these initiatives may depend on which areas of suboptimal quality performance require improvement.
Second, most NQIs had a rather focused impact, typically being associated with only one SQM, occasionally with two. The exception is regular review of quality measures on HHC NQI, which had a positive effect on quality improvement for all SQMs, except for patient experience. This finding is consistent with a prior study on the impact of the publication of HHC on quality measures 3 although this study did not address agency monitoring of quality measures. This finding might be because HHC promotes public awareness of potential performance gaps between HHAs that could influence consumers’ choices. The lack of association between the review of HHC measures and the patient experience SQM may be because this measure was added to HHC only in January 2016, 19 thus reflecting the minimal experience of patients and agencies with these measures on this platform during our study period. This NQI was also one of the most likely to be implemented by HHAs, as reported by 91% to 93% of survey respondents. This suggests that a review of HHC, because it has the potential to reveal previously unrecognized performance gaps relative to competitors, may be a precursor to other initiatives designed to remedy such gaps.
Third, whether web-based technology has a beneficial impact on quality may depend on the intended user. When the intended user is staff or the caregiver the impact on performance tends to be positive. When the intended user is the patient the impact tends to be negative. The web technologies and applications that had a significant positive impact on the clinical SQMs included the use of EMRs, web-based admission referrals into the agency, telehealth services provided to the patients in their homes, education provided to caregivers over the web, support groups for caregivers, and support groups for staff. These activities place technology in the hands of clinical and administrative HHA staff. One other NQI, staff scheduling via the web, was also beneficial, improving the score of the patient experience. In contrast, the potential for technology to be alienating when the patient is the intended user is most apparent concerning the patient experience SQM. While web-based staff scheduling, which may work behind the scenes to facilitate reliability of care, enhances patient experience, web or internet-based patient billing and scheduling appear to diminish it. Web-based support groups for patients also diminishes performance on all the SQMs, although only marginally (P = .06) on the ADLs/Pain SQM.
These findings suggest that the use of web-based technologies in the hands of staff has the potential to improve quality, perhaps because of the potential to free up administrative time for patient care. Discussions with home health directors support this interpretation that “efficiency leads to quality”. For example, standardization of workflow through the use of electronic resources not only streamlines the staff scheduling process but also lowers the risk of human error that could disrupt coordination and continuity. Similarly, the EMR improves communication between staff, makes their interactions more efficient, and scheduling of treatments timelier, leading to fewer errors in care and more timely treatments. Web-based initiatives that streamline the admission and intake process also increase efficiency and free up time for patient care reflected in quality measure improvement. However, patients may find interactions with agency staff through the web too impersonal and unsatisfying as an alternative to direct provider encounters. This may be a reflection of the relatively advanced age of many of the patients or perhaps the poor internet connection in many homes which some agency directors noted, and which may make web interaction less attractive. It should be noted that our survey was conducted in 2018, pre-Covid. Anecdotal evidence 20 suggests that during the Covid period, many home health patients welcomed telehealth services and preferred them to in-person home visits. It remains to be seen if this trend will continue once the pandemic is under control.
Fourth, because of their visibility in cost containment and reimbursement reform efforts, hospitalization rates are of particular interest to public and private payers as well as referring hospitals seeking to reduce their own exposure to the financial risks of rehospitalizations. However, our findings do not provide a clear path to performance improvement on this SQM for HHAs. The only NQI associated with improvement in hospitalization is the web-based support groups for staff, which perhaps makes the sharing of information about non hospital-based treatment options among staff more efficient, thus lowering hospitalization rates.
Conclusion
In summary, this study finds that new quality improvement initiatives undertaken by HHAs have a differential impact on quality: some improve quality, some do not have an impact, and some actually lead to deterioration in quality. As discussed above, one of the limitations of this study is its lack of sensitivity, a tradeoff necessitated by its national scope. This study should, therefore, be viewed as hypothesis-generating, providing the impetus for future studies that can offer more specificity in describing NQIs and identifying the specific quality measures they affect. In light of the scarcity of resources typically available for quality improvement studies, there is a need to prioritize the areas for studies and investment in quality improvement. The contribution of this study is in providing guidance and highlighting those areas most likely to be the most promising and have the greatest potential to improve quality.
Footnotes
Acknowledgements
We gratefully acknowledge support from the National Association for Home Care and Hospice and insightful comments from Molly McDonald MPT, DPT, MBA, Clinical Quality and Education Evergreen Home Care; Carolyn Flietstra, Exec VP, HCBS Holland Home; Cheryl Meyer Nelson, PT, MPH, GHC, CDP, Clinical Manager Telemedicine Department NVNA & Hospice; and Patty Upham, Vice President of Clinical Services at Health Recovery Solutions.
Funding:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute on Aging [grant no. 1 R01 AG051719].
Declaration of conflicting interests:
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Authorship
Each of the listed authors meets the four authorship conditions.
