Sage Journals: Discover world-class research

Abstract

As mental health care transitions from facility-based care to community-based services, methods to classify patients in terms of their expected health care resource use are an essential tool to balance the health care needs and equitable allocation of health care resources. This study performed a scoping review to summarize the nature, extent, and range of research on case-mix classifications used to predict mental health care resource use in community settings. This study identified 17 eligible studies with 32 case-mix classification systems published since the 1980s. Most of these studies came from the USA Veterans Affairs and Medicare systems, and the most recent studies came from Australia. There were a wide variety of choices of input variables and measures of resource use. However, much of the variance in observed resource use was not accounted for by these case-mix systems. The research activity specific to case-mix classification for community mental health care was modest. More consideration should be given to the appropriateness of the input variables, resource use measure, and evaluation of predictive performance. Future research should take advantage of testing case-mix systems developed in other settings for community mental health care settings, if possible.

Keywords

Mental health community case-mix resource allocation health care costs review

Introduction

Globally, the burden of mental illness accounts for 32% of years lived with disability,¹ costing about US$2.5 trillion in 2010.² As in other health care sectors, funding for mental health care is limited. The challenge is how to match resources to the needs of the population.^2,3 Methods to classify patients in terms of their expected resource use is an important pre-requisite in addressing this challenge.

Although each person in the population is unique, there are shared characteristics that determine the types of treatments or services that individuals receive from the health care system.⁴ In other words, there are groups of people with similar characteristics, which will consume similar amount health care resources and, by extension, incur similar costs of care. These groups represent the mix of cases that are observed in a health care system, or a “case-mix,”⁵ which can be viewed as a proxy for the types of health care needs of the population.

Case-mix classification is commonly used in payment systems to reimburse health care providers based on the type of patient,⁶ also known as activity-based funding. Other applications of case-mix classification include risk adjustment models for health outcomes or other quality measures, staffing, program evaluation, and long-term planning and budgeting tools for policy makers.⁶

Case-mix classification systems can be of two types: grouping or index systems.⁵ Grouping systems assign cases into relatively homogeneous groups in terms of their expected resource use.⁵ Each group has a weight associated to represent its expected resource use relative to the average case in the population, also known as “case-mix index.”⁵ For example, the Resource Utilization Group Version III (RUG-III) is commonly used in the United States and Canada for nursing homes reimbursement.⁷ Index systems, instead, combine different characteristics of a case to produce a numerical value for each case that represents the expected level of resource use, which can then be mapped it to a “case-mix index” value to represent expected level of resource use relative to the average case in the population.⁵ For example, the Outcome and Assessment Information Set (OASIS) is used by Medicare to reimburse home care services.⁸ A previous study also provided further discussion of these two types of systems.⁹

It is worth noting that a funding formula is distinct from a case-mix classification system. A funding formula may work by assigning a monetary amount to the case-mix index and further adjusted based on numerous factors external to the case-mix classification system, such as available funding, inflation, geographic and provider characteristics, or negotiations between health system administrators and the providers. On the other hand, the case-mix index values are expected to remain constant because the health care needs of one group relative to another should not change drastically from year to year. Case-mix index values can change in rare occasions, such as changes in technologies or clinical practices, which may affect only one or a few groups by making them either more or less expensive to care for compared to the rest of the population. In addition, a funding formula may not be composed solely of case-mix classification, other designs are possible, such as a blend of case-mix and global budget.¹⁰

For mental health, the delivery of care can take place in multiple settings as de-institutionalization has shifted mental health services from facility-based inpatient care to community-based care.^11,12 Facility-based inpatient care provides intensive observation, diagnosis, and treatment typically in times of crisis,¹³ and usually requires a hospital admission with one or more overnight stays.¹⁴ Community-based care typically employs a care team that provides a wider range of services, including both urgent and ongoing care, such as assertive treatment services, crisis management, outreach, recovery, housing, occupation training, and day programs.¹³

Previously, Jones et al¹⁵ reviewed 16 studies between 1990 and 2005 studying predictors of mental health service utilization and costs. Hermann et al¹⁶ reviewed 36 studies between 1980 and 2002 focusing on risk adjustment models of psychiatric health outcomes and costs, which included some case-mix systems. Mason and Goddard reviewed 5 international examples of activity-based funding systems for mental health between 2006 and 2008.¹⁷ Harris et al¹⁸ reviewed 13 case-mix classification systems for all mental health care settings, but only in some Western countries published between 1995 and 2012.

To date, most mental health case-mix classification systems have predominantly focused on care in acute or inpatient settings. However, case-mix classification systems for community settings have received little attention. Therefore, this review summarized the nature, extent, and range of the up-to-date research on mental health care resource prediction using case-mix specifically in community settings, and identified the gaps in the current research.

Methods

In alignment with scoping review methods by Arksey and O’Malley,¹⁹ and PRISMA,²⁰ four academic literature databases were searched: PubMed, Web of Science, PsychInfo, and SCOPUS. Keywords were used to search the title and abstract for the presence of mental health, case-mix, and community settings concepts: (“mental health” OR “mental ill*” OR “mental disorder?” OR “psychiatr*” OR “behavio* care” OR “behavio* health”) AND (“casemix” OR “case mix” OR “case-mix” OR “case type?” OR “diagnosis related group*” OR “patient mix” OR “patient? group*” OR “patient? classification?” OR “patient? cluster*” OR “case? cluster*” OR “risk adjust*” OR “case adjust*”) AND (“communit*” OR “outpatient?” OR “out-patient?” OR “ambulatory”). Searches were done in October 2018 and included all date ranges. Duplicates and non-English full-text articles were removed. Database searches were also supplemented by scanning references of the eligible articles, consulting with experts and co-authors.

Articles’ titles and abstracts were then screened for relevance, followed by a screen of the full-text by the lead author and a review by the co-authors. Articles were included if a case-mix classification system was used to predict resource use of community mental health care or health care resource use of people with mental health disorders in community settings. This review used the World Health Organization’s definition of health care resources as the three main inputs of a health care systems as human resources, physical capital, or consumable resources.²¹ As in similar reviews,^18,22,23 this review considered studies that predict resource use using case-mix classification, rather than to simply describe the differences in resource use among sub-groups of the study sample, or to explain the variation in resource use by adjusting for different variables. In addition, a predictive study should provide a quantitative assessment of how well the predicted resource use explained the observed resource consumption, such as the R² value.¹⁵ The community settings were defined as care settings that do not require an overnight stay at the facility,¹⁴ which may include outpatient treatments or day programs.

To capture the scope of the case-mix classification systems presented, we collected key characteristics from each eligible article. Specifically, we collected information regarding the bibliography (authors, year of publication), sample data (geographic jurisdiction, care settings, age groups, sample size), case-mix system (name, input variables, type), resource use measure (definition of measure), and predictive performance (type, reported value). Data were then recorded and reviewed with the co-authors.

Results

This study identified 17 articles matching the criteria (Figure 1), which presented 32 case-mix classification models (Table 1). Most were from academic sources, except for the technical reports of the case-mix systems developed in Australia and New Zealand.^14,46,50 Most studies (11 out of 17) focused only on adult population.

Figure 1.

Search procedures for relevant articles.

Table 1.

Eligible studies, ordered by year of publication.

Author(s)	Context	Sample size	Case-mix system(s)	Type
Wood and Beardmore²⁴	USA, adult outpatient service at an university affiliated mental hospital	1000 adults	DRGs: 8 mental health and substance abuse DRGs²⁵	Grouping
Wittman and Lerner²⁶	Israel, mentally ill outpatients	2118 outpatients, age: 15 to 65	Chronicity: 6 terminal groups classified by long-term service, age, disability, diagnosis, and prior hospitalizations	Grouping
Barker et al²⁷	USA, Oregon’s local community mental health agencies	240 adults	MCAS: 4 domains: interference with functioning, adjustment to living, social competence, behavioral problems	Index
Uehara et al²⁸	USA, Washington’s Community Psychiatric Clinic	598 adults	LONCA: clients were assessed for 10 key needs; each has 4 levels (none, low, moderate, intense). These needs were then grouped according to physical, psychological, and social functioning	Grouping
Ettner and Notman²⁹	USA, New Hampshire Medicaid enrollees	12 218 adults, 17 901 children	ACGs: 51 mutually exclusive ACGs³⁰ based on ICD-9 codes, age, gender, and intermediate ADGs of similar expected resource consumption	Grouping
Ettner et al³¹	USA, claim records from a private insurer provided plans for employer-sponsored health insurance	51 621 adults, 14 145 children	Demographics	Grouping
			Demographics and ACG	Grouping
			Demographics and ADG	Grouping
			Demographics and HCC³²	Grouping
			Demographics, diagnosis, and comorbidity	Grouping
Trauer et al³³	Australia, Melbourne public psychiatric service registration list	200 adults	Diagnosis (schizophrenia, personality disorder, and social withdrawal)	Grouping
			LSP functional assessment,^34,35 which contained 5 sub-scales: antisocial, bizarre, compliance, withdrawal, and self-care	Index
Samuels³⁶	USA, New York’s licensed mental health service providers	24 463 adults	URG: high/medium/low user groups based on historical usage, diagnosis and insurance type	Grouping
			URG: high/medium/low users groups based on historical usage, insurance type, diagnosis, and age	Grouping
			URG: high/medium/low user groups based on historical usage, insurance type	Grouping
Buckingham et al¹⁴	Australia, 22 sites (inpatient and outpatient)	Adults: 9806 episodes (outliers trimmed: 9096), children/adolescents: 2098 episodes (outliers trimmed: 1956)	MH-CASC: 19 community terminal groups (adults: 10, children/adolescents: 9), out of 42 groups for all settings. Adult variables: focus of care, legal status, HoNOS assessment,³⁷ and LSP-16.³⁸ Children/adolescents variables: age, HoNOSCA assessment,³⁹ CGAS assessment,⁴⁰ and FIHS assessment⁴¹	Grouping
	Australia, integrated mental health care sites	8067 adult episodes (outliers trimmed: 7244)	Experimental Bundled Episodes: 12 terminal groups. Variables: legal status, HoNOS assessment,³⁷ diagnosis, suicidal risk, psychotic symptoms, and age	Grouping
Leslie et al⁴²	USA, Veterans Affairs mental health outpatient clinics	53 700 adult patients	GAF⁴³: assessment of severity of mental illness with a scale from 0 to 100 on dimensions such as symptoms, impairments, and ability to perform daily activities	Index
			Service-connected status: assessment of disability linked to military service	Index
			Service-connected status, but if patients were not service-connected, use GAF	Index
			Diagnosis: 12 groups (alcoholism, bipolar, dysthymia, generalized anxiety, major depressive, organic brain syndrome, other substance abuse disorder, panic disorder, personality disorder, post-traumatic stress disorder, schizophrenia, and other)	Grouping
DeLiberty et al⁴⁴	USA, Indiana Division of Mental Health	>60 000 adults and children/adolescents	SMI: 9 groups. Level 1: by diagnoses. Level 2: by levels of difficulties	Grouping
Rosen et al⁴⁵	USA, Veteran Affairs inpatients and outpatients	1 571 264 adult patients (66.6% development, 33.3% validation)	DCG/HCC: ICD-9CM maps to 37 diagnostic groups, then aggregate into conditions categories (which a person can have multiple). Five hierarchies of conditions were then imposed so that minor diagnoses do not add to cost prediction	Grouping
Gaines et al⁴⁶	New Zealand, 8 district health boards	Adults: 9199, children/youths: 2868	NZ-CAOS: 22 community terminal groups, out of 42 groups for all care settings. Adults (13 groups): assessment only, legal status, ethnicity, focus of care, and age. Children/youths (9 groups): assessment only, ethnicity, age, HoNOSCA assessment³⁹	Grouping
			MH-CASC¹⁴	Grouping
Selim et al⁴⁷	USA, Veterans Affairs ambulatory care at 4 sites in Boston	2425 adults	PCI/MCI: count of 30 physical diagnoses and 6 mental diagnoses	Index
			CCI/MCI: count of 30 physical diagnoses (with symptoms) and 6 mental diagnoses	Index
Sloan et al⁴⁸	USA, Veterans Affairs inpatients and outpatients	914 225 adult patients (60% development, 40% validation)	PsyCMS: 46 categories based on ICD-9CM codes, with 4 hierarchies (alcohol use, drug use, anxiety disorder, and mood/psychotic disorder) imposed to assign patients into the highest expected cost category in a given hierarchy	Grouping
			Age (9 groups) and gender	Grouping
			VA-MH12: 12 categories of mental health diagnosis based on ICD-9CM codes	Grouping
			Adjusted Clinical Group/Aggregate Diagnostic Group (ACG/ADG)	Grouping
			DCG/HCC: 2 hierarchies (substance abuse and psychiatric disorders)	Grouping
			CDPS⁴⁹: 2 hierarchies (substance abuse and psychiatric) that grouped patients’ ICD-9CM codes based on diagnosis and expected cost	Grouping
Independent Hospital Pricing Authority⁵⁰	Australia, ambulatory episodes from 3 states	9976 community episodes (adults and children)	AMHCC: 46 community terminal groups, out of 91 groups for all care settings. Variables: 5 phases of care, age, HoNOS,³⁷ Life Skills Profile (LSP-16)³⁸	Grouping
			MH-CASC¹⁴	Grouping
Martin et al⁵¹	UK, 11 child and adolescent mental health service sites	4573 completed outpatient periods (50% development, 50% validation)	CAMHS Need-Based: 19 terminal groups. Variables: getting advice/help/more help, diagnosis, and NICE guidance for mental health and substance use disorders⁵²	Grouping

Abbreviations: ACG, Ambulatory Care Groups; ADG, Ambulatory Diagnostic Groups; AMHCC, Australia Mental Health Care Classification; CAMHS, Child and Adolescent Mental Health Services; CCI, Conditional Comorbidity Indices; CDPS, Chronic Illness and Disability Payment System; CGAS, Children’s Global Assessment Scale; DCG, Diagnostic Cost Group; DRG, Diagnosis Related Groups; FIHS, Factors Influencing Health Status; GAF, Global Assessment of Functioning; HCC, Hierarchical Condition Category; HoNOS, Health of the Nation Outcome Scales; LONCA, Level of Need-Care Assessment; LSP, Life Skills Profile; MCAS, Multnomah Community Ability Scale; MCI, Mental Comorbidity Indices; MH-CASC, Mental Health Classification and Service Costs; NICE, National Institute for Health Care Excellence; NZ-CAOS, New Zealand Mental Health Classification and Outcomes Study; PCI, Physical Comorbidity Indices; SMI, Serious Mental Illness; URG, Utilization Risk Groups.

Most of the research came from the United States, and the largest studies came from the USA Veterans Affairs and Medicare systems.^42,45,48,53 However, it is worth noting that the samples from the Veteran Affairs system were mostly adult males, and samples from the Medicare system were adults aged 65 or older, which were not representative of the US population. The most recent major effort came from Australia with their Australian Mental Health Care Classification (AMHCC),⁵⁰ which was developed to predict resource use for both inpatient and community settings, and all age groups.

The input variables for the case-mix classification systems were varied. Most common variables were diagnosis, demographics, variables related to severity, comorbidity, or functional status. Most case-mix systems were grouping system, and index systems were less common.

There were also a wide range of measures of resource use from the studies identified. These measures can be roughly classified into 2 types: proxy measures (such as number of visits) (Table 2) and direct measures (such as claims data or wage-weighted staff time) (Table 3).

Table 2.

Empirical results of case-mix systems predicting proxy measures of resource use, ordered by name of the case-mix system and year.

Case-mix system	Resource measure	Performance measure
ACG/ADG⁴⁸	Annualized mental health and substance abuse outpatient visits	R² (retrospective) = 11.1%, R² (prospective) = 2.8%
CAMHS + complexity factors, contextual problems, education, employment, training⁵¹	Number of appointments for closed-cases (without activities for ⩾6 months)	R² = 5.0%, R² (with provider effect) = 12.1%
CCI/MCI⁴⁷	Number of total visits (6 months)	R² = 5.7%
	Number of medical visits (6 months)	R² = 3.4%
	Number of mental health visits (6 months)	R² = 14.3%
CCI/MCI and demographics variables⁴⁷	Number of total visits (6 months)	R² = 6.7%
	Number of medical visits (6 months)	R² = 4.6%
	Number of mental health visits (6 months)	R² = 14.6%
CCI/MCI, demographics variables, and patient self-reported health status⁴⁷	Number of total visits (6 months)	R² = 7.5%
	Number of medical visits (6 months)	R² = 5.3%
	Number of mental health visits (6 months)	R² = 15.9%
CDPS⁴⁸	Annualized mental health and substance abuse outpatient visits	R² (retrospective) = 14.7%, R² (prospective) = 4.0%
Chronicity²⁶	Number of prior hospitalizations	F = 4.64 (P = .01)
Chronicity²⁶	Prescription of major psychotropic drugs (binary)	$χ^{2} = 419.5$ (P = .000)
DCG/HCC,⁴⁵ substance abuse indicators	Annualized contacts with providers	R² = 27.9%
DCG/HCC⁴⁸	Annualized mental health and substance abuse outpatient visits	R² (retrospective) = 15.7%, R² (prospective) = 4.0%
Demographics (age groups and gender)⁴⁸	Annualized mental health and substance abuse outpatient visits	R² (retrospective) = 2.1%, R² (prospective) = 0.8%,
DRG²⁴	Number of outpatient sessions	Hartley’s Fmax P-value < .01, Cochran’s C P-value < .01, Barlett-Box F P-value < .01 (groups variances were not homogeneous)
LONCA²⁸	Number of hospitalization, past 12 months	Cramer’s V = 0.17
MCAS²⁷	Hospitalizations admission (next 2 years) or involuntary admission (next 18 months) to state hospital	$χ^{2} > 6.05$ (P = .01)
PCI/MCI⁴⁷	Number of total visits (6 months)	R² = 5.4%
	Number of medical visits (6 months)	R² = 3.3%
	Number of mental health visits (6 months)	R² = 14.4%
PCI/MCI and demographics variables⁴⁷	Number of total visits (6 months)	R² = 6.6%
	Number of medical visits (6 months)	R² = 4.6%
	Number of mental health visits (6 months)	R² = 14.6%
PCI/MCI, demographics variables, and patient⁴⁷ self-reported health status	Number of total visits (6 months)	R² = 7.7%
	Number of medical visits (6 months)	R² = 5.5%
	Number of mental health visits (6 months)	R² = 15.8%
PsyCMS⁴⁸	Annualized mental health and substance abuse outpatient visits	R² (retrospective) = 24.4%, R² (prospective) = 6.5%
VA-MH12⁴⁸	Annualized mental health and substance abuse outpatient visits	R² (retrospective) = 17.0%, R² (prospective) = 4.6%

Abbreviations: ACG, Ambulatory Care Groups; ADG, Ambulatory Diagnostic Groups; CAMHS, Child and Adolescent Mental Health Services; CCI, Conditional Comorbidity Indices; CDPS, Chronic Illness and Disability Payment System; DCG, Diagnostic Cost Group; DRG, Diagnosis Related Groups; HCC, Hierarchical Condition Category; LONCA, Level of Need-Care Assessment; MCAS, Multnomah Community Ability Scale; MCI, Mental Comorbidity Indices; PCI, Physical Comorbidity Indices.

Table 3.

Empirical results of case-mix systems predicting direct measures of resource use, ordered by name of the case-mix system and year.

Case-mix system	Resource measure	Performance measure
ACG²⁹	Total annual Medicaid claims (in- and out-patient), except nursing homes, drug claims, and intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 2.0%, $R^{2}$ (children) = 4.1%
ACG²⁹	Total annual Medicaid mental health and substance abuse claims	$R^{2}$ (adults) = 2.1%, $R^{2}$ (children) = 1.7%
ACG³¹	Total annual mental health and substance abuse-related insurance claims, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 2.5%, $R^{2}$ (children) = 1.3%, $R^{2}$ (combined) = 2.3%
ACG³¹	Total annual mental health and substance abuse related insurance claims and out-of-pocket payments, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 3.0%, $R^{2}$ (children) = 1.4%, $R^{2}$ (combined) = 2.7%
ACG/ADG⁴⁸	Total annualized inpatient and outpatient cost of mental health and substance abuse care	$R^{2}$ (retrospective) = 4.8%, $R^{2}$ (prospective) = 2.6%
ADG³¹	Total annual mental health and substance abuse-related insurance claims, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 7.6%, $R^{2}$ (children) = 3.9%, $R^{2}$ (combined) = 6.8%
ADG³¹	Total annual mental health and substance abuse-related insurance claims and out-of-pocket payments, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	R² (adults) = 9.0%, $R^{2}$ (children) = 4.1%, $R^{2}$ (combined) = 7.9%
AMHCC⁵⁰	Direct cost: wage-weighted staff time, indirect cost: allocated equally among all contacts at a unit for an episode of care (various lengths)	R² = 26.6%
CDPS⁴⁸	Total annualized inpatient and outpatient cost of mental health and substance abuse care	$R^{2}$ (retrospective) = 8.3%, $R^{2}$ (prospective) = 5.4%
Demographics³¹	Total annual mental health and substance abuse-related claims, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 0.3%, $R^{2}$ (children) = 0.3%, $R^{2}$ (combined) = 0.3%
Demographics³¹	Total annual mental health and substance abuse-related insurance claims and out-of-pocket payments, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	R² (adults) = 0.3%, $R^{2}$ (children) = 0.4%, $R^{2}$ (combined) = 0.3%
Demographics, diagnosis, and comorbidity³¹	Total annual mental health and substance abuse-related insurance claims, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	R² (adults) = 8.6%, $R^{2}$ (children) = 4.2%, $R^{2}$ (combined) = 7.6%
Demographics, diagnosis, and comorbidity³¹	Total annual mental health and substance abuse-related insurance claims and out-of-pocket payments, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 9.9%, $R^{2}$ (children) = 4.7%, $R^{2}$ (combined) = 8.7%
Demographics (age groups and gender)⁴⁸	Total annualized inpatient and outpatient cost of mental health and substance abuse care	$R^{2}$ (retrospective) = 0.4%, $R^{2}$ (prospective) = 0.4%
Diagnosis (schizophrenia, personality disorder, and social withdrawal)³³	Log of community care cost (which included total annual clinic cost allocated to patients based on their contact duration for the year)	R² = 13.9%, $R^{2}$ (schizophrenia) = 2.6%, $R^{2}$ (personality disorder) = 6.2%, $R^{2}$ (social withdrawal) = 5.8%
Diagnosis (12 groups)⁴²	Annual direct and indirect costs of outpatient care	R² = 7.0%
DCG/HCC⁴⁸	Total annualized inpatient and outpatient cost of mental health and substance abuse care	$R^{2}$ (retrospective) = 9.5%, $R^{2}$ (prospective) = 5.7%
Experimental Bundled Episodes¹⁴	Wage-weighted staff time over 8-week long bundled episodes (across all care settings)	R² = 12.6%, $R^{2}$ (outliers trimmed) = 27.9%
GAF⁴²	Annual direct and indirect costs of outpatient care	R² = 3.1%
HCC³¹	Total annual mental health and substance abuse-related insurance claims, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	$R^{2}$ (adults) = 5.5%, $R^{2}$ (children) = 2.8%, $R^{2}$ (combined) = 4.9%
HCC³¹	Total annual mental health and substance abuse-related insurance claims and out-of-pocket payments, for both inpatient and outpatient settings, excluding nursing home or intermediate care facility for the mentally retarded	R² (adults) = 6.2%, $R^{2}$ (children) = 3.1%, $R^{2}$ (combined) = 5.5%
URG by diagnosis and funding source³⁶	Log of 3-year utilization of all mental health services, including inpatient settings	Misclassification = 35.6%, R² = 18.3%
URG by funding source, diagnosis, and age³⁶	Log of 1-year utilization of all mental health services, including inpatient and outpatient settings	Misclassification = 56.0%, R² = 4.3%
URG by funding source³⁶	Log of 1-year utilization of all mental health services, including inpatient and outpatient settings	Misclassification = 40.5%, R² = 4.0%
LSP sub-scales (antisocial and bizarre behavior)³³	Log of community care cost (which included total annual clinic cost allocated to patients based on their contact duration for the year) and inpatient cost	R² = 14.9, $R^{2}$ (antisocial) = 12.9%, $R^{2}$ (bizarre behavior) = 2.0%
MH-CASC¹⁴	Wage-weighted staff time over 8-week long episode	Adult: R² = 5.7%, $R^{2}$ (outliers trimmed) = 12.7%Children/adolescents: R² = 12.4%, $R^{2}$ (outliers trimmed) = 18.8%Combined: R² = 4.1%, $R^{2}$ (outliers trimmed) = 14.8%
MH-CASC⁴⁶	Cost based on staff activity data attributable to clients for an episode of care (various lengths)	Adults: R² = 3.5%Child/youth: R² = 5.3%Combined: R² = 4.1%
MH-CASC⁵⁰	Direct cost: wage-weighted staff time, indirect cost: allocated equally among all contacts at a unit for an episode of care (various lengths)	R² = 5.9%
NZ-CAOS⁴⁶	Cost based on staff activity data attributable to clients for an episode of care (various lengths)	Adults: R² = 13.2%, $R^{2}$ (outliers trimmed) = 14.5%Child/youth: R² = 12.9%, $R^{2}$ (outliers trimmed) = 14.2%Combined: R² = 13.5%, $R^{2}$ (outliers trimmed) = 15.1%
PsyCMS⁴⁸	Total annualized inpatient and outpatient cost of mental health and substance abuse care	R² (retrospective) = 11.2%, R² (prospective) = 6.4%
Service-Connected Disability⁴²	Annual direct and indirect costs of outpatient care	R² = 1.6%
Service-Connected Disability and GAF⁴²	Annual direct and indirect costs of outpatient care	R² = 2.5%
SMI⁴⁴	Difference between reimbursement based on average cost vs case-mix adjusted rates	Difference range = −40.0% (approx. –US$700 000) to 30% (approx. US$1 000 000)
VA-MH12⁴⁸	Total annualized inpatient and outpatient cost of mental health and substance abuse care	$R^{2}$ (retrospective) = 9.6%, $R^{2}$ (prospective) = 5.9%

Abbreviations: ACG, Ambulatory Care Groups; ADG, Ambulatory Diagnostic Groups; AMHCC, Australia Mental Health Care Classification; CDPS, Chronic Illness and Disability Payment System; DCG, Diagnostic Cost Group; GAF, Global Assessment of Functioning; HCC, Hierarchical Condition Category; LSP, Life Skills Profile; MH-CASC, Mental Health Classification and Service Costs; NZ-CAOS, New Zealand Mental Health Classification and Outcomes Study; SMI, Serious Mental Illness; URG, Utilization Risk Groups.

For the direct measures, all studies used episodic basis for their resource use measures, which summed all the relevant costs over an episode of care. Only two studies attempted to define episodes of care that were variable based on the group or case,^46,50 while the others pre-defined a fixed episode length for the entire sample. There were also a wide range of follow-up times used for measuring resource use (Tables 2 and 3), ranging from a few weeks to up to 3 years. Alternatively, another option is to calculate a direct resource use measure on a per-diem basis, which predicts resource use per day or per visit,⁵ such as the System for Classification of In-Patient Psychiatry (SCIPP) developed in Canada.⁵⁴

The measures of resource use could also be expressed as a continuous variable or a categorical variable. As a result, there were also various performance metrics used to evaluate the case-mix classification systems, but most common was the coefficient of determination (R²) for the measures of resource use expressed as a continuous variable (Tables 2 and 3). The R² was sometimes referred to as the reduction in variance (RIV), or the amount of variance in resource use explained by the case-mix classification system. Although the R² was commonly reported, the differences in the measures of resource use and follow-up duration did not allow for a meaningful comparison.

Since the distribution of the resource use was often positively skewed, some studies attempted to approximate a symmetric distribution with a log transformation^33,36 (Table 3). Some studies also trimmed the outliers to improve their predictive performance^14,46 (Table 3).

There were also other notable case-mix classification systems currently being used, where activity-based funding has been implemented, such as the Netherland’s Zorgzwaartepakketten (ZZP) and the UK’s Mental Health Clustering Tool (MHCT) but, to our knowledge, these did not have empirical results regarding their predictive performance. The ZZP has 38 psychosocial care packages which classifies all ages based on psychosocial or cognitive functioning, social skills, mobility, activities of daily living, and behavioral problems.¹⁷ The MHCT has 21 groups which used the Health of the Nation Outcome Scales (HoNOS)³⁷ as input, then classifies adults using diagnosis, severity, chronicity, and cognitive impairment.⁵⁵ An earlier version of the MHCT with 13 groups reported an R² = 10.9%.⁵⁶

Discussion

Principal results

A modest number of studies examined case-mix classification systems to predict mental health care resource use in the community settings. A direct comparison in terms of predictive performance was not possible due to the variation in the measures of resource use, the follow-up duration, and performance metrics. In general, it can be said that the large majority of the variation in community mental health resource use was still not accounted for by these case-mix classification systems.

Although most research on this topic came from the Unites States, the Australian system (AMHCC) was most comprehensive, covering all ages and care settings (inpatient and community settings).⁵⁰ The most recent refinement was the addition of five phases of care (assessment only, acute, functional gain, intensive extended, and consolidating gain) which reflects the goal of care.⁵⁰ These phases of care are intended to serve as a proxy for a person’s health care needs and, by extension, a person’s expected resource use driven by health care needs.

Input variables

It is worth acknowledging that when a case-mix classification system is used in a funding formula, it must ensure that resources are allocated equitably. Therefore, whether a variable should be a case-mix input variable is an important consideration. In the literature, the variables used for classification were often grouped into only a few categories such as demographics, diagnosis, clinical status, or treatment variables. Discussions regarding their appropriateness as case-mix variables were also rare. Using an alternative classification of these variables, this study summarized the scope of case-mix variables used in the literature and discussed how case-mix variables can influence funding allocation (Table 4).

Table 4.

Input variables and their alternative case-mix classifications.

Variable	Number of models	Needs	Individual	Provider	Process	Historical
Diagnosis	22/32	x	x
Age	12/32		x
Health conditions	10/32	x	x
Social relations	9/32	x	x
Mental status	8/32	x	x
Gender	7/32		x
Harm to self or others	7/32	x	x
Functional status	6/32	x	x
Substance use	6/32	x	x
Behavior	6/32	x	x
Service history	4/32		x			x
Medication usage	4/32	x	x
Legal status	4/32		x
Treatments	3/32				x
Roles functioning and finances	3/32	x	x
Care settings	3/32			x
Cognition	3/32	x	x
Living conditions	3/32	x	x
Insurance benefits	3/32		x
Veterans status	2/32		x			x
Ethnicity	1/32		x
Communication and vision	1/32	x	x
Stress and trauma	1/32	x	x			x

Needs Variables

Variables that indicate the level of health care needs are those that not only have high explained variance of the resource use, but should also be variables that directly drive the resource use. For example, ethnicity⁴⁶ and gender^29,31,48 may have high correlation with resource use, but such correlation may be confounded by other factors such as systematic marginalization in the society that can make someone more vulnerable to mental health disorders and, by extension, to have higher expected level of resource use. Therefore, future research should consider variables that directly drive resource use, such as diagnosis, functional status or severity of illness, instead of those that simply correlate with resource use.

Individual vs Provider Variables

Provider variables, in essence, describe why it costs more in one facility compared with another, regardless of the person’s health care needs. For example, these can be care setting, facility type, regional characteristics, staff qualifications, or teaching status. Using these variables as case-mix variables essentially reinforces the systematic inequalities that exist among the providers. Therefore, using variables related to the individuals, whenever possible, may help avoid this reinforcement. However, in some cases, reinforcing systematic inequalities may be desirable, such as adjusting for facilities located in rural areas where resources and supplies may cost more to be delivered. As mentioned, these may also be adjusted for in a funding formula as external factors, so that the case-mix system maintains its focus on the health care needs. Only the case-mix classification systems from Australia and New Zealand used care setting as a case-mix variable, but they were used as the first split to essentially join 2 separate case-mix systems for inpatient settings and community settings together.^14,46,50

Process Variables

Process variables are those that describe treatments or services given to a person.^14,46,50,51 When using treatments or services as case-mix variables, they may encourage providers to do more of them for financial gain if they are under the control of the providers. Similar to provider variables, consideration should be given to whether variables that describe the needs of the individuals should be used as much as possible, or if there is a valid rationale for reinforcing differences in such variables (eg, to incentivize certain practices).

Historical Variables

Variables that describe historical use of services or treatments provided can be viewed as proxies for historical needs, such as prior hospitalization,²⁶ or usage in a prior year.³⁶ The shortcoming of these variables is that they have limited ability to be modifiable and change with current needs. On the other hand, there are historical variables that are continued to be relevant to current needs, eg, past history of abuse or violence.¹⁴ Historical variables therefore should not be entirely discounted, but the important consideration is whether historical variables have long-term relevance in describing a person’s current health care needs, or whether another variable that is more dynamic and could change with a person’s health care needs may be more appropriate.

Ambiguity of Variables

Ambiguity may arise if the variables chosen to describe the patient type result in more than one way to classify an individual. This ambiguity may give providers an incentive to choose the classification that maximizes the reimbursement, especially if the differences in the expected resource use or reimbursement of the possible classifications are significant. Given the same input, a good case-mix system should be able to consistently classify a person to only one group.

Studies in this review primarily obtained their input data from administrative sources or clinical assessment data. While administrative datasets offered convenience, they may only contain a few variables of health care needs, such as diagnosis codes. Instead, it may be more advantageous to use clinical assessment data designed to measure needs as part of the care plan creation. However, not all clinical assessments are created equal. Future research should consider the assessment that appropriately matches the clinical context (eg, mental health specific vs general medical assessment, inpatient vs community, children vs adults), and as comprehensive as possible to capture: needs, individual, provider, process, and historical variables describing an individual.

Output variables

The use of proxy measures of resource use was common in this review, such as the number of visits or appointments (Table 2). In fact, one of the first case-mix classification systems, DRG, used length of stay as a proxy for an inpatient episode’s cost.²⁵ This approach assumed that costs of care do not vary day-to-day during the hospitalization.⁵⁴

Similarly, for direct measures of resource use, when assuming that the costs of care do not vary day-to-day or visit-to-visit, it is also possible to calculate the costs of care for a particular case on a per-diem basis by multiplying the number of days/visits with expected cost per day/visit. However, the studies using direct measures of resource use found in this review all calculated costs of care on an episodic basis with a pre-defined follow-up length (Table 3). In an analysis from Australia, the preferred method of predicting resource use in community settings was a pre-defined episode with fixed length, due to the chronic nature of mental health care and intermittent provision of community-based services.¹⁴ This differs from the continuous service consumption of inpatient settings.

The class of direct measures of resource use can be further divided into billed costs (eg, claims data) or observed costs (eg, staff time study). Billed costs have three main limitations.⁵⁴ First, they often include non-clinical administrative costs (such as management, claims department), which could reduce the variance in the resource use measure if the administrative costs are high relative to costs of clinical care.⁵⁴ Second, sometimes billed costs are derived by averaging over a large number of patients rather than the actual amount an individual patient consumed,⁵⁴ which could also reduce the variance in the resource use measure. Third, additional variance can be added if there is a lot of variation in accounting practice across different organizations.⁵⁴ On the other hand, observed costs like staff time activities are more likely to closely match the actual resource consumption by individual patients and potentially more responsive to patients’ characteristics,⁵⁴ but these may be harder to obtain than available administrative data.

Gaps in current research

There are many available case-mix systems that were developed for inpatient settings but were not tested for community settings. Creating a case-mix system is not a trivial process; however, considerable progress can be made by experimenting with existing case-mix systems developed for use in another setting. For example, Canada’s SCIPP is a good candidate for testing in community settings. This system was able to explain 26.3% of variance in inpatient psychiatry cost, which is higher than most of the identified case-mix systems.⁵⁴

It has been shown that children and adolescents also have unmet mental health care needs.⁵⁷ Most of the studies only focused on adult populations (Table 1). Therefore, future case-mix classification systems should also consider children and adolescent populations in the development of new case-mix systems.

Only three of the studies cross-validated the predictive performance of their systems on a different data set than the one used for model derivation.^45,48,51 Cross-validation can serve two purposes: to evaluate the generalizability of the model on unseen observations or future users of the health care system, and to compare competing models.⁵⁸ Future research should consider using cross-validation when evaluating the predictive performance because the uncross-validated performance metric may give an overestimation.

Finally, it was not always clear if there exists a process or mechanism for updating the case-mix systems and exchanging knowledge. The pattern of health care resource consumption could potentially change between pre- and post-implementation of case-mix funding. Therefore, it is important to have a robust feedback loop by conducting more replication studies to validate case-mix systems under different conditions, as new data become available if using administrative data, or with more participating sites and over different time periods if using staff time activity data. For example, Australia has an organization dedicated to continuous improvement of case-mix classification systems with more replication studies planned.⁵⁰

Limitations

This study was not without limitations. First, this study only examined articles written in English, which also limited our review to only English-speaking jurisdictions. Second, the future of case mix classification systems within funding formulas was not clear from this review. This study did not consider the implementation outcomes and policy impacts of the identified case-mix systems, which deserves a separate review in the future.

Conclusions

This study provided a summary of the scope of research in community mental health care case-mix classification. The research activity was modest, while the transition from institutionalization to community care continues to evolve. Consideration should be given to appropriateness and assumptions of the case-mix variables, resource use measure, and evaluation of predictive performance. More research, especially of replication type, is needed in community mental health to ensure resources are meeting the needs of the population as new data become available and as the health care system evolves overtime. The introduction of standardized assessment systems into community mental health services should be considered as a foundational step toward establishing a pervasive common data source for clinical variables to be used in community-based case-mix systems.

Footnotes

Funding:

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: NT was supported by the Ontario Trillium Scholarship by the Government of Ontario, Canada. This research was also partially supported by interRAI Canada. The opinions expressed were only of the authors.

Declaration of Conflicting Interests:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Author Contributions

NT and JWP created the search criteria. NT, CP, and JPH contributed to the literature search. NT performed the primary screening. All authors contributed to the writing of the manscript and approved the final manuscript.

References

Vigo

Thornicroft

Atun

Estimating the true global burden of mental illness. Lancet Psychiatry. 2016;3:171–178.

Hewlett

Moran

Making mental health count: the social and economic costs of neglecting mental health care. https://www.oecd-ilibrary.org/content/publication/9789264208445-en. Updated 2014.

Knapp

Economic evaluation and mental health: sparse past. J Ment Health Policy Econ. 1999;2:163–167.

Sanderson

Mountney

Lees

Casemix for All. London, England: CRC Press; 2018.

Costa

Poss

McKillop

Contemplating case mix: a primer on case mix classification and management. Healthc Manage Forum. 2015;28:12–15.

Hornbrook

MC.

Hospital case mix: its definition, measurement and use: part I. The conceptual framework.Med Care Rev. 1982;39:1–43.

Fries

Schneider

Foley

Gavazzi

Burke

Cornelius

Refining a case-mix measure for nursing homes: resource utilization groups (RUG-III). Med Care. 1994;32:668–685.

Centers for Medicare Medicaid Services. OASIS User Manuals. Baltimore, MD: Centers for Medicare Medicaid Services; 2014.

Fries

BE.

Comparing case-mix systems for nursing home payment. Health Care Financ Rev. 1990;11:103–119.

10.

Sutherland

Repin

Crump

RT.

Reviewing the potential roles of financial incentives for funding healthcare in Canada. https://www.cfhi-fcass.ca/Libraries/Reports/Reviewing-Financial-Incentives-Sutherland-E.sflb.ashx. Updated 2012.

11.

Mechanic

Rochefort

DA.

Deinstitutionalization: an appraisal of reform. Annu Rev Sociol. 1990;16:301–327.

12.

Grob

GN.

Community mental health policy in America: lessons learned. Isr J Psychiatry Relat Sci. 2016;53:6–13.

13.

Lin

Coldefy

et al . Medical practice variations in mental health and addictions care. Med Pract Variat. 2016:1–41.

14.

Buckingham

Burgess

Solomon

et al . Developing a casemix classification for mental health services: summary. https://www.amhocn.org/sites/default/files/publication_files/mh-casc_summary.pdf. Updated 1998.

15.

Jones

Amaddeo

Barbui

Tansella

Predicting costs of mental health care: a critical literature review. Psychol Med. 2007;37:467–477.

16.

Hermann

Rollins

Chan

JA.

Risk-adjusting outcomes of mental health and substance-related care: a review of the literature. Harv Rev Psychiatry. 2007;15:52–69.

17.

Mason

Goddard

Payment by results in mental health: a review of the international literature and an economic assessment of the approach in the English NHS. http://eprints.whiterose.ac.uk/139377/. Updated 2009.

18.

Harris

Legge

Diminic

et al . Mental health service cost drivers: an international literature review. Final report for stage B of the definition and cost drivers for mental health services project. Volume 2. https://espace.library.uq.edu.au/view/UQ:344069. Updated 2013.

19.

Arksey

O’Malley

Scoping studies: towards a methodological framework. Int J Social Res Methodol. 2005;8:19–32.

20.

Tricco

Lillie

Zarin

et al . Prisma extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–473.

21.

World Health Organization. The World Health Report 2000—Health Systems: Improving Performance. Geneva, Switzerland: World Health Organization; 2000.

22.

Dunn

Mirandola

Amaddeo

Tansella

Describing, explaining or predicting mental health care costs: a guide to regression models: methodological review. Br J Psychiatry. 2003;183:398–404.

23.

Mirandola

Amaddeo

Dunn

Tansella

The effect of previous psychiatric history on the cost of care: a comparison of various regression models. Acta Psychiatr Scand. 2004;109:132–139.

24.

Wood

Beardmore

DF.

Prospective payment for outpatient mental health services: evaluation of diagnosis-related groups. Community Ment Health J. 1986;22:286–293.

25.

Fetter

Shin

Freeman

Averill

Thompson

JD.

Case mix definition by diagnosis-related groups. Med Care. 1980;18:iii, 1–53.

26.

Wittman

Lerner

Classification system for the chronic mentally ill outpatient. Isr J Psychiatry Relat Sci. 1990;27:17–28.

27.

Barker

Barron

McFarland

et al . A Community Ability Scale for chronically mentally ill consumers: part I. Reliability and validity. Community Ment Health J. 1994;30:363–383.

28.

Uehara

Smukler

Newman

FL.

Linking resource use to consumer level of need: field test of the level of need-care assessment (LONCA) method. J Consult Clin Psychol. 1994;62:695–709.

29.

Ettner

Notman

EH.

How well do ambulatory care groups predict expenditures on mental health and substance abuse patients. Adm Policy Ment Health. 1997;24:339–357.

30.

Weiner

Starfield

Steinwachs

Mumford

LM.

Development and application of a population-oriented measure of ambulatory care case-mix. Med Care. 1991;29:452–472.

31.

Ettner

Frank

McGuire

Newhouse

Notman

EH.

Risk adjustment of mental health and substance abuse payments. Inquiry. 1998;35:223–239.

32.

Ash

Porell

Gruenberg

Sawitz

Beiser

Adjusting Medicare capitation payments using prior hospitalization data. Health Care Financ Rev. 1989;10:17–29.

33.

Trauer

Duckmanton

Chiu

Economic grand rounds: estimation of costs of public psychiatric treatment. Psychiatr Serv. 1998;49:440–442.

34.

Rosen

Hadzi-Pavlovic

Parker

The life skills profile: a measure assessing function and disability in schizophrenia. Schizophr Bull. 1989;15:325–337.

35.

Trauer

Duckmanton

Chiu

The life skills profile: a study of its psychometric properties. Aust N Z J Psychiatry. 1995;29:492–499.

36.

Samuels

Risky Business: Classifying Mental Health Consumers for Capitation Reimbursement Under Managed Care [PhD Thesis]. New York, NY: New York University; 1996.

37.

Wing

Curtis

Beevor

HoNOS: Health of the Nation Outcome Scales: Report on Research and Development, July 1993–December 1995. London, England: Royal College of Psychiatrists; 1996.

38.

Coombs

Australian mental health outcomes and classification network: training manual adult ambulatory. https://www.amhocn.org/sites/default/files/publication_files/adult_ambulatory_manual_0.pdf. Updated 2005.

39.

Gowers

Harrington

Whitton

et al . Brief scale for measuring the outcomes of emotional and behavioural disorders in children: health of the nation outcome scales for children and adolescents (HoNOSCA). Br J Psychiatry. 1999;174:413–416.

40.

Shaffer

Gould

Brasic

et al . A Children’s Global Assessment Scale (CGAS). Arch Gen Psychiatry. 1983;40:1228–1231.

41.

Department of Health and Ageing, Quality and Effectiveness Section, Mental Health Suicide Prevention Branch. Mental health national outcomes and casemix collection: overview of clinician rated and consumer self-report measures and data items version 1.50. https://www.amhocn.org/sites/default/files/publication_files/mhnocc_overview_doha.pdf. Updated 2003.

42.

Leslie

Rosenheck

White

WD.

Capitated payments for mental health patients: a comparison of potential approaches in a public sector population. J Ment Health Policy Econ. 2000;3:35–44.

43.

Endicott

Spitzer

Fleiss

Cohen

The global assessment scale: a procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry. 1976;33:766–771.

44.

DeLiberty

Newman

Ward

EO.

Risk adjustment in the Hoosier assurance plan: impact on providers. J Behav Health Serv Res. 2001;28:301–318.

45.

Rosen

Loveland

Anderson

Hankin

Breckenridge

Berlowitz

DR.

Diagnostic cost groups (DCGs) and concurrent utilization among patients with substance abuse disorders. Health Serv Res. 2002;37:1079–1103.

46.

Gaines

Bower

Buckingham

et al . New Zealand mental health classification and outcomes study: final report. https://www.tepou.co.nz/resources/new-zealand-mental-health-classification-outcomes-study-caos-final-report/91. Updated 2003.

47.

Selim

Fincke

Ren

et al . Comorbidity assessments based on patient report: results from the veterans health study. J Ambul Care Manage. 2004;27:281–295.

48.

Sloan

Montez-Rath

Spiro

III et al . Development and validation of a psychiatric case-mix system. Med Care. 2006;44:568–580.

49.

Kronick

Gilmer

Dreyfus

Lee

Improving health-based payment for Medicaid beneficiaries: CDPS. Health Care Financ Rev. 2000;21:29–64.

50.

Independent Hospital Pricing Authority. Development of the Australian mental health care classification—public consultation paper 2. https://www.ihpa.gov.au/files/development-australian-mental-health-care-classification-public-consultation-paper-2-november. Updated 2015.

51.

Martin

Davies

Macdougall

et al . Developing a case mix classification for child and adolescent mental health services: the influence of presenting problems, complexity factors and service providers on number of appointments [published online ahead of print September 1, 2017]. J Ment Health. doi:10.1080/09638237.2017.1370631.

52.

National Institute for Health Care Excellence. Guidance list—clinical guidelines. www.nice.org.uk/guidance. Updated 2015.

53.

Averill

Goldfield

Wynn

et al . Design of a prospective payment patient classification system for ambulatory care. Health Care Financ Rev. 1993;15:71–100.

54.

Hirdes

Fries

Botz

et al . The System for Classification of In-Patient Psychiatry (SCIPP): A New Case-Mix Methodology for Mental Health. Waterloo, ON, Canada: University of Waterloo; 2002.

55.

Cohen-Tovée

Care Packages and Pathways/Payment by Results for Mental Health Services for Adults, Implications for Psychological Services. Leicester, UK: British Psychological Society; 2012.

56.

The Casemix Service. End Stage Report—Mental Health Casemix Classification Development. London, England: The Casemix Service; 2006.

57.

Patel

Flisher

Hetrick

McGorry

Mental health of young people: a global public-health challenge. Lancet. 2007;369:1302–1313.

58.

Browne

MW.

Cross-validation methods. J Mathemat Psychol. 2000;44:108–132.

Case-Mix Classification for Mental Health Care in Community Settings: A Scoping Review

Abstract

Keywords

Introduction

Methods

Results

Discussion

Principal results

Input variables

Needs Variables

Individual vs Provider Variables

Process Variables

Historical Variables

Ambiguity of Variables

Output variables

Gaps in current research

Limitations

Conclusions

Footnotes

Funding:

Declaration of Conflicting Interests:

Author Contributions

References