Abstract
As treatment for HIV improves, an ageing population is experiencing comorbidity which often leads to complex clinical presentations requiring an interdisciplinary care approach. This study sought to quantify clinician assessment of the level of clinical complexity, through the development of a rating scale for people living with HIV (PLHIV), to improve client care through an interdisciplinary care model. An existing alcohol and other drug complexity rating scale was selected and modified for use with PLHIV. HIV-specific items were included through consultation with an interdisciplinary team. A risk-prediction model was developed and validated using clinician ratings of clients attending The Albion Centre, a tertiary HIV clinic in Sydney, Australia, resulting in the development of the Clinical Complexity Rating Scale for HIV (CCRS-HIV). Multivariable logistic regression models identified eight characteristics based on clinician assessment of complexity in PLHIV: financial instability, social isolation, problematic crystal methamphetamine use, mental illness and/or other problematic substance use, cognitive/neurological impairment, polypharmacy, current hepatitis C infection and/or cancer, and other physical health comorbidity. A weighted risk-prediction model was developed and validated. The final model accurately predicted 85% of complex clients, with a sensitivity of 80% and specificity of 91%. This study developed an HIV-specific clinician-rated complexity scale. Further investigations are required to validate the CCRS-HIV with broader HIV populations. This simple complexity screening tool is a promising adjunct to clinical assessment to identify clients with complex physical and psychosocial needs who may benefit from interdisciplinary care interventions and allocation of resources.
Introduction
HIV medical treatment has advanced significantly, reshaping HIV management. These advances improve viral suppression, such that life expectancy for people living with HIV (PLHIV) is approaching that of the general population. 1 With the potential to manage HIV as a chronic health condition, comorbidity from other challenging chronic medical and psychosocial conditions can have a significant impact on retention in care and quality of life (QoL). 2
Concurrent chronic physical (e.g. other physical diagnoses and associated treatments) and psychosocial (e.g. psychological, social and economic circumstances, and stigma) conditions complicate client presentations, particularly within the public health domain. 3 HIV primarily impacts marginalised and often stigmatised communities where the mental health burden is higher than the general population,4,5 and HIV can compound this inequity. 6 Discrimination, stigma, poor mental health, substance misuse and QoL, and social isolation have all been shown to have a negative impact on HIV health outcomes, including poor adherence to antiretroviral therapy (ART), poor retention in care and mortality.7–12
Older age has also shown to be a significant factor affecting HIV outcomes and is associated with poorer clinical outcomes and a higher mortality rate, with a 16 times risk of death within first year of diagnosis in those aged 50 years or older. 13 Ageing in PLHIV is associated with social isolation, loneliness, stigma and increased risk of mental illness. 14 Identifying risk factors for comorbidity and frailty, and easy identification of client complexity, has become a clinical care priority. It is evident, therefore, that PLHIV, particularly those with past prolonged immunodeficiency, have complex care needs as they age.
Studies of client complexity in HIV care have suggested that primary care providers are better able to identify client complexity compared with rule-based systems, such as identifying the number of comorbid diagnoses. 3 Multifactorial interactions of comorbid diagnoses can impact a person’s capacity to effectively manage HIV infection, including adherence to ART or appointment attendance. Two issues arise from complexity issues within HIV care. First, clients who disengage from care have difficulties suppressing HIV and are at risk of life-threatening HIV complications 15 and transmitting the virus. Second, QoL can be adversely affected irrespective of HIV suppression if comorbidity issues are not addressed.
Current research reflects a general consensus that interdisciplinary care improves outcomes for clients with chronic and/or comorbid health conditions.16,17 The investigation of the clinical and economic effectiveness of interdisciplinary care for complex health conditions is an emerging research field. A recent Cochrane review 18 concluded that there is currently insufficient evidence to definitively interpret the impact of interdisciplinary interventions; however, compared with usual care, interdisciplinary care of complex clients can improve elements of disease control and service delivery at a modestly increased cost, 17 supporting the development of a holistically healthy population of PLHIV. In terms of the experience of interdisciplinary care, clients report enhanced satisfaction and improved health outcomes while health workers report enhanced job satisfaction, greater role clarity and enhanced well-being at work. 19 Much of the literature reviewed here was conducted in the United States, and there is a paucity of Australian-based research. Furthermore, there is no validated clinician-administered rating scale to identify clinical complexity in HIV.
Risk-prediction models offer a mechanism to identify risk factors and their role on potential outcomes. 20 To determine effective treatment plans, including developing an appropriate schedule of attendance, clinicians routinely make assessments about the degree of intervention and resource allocation required based on clinical complexity of clients. There is also benefit in proactively identifying potential risk factors rather than reacting once comorbidity has already impacted on HIV management. The primary aim of the present study was to quantify the components of specialised HIV clinician assessments, to develop and validate a simple screening tool. Clinicians determine treatment plans and appointment schedules based upon their assessment of clients’ health care needs and capacity to adhere to a regimen. Inherent in this assessment is an estimation of the degree and impact of client comorbidity and clinical complexity. This methodology has been used in previous studies 3 seeking to quantify clinician assessment to improve patient outcomes.
Methods
Development phase
The Albion Centre (Albion) cohort of 1125 regular PLHIV clients was divided into a development database (n = 189) and a validation database (n = 936). During the initial phase of the study, two attending medical officers (AMOs) at Albion assessed their clients who had attended for a medical review over a nine-month period. Both AMOs had over 20 years’ experience working with PLHIV. Each client was assessed as ‘complex’ or ‘not complex’ by their respective AMO, based on the clinician’s long-term knowledge of the clients. For the purpose of the present study, ‘complexity’ was defined as ‘the presence of comorbidity currently impairing functioning, and potentially warranting an interdisciplinary team response’.
An existing rating scale comprising nine items and used in drug and alcohol settings, the Clinical Complexity Rating Scale (CCRS), 21 was adapted. This scale had a mixed binary and non-binary scoring system and was not specifically assessing functional impairment; therefore, the scoring was modified to ensure it was binary, to have consistency across variables, and the definitions were adapted to specifically address the issue of functional impairment. Clinicians from different disciplines (medical, nursing and psychology) independently identified further HIV-specific items which were of conceptual interest. An additional eight HIV-specific items were added to form a composite scale. All 17 items are listed in Table 1.
List of variables used to assess clinical complexity in the present study.
CCRS: Clinical Complexity Rating Scale.
Of this cohort, 16% were considered ‘complex’ according to clinician assessment and were subsequently rated across the 17 variables (Table 1). Definitions of each variable can be found in Appendix 1. An age- and sex-matched ‘non-complex’ cohort was identified and rated in the same way.
The AMOs rated functional impairment according to each of the 17 variables as a binary code. Non-binary variables from the original CCRS scale were re-coded in the analyses, such that ‘0’ denoted ‘the variable did not cause functional impairment’ and ‘1’ denoted ‘the variable caused functional impairment’. The only exception to this was ‘polypharmacy’, for which clients received a score of ‘1’ solely for the presence of this factor (i.e. being prescribed at least five medications). For the purpose of this study, functional impairment was considered to be a loss of functional capacity, as a result of the variable in question, in the context of their everyday life. AMOs agreed that functional impairment was evident when any of the complexity items in question were inhibiting the client’s ability to fully engage in social, occupational or physical activities, or the AMO believed the extent to which the client could function in these domains was significantly reduced as a result of complexity. For example, AMOs agreed that mental health or substance use issues that impacted the client’s ability to regularly engage in employment or led to the repeated request for medical certificates was deemed ‘functional impairment’. Given scores can, and should, be modified over time in response to changing client characteristics, receiving a score of ‘1’ denoted functional impairment at the time of assessment.
Multivariable logistic regression models were used to create a predictive model and identify the key variables of interest. After fitting the univariate logistic regression models, variables were entered into the multivariate model if they had a p-value of less than 0.10 in the univariate analysis. The final multivariate model was obtained using a forward stepwise approach and included statistically significant covariates (p < 0.05). The resulting scale, the Clinical Complexity Rating Scale for HIV (CCRS-HIV; Appendix 2), comprised eight variables, four psychosocial and four physical health variables. The model’s goodness of fit was assessed using the Hosmer–Lemeshow goodness-of-fit test. 22 High p-values (>0.05) were considered a reasonable fit.
After the final model was identified, a weighted risk-prediction screening tool was developed by rounding each regression coefficient (β-coefficient) to the nearest integer and multiplying by 10. This scoring procedure has been used in multiple other studies, with statistically acceptable robustness.20,23 The total CCRS-HIV score for each client was derived by summing the individual weighted risk factor scores. There are precedents and rationales for the use of summary comorbidity measures which use weighted scores to assess the presence of comorbidity. 24
Statistically and clinically relevant ‘cut-offs’ to identify clinically complex individuals with acceptable robustness and accuracy were determined using the development dataset. The predictive power of these cut-offs was assessed using the validation dataset, using standard statistical measures to determine sensitivity/specificity and the area under the receiver-operating curve (AUC) to assess the accuracy of the model and the discriminative power of the risk-prediction screening tool.
Validation of the screening tool
The purpose of the validation phase of the study was to confirm whether factors identified during the development phase could still be considered significant predictors of the primary outcome of interest, clinical complexity. Following identification of the statistically and clinically relevant variables during the development phase, data for the remaining clients accessing Albion for medical care (n = 936) were collected using the same methodology described above. During this phase, the remaining AMOs within the service assessed their clients according to the variables identified.
Validation of the screening tool was conducted using the original eight variables and cut-offs determined during the development phase. All analyses were conducted using Stata statistical software v14.0 (College Station, TX, USA).
Results
The total sample (N = 1125) was comprised of 1024 (91%) males and 101 (9%) female clients of Albion. The mean age of clients attending Albion is 48 years (range 19–88 years). The profile of sex distribution shown here is consistent with other HIV services in Australia. Results from the coding across all 17 variables during the development phase (n = 189) are presented in Table 2. The distributions of these variables were analysed to provide unadjusted odds ratios to show the strength of the associations with the primary outcome, clinician-assessed ‘complexity’.
The distribution of all variables in the development dataset (shaded items denote HIV-specific items added to the original scale).
CMA: crystal methamphetamine; CVD: cardiovascular disease; HCV: hepatitis C virus.
aModel did not converge.
Deriving the complexity screening tool
Multivariable regression models were conducted during the development phase to determine the final model which included four psychosocial factors: financial instability (adjusted odds ratio [aOR]: 14.78, 95% CI: 2.62, 83.51); problematic crystal methamphetamine use (aOR: 7.05, 95% CI: 1.25, 39.65); mental illness and/or other problematic substance use (aOR: 4.22, 95% CI: 1.45, 12.27); and social isolation (aOR: 10.78, 95% CI: 2.47, 47.14), and four physical health factors: cognitive/neurological impairment (aOR: 6.19, 95% CI: 1.94, 19.81); polypharmacy (aOR: 4.27, 95% CI: 1.43, 12.76); current (problematic) hepatitis C infection and/or cancer (aOR: 5.65, 95% CI: 1.47, 21.79); and other physical health comorbidity (aOR: 5.33, 95% CI: 1.69, 16.76). The data from the development and validation phases for the resulting eight variables comprising the CCRS-HIV are outlined in Table 3, and the scale is provided in Appendix 2.
Results from multivariable logistic regression analyses for the development and validation datasets.
AUC: area under the receiver-operating curve; CMA: crystal methamphetamine; HCV: hepatitis C virus.
Re-fitting these variables for the remaining 936 participants during the validation phase was shown to have good predictive value. High p-values (0.325 and 0.182 for the development and validation models, respectively) indicated reasonable model fits for both development and validation cohorts.
Given the consistent results observed between the development and validation datasets, it was decided to maintain the model derived during the development phase as the final model, as this was validated during the second phase of the study.
Performance of the complexity screening tool
Analyses were conducted to assess the performance of the model for various cut-offs to identify complexity. The development and validation dataset results are summarised in Table 4 and show that high cut-offs resulted in high specificity, but relatively low sensitivity. On balance, a cut-off of 40 (total CCRS-HIV score) was used to determine ‘complex’ presentations with an acceptable level of statistical precision. Table 4 shows the final model to determine complexity, balancing appropriate sensitivity with specificity. During the validation phase of the study, the data were fitted according to the cut-offs determined during the development phase and were shown to have good predictive ability. The final model accurately predicted 85% of complex clients, with a sensitivity of 80% and specificity of 91%. The overall AUC was 95% (95% CI: 92%, 98%) for the final model, demonstrating high discriminative power.
Performance of the development and validation models (shaded line denotes chosen cut-off).
aAUC: area under the curve for score as a continuous variable.
Clinicians were consulted regarding the clinical utility of the scale to identify complexity and target interventions. Accordingly, a three-tiered ‘traffic light’ system was developed to categorise complexity. As per results in Table 4, a score of 40 or above was used to classify someone as ‘complex’ (Red) which determined that this cohort required additional interdisciplinary interventions, more frequent follow-ups and more frequently reviewed care plans. The remaining two tiers were developed to classify those as ‘not complex’ (Green, score 0–29), and those with ‘some complexity evident’ (Orange, score 30–39), requiring monitoring and care plans of lower intensity.
Discussion
The present study is the first, to our knowledge, to quantify experienced HIV clinician assessment of clinical complexity and determine the individual variables which comprise such an assessment. Higher scores on the CCRS-HIV were associated with increased prevalence of complexity, which provides a platform for a more coordinated, enhanced and targeted interdisciplinary care team approach. The scale also potentially provides the opportunity to separately identify physical and psychosocial comorbidity, which could be useful in directing interdisciplinary care.
PLHIV now have a similar life expectancy to that of the non-infected population due to ART but the incidence of non-AIDS comorbidity and malignancies amongst PLHIV is increasing. 25 Reasons include mental health and other psychosocial factors, ageing, ongoing immune activation and inflammation despite virological suppression, 26 high rates of modifiable risk factors (e.g. smoking) and the long term effects of ART. 27 In addition, PLHIV are vulnerable to stigma, social isolation and mental health issues. As Albion is publicly funded, it also has a significant number of clients who are from culturally and linguistically diverse (CALD) backgrounds, are isolated, homeless, and/or have mental health and/or alcohol and other drug problems.
Specific rates of non-AIDS comorbidity in our clinic population are published elsewhere 28 and are similar to the rates within the Australian HIV Observational Database (an observational cohort study of more than 4000 PLHIV across Australia under routine clinical care), including rates of psychological disorders such as depression (35%) and anxiety (18%). 29 This highlights that contemporary HIV management has moved away from treating the virus alone to encompassing the psychosocial and lifestyle factors of clients. Thus, it is important to have available a validated complexity screening tool to identify those clients who require more intense interdisciplinary care, support and follow-up. Not only does this attend to all the health needs of the client but it also feeds back into the maintenance of virological suppression by ensuring clients’ adherence to ART and prevention of transmission of HIV.
This study has developed a valid and standardised scale of clinician-rated complexity in PLHIV that can be used to guide interdisciplinary HIV management. The scale allows any member of the clinical team to detect complexity with accuracy similar to that of a specialist HIV physician. By identifying specific, measurable factors, the scale may further assist in the development of interventions that will potentially maximise retention in care. As all members of the clinical team can see a client’s complexity score, it provides more opportunity for interdisciplinary engagement. For example, if a client with a high score attends to see the Triage Nurse, their score would trigger an alert to other relevant clinicians; this allows more opportunistic engagement of vulnerable clients.
Specific potential intervention strategies are likely to include: (1) Tailored monitoring programmes guided by the specific factors identified for individual clients, (2) development of individualised interdisciplinary case management plans and (3) referral for specific medical and/or psychosocial interventions. The development of a risk-prediction model such as this has the benefit of minimising prediction error; it does not, however, provide evidence of a causal relationship between variables and the outcome which might be more useful in determining prevention and intervention strategies. Further research is needed to identify whether the combined use of this scale, and related intervention strategies, contribute to a significant increase in retention in care and other client outcomes. Arguably, effective use of such a scale may prevent negative outcomes of complexity by alerting the need for intensive care needs as soon as a client enters a service. In addition, the use of such scales in conjunction with improved interdisciplinary care coordination models should improve QoL and well-being.
To our knowledge this is the first complexity rating scale in HIV to be based on an Australian population from a large, publicly funded HIV clinic in metropolitan Sydney. The present clinical setting and diversity of client presentations lends itself to a more predictive and appropriate model of client complexity. The scale has now been incorporated in the state-wide electronic medical record, allowing clinicians to input data directly into the model during the client consultation, identifying areas of need that can be addressed by other members of the interdisciplinary team. This model can also be used to further quantify the impact of clinical interventions over time, for example to measure changes in a client’s complexity rating following an intervention. These are some of the advantages of quantifying complexity and progress within HIV management, as opposed to prior methods of qualitatively describing progress.
It is worth noting that similar biomedical models have been developed for the clinical assessment and management of HIV clients. Consequently, precedents do exist for this form of clinician-administered rating scale, 3 even to the extent of using summary (additive) comorbidity measures, 24 which some have suggested are not ideal. 30 An important and contemporary example of a clinical tool that is applicable to ageing PLHIV is the ‘frailty index’ validated by Guaraldi et al., 31 which predicts survival and risk factors for incident comorbidity that are independent of viral load and CD4 among PLHIV. Likewise, the Veterans Ageing Cohort Study (VACS) index 32 predicts mortality in a cohort of veteran PLHIV. The present study, however, is unique in its development of a tool to screen for complexity in PLHIV within Australia, and also for its inclusion of psychosocial as well as medical/physical health factors. This is especially important to develop a coordinated and collaborative interdisciplinary response to holistic care needs.
There are limitations to this research. First, the specific scoring criteria are not clearly defined for every item on the tool, which is largely a nominal measurement scale (i.e. requiring yes/no responses). For example, the exact clinician-reported threshold of what constitutes social isolation may not itself be uniform. Second, it is unclear how representative the sample population is in relation to the entire Australian HIV population, given that our clients are predominantly Caucasian men who have sex with men, living in metropolitan Sydney. However, Albion is one of the largest publicly funded HIV clinics in the state, with at least 40% of the service’s clients come from across the state and from different socioeconomic and CALD backgrounds. 28 It is therefore reasonable to infer the tool is sufficiently validated to be used throughout New South Wales.
A third limitation of the present methodology is that it is subjective in nature, inherent in quantifying clinician assessment. Comparing clinician ratings in specific domains against clinicians’ broad assessment of whether a client is ‘complex’ or ‘not complex’ can be argued to be problematic. This, however, is the reality of clinical consultations, where the clinician is required to make such assessments regarding a person’s complexity in order to determine a course of treatment, visit schedule and appropriate referrals. The scale developed in the present study aims to quantify such an assessment; however, future studies should seek to validate these assessments against other more objective measures of complexity. Likewise, clinician assessment of functional impairment is also subjective in the present study. Future research should consider more objective measure of a loss of functional capacity.
The scale primarily identifies accumulative scores of issues to represent complexity and may not account for other factors around clinician engagement, chronicity and acuteness of other comorbidity and available supports and resources. Complex trauma and other psychological conditions and their related behaviours may also be considered the actual drivers of the symptoms being presented as contributing to complexity. It is therefore possible that the present scale does not accurately assess all the relevant factors that impact PLHIV. Future modifications of the scale could seek to adapt and modify the variables assessed, as well as assessing the correlation between clinician and self (client) reported concerns.
In the present context, ratings relied upon prior knowledge of existing clients for clinicians to appropriately estimate the extent of their comorbidity. However, it is also possible for the scale to be utilised during an initial consult and can be used to target clinical assessment questions accordingly.
Conclusions
This study demonstrates promising results for the development and use of a complexity rating scale for PLHIV. Use of this scale to target clinical interventions, with an increasing move towards proactive early intervention for non-AIDS comorbidity, is an important next step. The tool, while validated in its current form, requires ongoing modification and review informed by evolving data. It does, however, provide clinicians with a simple and brief means of assessing the most relevant non-AIDS comorbidity, with a view to targeting resources and interventions accordingly, and can readily be inserted into an electronic medical record. What makes this model unique in the field of interdisciplinary HIV care is that it has potential advantages for both the individual (improvement of general health outcomes) and public health (enabling adherence to ART and thereby reducing onward transmission of HIV).
Footnotes
Acknowledgements
The authors would like to thank Tom Hoglund for extensive support and guidance throughout the process of this study, and Dr Peter Baldwin for initial statistical support and advice. In addition, we would like to acknowledge the clinical team at Albion, whose hard work and engagement throughout this process have been invaluable. The lead author is a 2019 recipient of the NSW Health Education and Training Institute (HETI) Mental Health Research Award. The Albion Centre is a NSW Health entity and WHO Collaborating Centre for capacity building on HIV and STI Care, treatment and support.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
This study has received quality assurance or quality improvement (QA/QI) approval from South East Sydney Local Health District Human Research Ethics Committee (HREC), as well as HREC approval from the University of Technology Sydney.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was partially supported by an unrestricted research grant from Gilead Sciences.
