Abstract
People with psychotic disorders are impaired by their unsatisfactory social and instrumental skills apart from their psychotic symptoms in pursuing age-appropriated goals [1, 2]. Unfortunately, the advantages of medication in helping them improve their social adjustment and thus obtain competitive employment have not been clearly shown [3]. Psychosocial intervention is an adjunct strategy to equip them for independent functioning and improve their subjective quality of life [4]. In this study, we refer psychosocial interventions to evidence-based practices which include vocational rehabilitation, family intervention, skills training, cognitive behavioural therapy and case management [5].
Compliance to treatment is an essential factor for therapeutic success for those with psychotic disorders [6]. Research suggests that many mental health consumers quitted prescribed psychosocial treatment before they completed the course of treatment [7, 8]. Non-compliance has been demonstrated to increase the rate of relapse and re-hospitalization which will then increase the clinical, social and economic costs [9]. Compliance behaviour of mental health consumers has therefore recently generated substantial interest among researchers.
Compliance is the extent to which a person's behaviour coincided with given medical advice [10]. Compliance behaviour can be explained by two theories: the role theory [11] and the health belief theory [12]. Role theory assumes that patient is possessed to sick role. Patient is incapable to manage his health problems alone, and is constrained to seek help from physician in following the rational instruction. Patient who does not comply with prescribed treatment is deemed to be irrational and responsible for the non-compliance [13]. Health belief theory however, alleges that health behaviour is the readiness to act. Compliance behaviour is based on the perception that health is being threatened and the threat can be reduced by a particular action. The weighing between the advantages of action and the negative consequence is the indicator for making decision to comply or not [10].
Treatment compliance is measured through attendance and appointment keeping [14–16], willingness to follow physicians' advice [17, 18], participation in treatment sessions [16, 19, 20], and motivation in joining recommended treatment [16]. Previous studies adopted one or two of these components in measuring treatment compliance. Those methods are not comprehensive enough to reflect the real compliance level. A psychometrically valid compliance scale for psychotic disorders does not exist in the literature and is therefore required. Although self-report measure of compliance is easy to administer, this method has the drawback of being inaccurate [21]. We aimed at developing a compliance scale which is to be completed by case managers.
Two theoretical models explain compliance behaviour in adhering to psychosocial treatment. We assume that target compliance behaviours should be manifested in certain circumstances according to perceived benefits of consumers. A list of target compliance behaviours in psychosocial treatment is to be generated.
The influential factors of treatment compliance can be differentiated into patient, treatment, environment and physician related domains [22]. The personal characteristics of consumers have a strong impact on treatment acceptance. The public stigma is so severe [23] that mental health consumers may avoid participating in psychiatric services to reduce the chance of being labelled [24, 25]. This problem is most prominent for those with high self-stigma, low self-esteem and low self-efficacy [26]. Self-esteem and self-efficacy should enhance treatment adherence [27]. The negative effect of those factors will undermine psychosocial treatment compliance [28]. Furthermore, poor insight on their mental illness may act as a mediating factor that leads to adverse effect on psychosocial treatment compliance [16]. This explains why insight plays such an important role in the course of mental illness [29]. Similarly, negative attitudes of consumers towards prescribed treatment may cause medication non-compliance [30].
This study aimed at developing a scale for people with psychotic disorders to measure their compliance as to psychosocial interventions and evaluating its psychometric properties.
Method
Item selection and scale development
Follow standard procedures on scale development [31–33], item pool was generated by literature review and input from related professionals. Computerized search of CINAHL, MEDLINE, PsycINFO, and Social Sciences Citation Index from January 1966 to March 2004 using key words ‘compliance’, ‘treatment’, and ‘mental illness’ generated 13 relevant items. Another 12 items were collected from mental health experts. After initial screening, 25 items for measuring psychosocial treatment compliance were generated.
A 5-point Likert summated rating scale [33] ranging from Never (1) to Always (5) was used to record the compliance behaviours. The Psychosocial Treatment Compliance Scale (PTCS) was written in English to improve its practicability to English-speaking countries including Australia and New Zealand. This scale should be easily administered by informants who have a good working knowledge on English. The actual compliance behaviour of participants is scored according to the day-to-day observation by their case managers.
Two seasoned PhD level researchers who have had extensive experiences in developing tools for mental health consumers and a master level research student were involved to assess the content validity and cultural relevancy of this scale. To improve the content validity of the PTCS, some items were deleted because of duplication. Some wordings of the items were revised according to their advice. This resulted in the 17-item PTCS.
Another three experienced mental health professionals who were experienced in providing psychosocial treatment were invited to join the expert panel at this stage and examine the content validity and cultural relevancy of the scale. This expanded the panel to be consisting of six members. The method of content validity ratio (CVR) [34] was used to determine whether the items were valid and culturally neutral. The CVR for each of the items that we obtained was 1.00 which reached the critical value suggested by Lawshe [34] and indicated that the scale did not consist of any culturally sensitive items. The final version of the PTCS was then developed.
Participants
A total of 108 mental health consumers were randomly recruited from Kwai Chung Hospital and her clustering units including South Kwai Chung Psychiatric Centre, East Kowloon Psychiatric Centre and Yaumatei Psychiatric Centre, and Lai Kwan Day Training Centre of Baptist Oi Kwan Social Service between July 2004 and February 2005. Kwai Chung Hospital is one of the two mental hospitals in Hong Kong which provides 1472 psychiatric beds and mainly serves adults with acute or chronic mental illness. The mean age of participants was 38.47 (SD = 8.13), and 47.2% of them were male. Most of them were single (76.9%) and they were either inpatients (42.6%), day-patients (46.3%), outpatients (3.7%), or recipients of other community psychiatric services (7.4%). Their average length of stay in psychiatric hospital was 24.89 months (SD = 31.99 months), and their number of previous admission was 4.03 times (SD = 4.16 times). All of them had received a major DSM-IV diagnosis of psychosis given including schizophrenia (79.6%), schizoaffective disorder (3.7%), depressive disorder (6.5%), bipolar affective disorder (8.3%) and delusional disorder (1.9%) according to the medical records filled out by certified psychiatrists in the mental hospital. All of them were currently taking psychotrophic medication. Their symptoms were provided by case therapists based on participants' medical records, their day-to-day clinical assessment and observation. They included hallucination (18.5%), delusion (12.0%), blunt affect (24.1%), avolition (20.4%), anhedonia (12%), poor attention (12.0%), poor social relationship (17.6%) and poor social judgement (20.4%). They obtained at least primarily education and had received psychosocial treatment for the past 3 months which included vocational rehabilitation, social skills training, cognitive behavioural therapy and family intervention. The participants were engaged in one or more types of psychosocial treatment. They were free from developmental disabilities, dementia, substance abuse of illicit drug and alcohol and hearing impairment. Eighteen qualified case occupational therapists who were fully aware of the participants' psychosocial treatment compliance for the past 3 months were invited to complete the PTCS. A training session on the administration of questionnaire was offered by the research personnel to make sure that they understood how the scale should be completed. The training consisted of instruction and discussion on the key administration procedure of the questionnaire. Queries from the test administers were addressed as to how the scale should be used. All the case therapists have undergone this process. On the whole, they expressed at the end of the training that the instruction of completing the form was easy to understand and implement. They had an average of 8.83 years (SD = 4.21) of experiences working with mental health consumers and they had provided psychosocial treatment on average to participants for 11.83 months (SD = 11.36).
Instruments
The 17-item PTCS was used as the main instrument for the psychometric study. It rates on 5-point Likert scale with 1 meaning ‘Never’, 2 meaning ‘Infrequently’, 3 meaning ‘Sometimes’, 4 meaning ‘Frequently’ and 5 meaning ‘Always’. Higher scores represent better psychosocial treatment compliance. This scale is attached in Appendix I. Its psychometric properties were to be evaluated by this study.
The Self-stigma of Mental Illness Scale (SSMIS) [35] was used for measuring perceived discrimination and self-stigma for the participants. Perceived discrimination was measured through the subscale of ‘stereotype awareness’. Self-stigma, as a three-level process, was measured via the three corresponding subscales on ‘stereotype agreement’, ‘self-concurrence’ and ‘self-esteem decrement’. Each of the four subscales contains 15 items rated on a 9-point Likert scale. The internal consistency and test–retest reliability of the four subscales ranged from 0.64 to 0.87 and 0.62 to 0.75 respectively. The construct validity of this scale has been satisfactorily demonstrated.
The Rosenberg Self-esteem Scale [36] was adopted to assess global self-esteem. It is a reliable and valid tool [37] which consists of 10 items rated on a 4-point Likert scale. The test–retest reliability (0.82–0.88) and internal consistency (0.77–0.88) for this scale were of good statistical quality [37, 38].
The Self-efficacy Scale [39] consists of 23 items in assessing general self-efficacy (17 items) and social self-efficacy (6 items). Satisfactory internal consistency was demonstrated for the general domain (α = 0.86) and the social domain (α = 0.71).
The Scale to Assess Unawareness of Mental Disorder (SUMD) [40] is a reliable and valid tool in assessing discrete and global aspects of insight of people with psychotic disorders. It contains three general global questions on insight and 17 items on awareness and attribution of specific signs and symptoms for both current and past time period. The interrater reliability and test–retest reliability for the first general item were 0.95 and 0.62 respectively. The median interrater reliability and test–retest reliability were 0.89 and 0.51, respectively, for other insight items [41].
The Drug Attitude Inventory-10 (DAI-10) [42] is a 10-item measure on subjective experiences of neuroleptic medication. It has good internal consistency and test–retest reliability.
The Rating of Medication Influences (ROMI) [43] has seven items reporting the reasons of medication compliance and 13 items reporting medication non-compliance by a 3-point Likert scale. The kappa for the items ranged from 0.63 to 1.00, and the internal consistencies of reason for medication compliance and non-compliance at admission were 0.57 and 0.55 respectively.
Translation and Cultural Relevance of Instruments. All instruments used in this study were assessed for cultural relevance by two PhD level researchers. It was concluded that all instruments used were culturally relevant for use in Hong Kong. The instruments were then translated into Chinese and back translated into English by two independent translators. The Chinese version was finalized by reconciliating the original and translated English versions. Pilot studies were conducted to make sure all instrument items were clear and understandable to the target participants of this study.
Data collection
Written consent from each participant was obtained before data collection began. The PTCS, demographic data, and signs and symptoms of each participant were completed and recorded by the case therapist. The PTCS was completed retrospectively based on observations of the participant's compliance behaviours in various psychosocial treatments for the past 3 months. The scoring was based on individuals' compliance behaviours as to their fulfilment with our standard expectation in psychosocial treatment. Two research assistants who were qualified occupational therapists completed the remaining scales through interviews with the participants. All scales were administered again by single raters within 1–2 weeks after the first administration to assess their test–retest reliability for the first 31 participants. The remaining participants were only required to answer the questionnaires once.
Data analysis
The data were analysed by the SPSS version 11.0. All analyses were done based on the sample consisting of the 108 participants, except for the analysis of test–retest reliability (n = 31). Descriptive and frequency statistics were used to summarize the demographic data of the participants and informants. Skewness and kurtosis were used to test the normality of the mean scores of all measurements. Z-score was used to detect the presence of outliers. Intraclass correlation coefficient and Cronbach's coefficient alpha (α) were used to assess the test–retest reliability and internal consistency of the PTCS respectively. Factor analysis was performed to improve our understanding on the structural validity of the scale. As pointed out by Portney and Watkins [44], this analytic method can superimpose the underlying complex phenomena of an abstract construct such as compliance to psychosocial treatment. The Kaiser-Meyer-Oklin value [45] and Bartlett's Test of Sphericity [46] were used to test whether the dataset was suitable for factor analysis. Exploratory factor analysis with varimax rotation was used to explore the interrelationships among the items of the scale. The Kaiser-Guttman criterion [47] and the Cattell's scree test [48] were applied to retain factors for investigation. The relationships between psychosocial treatment compliance and self-stigma, self-esteem, self-efficacy, insight and attitudes to medication were examined by Pearson product-moment coefficient of correlation. To exclude the possibility that heterogeneity of diagnostic groups and treatment groups influenced the variance of the PTCS, t-test was performed for comparison. ANOVA was not appropriate for the analysis, as there was a marked uneven distribution of samples in different groups. Hence, for the diagnostic groups, the schizophrenia group (n = 86) was compared with the group of schizophrenia plus other diagnoses (n = 108). According to the similarity of service utilization, the groups of day-patient, outpatient and others were combined (n = 62), and compared with the group of inpatients (n = 46).
Results
Normality of data and outliers
The normality of the mean of sample distribution of all measurements was assumed. The results of skewness and kurtosis ranged from − 0.701 to 1.154 and − 0.932 to 0.840, respectively, which were within the satisfactory limit. By using the criterion of z-score of 3.29 [49], no univariate outliner was detected.
Factor structure
The data were found to be suitable for factor analysis by the Kaiser-Meyer-Oklin value (0.928) and the Barlett's Test of Sphericity (0.000). Principal components analysis with varimax rotation of PTCS revealed two factors with eigenvalue greater than 1. Catell's scree test also suggested a two-factor solution which accounted for 70.74% of the total variance with factor one and two explaining 40.08% and 30.66% of the variance respectively. The finalized items, factor structure and the correlation between items and their corresponding factor are shown in Table 1.
Factor loadings of PTCS items and correlation between items and the corresponding factors
∗p < 0.01. PTCS, Psychosocial treatment compliance scale.
Factor 1: participation
Twelve items belonged to this factor. This fact was related to their participation in psychosocial treatment such as cooperating with others, following instructions and seeking for improvement. The internal consistency of this factor was 0.96 and the test–retest reliability was 0.90. The mean scores of the items ranged from 3.31 to 4.04 (SD = 0.76–0.98).
Factor 2: attendance
There were five items belonging to this factor. Attendance and punctuation were the main concerns for this factor. The internal consistency was 0.87 and the test–retest reliability was 0.86. The mean scores of the items ranged from 3.62 to 4.22 (SD = 0.74–0.95).
The factor loading for most of the items exceeded 0.63 which may statistically be considered very good [50]. Although the factor structure is on the whole clear and easy to interpret using the two-factor solution, some items appeared to be equivocal in nature. Item 6 (active participation), item 7 (being attentive) and item 15 (avoiding premature termination) loaded evenly on the two factors. We followed Kim and Mueller's rule [51] that these three items were allocated based on their underlying meaning rather than factor loadings. As such, we included items 6 and 7 to factor 1 whereas item 15 to factor 2. By the same token, we allocated item 4 to the factor ‘participation’ although it had a high loading on ‘attendance’. After these allocations, internal consistency of the two factors and hence their alpha values did not differ significantly which still stood at the excellent level.
Correlations with other measures
The correlations between the PTCS and the measurements of selfstigma, self-esteem, self-efficacy, insight and attitudes to medication are shown in Table 2. Statistical significant associations were found on all pairs, except for the correlations between the ‘participation’ of PTCS and the ‘stereotype awareness’ of SSMIS, and the ‘current attribution of symptoms’ of SUMD and the two subscales of PTCS.
Correlations between psychosocial treatment compliance and self-stigma, self-esteem, self-efficacy, insight and attitude toward medication
∗p < 0.05, ∗∗p < 0.01. PTCS, Psychosocial treatment compliance scale.
Comparison of compliance for different diagnostic groups
It was shown by the t-statistics that there were no significant differences between the different diagnostic groups (schizophrenia vs all diagnoses: participation, t(192) = −1.230, p = 0.220; attendance, t(192) = −1.067, p = 0.287) and the different modes of service utilization (inpatients vs day-patients, outpatients and other patients: participation, t(106) = 1.165, p = 0.246; attendance, t(106) = 0.458, p = 0.648) in psychosocial treatment compliance.
Discussion
Reliability and structural validity
The excellent internal consistency and test–retest reliability of the two subscales of PTCS suggest that the items of the subscales are measuring the same construct and are reliable as far as stability over time is concerned.
‘Participation’ is the pivotal element of psychosocial treatment compliance which refers to cooperating and engaging fully in the prescribed treatment. This factor is assessed by adding the scores of item 4, 6–14 and 16–17. ‘Attendance’ is another significant indicator for compliance. It is measured by summating the scores of item 1–3, 5 and 15. This is consistent with the previous attempts that non-compliance was usually measured by appointment keeping and record of attendance [14–16].
Convergent validity
The results of normality and detection of outliers eliminated the possibility that the significant correlations between the PTCS and other measures are due to unrepresentative subjects. The two subscales are negatively correlated with ‘stereotype agreement’, ‘selfconcurrence’ and ‘self-esteem decrement’ of SSMIS. These three subscales of SSMIS represent different levels of self-stigma [35]. Self-stigma begins with stereotype agreement. People with psychotic disorders may endorse the perceived stereotype to be common in public, and then concur with this adverse endorsement. This internalized effect would result in their drop in self-esteem [35]. Self-stigmatized mental health consumers are more likely to avoid social interaction with people outside family [52] and avoid seeking help from psychiatric services [53]. Their tendency to participate properly in psychosocial treatment would drop accordingly. These associations substantiated the phenomenon that self-stigma encumbered mental health consumers in complying with psychosocial treatment. In addition, the significant correlation between the ‘attendance’ subscale of PTCS and ‘stereotype awareness’ of SSMIS pointed to the same direction. Stereotype awareness refers to the perceived discrimination by the public [35]. Mental health consumers who internalized this belief would avoid adhering to psychosocial treatment.
Our results showed that high self-esteem, high general self-efficacy and high social self-efficacy were correlated with good participation and attendance of psychosocial treatment. This supported the suggestion thrashed out by Corrigan and Watson [26] that individuals who have low self-esteem and general self-efficacy are more likely to be undermined by self-stigma, and thus more reluctant in adhering to psychosocial treatment so as to avoid the effect of labelling. Higher level of social self-efficacy is associated with higher levels of social support and lower stress [54]. Better social self-efficacy should enhance their competence in groups and may improve their compliance, especially for their participation in treatment.
Except for the subscale of ‘current attribution of symptoms’, all remaining subscales of SUMD are negatively correlated with the two subscales of the PTCS. It implies that poor psychosocial treatment compliance is more likely to be associated with the poor recognition of current symptoms, rather than the poor attribution of current symptoms specifically to the psychotic disorders. The results emanated are consistent with the literature that poor insight is correlated with poor treatment compliance. Akin to medication compliance, mental health consumers who have better insight of their illness and more benefits they could gain from treatment would lead to better treatment compliance and hence treatment outcome [55].
Attitudes to medication are significantly correlated with the two factors of the PTCS in the expected direction. It seems that mental health consumers who have positive attitudes towards medication are more compliant to prescribed psychosocial treatment.
The correlation between medication compliance and the mentioned constructs have been examined in another related study and would be reported elsewhere. In short, good medication compliance was associated with good psychosocial treatment compliance, less stereotype agreement, better self-esteem, better insight and more positive attitudes to medication.
It makes sense to see that mental health consumers will refuse adhering to prescribed psychosocial treatment when they are self-stigmatized, have low self-esteem and have diminished self-efficacy. This is because they may be afraid of being stigmatized by the public because of their mental illness. The natural consequence is to stop attending the prescribed treatment in order not to be labelled. Furthermore, they would refuse to believe the effectiveness of treatment in improving their condition, and thus hold negative attitude towards their intervention. This finding is consistent with the health belief theory [12] which stipulates that the above-mentioned negative beliefs would decrease their treatment compliance. In addition, our results parallel the role theory [11] that poor insight of the mental health consumers may further worsen their compliance. This is simply due to the fact that they did not regard themselves sick and hence did not see the need to play the role as the help seeker.
The PTCS is intended to apply to people with severe mental illness. After random sampling, it shows that the majority of participants were diagnosed with schizophrenia (n = 86). We conducted t-test to see if there is a statistical difference on psychosocial treatment compliance among different diagnostic groups which did not reveal any difference. We are of the opinion that PTCS is applicable to those with severe mental illness.
The results of this study suggest that the 17-item PTCS is reliable and valid to measure compliance behaviour of mental health consumers on psychosocial treatment. However, there are points for further improvement in the present study. We did not offer any details on participants' actual compliance (e.g. attendance rate) for establishing the psychometric properties of the PTCS because of administrative inconvenience. We will try to include this information in our further study. In addition, confirmatory factor analysis cannot be implemented because of the limited sample size. We plan to apply this statistical technique in our further validation exercise of this scale.
Finally, the score of this scale can act as a reference point to study the effectiveness of other compliance enhancement strategies such as motivational interviewing [56] and educational intervention [57] for specific psychosocial treatment, and explore other mediating variables such as therapeutic alliance and social functioning in influencing compliance.