Abstract
Patient autonomy is a central value in medicine and critical component of adherence to medical advice. This article reports on a validation of the 6-item version of the Health Care Climate Questionnaire (HCCQ), a measure of autonomy support in health care settings, in a sample of 242 melanoma patients. The HCCQ showed excellent internal consistency (α = .91), structural validity (Tucker–Lewis index = .99; comparative fit index = .99; root mean square error of approximation = .06, 90 per cent confidence interval (.00, .11); standardized root mean square residual = .02; χ2 (8, N = 235) = 13.99, p = .08) and construct validity (92.31% of hypothesized correlations with other measures confirmed). Acceptable 3-month test–retest reliability was observed (r = .55, p < .001; intraclass correlation coefficient (A, 1) = .54, p < .001). The French version was found equivalent to the English version.
Keywords
Introduction
The purpose of this article is to validate the Health Care Climate Questionnaire (HCCQ) – a unique measure of the patient perspective concerning their autonomy in medical settings. Patient autonomy is a central value in Western medicine (Beauchamp and Childress, 2001; Varelius, 2006) and one of the three fundamental principles of the Charter on Medical Professionalism (Medical Professionalism Project, 2002). Accordingly, this principle states that “physicians must have respect for patient autonomy … be honest with their patients and empower them to make informed decisions about their treatment” (Medical Professionalism Project, 2002: p.1). This recent official adoption of patient autonomy as a part of the physicians’ charter thus indicates that health care practitioners are now responsible for promoting both autonomy and well-being in their patients (Beauchamp and Childress, 2001; Ng et al., 2012; Medical Professionalism Project, 2002).
According to Self-Determination Theory, patients’ sense of autonomy represents a critical component of their motivational profile (Patrick and Williams, 2012; Ryan and Deci, 2006; Ryan et al., 2008; Williams et al., 1996, 1998a, 1998b, 2000). In health care settings, autonomous motivation is of particular importance concerning patient adherence to medical advice, treatment and disease prevention. According to Sox (2002), “‘the center of patient care is not in the physician’s office or the hospital. It is where people live their lives, in the home and the workplace. There, patients make the daily choices that determine their health” (p. 243). Unfortunately, the choices made by patients do not always contribute to better health outcomes. For example, non-adherence to medication ranges from 25 to 50 per cent and is associated with more hospital admissions and medical complications (Beck et al., 1985; Claxton et al., 2001; Haynes et al., 2002; Nieuwlaat et al., 2014; Payero et al., 2014; Sokol et al., 2005). In addition to health problems experienced by the patients, there are significant financial costs associated with greater hospitalization, medical tests, and treatments due to patient non-compliance (Berg et al., 1993; Iuga and McGuire, 2014; Roebuck et al., 2011; Sokol et al., 2005). Moreover, the current methods of increasing adherence tend to be complex in nature and largely ineffective (Haynes et al., 2002; McDonald et al., 2002; Nieuwlaat et al., 2014).
Why do patients not follow medical advice? From the perspective of the patient, the seemingly irrational act of non-compliance may be perceived as having a rational basis (e.g., unfavourable cost–benefit analysis or lacking sufficiently persuasive reasons to comply; DiMatteo et al., 2007; Donovan and Blake, 1992). Moreover, given that patient beliefs concerning their personal circumstances, prior knowledge and medical options are salient antecedents of their subsequent health-related decision-making; their personal beliefs concerning medical issues and physician support are indeed worthy of empirical study (Donovan and Blake, 1992; Vermeire et al., 2001). Patient involvement in the treatment process and the relationship between the patient and health care professionals are among the core elements of patient-centered care internationally (Kitson et al., 2013). Furthermore, an autonomy-supportive climate of mutual understanding, trust, and shared decision-making between the patient and the physician has been found to be conducive to discussing patients’ concerns, increasing adherence and facilitating patient participation, each of which corresponds to various positive outcomes (Greene and Hibbard, 2012; Hibbard and Greene, 2013).
For instance, greater communication between the patient and the health care team has been found to improve not only treatment adherence but also patient satisfaction (Burnier, 2000; Donovan, 1995; Fuertes et al., 2007; Ha and Longnecker, 2010; Hillen et al., 2011). Additionally, more effective provider–patient communication is associated with greater work satisfaction, less work-related stress, and lower burnout for medical professionals that, in turn, can facilitate patient adherence and satisfaction with care (DiMatteo et al., 1993; Ha and Longnecker, 2010; Pepper et al., 2012; Stavropoulou, 2011; Zolnierek and DiMatteo, 2009). With respect to the qualities of effective communication, findings suggest that shared decision-making, presenting options, addressing patients’ concerns, and mutual trust are critical to patient–physician relationships and health outcomes (Ha and Longnecker, 2010; Hillen et al., 2011; Stavropoulou, 2011). It is these key aspects of communication between a health care provider and a patient that are captured by the Health Care Climate Questionnaire developed by Williams et al. (2000) in which patients’ perceptions concerning autonomy-supportive nature of the health care climate are specifically assessed.
The HCCQ has been used in research for over 20 years to measure patients’ perceptions concerning the extent of autonomy support provided by their health care providers (Fortier et al., 2007; Kasser and Ryan, 1999; Williams et al., 1996, 1998b, 1998c, 1999, 2002, 2006). According to Williams et al. (2000: p. 81), autonomy support in health care settings “refers to providers’ interacting with patients by taking full account of their perspectives, affording choice, offering information, encouraging self-initiation, providing a rationale for recommended actions, and accepting the patients’ decisions” (p. 81). The HCCQ was designed specifically to assess these specific aspects of autonomy support in health care environments. It was developed based on prior work with similar questionnaires in non-health care settings (Williams et al., 1996), and can be adapted for use with various medical professionals including physicians, nurses and dieticians (Williams et al., 1998b). However, given limited published research to date on the empirical validation of the HCCQ, the purpose of this study was to assess the validity and reliability of this self-report measure. Given that researchers and clinicians alike typically prefer less intrusive measures that perform optimally, the short 6-item version of the original 15-item scale was examined with respect to internal consistency, structural and construct validity, test–retest reliability, as well as descriptive and item-specific statistics.
Scale validation was conducted in a hospital-based sample of patients diagnosed with melanoma skin cancer. This population was selected due to clinical guidelines recommending long-term adherence to a skin self-examination (SSE) regimen for melanoma survivors (Coit et al., 2009), thereby affording a sufficiently intensive examination of patient autonomy in the context of medical adherence. In addition to scale validation, the second study objective was to examine English–French language equivalency in terms of mean level differences as well as contrasting internal consistency, structural validity, and item/scale descriptives between the English and French samples. French was chosen as a second language due to the measure having previously been employed in French samples (e.g. Fortier et al., 2007) despite no published scale validation for the French version.
Method
Participants
The study sample consisted of patients diagnosed with melanoma skin cancer treated at two major teaching hospitals in Montreal, Canada. Eligibility criteria included the ability to read either English or French, being at least 18 years of age and having a medically confirmed diagnosis of melanoma. In total, 15 per cent were diagnosed with melanoma in situ (stage 0), 48.7 per cent with stage I, 18.7 per cent with stage II, 6.2 per cent with stage III and 3.6 per cent with stage IV melanoma.
Materials
Health Care Climate Questionnaire (HCCQ)
The HCCQ was utilized to assess the degree to which the participants perceived their physicians as supportive of their autonomy as opposed to controlling in nature (Williams et al., 1996). Sample items include ‘I feel that my doctor has provided me choices and options’, ‘My doctor tries to understand how I see things before suggesting a new way to do things’ and ‘My doctor encourages me to ask questions’. Each item was rated on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The language of the scale is devoid of jargon, double negative statements, and advanced vocabulary to optimize accessibility for individuals across education levels. Similar to the original 15-item scale having excellent internal consistency (Cronbach’s alpha (α) = .95) and a 1-factor structure (Williams et al., 1996), high internal consistency (Cronbach’s α = .96) and a 1-factor structure were also obtained in the current sample.
Comprising items from the original 15-item version, the shorter 6-item scale includes items 1, 2, 4, 7, 10 and 14 (Self-Determination Theory, 2017). Similar short versions with four (Williams et al., 1999), five (Williams et al., 1996, 1998b, 2002) and seven items (Kasser and Ryan, 1999) have previously demonstrated acceptable to excellent internal consistency (.72 ≤ Cronbach’s α ≤ .96) and reflected a 1-factor structure (Fortier et al., 2007; Kasser and Ryan, 1999; Williams et al., 1996, 1998b, 1998c, 1999, 2002, 2006). For the purposes of this study, the 6-item version was administered following from a recommendation of this version by the scale authors (Self-Determination Theory, 2017). The 15-item version was additionally administered to examine correlations between the full and 6-item scale versions.
Other measures
Socio-demographic data (age, gender, years of education, etc.) and need for additional services (e.g. to spend more time with the physician) were assessed via a self-report survey. Time since diagnosis and cancer stage were additionally collected from patients’ medical files and hospital tumour registries. The Physician Support of Skin Self-Examination Scale assessed the patients’ perspective of their doctors’ encouragement of skin self-examination (SSE) (Coroiu et al., 2017). Patients’ self-efficacy for SSE was assessed using Skin Self-Examination Self-Efficacy Scale (Bergeron et al., 2017). The Skin Cancer Index (SCI) was used to measure patients’ worries specific to skin cancer (proposed by the original authors as a measure of quality of life; Matthews et al., 2006; Rhee et al., 2007), while the Patient Health Questionnaire-4 was used to assess psychological distress with respect to depressive and anxiety symptoms (Kroenke et al., 2010).
Procedure
Design
Ethics approval was obtained from the university’s faculty of medicine research ethics board. This specific study was conducted at two teaching hospitals as part of a larger, longitudinal project examining how psychosocial variables correspond with adherence to medical advice during melanoma follow-up clinical care. Further procedural details are outlined in a published study protocol (Körner et al., 2013).
Translation
There exists no official validated French version of the HCCQ despite its previous use in English–French bilingual settings (e.g. Fortier et al., 2007). Consequently, a professional translator was employed to translate the HCCQ as per a cross-cultural translation technique (Banville et al., 2000). First, a professional translator from Québec, Canada (study location) fluent in both English and French translated the HCCQ from the original English into French. Second, a bilingual individual without access to the original version back-translated the measure into English. Third, the original and back-translated versions were compared and consensus on the final version was reached between both translators. The 7-item response format was retained from the English version (1 = fortement en désaccord, 3 = neuter, 7 = fortement en accord). The translated questionnaire items are presented in Table 1.
Items of the HCCQ in English and French.
HCCQ: Health Care Climate Questionnaire.
6-item version items are italicized.
Analyses
The validation analyses were principally conducted using baseline data (Time 1) from the longitudinal project to afford the greatest number of participants (N = 242). To evaluate test–retest reliability, data from Time 2 (6-month lag, N = 109) and Time 3 (9-month lag, N = 112) were additionally assessed. First calculated were item means, standard deviations (SDs), Cronbach’s α if item deleted, inter-item correlations, corrected item-total correlations and factor loadings, as well as the scale mean and SD. Second, the correlation with the full scale (15-item version) was calculated, followed by Cronbach’s α and a confirmatory factor analysis (CFA) for the 6-item scale. Construct validity was evaluated by comparing the HCCQ to similar and dissimilar constructs using Pearson’s r and point biserial correlations. Reproducibility of the HCCQ scores (test–retest reliability) was assessed by correlating Time 2 and Time 3 data using the intraclass correlation coefficient (ICC). Additionally, floor and ceiling effects were investigated.
Equivalency between the English and French versions was examined by (a) correlating language with the HCCQ score, (b) comparing group means, (c) comparing CFA fit and factor loadings between the English and French subsamples, (d) comparing overall Cronbach’s αs and item-corrected Cronbach’s αs of the English and French groups and (e) comparing item-level analytics (inter-item correlations, item-total correlations, item means). Confirmatory factor analyses were conducted using Mplus statistical software (version 7.0; Muthén and Muthén, 1998–2015), with the remaining analyses conducted using Predictive Analysis Software (PASW version 18; SPSS, Inc., 2009).
Results
Descriptive and item analyses
Item means were similar in magnitude ranging from 4.74 to 5.64 (SD = 1.44–1.68) with an average item mean of 5.29 (average mean across all individual item means). Similarly, the means for the entire scale (regardless of single items deleted) remained within a range of 1 point (from 26.09 to 26.98). The inter-item correlations ranged from .55 to .75 (p < .001 for all). Furthermore, corrected item-total correlations were high and comparable across items (ranging from .68 to .82) as were the Cronbach’s αs if item deleted (ranging from .88 to .90). The short, 6-item version of the questionnaire correlated very highly with the original, 15-item version (r = .95, p < .001). Additionally, all items showed excellent factor loadings (Tabachnick and Fidell, 2013; for more details on the structural validity see the next section). Additional details regarding the analyses described above can be found in Table 2.
Health Care Climate Questionnaire – item analyses based on the total sample (N = 222).
HCCQ: Health Care Climate Questionnaire; ITC: item-total correlation.
Internal consistency and structural validity
Cronbach’s α was high (α = .91) indicating excellent internal consistency (Streiner, 2003; Terwee et al., 2007). Furthermore, as the HCCQ is considered a 1-factor scale (Williams et al., 1996, 1998b), a confirmatory factory analysis (CFA) was conducted to corroborate previous findings. A total of 235 participants’ data were used in this analysis allowing for sufficient power to detect a model fit (Terwee et al., 2007). A CFA with maximum likelihood estimation showed an excellent fit following Hu and Bentley’s as well as Kline’s criteria (Hu and Bentler, 1999; Kline, 2013; Tucker–Lewis index (TLI) = .99; comparative fit index (CFI) = .99; root mean square error of approximation (RMSEA) = .06, 90 per cent confidence interval (CI) (.00, .11); standardized root mean square residual (SRMR) = .02; χ2 (8, N = 235) = 13.99, p = .08). The factor loadings ranged from .74 to .90.
Construct validity
As per recommendations by Terwee et al. (2007), construct validity was established by testing hypotheses on how the HCCQ should correspond to similar and dissimilar constructs through correlations, pertinent single items as well as sociodemographic and medical data (see Measures). In total, 12 of 13 hypotheses (listed in Table 3) were supported (92.31%) providing evidence for the construct validity of HCCQ (for more details see Table 3; Terwee et al., 2007). One hypothesis was not supported, specifically that involving the Emotional subscale of the Skin Cancer Index (SCI) such that a moderate correlation was expected but no significant correlation was observed. Thus, general sense of distress (Patient Health Questionnaire-4 (PHQ-4)) did correspond negatively with the HCCQ, whereas specific symptoms of distress as assessed by the emotional subscale of SCI (e.g. embarrassment about cancer and worries) did not relate to perceived autonomy as measured by the HCCQ.
Correlations between HCCQ-6 and other variables.
HCCQ: Health Care Climate Questionnaire; SD: standard deviation; SSE: skin self-examination; SCI: Skin Cancer Index; PHQ: Patient Health Questionnaire; ns: not significant; n/a: not applicable.
Point biserial correlation was performed for the dichotomous variables.
Cronbach’s α.
Significant after applying Bonferroni adjustment for multiple comparisons requiring p < .004.
Magnitude of a correlation is based on Cohen’s (1988) criteria.
p ≤ .001.
Reproducibility
Test–retest reliability was established by correlating Time 2 and Time 3 data (gathered 3 months apart) and computing the intraclass correlation agreement (ICC). The data from T2 and T3 showed a strong correlation (Cohen, 1988) which is notable given the time lag and the possibility of some patients having consulted different physicians during this period (r = .55, p < .001). Additionally, the ICC was calculated to assess test–retest reliability (Weir, 2005). Unlike Pearson’s r, the ICC accounts for both consistency of performances from test to retest (within-subject change), as well as change in average performance of participants as a group over time (i.e. systematic change in the mean; Vaz et al., 2013). Using a two-way random effects model (McGraw and Wong, 1996), ICC values showed reliability to be adequate (ICC (A, 1) = .54, p < .001 – equivalent to ICC (2, 1) in Shrout and Fleiss’ (1979) notation).
Floor and ceiling effects
The lowest possible score of 6 was obtained by 1 per cent of the sample, and the highest possible score of 42 was obtained by 10.8 per cent of the sample. Thus, based on McHorney and Tarlov’s (1995) criteria, no floor or ceiling effects were detected.
English–French equivalency
We further examined differences between the English and French versions of the HCCQ concerning group means, dimensionality, and internal consistency. First, the correlation between the HCCQ score and language was not significant (rpb = –.03, p = .67). Second, HCCQ scale means were very similar between the English and French subsamples (MEn = 31.92, SD = 7.78; MFr = 31.48, SD = 7.51) as indicated by a nonsignificant t-test (t(218) = .43, p = .67). Third, average item means (average across all individual item means) were only .07 apart with each item mean falling into similar ranges for each subgroup (MEn = 5.32, range: 4.72–5.79, SD = 1.42–1.75; MFr = 5.25, range: 4.76–5.59, SD = 1.42–1.62). Similarly, inter-item correlations for the subgroups showed comparable ranges (English: .53–.76; French: .54–.79). Finally, corrected item-total correlations were high and comparable across the items (English: .67–.81; French: .69–.83).
Confirmatory factor analyses were conducted for the English and the French subgroups separately using a maximum likelihood estimation and showed very good model fits (see Hu and Bentler, 1999; Kline, 2013): English: (n = 119): TLI = .96; CFI = .97; RMSEA = .11, 90 per cent CI (.05, .17); SRMR = .03; χ2 (9, N = 119) = 22.12, p < .01; French: (n = 115): TLI = .99; CFI = .996; RMSEA = .04, CI (.00, .12); SRMR = .02; χ2 (8, N = 115) = 9.62, p = .29. Data from the French subgroup indicated slightly better model fit than for the English subgroup. The factor loadings in the English subsample ranged from .69 to .87 and in the French subsample ranged from .75 to .92. Finally, observed Cronbach’s αs for the two subscales were excellent (English: α = .91; French: α = .90) and remained in similar ranges when an item was deleted (English: .89 ≤ α ≤ .91; French: .87 ≤ α ≤ .90). Overall, it can be concluded that the scale performed equally well in English and French, and thus can be considered valid for use in these two languages.
Discussion and conclusion
Discussion
The HCCQ was first introduced over 20 years ago (Williams et al., 1996) and has been used in numerous studies (Fortier et al., 2007; Kasser and Ryan, 1999; Williams et al., 1996, 1998b, 1998c, 1999, 2002, 2006). Despite its popularity, however, the measure has not been formally empirically examined as to its reliability and validity. First, there is little by way of detailed psychometric information on the 6-item version of the HCCQ, and only limited data on other short versions with four (Williams et al., 1999, 1998c) five (Williams et al., 1998b, 1996c, 2002) and seven items (Kasser and Ryan, 1999). Accordingly, the present validation analyses of the 6-item official (Self-Determination Theory, 2017) brief HCCQ were conducted to address this research gap.
Second, whereas the scale has been used in French, there is presently no published information as to its psychometric properties. Thus, not only is a valid French version necessary for conducting research using the HCCQ with Francophone participants, one must evaluate the correspondence between the English and French versions to be able to directly compare data from French and English studies using this measure. Consequently, we translated the HCCQ and evaluated the reliability and validity of the scale in both languages in samples of English- and French-speaking melanoma patients.
Overall, the 6-item HCCQ demonstrated excellent validity and reliability with respect to internal consistency, structural validity, and construct validity. Additionally, acceptable 3-month test–retest reliability was observed, and floor or ceiling effects were not detected. Furthermore, the comparison of French and English versions in terms of scale means, structural validity, and internal consistency did not reveal significant differences between the two subsamples indicating equivalence of performance in these two languages.
Nevertheless, specific limitations warrant discussion. First, the data in the present analyses are based on a sample of modest size, that although is adequate (minimum 10 participants per item, minimum 100 data points overall for the CFAs; Terwee et al., 2007), did not significantly exceed standard benchmarks (115 in the French subsample, 119 in the English subsample). Second, in our examination of construct validity, it was not possible to compare the HCCQ to another gold standard measure of the ‘health care climate’ or ‘health care provider support of patient autonomy’ as no such measures exist. Thus, the measures used to establish construct validity were not redundant but instead represented close approximations to the construct proposed to be captured by the HCCQ.
Third, whereas it is recommended that reliability testing of health status questionnaires be conducted 1 or 2 weeks apart – long enough for the participants to forget scale items but short enough for no significant change to occur (Terwee et al., 2007) – the present lag was 3 months with some patients likely having consulted different physicians between Time 2 and Time 3. Given the substantial time lag, we consider the strong correlation and the adequate ICC as indicative of robust test–retest reliability. Given these limitations, future research to validate the HCCQ in different populations with larger samples and shorter retest lags are encouraged. Finally, as the analyses were conducted for patients specifically diagnosed with melanoma skin cancer, the utility of the HCCQ in other populations warrants investigation.
Conclusion
The brief, 6-item version of the Health Care Climate Questionnaire is a reliable and valid measure of perceived physician support of autonomy among patients diagnosed with melanoma, as administered in both English and French languages.
Practice implications
To our knowledge, the use of the Health Care Climate Questionnaire in clinical practice has not been reported in the existing literature, and we hope that validating the measure will open doors to its potential use outside of research. The HCCQ-6 is recommended as a reliable and valid measure for use in future research and clinical practice to assess patients’ perceived autonomy in both Anglophone and Francophone populations.
Footnotes
Acknowledgements
Patients’ personal identifying information has been removed to ensure participant anonymity. All authors have contributed substantially to designing the study, study execution, and/or manuscript preparation. Furthermore, all authors have approved the final version of this paper. The authors would like to thank all the participants and research assistants for their valuable input.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research was funded by the Fonds de recherche du Québec – Santé (no. 22715) and by the Canadian Institutes of Health Research (doctoral fellowship).
