Abstract
The aim of this study is to recommend a common data element (CDE) to measure supervisory effectiveness of staff working in LTC homes that can be used in international research. Supervisory effectiveness can serve as a CDE in an effort to establish an international, person-centered LTC research infrastructure in accordance with the aims of the WE-THRIVE group (Worldwide Elements to Harmonize Research in Long Term Care Living Environments). A literature review was completed and then a panel of experts independently reviewed and prioritized appropriateness of the measures with mindfulness of their potential applications to international LTC settings. The selection of a recommended CDE measure was guided by the WE-THRIVE group’s focus on capacity rather than deficits, the expected availability of internationally comparable data and the goal to provide a short, ecologically viable measurement, specifically for low- and middle-income countries. Two measures were considered as the CDE for supervisory effectiveness, Benjamin Rose Relationship Scale and the Supervisory Support Scale; however, given that the latter measure has been translated in Spanish and Chinese and has been tested with nursing assistants in both of these countries with good psychometric properties, our group recommends it as the CDE going forward.
Researchers from various international countries founded the WE-THRIVE initiative (Corazzini et al., 2019) whose mission is to develop core measurement domains for LTC environments in order to address gaps in the provision of care and to contribute to person-centered care (Corazzini et al., 2019). The call for the creation of standardized domains stem from the need to account for experiences of staff members as well as the residents living in the environment, linking the interactions between the two, and addressing internationally low- and middle-income LTC homes (LTCHs). Through the creation of what is known as common data elements (CDEs), defined as variables that are operationalized and measured in identical ways across studies (Redeker et al., 2015), the sharing of data and aggregation can be promoted, leading to improved quality of data and to more persuasive comparisons (Corazzini et al., 2019).
To drive this forward, in 2017 the WE-THRIVE Group identified four core measurement domains: (i) Organizational Context, (ii) Workforce and Staffing, (iii) Person-centered Care, and (iv) Care Outcomes (Corazzini et al., 2019). The domain workforce and staffing includes five concepts: staff skills, attitudes, and knowledge; staff collaboration and teamwork; training and self-efficacy; staff retention and turnover; and leadership and supervisory effectiveness (Corazzini et al., 2019) of which the latter is the focus for this paper. In terms of distinguishing between the concepts of leadership and supervisory effectiveness, there is ambiguity between these concepts in terms of profiles, theories, and how these roles often intersect in positions of authority (Curtis et al., 2011; Jeon et al., 2015). Leadership can be viewed as the ability to effectively engage individuals to achieve organizational goals using tasks and oriented behaviors, whereas supervision is viewed more as a role that involves less subtle task focused activities such as communication, team-building, and developing supportive relationships with the supervised staff (Hawkins & Shohet, 2006; Mintzberg, 1998).
The most influential factor in whether workers feel valued and respected at work is their relationship with their nursing supervisors (Hawkins & Shohet, 2006). As the nursing supervisor is often the first workplace contact for educational, instrumental, and moral support of nursing assistants (NAs) in LTCHs (McGilton, 2010) a decision was made to focus on the effectiveness of the supervisory role, as opposed to the leader in the LTCH. An integrative review focused on supervisory effectiveness revealed that nurse supervisors are in pivotal positions to influence the quality of LTCH and resident care (McGilton et al., 2016). Additional evidence suggests that NAs provide better resident-centered care and a higher quality of care when they have supportive supervisors (Ericson-Lidmann et al., 2014). Supervisory nurse performance is related to NAs’ job satisfaction, (Bishop et al., 2009; Choi, 2010), intent to turnover (Bethell et al., 2018), and actual turnover (Chu et al., 2014), which in turn influences quality of resident-centered care (Ericson-Lidman et al., 2014). From a more recent study focused on LTCH that had supervisors who were rated highly supportive (Escrig-Pinol, Corazzini, et al., 2019), results indicate that LTCHs could improve their resident’s quality of life by prioritizing and strengthening the relationships between nurse supervisors and NAs. With such strong evidence for the need of effective supervisors in LTCH, a decision was made by the expert panel to focus on the effectiveness of the supervisory role. Leadership is outside the purview of this paper, as there is WE-THRIVE working group that will focus on the organizational domain related to leadership in LTCHs.
The nurse supervisor in LTCH directly oversees front-line care staff and encourages individual self-reflection and learning from practice; provides on-going education, socioemotional, and instrumental support; and evaluates quality of care and resident outcomes (Orgambídez & Almeida, 2019; Williams & Irvine, 2009; Wong et al., 2013). Accruing evidence suggests that the most effective supervisory behaviors embrace values of empathy and respect (McGilton et al., 2007). Supervisors acknowledging staff personhood has been positively correlated to employee job satisfaction and perception of their work environments, which in turn may decrease average staff turnover rates of 70% to 100% and thus result in improved quality care delivered to residents (Anderson et al., 2004; McGilton, 2010; Noelker et al., 2009). Examples of such activities include active listening to employees’ unique aspirations and interests, as well as establishing policies that assist individuals in balancing their occupational and familial needs with their health and safety (Berta et al., 2018; McGilton et al., 2007; Chu et al., 2016). Although there is consistency in common attributes of supervisors in LTCHs, there is a gap on the best measure of supervisory effectiveness.
To date, several researchers have operationalized effective supervision and created various measures based on these conceptualizations. Researchers have used a variety of different measures to assess the concept of supervisory effectiveness which makes international comparisons difficult to achieve. Supervisors in LTCHs have been identified as having four dimensions to their role: clinical, supervisory, team support, and managerial (Escrig-Piscol, Hempinstall, et al., 2019). To this end, researchers have developed measures that focus on various aspects of these roles. Many measures have focused on the managerial role such as delegation, conflict management, coaching, and intellectual stimulation (Backman et al., 2017; Kiefer et al., 2005; Keisu et al., 1999). Other researchers have focused measurement on the supportive role of the supervisor, being reliable, empathic, with a focus on connecting with their staff (McGilton, 2010; Noelker & Ejaz, 2001).
However, despite the inconsistencies in the measurement of effective supervisors, similar NA outcomes have been realized, most notably, that effective supervisors have influence on NA retention and turnover (Anderson et al., 2003; Kiefer et al., 2005; Tourangeau & McGilton, 2004). These universal results provide evidence that effective supervisors make a difference in LTCHs. However, in terms of selecting a CDE for effective supervision, it would be preferable to have only one measure which has demonstrated feasibility, reliability, and validity, and has the potential to be easily translated into numerous languages to enable the aggregation of data for cross-national comparisons.
The aim of this study is to recommend a CDE to measure supervisory effectiveness of staff working in LTCHs that can be used in international research.
Methods
In terms of selecting the best candidate measure we followed the Best Practices for Identifying Common Data Elements identified by Redeker et al. (2015) which involved: (1) ensuring there is conceptual consistency of a measure, which involved conducting a literature review focused on supervisory effectiveness measures; and the (2) convening a group of WE-THRIVE experts to review the CDE providing them with the measures to review, to rank them and select a measure.
Literature Review on Supervisory Effectiveness Measures
For the first phase of identifying measures of supervisory effectiveness within LTCH, we reviewed the available literature on nursing authority positions and identified scales used to measure them. Concepts identified as appropriate for this review of nursing supervisory effectiveness used search terms “leadership,” “leader,” “lead,” “charge,” “manager,” “nursing,” and “nurse,”. These were further paired with terms “nursing home” and synonyms, “questionnaires,” “psychometry,” and “measurement.” We, conducted the review on June 2019 on Medline, Embase, CINAHL, and HAPI.
Inclusion criteria for selected measures included publications in English, possibility for or previous use in LTC/dementia settings, and surveyors as front-line workers (e.g. registered nurses, registered practical nurses, unregulated NAs) and/or LTC residents and their relatives. Exclusion criteria included use of open-ended questions, community, acute, and sub-acute/rehabilitation care settings, supervisors other than nurses, and nursing students as NAs. The decision to focus on LTCHs as opposed to community was based on the rapidly growing nature of these LTCHs that is projected to eclipse all other health sectors growth (York & MacAlister, 2015).
Ranking Process and Selection of Candidate Measures
Two reviewers independently analyzed the literature review and by using the inclusion and exclusion criteria, eleven measures were identified (n = 11 measures). Two measures were removed retroactively, given they were not financially accessible to LTCHs in countries of low-income and thus did not complement the WE-THRIVE values. The nine identified measures from the literature review were listed into an Excel workbook containing an evaluation grid for each measure. The table containing the measures were then sent out to five WE-THRIVE steering committee members to review and offer reasoning to support their choices. Members were provided with information about each measure and published studies that had utilized them. The members were then asked to individually review the remaining measures (n = 9) and rank according to those being most appropriately suited for international use in LTCH. Several aspects were considered when evaluating the measures, such as conceptual consistency of the measure, the number of items of the scales, the relevance of scale items, accessibility of the measure in different languages, demonstrated relationships of the measure to outcomes, and the psychometric properties of the scales. Lastly, members’ rankings were influenced by their related academic knowledge and/or experience in this area of nursing research, and in analysis of how well the measures operationalize the concept of supervisory effectiveness.
Consulting participants in this review were asked to rank measures they believed to be the best as “1,” and least appropriate as “9” within the table. Once the reviewers had submitted their rankings, measures were evaluated by averaging members’ respective scores. The lowest scores represented the preferred measures for evaluating supervisory effectiveness.
Results
The search generated nine measures that were reviewed by the experts and all of them were self-report measures asking NAs to rate their primary supervisors’ behaviors. For the majority of the measures, supervisory effectiveness was measured with a subscale that was a part of a larger questionnaire that focused on measuring work environment and performance, hence the rating of the supervisors behaviors were not the primary focus. Of the nine measures identified, all had good psychometric properties, and all have published results on the positive influence of supervisors on NA outcomes. In terms of common conceptualizations of the measures, seven of the supervisory scales were primarily focused on the management skills of the supervisor such as delegation and conflict management, while deemed important by the experts, were not deemed as essential as the supportive skills of the supervisor.
The two highest ranking measures were developed exclusively to measure the supportive skills of the supervisory, the Benjamin Rose Relationship with Supervisor Scale (ranked 1.8, the highest) (Noelker & Ejaz, 2001) and the Supportive Supervisory Scale (ranked 2.4, the second highest) (McGilton, 2010). They both had a focus on the relational aspects of their role; listening, being empathic, reliable, respectful and recognizing staff for their contributions which experts perceived as essential qualities for supervisors in LTCHs. Both scales were ranked highest in terms of usability, good psychometric properties, relevant conceptualizations of supervisory behaviors in LTCHs, demonstrated relevancy to outcomes, short time to complete, and easily translatable. The proposed measures are discussed in detail below. All of the other measures considered in this review and their rankings can be found in Appendix 1.
Benjamin Rose Relationship with Supervisor Scale
The Benjamin Rose Relationship with Supervisor Scale is a measure designed by the Margaret Blenkner Research Institute that is utilized to determine the perceptions of NAs relationships with their supervisor (Noelker & Ejaz, 2001). An 11-item scale, it evaluates aspects such as provision of recognition for contributions, quality of communication, active listening, respect of abilities, as well as the practice of empathy via a 3-point Likert-style scale (Noelker & Ejaz, 2001). Options for prompts regarding the participants’ supervisor include “most of the time,” “some of the time,” and “hardly ever/never.” This scale’s score ranges between 0 and 22, with scores on the highest end indicating the most positive reviews of participants’ supervisors (Kiefer et al., 2005; Noelker & Ejaz, 2001). Outcomes with use of this scale have demonstrated that higher supportive relationships between NAs and their direct supervisors result in both improved relations with collegial staff and job satisfaction (Kiefer et al., 2005). Psychometric properties have been established with use in LTC over 10 years, with an acceptable internal consistency and alpha co-efficient of 0.90 (Kiefer et al., 2005). Permission must be granted prior to use. As of the time of publishing, this scale is only offered in English.
Supervisory Support Scale
The Supervisory Support Scale (McGilton, 2010) is based on the definition of a supportive supervisor as an individual who is dependable, empathetic, and nurtures the personal side of the relationship with his or her staff. A 15-item scale was thus refined, with five items operationalizing each domain. A 5-point Likert-style scale was used to measure the supervisors’ behaviors. The response options included “always,” “often,” “occasionally,” “seldom,” and “never.” Content validity of the 15-item instrument was established by a panel of experts. Psychometric properties were established with a sample of 75 staff in three LTCHs, with an internal consistency of 0.94 and test-retest score of 0.70 (McGilton, 2010). The supportive supervisor scale is a summated rating scale out of a possible 75, with higher scores indicating more positively perceived relationships (McGilton, 2010).
The experts felt that either the Benjamin Rose Relationship with Supervisor Scale or the Supervisory Support Scale could be used as a CDE, to measure supervisory effectiveness. Since the ranking of the scales with the expert was conducted, the Supervisory Support Scale has been translated into Spanish (Alconada-Romero et al., 2020) and has been used in 37 LTCH in Spain demonstrating a significant positive relationship with job satisfaction of 394 NAs in Spain (McGilton et al., 2020). The scale has also been recently translated into Chinese (Tian et al., 2020) and validated with 300 NAs in four LTCHs and has established good psychometric properties which makes the Supervisory Support scale more appropriate and feasible in different regions of the world for international comparisons.
Discussion
This article presents the process of recommending a CDE for supervisory effectiveness, from the perspective of a consortium of international researchers. With utilization of a systematic approach, a literature review was completed that reflected the interests of the effective supervisors in LTCHs with a focus on their relational skills. A panel of experts independently reviewed and prioritized appropriateness of nine potential CDEs with mindfulness in their potential application to international LTCHs. Two major conceptualizations of the supervisor role were highlighted in the nine measures found; seven measures focused on the managerial role of the supervisor, while two focused on the relational role of the supervisory. The Benjamin Rose Relationship with Supervisor Scale and the Support Supervisory Scale included the following characteristics of a supervisor: attentive listening; provision of credit, recognition, praise and vocalized appreciation in appropriate contexts; encouragement of individual self-growth and fullest application; and receptiveness to staff remarks, concerns, and ideas. As the WE-THRIVE is focused on person centeredness, the experts were more in favor with these two measures of effective supervision which took into account the relationship between the individual NA and supervisor. In terms of making a final recommendation for the CDE, the Supportive Supervisory Scale which has been cross culturally validated and has demonstrated preliminary construct validation was selected as the candidate CDE measure.
Our results highlighted the lack of any international comparison research conducted on the role of the supervisor in LTCHs. Most of the research comes from North America and high-income countries. The questions of whether supervisors in LTCHs internationally are seen as important or whether low socioeconomic countries have supervisors in their homes are worth pointing out. Another concern with this CDE is the power differentials in the supervisor-supervisee relationship and how this hierarchy may skew the supervisee ratings. Rating their supervisors may be perceived as a risk by staff so ensuring anonymity of the data will be essential when collecting this information.
Strengths of this article include the international and multidisciplinary nature of the WE-THRIVE initiative, and the expert-judge review panel of this article spanning various countries such as Canada, Spain, Sweden, Switzerland, and the United Kingdom. The authors of this article with variant academic and experiential backgrounds were able to consider measures in a number of contexts relatable to low- and middle-income countries and agree on proposed CDEs. Further, the systemic approach in which CDEs were identified and captured should strengthen confidence in, and relevance of, the proposed CDEs for international studies in this field.
While we were able to recommend a candidate CDE measure for supervisory effectiveness, a number of limitations are noted. Firstly, the systematic process of identifying candidate measures was based on a limited number of search terms, and only in the English language. Therefore, there is a risk that all possible measures appropriate for this research may not have been included in the review. Only WE-THRIVE experts ranked the measures but perhaps NAs’ perceptions are required to determine the ranking of measures as they are the target population responding to the survey items. They have the unique ability to assess item relevance and potential realities of completing these measures. Another limitation is that the expert-judge panel of reviewers may not have accurately represented the full lengths to which LTCHs may differ internationally, as the panel did not include representatives from Africa, Asia, Australia, or Southern America even though we made strides to do so. As such, the process of review may not echo all relevant research in the WE-THRIVE concept of supervisor effectiveness, or accurately embrace relevant values of equivalent LTC organizational structures in other contexts. There is a continued need to accentuate research into the nursing supervisor role in low- and middle-income countries, as well as their organizational contexts and professional practice models of LTCH. Our ultimate goal is to test these measures with LTC stakeholders in low-and middle-income countries (Corazzini et al., 2019) to ensure they are applicable to their LTCHs.
Implications for Practice, Policy, and/or Research
This CDE offers the opportunity for capacity building within LTCH, which may in turn promote productive changes to bring about improved perceived outcomes via supervisees. Previous research on the role of the nursing supervisor has articulated the complexity of the role and the lack of preparedness of those in the role (Escrig-Piscol, Hempinstall, et al., 2019). A better understanding of how supervisors effectively work with their NAs across countries will provide a baseline understanding of what is required to inform the development of training programs for supervisors and what factors need to be considered to enhance their role globally. While our experts believe the role of a nursing supervisor exists in most LTCHs internationally the role of supervisor can be held by different health care personnel other than nurses; however, the main qualities espoused by the supervisory scales selected should be universal to all supervisors in LTCH, but further investigation is warranted to support this claim.
In summary, the recommended CDE for supervisory effectiveness is the Supportive Supervisory Scale. With the use of CDEs generated by WE-THRIVE members, comparison between various settings will become possible and persuasive. With the ability to compare and contrast research data and studies, areas with optimal development and LTC work can be identified. Further work is needed to explore the usefulness of the WE-THRIVE CDEs from the perspective of LTC providers and also, other low- and middle-income countries in non-English speaking countries.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The PA is the original author of the Supportive Supervisory Scale and was not involved in the rankings of the scales. The other authors have no conflicts of interest to declare.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
