Abstract
Coaching as a practice and process has grown. Coach training and certification is currently based on competency models derived from opinion surveys not research on coaching outcomes. Competency models developed on expert opinion were about 50% accurate in terms of predicting managerial effectiveness in prior studies. To address this gap, we tested behavioral emotional and social intelligence competencies of coaches, not self-assessed, that predicted client behavior change over 2 years. The sample was 240 coach-client dyads involving 60 different coaches. Effective coach competencies were achievement orientation, adaptability, emotional self-control, empathy, organizational assessment, and influence. To rule out general mental ability (GMA) as a component, a subsample of 135 dyads based on 27 different coaches showed that GMA was not significant. In the subsample, effective competencies were the above list and conflict management, with near significant findings for coaching/mentoring and teamwork. More studies are needed to guide training and certification programs.
Introduction
Coaching as a practice and process has increased dramatically in recent decades to help members of organizations and professionals develop their capability. People want and need development for performance maximization and expansion of their talent and activities. In surveys of European MBAs, the most frequent reasons cited for choosing a company to work for and staying were opportunities to develop (Boyatzis et al., 2019). Development was a higher ranked reason than compensation. At the same time, there was an engagement crisis pre-COVID showing that most people with full-time jobs were not motivated and engaged in their work from every country surveyed (Harter, 2023). It was followed by the great resignation, quiet quitting, and retiring in place movements. Development opportunities offered and available to employees appear key to selection, retention, and motivation.
Other than coaching, there are three major approaches to development of people at work: education, training, and developmental assignments. Formal degree-based education does not have a track record of providing retained learning. One study showed that the half-life of knowledge from the first required accounting course at a top 20 ranked full-time MBA program was 6 ½ weeks (Specht & Sandlin, 1991). Outcome assessment studies of 4-year colleges suggest little is developed of the skills and competencies needed for effective job performance (Astin, 1993; Winter et al., 1982). Training programs in organizations do not appear to provide sustained development suffering from the “honeymoon effect” of noticeable behavior changes that last 3 weeks to 3 months and then atrophy (Campbell et al., 1970). Developmental assignments are jobs or partial jobs offered to help a person practice new behavior considered crucial to their effectiveness in future jobs or roles. They are often provided through formal career pathing or temporary project assignments and have been used for hundreds of years. They are believed to work, but there is little research to support its efficacy. Clearly it works for some people, but that does not account for thousands or millions of people for whom such assignments have yielded positive benefits that appear to be more haphazard and possibly wasteful.
As a result, organizations have turned to coaching for the next wave of possible developmental activities. Career, leadership, or executive coaching is emerging as a popular form of development in organizations and for individuals wanting to change their behavior and perspective to better perform in future roles and earn a wider variety of future promotions or job possibilities. It is personalized and more acceptable than training programs in hierarchical cultures. The early results and outcomes from random assignment, control group studies are promising (Boyatzis et al., 2022; De Haan et al., 2019; De Haan & Nilsson, 2023).
Supporting this is many training programs to develop coaches and associations offering certification as coaches. The training programs and certification processes organize themselves around a competency model (i.e., a list of competencies or capabilities needed for a person to be an effective coach). To date, it is widely acknowledged by experts in coaching and coaching research that none of these models are based on research on coaching impact other than expert or client opinion as to which competencies are useful and effective (Boyatzis et al., 2022).
Prior research on competency models on management jobs showed that models developed on expert opinion were about 50% accurate in terms of predicting managerial effectiveness with performance measures (Boyatzis, 1982). Other elements in the models were irrelevant or opposite to those that predicted effectiveness. Competency models based solely on expert opinion or values (e.g., Delphi techniques) are misleading and will waste time in training programs and possibly certify the quality of people in the field of coaching inappropriately.
The current competency models and practices in coaching rely on inappropriate definitions of competencies that confuse behavioral characteristics of a coach with other factors. The lack of adherence to a sound theory of competencies results in contamination with job tasks, values, style, and knowledge. While possibly relevant to coaching, such other variables are often improperly assessed which render them vulnerable to extraneous factors not causally related to effectiveness in helping others, such as imposed current fads, prejudices, culturally biased, and stereotypical attributions. These theoretical and research challenges result in practices that waste time and money developing inappropriate or irrelevant characteristics. Historically, they even limit access to professions, like coaching, based on prejudicial reasons not performance or effectiveness.
To address this gap in the literature, a study was designed to examine which competencies of coaches enable them to be effective in helping clients change their behavior in ways needed in their intended and future careers. A definition of a competency is used as a set of related behaviors organized around an underling intent based on a theory of performance (Boyatzis, 1982; McClelland, 1973), as discussed later in the paper. Appropriate behavioral measures of the coach's competencies, not self-assessment, were used. These coach competencies were then analyzed as to how much of their clients’ desired, behavior change occurred over 2 years, again, not self-assessed, to determine which coach competencies help their clients change.
One contribution of this study to coaching and helping theory is to clarify the definition of a competency and re-establish needed research design and measurement approaches to validate them against coach effectiveness. A contribution to the research literature is to conduct the first empirical study of coach competencies that should inspire more studies and help to focus coach training and certification on the relevant competencies instead of wasting time on fads and popular themes. A contribution to the practice of coaching and helping is to provide empirically based guidance in design of training programs by devoting more time and experiences to the competencies that make a difference. These results can also provide guidance for certification programs by focusing on competencies that actually matter to coach effectiveness.
Coaching to Help People Change
Coaching is a role and process used to help people learn, grow, and change. The International Coaching Federation defines coaching as, “partnering with an individual or group in a thought-provoking and creative process that inspires them to maximize their personal and professional potential.” (ICF, 2019). This study is based on a stream of research in which coaching is defined as “a facilitative or helping relationship with the purpose of achieving some type of change, learning, or new level of individual and organizational performance” (Boyatzis et al., 2019, p. 15). It differs from mentoring in that it is often shorter in duration and often focused on either a person's life/career or their functioning in specific settings, like health.
The desired outcome of coaching includes: an affirmation or change in a client's dreams, vision, and purpose in life; an improvement in their behavior relevant to effectiveness in their chosen job, role, profession, or career; higher quality relationships with others with whom they live and work; a greater sense of well-being (Spence & Grant, 2007); better health (Hammoud et al., 2023); and related factors like increased engagement and motivation at work (Boyatzis et al., 2022). In a qualitative study of executives and their coaches, they hoped for improvement in their vision, purpose, and dreams during the months following the coaching (Passarelli et al., 2022). But 1 year or more after the coaching, they focused on how much the client's behavior had changed (Passarelli et al., 2022). It is analogous to other helping professions, like teaching, therapy, trainers, physicians, nurses, or clerics. The consequence of achieving these desired outcomes of coaching should include performance improvement (Boyatzis et al., 2022). Randomized assignment, comparison group studies have documented outcomes that clients feel as if the process of coaching helped them in their goal aspiration (Spence & Grant, 2007), and they felt satisfied with the coaching (De Haan et al., 2019; Jones et al., 2015).
This approach is based on a contingency theory of performance (Boyatzis, 1982) in coaching. It claims when a person uses their individual characteristics (i.e., values, style, traits, competencies) that are appropriate for the tasks and responsibilities of coaching, and is consistent with cultural and system context of the organization, effective coaching will occur. Competencies of coaches are, therefore, proximal outcomes of coaching toward the distal outcome of increased effectiveness (Bozer et al., 2013).
Competencies and Their Measurement
Competency-based human resources have gone from a new technique to a common practice in the 50 years since David McClelland (1973) first proposed them as a critical factor in performance. The quality of the research has trailed the popularity of applications. One confusion has been the nomenclature. A competency is defined as a capability or ability of a person, not a characteristic of a job, that results in effective performance of a role or job (Boyatzis, 1982; McClelland, 1973). It is a set of related but different sets of behavior (i.e., a collection of skills) often called alternate manifestations organized around an underlying construct or intent.
For example, listening to someone and asking them questions are two skills. A person can use these skills for multiple reasons or desired results. A person can ask questions and listen to someone to ingratiate themselves or to appear interested and be seen by the other person more positively. A person can ask questions and listen to someone because they are interested in understanding the other person. They may ask questions and listen to another person to trick them into confessing or revealing a lie. A competency, therefore, is more than a skill or specific behavior. The underlying intent around which the set of skills are arrayed is critical to understanding the functions, benefits, and measurement of a competency.
In this way, a competency includes action (i.e., a set of alternate behaviors or skills varying according to the situation) and intent. Some have confused competencies with proficiencies. The latter are often determined as the minimal effort needed to accomplish a task and skills (Bachkirova & Smith, 2015). Even the label often requires translation in languages other than English. For example, in Italy, a competency with the meaning of our research is not translated to “competenze” but to “capacitá.”
The confusion as to definition was exacerbated by different methods of assessment and claims made on the basis of the various methods. The early work on identifying competencies was inductive and started with measuring performance effectiveness. Often, a sample of outstanding or superior performers was identified with multiple performance criteria, and another sample of “average” or “poor” performers was also identified from the remaining population. The criterion sampling was crucial for this inductive method. While assessment against effectiveness from work output data was best, like sales, profits, waste reduction, or new products launched, the study of many jobs and roles needed something else.
It was often convenient to ask people what competencies they thought were needed for effective performance (Cavanaugh, cf Boyatzis et al., 2022; Grant & Cavanagh, 2008). This collected what was called expert opinion, values, or attitudes about the competencies needed. All of the current competency models used for training to comply with certifying associations or certification of coaches are based on such data. In a major study of the competencies of managers at all levels based on samples from 12 large organizations in the public and private sectors showed a troubling consequence of expert opinion and value-based competency models (Boyatzis, 1982). When managers and executives were asked what competencies were needed for effective performance, or consultants and trainers to them were asked, they identified a list of characteristics which they might have called competencies.
Such lists were often a confusing mixture of skills, job tasks, style, values, and some actual competencies. For example, someone might say that a needed competency was selecting staff. This is clearly an important task of a manager, but not a personal characteristic. To select staff well, a manager would have to use a number of actual competencies, like empathy, organizational awareness, and pattern recognition. In fact, when the lists generated from opinion and attitude polls were compared against competencies that predicted effectiveness from performance measures, a startling discovery was made (Boyatzis, 1982). About half of the items were accurate and useful. But 25% were irrelevant and 25% were opposite to the competencies that predicted managerial performance.
The challenge is that when training or certifying to the opinion and attitude-based lists, a great deal of time and effort was wasted and some items were used that would diminish a manager's actual effectiveness. Such efforts were shown to institutionalize mediocrity in various occupations and limit access of those groups not deemed appropriate at the time (Fallows, 1985). In other arenas, commonly held beliefs about the characteristics that produce performance, even in the face of data refuting those contentions, have resulted in massive waste of money and sometimes lives (Fallows, 1981).
The inductive studies to determine which competencies predicted effectiveness in various jobs was began with a variation on the critical incident interview (Boyatzis, 1982; Flanagan, 1954; Spencer & Spencer, 1993). It was adapted using the inquiry sequence from the Thematic Apperception Test (Murray, 1938) and a focus on specific events from one's life from the biodata method (Dailey, 1971). This interview was called a behavior event interview (BEI). It was a semi-structured interview in which the respondent was asked to recall specific events in which they felt effective or ineffective. The resulting stories were coded for evidence of competencies in terms of actions and intent (Boyatzis, 1982). The competency definitions were, therefore, developed inductively from thousands of such studies on various jobs in public, private, and nonprofit organizations in countries throughout the world reflecting a vast array of different genders, races, ethnicities, faiths, cultural context, and so forth.
The BEI interview method was time consuming and costly for both data collection and coding. After 20 years of compiling a list from such inductive studies, competencies were compiled into generic models organized by the nature of the work (Boyatzis, 1982; Goleman, 1998; Spencer & Spencer, 1993). Since new competencies were seldom identified and the behavioral indicators were known, 360 degree assessments could be constructed with those behavioral indicators. The use of 360 degree assessments revealed observations from others with whom a person worked or lived as describing the frequency and consistency of a person's behavior. The observations of the behavioral indicators were organized into scales reflecting a competency and used with far less labor and expense.
Competencies Matter
Most attempts at helping others learn and change are not as effective as the effort and energy devoted would suggest (Osman et al., 2020). As mentioned earlier, the retention of knowledge in graduate courses is abysmal. A top 20 ranked MBA program in the US assessed retention of learning from their first required course in accounting for the 28-year-old full-time MBAs. The half-life of knowledge was 6 ½ weeks. They could only produce half of what they had earlier produced on the same final exam given 6 ½ weeks later (Specht & Sandlin, 1991). When comparing value-added outcomes of graduates of full- and part-time MBA programs (i.e., assessing them at graduation versus when the entered the program) on behavioral evidence of learning the emotional, social, and cognitive competencies that predict effectiveness in management and professional jobs, the average impact was a 4% improvement over 1–2 years (Boyatzis et al., 2002). Earlier competency outcome studies of thousands of graduates of 4-year undergraduate programs in the US showed that including measures of various forms of cognitive reasoning, as well as emotional and social competencies needed in most jobs to be effective, the median number of competencies on which there was significant value added was 1 (Astin, 1993; Winter et al., 1982). That was over 4 years!
The impact of leadership training programs in government and industry do not fare much better. The documented impact of such training is about an 11% improvement over the 3 weeks to 3 months following training. Sadly, these numbers deteriorate quickly in what is called the honeymoon period of training (Boyatzis, 2008; Campbell et al., 1970). In health care, treatment adherence means the degree to which patients do what their physicians and nurses tell them to do post-surgery or to treat various diseases. Treatment adherence is typically between 20 and 50%, which results in prolonged difficulties and often progressive deterioration or disease (Khawaja, 2011).
The efficacy of psychotherapy has been a puzzle for decades. In his classic review of the literature, Hans Eysenck (1952) concluded that, “roughly two thirds of a group of neurotic patients will recover or improve to a marked extent within about two years of the onset of their illness, whether they are treated by means of psychotherapy or not.” (p. 319). He showed how being on the waiting list was just as beneficial as psychotherapy. Of course, the practices have advanced in recent decades but efficacy of outcomes is still in question (Osman et al., 2020) with the notable exception of cognitive behavior therapy.
Studies of factors resulting in effective impact of helping eliminate “approaches” or types of helping, duration of help, and focus of the help as useful. They indicate that it is the behavior, skills, or competencies of the helper that matter. Emrick (1974, 1975) attempted to determine whether type of counseling/therapy, duration, or focus had differential benefits for treating alcoholics. He found that none of the factors examined predicted effectiveness except the skills of the counselor. In the domain of psychotherapy, Rogers (1961) contended that accurate empathy, genuineness, and unconditional positive regard were crucial for a therapist to be effective. Empathy seemed to be the clearest and most consistent predictor of successful helping (Carkhuff, 1969). In a study of alcoholism counselors in the US Navy, competencies that predicted performance of their clients 1–2 years later, Boyatzis and Burruss (as cited in Boyatzis, 2005) showed that emotional self-awareness (ESA), empathy, and pattern recognition of the counselors predicted more effective treatment. It suggests that the competencies of the coach may have a substantial impact on the degree to which the client is helped and effectively changes in desired directions.
How Coach Competencies Matter
A coach's competencies affect the client in numerous ways. As in any interaction, each person affects the other. When a power distance is present (i.e., one person has higher social status in the organization or social system or one person is asking the other for help), the person in the more powerful position, in this case, the coach, has more influence on the client than vice versa. The coach will invoke and evoke emotional reactions in the client through their questions, observations, and even non-verbal postures and gestures.
Emotional contagion occurs when there is an emotional or neural activation in another person in which an emotion similar to the first person's is elicited in the other person (Lewis et al., 2000). This may occur as a form of neural activation in thousands of a second and be unconscious (i.e., below conscious awareness) through the Default Mode Network (Boyatzis & Jack, 2018; LeDoux, 2002) or be a result of visual and behavioral cues (Elfenbein, 2014; Hatfield et al., 1994). A person may also be caught up in social contagion in which watching or being present when another person acts in a certain way stimulates the same action as in the effect of the mirror neuron network (Boyatzis & Jack, 2018), or social copying (Fowler & Christakis, 2010).
The coach's competencies (i.e., the coach's behavior) can directly affect the client's thoughts, feelings, and actions by asking questions, giving advice, or sharing experiences and perspectives. The process begins with what the coach notices about the client's ideas or emotions, how and when they react and/or support them. The coach's competencies determine how, when, and why a coach attends to the client. At the behavioral level, the coach's actions become a role model for the client (Bandura, 1977).
A coach's mood as well as verbal and non-verbal actions creates a context within which the client is functioning. The coach's competencies determine the range of likely actions of the coach in creating an atmosphere of openness to new ideas or being defensive and being closed to new ideas. As suggested previously, if the coach is worried, preoccupied or stressed, the client will pick this up and likely reflect it or feel and act in the same way. If the coach is open, curious, and reflective, the client will also likely pick this up and reflect it in their feelings or actions.
Possible Competencies of Effective Coaches from Association Models
To determine which competencies should be studied and have a likely impact on a coach's effectiveness, two approaches were taken. First, a review of dominant competency models used by coaching associations was made that have been developed on the basis of value, opinion surveys of coaches and clients. Second, a review of evidence from competency studies of other helping professions was made. The result was a focus on a selected group of emotional and social intelligence (ESI) competencies used in this study.
A detailed review of what is perhaps the most widely used competency model, that of the International Coaching Federation (ICF, 2019), revealed a collection of skills, tasks, values, knowledge, style, and components of competencies. Similarly, competency models used by the World Association of Business Coaching (WABC, 2022), the European Mentoring and Coaching Council (EMCC, 2015), and Association for Coaching (AC, 2012) showed a set of activities and thoughts that a coach should have to be considered an effective coach. The review was not exhaustive but the patterns were consistent, suggesting that other such models would yield similar observations.
The definition and requirements for something to be considered a competency were reviewed above. A task is an activity that a coach should do. An example from the AC model is, “Provides observational feedback where relevant, leaving the client free to choose to act upon it or not.” Another example from the ICF model is, “Maintains confidentiality with client information per stakeholder agreements and pertinent laws.” Style is a mode of comportment of a coach. An example from the EMCC model is, “Critically reflects on practitioner paradigms and their impact on clients and client systems.” Another example from the ICF model is, “Celebrates the client's progress and success.” A skill is a specific behavior, like listening. Knowledge is a concept or process often referred to as declarative or procedural knowledge. For example, knowledge might be knowing the components of the Prochaska and Norcross model of change (Prochaska et al., 1992). A value is a belief. An example is from the WABC model, “Bring to the coaching situation a wide range of relevant knowledge about, and experience in SMEs and start-ups/entrepreneurial businesses.” A component of a possible competency could be a behavior indicator of a competency. For example, a component of Empathy as defined and operationalized in many competency studies of performance in various jobs (Boyatzis, 1982, 2018; Goleman, 1998; Spencer & Spencer, 1993) from the ICF model is, “Seeks to understand the client within their context which may include their identity, environment, experiences, values and beliefs.”
The thematic coding of the indicators within each of these models revealed a total of 473 indicators. The models each had 53 to 240 indicators organized into 8 to 35 supposed competencies. This review had to be done at the indicator level because most of the “competencies” in these models were a mixture of possible competencies, job tasks, style, values, and knowledge. Most indicators were job tasks (i.e., 38%) of all of the combined models. Next most frequent were style indicators (i.e., 27%). Likely components of competencies were 22% of the indicators with 1% claiming cognitive abilities. Values were 15% of the indicators.
As mentioned earlier, a detailed elucidation of the job tasks and responsibilities is useful in orienting people new to coaching. But such a laundry list is neither based on evidence that it matters in terms of client change, nor distinguishes from minimal requirements to do the “job” versus characteristics that distinguish effective performance as a coach. For example, “respecting the confidentiality of the client” is a task that a coach should perform and a value everyone in the field holds. But it would be difficult to determine variance in a coach's behavior in this regard and as a result, the thought that doing more of it results in better performance is unlikely. You can test whether a person understands that this is a job requirement or task of coaching. But it would be unlikely that you could ever see them demonstrating it in coaching sessions or recordings of them.
The components of likely competencies from these models could be classified as predominantly ESI competencies (Boyatzis, 2018; Boyatzis & Goleman, 2007; Goleman, 1998; Salovey & Mayer, 1990). This is not surprising since coaching is an interpersonal job and a helping profession (Lai & McDowall, 2014). A list of those ESI competencies which have been validated against performance in a wide variety of jobs and roles and compiled into several empirical (Boyatzis, 1982, 2018) and other conceptual generic models (Goleman, 1998; Spencer & Spencer, 1993) are shown in Table 1.
The Emotional and Social Intelligence Competencies (Boyatzis & Goleman, 2007).
Of those listed as emotional intelligence competencies, components of the following appearing in the four models from coaching associations reviewed were ESA, achievement orientation (AO), emotional self-control (ESC), adaptability, and positive outlook. Of those classified as social intelligence competencies, these association models included empathy, inspirational leadership, influence, and organizational awareness. It is useful to note that the most frequent indicators mentioned in these models were AO and empathy, in that order. Although only a few, several references were made to aspects of general mental ability (GMA) such as curiosity, systems thinking, and pattern recognition.
Possible Competencies of Effective Coaches from Helping Professions
Because there are no studies to date of coach competencies assessed against effectiveness as measured by client change, a search of the literature on related helping professions was used to supplement the review of the value, opinion-based association models (Kotte et al., 2022). The studies reviewed are only included if the assessment of the characteristic was behavioral (i.e., observed by others from actual events, recordings of critical incidents, video recordings of simulations, or 360 assessments with observations from informants).
Empathy (E) can be defined as seeking to understand and build a caring relationship with another person. As in these association-based competency models of coaches, empathy has been viewed as a needed capability of a therapist (Carkhuff, 1969), which built on earlier conceptual work of Rogers (1961). It also predicted effectiveness in alcoholism counselors (Boyatzis, 2005) and in a variety of human resource jobs (Spencer & Spencer, 1993). Empathy was shown to predict inspirational professors (Barco, 2019) and athletic coaches of Division I sports teams (Van Sickle, 2004).
AO can be defined as seeking to do better often characterized by setting goals, planning, evaluating progress, and seeking better options to increase the output to input ratio. AO predicted effectiveness of middle school and high school principals in an urban school system (Williams, 2008). Again, it was noted as predicting performance for various human service workers (Spencer & Spencer, 1993). AO was also predictive of the engagement and motivation of community college faculty (Babu, 2016). The aspect of a passion for feedback about one's performance was noted as predictive in various studies of helping roles (Goleman, 1998; McClelland, 1961).
Although mentioned less frequently within the association models, several other competencies were present in terms of requisite actions and intent in helping. That is, ESA and ESC were mentioned as important competencies. ESA is a coach being aware of their feelings, thoughts, values and how it affects their behavior and impact on others. It was predictive of effectiveness in alcoholism counselors (Boyatzis & Goleman, 2007) and personal, financial advisors (Goleman, 1998).
ESC is defined as the ability to inhibit one's feelings or actions for a “greater good” (Boyatzis, 1982). Spencer and Spencer (1993) noted it as a predictive competency for effectiveness of human service workers. The most detailed work on self-control and its positive impact on others and relationships was shown by Baumeister (2018). He showed that demonstration of self-control was far more predictive of success in life than self-esteem and other behaviors. He concluded that it was a crucial component for people to create and participate in society and organizations. Both ESA and ESC were predictive of effectiveness of athletic coaches (Van Sickle, 2004) and engagement and motivation of community college faculty (Babu, 2016).
Two other competencies documented as predicting effectiveness in human service workers are Positive Outlook (PO) and Adaptability (A). The former is an optimistic view of events and others (Seligman, 1998). It has been shown to predict effectiveness of athletic coaches (Van Sickle, 2004). Adaptability is being flexible and showing an ability to juggle options, and change actions, strategies, and opinions when appropriate. Both are mentioned in the WABC coach competency model. Both were found to predict motivation and engagement of community college faculty (Babu, 2016) and effectiveness of Division I athletic coaches (Van Sickle, 2004), while PO was predictive of inspirational professors in a Mexican University (Barco, 2019).
In addition, there were some studies showing the positive impact of the entire ESI cluster of competencies. When peers of dental graduate students described each other's behavior several months into their graduate program, the resulting observation predicted their grades in the third and fourth years of their program which are spent entirely in the clinic working with actual patients (Victoroff & Boyatzis, 2013). This full set of ESI competencies in aggregate of Turkish headmasters predicted job satisfaction of teachers (Camsarı Turanlı, 2007), US community college faculty (Babu, 2016), and parishioner satisfaction on multiple dimensions of Catholic Pastor priests (Boyatzis et al., 2011). The set of social intelligence competencies in aggregate were predictive of physician executives’ leadership effectiveness (Quinn, 2015), as this cluster did for athletic coaches (Van Sickle, 2004) and Olympic coaches (Cote & Sedgwick, 2003).
Possible Complications in Measurement
To validate a set of competencies of coaches, it is necessary to first evaluate the coach as to whether or not they show the desired competency and how much it is used in their behavioral repertoire. The most detailed assessment would be coding of actual behavior during the coaching sessions to know that the competencies were used at that time. But such recording and coding is often intrusive and inhibits both clients and coaches, so an assessment of the coach's likely behavior from their often used repertoire is used as an approximation.
Assessment of tasks will describe the job but not the coach. Assessment of values or style, as stated previously, is subject to current beliefs. Whether shared within a profession or not, values and attitudes show a mixed causal impact on behavior (Boyatzis, 1982, 2018). Assessment of a person's values or style may reveal compatibility within a cultural set of norms, but may have little relevance to effective performance. In the worst situations, values and style proscriptions emerge from prejudicial positions that not only do not predict effectiveness but may perpetuate stereotypes and attributions harmful to individuals not socialized into the majority group in power within a field, such as coaching.
Competencies, as stated earlier, are characteristics of a person and should be assessed by the behavior shown across a variety of situations and encounters. Self-assessment of one's behavior or skills was purported to be delusional in leadership studies as early as the work of Hollander in the 1950's. More recent studies have shown dramatic biases in self-assessment by gender within a culture (Taylor, 2014). Attempts to determine the most valid observations about a person's performance among all sources were shown to be peers (Lewin & Zwany, 1976). Beyond that, self-assessment and self-evaluation have been repeatedly shown to be unreliable, biased, and inflated (Baumeister, 2005; Dunning, 2005; Leary, 2004; Mabe & West, 1982; Podsakoff & Organ, 1986).
Therefore, the most likely method for assessing and testing the validity of competencies of coaches would be direct observation of the coaching sessions and subsequent reliable coding of video and audio recordings. But to establish a reasonable characterization of a coach's range of competencies used, a large number of coaching sessions, with different clients, over time and in differing situations would be required. Not only would the costs be prohibitive, but the recording of coaching sessions might affect the candor and comfort of the clients and possibly even the coaches.
A convenient and reliable approximation of a person's behavioral repertoire and the frequency of use can be obtained through informant reports in a process known as 360 degree assessment. With regard to coaches, where the hierarchy of jobs is not typical, they do not have consistent “bosses” or “subordinates.” Therefore, with reasonable caution, informant assessment from peers becomes the most viable and likely valid measure.
Hypotheses
The review in the previous sections revealed which specific competencies have been shown to relate to effectiveness in helping professions other than coaching and which specific competencies are thought to be relevant based on current expert opinion among coaching professionals. On the basis of the review, two hypotheses were used to design this study.
Hypothesis 1. Coaches demonstrating more frequent use of each of the ESI competencies (i.e., specifically Emotional Self-Awareness, Emotional Self-Control, Achievement Orientation, Adaptability, Positive Outlook, Empathy, Organizational Awareness, Inspirational Leadership, Influence, Conflict Management, Coaching & Mentoring, and Teamwork) would predict the degree of client behavior change. Hypothesis 2: Coaches demonstrating greater general mental ability (GMA) would not predict degree of client behavior change.
Because of the frequency with which AO and Empathy appeared in the existing expert opinion, association competency models, it was hypothesized that each of the competencies would not have equal impact on client behavioral change.
Hypothesis 3: Certain competencies of coaches would have a more potent effect on the degree of client behavior change, namely Achievement Orientation and Empathy.
Method
A longitudinal sample was used to test the hypotheses. Survey data was collected from coaches as they joined the faculty working in this program. Behavioral data was collected from clients early in their first year of their medical school 4-year graduate program and again 2 years later over a number of cohorts. Behavioral data is not self-assessment, but observations from others as to a person's behavior.
Samples
Data was collected data from 69 coaches and 314 medical students (hereafter referred to as clients) from a college of medicine in the US at two different campuses enrolled between the year of 2012 and 2018. The behavioral competency data on the coaches was available for 60 coaches. Among the 60 coaches in sample 1, 19 are male (31.67%) and 41 are female (68.33%). Among the 27 coach profiles in sample 2, which was a subsample of sample 1, 10 are male (37%) and 17 are female (63%). The age range of the coaches was 27–65. Among the 314 medical students, 172 were male (55%) and 142 were female (45%). The age range of the clients at entry into the program was 21–35.
The clients were each paired with two coaches for a coaching period of 4 years. Each coach and client met independently over the 4 years. Two coaches were used to ensure availability. Since most of the coaches were physicians, clinical demands on their time and availability at specific moments were uncertain. Also, two coaches provided a diversity of perspective and increased the likelihood of positivity; constructive relationships would be developed with each client. Client's observed behavioral data with the 360 described below was collected at Time 1 when they started the coaching a few months into their first year and Time 2 at the end of their second year or during their third year in the program. The specific clients with whom each coach worked were identified and classified as a coaching dyad.
Matching client profiles to coach profiles, 562 dyads were identified. The number of clients with whom a coach worked varied from 4 to 25 over the cohorts in sample 1. To ensure comparability within the sample, 4 client dyads were randomly selected from those available from each coach using SPSS's randomizing feature. Sample 1 consisted of 240 coach-client dyads.
GMA was only available for 27 of the coaches. Pairing them with clients, and randomly adjusting for the minimum of five clients available within this sample resulted in sample 2 consisting of 135 coach-client dyads. Sample 2 was a subsample of sample 1.
Measures
Independent Variable: Coach Competencies
The 60-item Emotional Social Competencies Inventory (Boyatzis & Goleman, 2007) which is a Likert scale rated from 1–5, typically administered as a 360 degree assessment. The hypotheses of the study were about behavioral competencies, so only data from informants (i.e., peers) were used. The self-assessment was discarded for this research. Scale scores were calculated for each of the 12 competency scales as an average item score. The 12 coach competencies were ESA (sample item, “Understands the connection between what is happening and own feelings”), ESC (sample item, “Remains composed, even in trying moments”), Adaptability (sample item, “Adapts to shifting priorities and rapid change”), AO (sample item, “Seeks ways to do things better”), Positive Outlook (sample item, “Sees opportunities more than threats”), Empathy (sample item, “Understands others from different backgrounds”), Organizational Awareness (sample item, “Understands social networks”), Coach and Mentor (sample item, “Cares about others and their development”), Inspirational Leadership (sample item, “Leads by articulating a compelling vision”), Influence (sample item, “Uses multiple strategies when trying to convince others”), Conflict Management (sample item, “Resolves conflict by bringing it into the open”), and Teamwork (sample item, “Works well in teams by encouraging cooperation”), as shown in Table 1.
Each scale score has satisfied all appropriate psychometric criteria for reliability and validity in previous publications (Boyatzis et al., 2015).
Independent Variable: Coach General Mental Ability
Coach GMA was measured with the ICAR, which is a 16 item test (Condon & Revelle, 2014). Right answers were coded as 1, and wrong answers as 0. Total ICAR score is the sum of all 16 questions for each coach. The ICAR has shown substantial reliability and validity in assessment against other GMA and IQ measures (Condon & Revelle, 2014; Revelle et al., 2020).
Dependent Variable: Client Behavior Change. Medical schools prepare physicians. In addition to the vast array of knowledge and technical skills needed to be an effective clinical physician, increasing literature has claimed that physicians need a behavioral repertoire that includes ESI competencies (Epstein & Hundert, 2002; Hammoud et al., 2023; Jacoby et al., 2021; Libbrecht et al., 2014; Nowacki et al., 2015). The perceived need is to increase the quality of communications and relationships with patients and their families, as well as peers and staff. This is often discussed as strengthening the physician's leadership and relationship ability which has been shown to increase engagement and motivation of others (Quinn, 2015), reduce stress and burnout of physicians (Dyrbye et al., 2019), reduce malpractice claims (Reiss & Neporent, 2018), and even increase treatment adherence (Khawaja, 2011). The Medical School at which this study was conducted introduced a curriculum to increase medical students’ ESI competencies. The curriculum included providing two separate coaches for each student. All coach and client data were collected under the auspices of the Institutional Ethical Use of Human Subjects board.
To provide feedback and encourage development of the needed competencies, as part of this curriculum, the entering medical students completed the ESCI in the 360 format within a few months of entry into the program. They also completed the ESCI in the 360 format 2 years later. The desire was to calculate how much a medical student's behavior has changed over the 2 years. As stated earlier, the self-assessment was discarded for this study.
Desired improvement was to be on all of the ESI competencies. Therefore, the degree of development of their ESI competencies was calculated as the client's (i.e., the student's) Time 2 total ESI score minus their Time 1 total score. Each competency scale score was calculated as an average item score, and then summed across the 12 competencies. This was labeled as their Client Behavior Change score.
To allow for increased robustness of the multivariate analysis, the Client Behavior Change score was also calculated for the sum of the competency scales in each of the four clusters (Boyatzis & Goleman, 2007; Boyatzis et al., 2015): self-awareness, self-management, social awareness, and relationship management. The cluster composition is in shown in Table 1.
Control Variables: Gender and Campus
Gender of coaches and their work location (two different campuses) were tested as control variables.
Results
To maximize robustness of findings, the data was analyzed first with the largest sample of 240 coach-client dyads for whom the ESI data was available. Then, it was analyzed with 135 coach-client dyads for whom the ESI and GMA data were available. The correlations of all of the coach's ESI competencies and Client Behavior Change are shown in Tables 2 and 3 for each of the samples. A coach's AO was positively correlated with Client Behavior Change over the 2 years for sample 1, and Conflict Management for sample 2.
Coach Competencies Means, Standard Deviations, and Correlations in Sample 1.
n = 240 dyads.
correlation is significant at the .01 level (two-tailed).
Coach Competencies Means, Standard Deviations, and Correlations in Sample 2.
n = 135 dyads.
correlation is significant at the .01 level (two-tailed).
correlation is significant at the .05 level (two-tailed).
To assess the distinctive contribution of each competency, the frequency of Client Behavior Change was trichotomized. The top third of clients showing improvement was then compared to the bottom third of clients. The top third showed a mean of Client Behavior Change of 2.55, while the bottom third showed a mean of −4.67 (n = 240, df = 158, t = −22.2, p < .001) for sample 1, and a mean Client Behavior Change of 2.64, while the bottom third showed a mean of −3.94 (n = 135, df = 88, t = −20.17, p < .001) for sample 2. T-tests were run comparing the top and bottom third of Client Behavior Change, as shown in Tables 4 and 5.
Coach Competencies Differentiating Highest Client Change and Lowest Client Change Sample 1 (n = 160 dyads).
Note: Groups are 1 = top third percentile of client behavior change, −1 = bottom third percentile of client behavior change.
Coach Competencies Differentiating Highest Client Change and Lowest Client Change Sample 2 (n = 90 dyads).
Note: Groups are 1 = top third percentile of client change self, −1 = bottom third percentile of client. Change self. Variables are standardized z-scores.
atwo-sided p-value.
Coach behavioral competencies that showed a significant and positive difference in Client Behavior Change in sample 1 were AO, adaptability, ESC, empathy, organizational awareness, and influence. Coach behavior competencies that showed a significant and positive difference in Client Behavior Change in sample 2 were AO, adaptability, ESC, organizational awareness, and conflict management. Coach behavior competencies that showed a near significant and positive difference in client behavior change in sample 2 were coach and mentor, empathy, influence, and teamwork. In sample 2, the coaches’ GMA did not show a significant effect on client behavior change comparing the top and bottom third of the distributions.
Partial support for Hypothesis 1 was found. To review, seven of the 12 coach competencies were found to significantly and two additional competencies near significantly differentiate the top and bottom third of the client behavior change in one or both of the samples. They were AO, adaptability, ESC, empathy, organizational awareness, influence, conflict management. Near significant findings in one sample included coaching & mentoring, and teamwork.
Hypothesis 2 was supported, GMA was not found to differentiate coaches that had more impact on client behavioral change.
To assess differential impact on client behavior change, multiple regressions were run on the two samples using Mplus 8.8 (Muthén & Muthén, 1998–2017). To increase the power of the analysis, the results were run separately on each of the competencies within each of the four ESI clusters, as shown in Tables 6 and 7, for the two samples. With sample 1, the coach's AO was most powerful in significantly predicting client behavior change in each of the four clusters. It was also shown that a coach's coaching and mentoring competency was negatively predictive of client behavior change for the social awareness cluster. In sample 2, neither the coach's GMA nor did most of the competencies appear distinctively more potent in the multiple regression analyses. Positive outlook showed a positive predictive effect for the social awareness cluster of client behavior change. ESC showed a negative relationship to client behavior change for the self-awareness, social awareness, and relationship management clusters.
Coach Competencies Predicting Client Behavior Change by Cluster for Sample 1 (n = 240 dyads) via MR.
Note: Estimates are standardized coefficients.
*p < .05 (two-tailed).
**p < .01 (two-tailed).
Coach Competencies Predicting Client Behavior Change by Cluster for Sample 2 (n = 135 dyads) Via MR.
Note: Estimates are standardized coefficients.
*p < .05 (two-tailed).
**p < .01 (two-tailed).
Partial support for Hypothesis 2 was found in that a coach's AO significantly differentiated client behavior change, but neither empathy nor GMA did. Because of the array of competencies shown to be significant and near significant predictors of client behavioral change from the top and bottom third of coach impact analysis and the paucity of distinctively potent competencies, it can be said that the coaches use of each of the competencies are likely relatively similar in impact (i.e., not explained by a subset of the competencies).
Discussion
To summarize the results, seven competencies often referred to as ESI competencies significantly and two near significantly predicted client behavior change over 2 years in one or more of the analyses. Given all of the other experiences and activities going on in the lives of the clients during the 2 years, to find significant and near significant differences in coach competencies suggests that they have an important effect on the coaching process, relationship and client outcomes. Significant findings in one or more of the samples were AO, adaptability, ESC, positive outlook, empathy, organizational awareness, influence, and conflict management. Near significant findings in one analysis included: coaching & mentoring and teamwork. The coach's mental neither ability predicted degree of client behavior change. A coach's competency of AO appeared to have a particularly strong impact on predicting client behavior change.
As reviewed earlier in the paper, coaching is an interpersonal process with many levels of emotional exchange between the coach and the client. The qualities of their relationship will likely affect the motivation of a client to change and the sustainability of that change. Therefore, how the coach handles their own emotions and literally manages their own behavior is key to the effect they will have on the relationship and client.
Of the competencies in the emotional intelligence clusters, AO had a dramatically strong effect. The clients were participants of a 4-year development program. They had to demonstrate knowledge and skill acquisition, ethical standards and comportment, and earn certification into their profession as a physician through passing numerous courses, clinical rotations, standardized patients assessments, and eventually through internships and residencies in a hospital. During their program, the clients are particularly stressed, fatigued, and preoccupied with earning their qualifications and being seen and experienced by others as credible and caring physicians. Helping the client juggle the many competing demands for their time and attention is part of what the coach is providing.
When the coach uses their AO, they ask questions and offer ideas to the client to help them organize and prioritize their competing demands. They role model attention to the highest priorities, and thinking through alternatives to reach their goals. It can help the client choose among options for their time. It quite literally enables the client to determine the most effective and efficient path to their desired outcomes given the inputs of their limited time and resources. The coach uses their AO to remind the client of their purpose and consider not just tasks. It can help a client avoid effort minimization and want to do better.
When coaching professionals or executives with multiple, competing demands for their time and focus, this finding may also apply. But in coaching people with less demanding work or home environments, with possibly more specific challenges, AO might be a less potent differentiator.
The coach also has to manage their own focus and emotional contagion with each client. This is when the coach's use of their other competencies is key. When the coach uses their ESC competency, it enables the coach to hold back their personal reactions to serve the client's needs. It also makes it possible for the coach to compartmentalize other things demands in their work or family life to focus on the client during the coaching sessions. If the client notices this, the coach's use of this competency becomes a role modeling of how they might act with patients and staff.
The coach's adaptability competency helps the coach shift approach when a particular approach is not helping the client sufficiently. This may take the form of changing questions or topics during coaching sessions. The more the coach is able to demonstrate being open to new ideas and the client's reactions; they are modeling an openness and flexibility needed in helping others. Even positive outlook keeps them keep going even when the stress and obligations seem overwhelming for the client and possibly even the coach.
When the coach uses their Empathy competency, they are seeking to understand the client. Often it appears as true listening in contrast to the possibility of being distracted when talking with someone. Tuning into the client's feelings, moods and state requires empathy. When the coach shows an interest in understanding the client, even if the client is visibly different than the coach, the coach is again modeling desirable behavior for clients, in this case physicians in training, to use with patients and staff. If the client experiences a rapport with the coach, they will be more open to listening to what the coach is asking or saying. The coach's use of the organizational awareness competency demonstrates to the clients an awareness of social networks and concern with the organizational norms and values of their medical center. Helping the client navigate the many networks, peer group pressures, evaluative supervisors, and cultural norms requires the coach to help the client tune into organizational awareness. It encourages the client to do the same because of modeling.
The coach's use of the Influence competency shows the client that there are different approaches to inspiring or influencing someone to act in a desired way. At the most fundamental level of the client's work, getting patients to comply with treatment recommendations or to helping families support a patient, a wide variety of ways to influence others is needed. It can provide approaches for the client to influence their superiors to change methods or practices. Similarly, when the coach uses the conflict management competency, it shows the client how it can feel to be in a conflict and have a functional and desirable resolution. The coach's behavior creates a possibility for the client to not merely avoid conflict with others, but actually attempt at resolution. The coach using their own conflict management and teamwork competencies helps the client perceive the importance of working with others, even when they disagree or emotions are intensely aroused, as occurs with colleagues as well as patients.
The coach's use of the coaching & mentoring competency is not only at the core of the coach's purpose, to develop others, but shows the client that even when not in physician–patient interactions, developing others is relevant and useful. It reminds the client of their primary purpose, that of helping and healing others. The coach's use of the teamwork competency reminds the client that seldom does a physician work with patients or colleagues alone. When working with others, there are ways to act to include and encourage others to participate on their shared purpose and objectives.
One competency that did not appear significantly predictive of client behavior change was ESA. It was opposite to what was expected. One possibility is that the coaches do not share their feelings and self-assessment with the clients, as a result, the clients do not observe this aspect of the coach's repertoire. Another possibility is that the informant measure (i.e., a 360 degree assessment) does not easily allow others to witness a coach's internal assessment of their state, their strengths, weaknesses, and moods. That is, they do not talk about it. A third possibility is that the coaches are focused on the clients and not themselves. It is also possible that ESA may be key to a coach's development, but not their actual coaching.
The observation that the inspirational leadership competency of the coach did not predict client behavior change can be understood. The coach is helping the client as a person, as a new entrant to a field, and during a maximum learning focused four to 6-year period. Although prior research has shown the inspirational leadership competency is key to effectiveness of physician executives, the clients were many years from being in such positions. So the coach's use of the competency may have less relevance at this stage in their development.
In the multivariate analyses of both samples, a coach competency appeared as oddly negatively related to client behavior change, but in each sample only within a subset of the client behavior change spectrum—those of certain clusters of competencies. In sample 1, the multivariate showed that a coach's coach and mentor competency was negatively loaded to client behavior change. That is likely because it had a higher inter-correlation with AO, which was quite strong in predicting client behavior change in the social awareness cluster, as well as all other clusters. In sample 2, a coach's ESC negatively predicted client behavior change in three of the clusters of client change. It is possible that the particularly high inter-correlation with two other coach competencies that had high loadings in the multivariate analysis meant that this was a statistical result of the other competencies pulling too much of the variance.
To find so few of the coaches’ competencies that had unique variance in the multivariate analyses suggests that the set of competencies and their inter-relationships create an overall impact on the client. As has been discussed in previous articles, the competencies are not thought to operate alone or separately from the others. A coach's use of Empathy will affect their use of Teamwork and organizational awareness. And so on for interaction among many of the competencies.
To check whether the samples used in this study were different in their behavioral change from the larger population of clients (i.e., medical students at this University), each of the samples was compared to the entire population. The t-tests were non-significant for both samples used in this study (sample 1: df = 535, t = .525, p = .600; sample 2, df = 430, t = −.812, p = .417).
The coaching occurred on two different campuses in two states. To check if this made a difference, t-tests were analyzed of client behavior change. It was non-significant for the 550 clients in sample 1 (n = 251 from one campus and n = 300 from the other campus, df = 549, t = .525, p = .598). Client behavior change was also not different for the sample 2 (n = 109 from one campus and n = 135 from the other campus, n = 244, df = 224, t = 1.306, p = .194). Gender of the coaches was not significantly correlated with client behavior change for either sample.
Limitations
This is the first study to examine a coach's competencies assessed against impact on client change. One limitation of this study is that the population from which the samples were drawn were medical school students. It may or may not be comparable to coach competencies that would predict client behavior change for managers, or other specific occupations. Additional studies and replications are needed to resolve this limitation.
Another possible limitation is that there are other competencies that were not assessed in this study that may be important. A qualitative study of the actual behavior and thoughts of coaches during coaching sessions, such as coding of critical incidents or coding of videos of coaching would help to address this limitation. While other possible competencies should be explored in future studies, the caution is that the competencies should conform to the definition of a competency.
Future studies should involve more coaches. This sample was limited to 60. More coaches should increase the variance of competencies demonstrated and could affect the impact on clients.
The certifications of the coaches from various associations, like ICF, WABC, EMCC, or AC, were not collected. Therefore, any conclusions about the benefit of certification on the coaches’ competencies could not be ascertained. Although all of the coaches in this sample were formally trained in many approaches to coaching, specific certifications were not required for their participation. Coaches certified by each association should be compared to others or those without formal certifications to fully appreciate the impact of such certifications.
Future studies should examine specific other characteristics of coaches to understand impact on specific groups of clients, such as race, gender, age, faith, ethnicity, and so forth.
Implications for Future Research
As stated above, the need for replication with additional types of samples is critical to generalizability to all coaching. As is the need for studies of coach competencies against an array of desired outcomes, and an array of different types of coaches and clients. For example, studies should be conducted with performance improvement as the ultimate outcome from an organizational perspective, or finding and affirming one's vision or purpose in life from a personal perspective. Other intermediary outcomes should also be tested, such as the coach's perception of the degree of client change in behaviorally anchored scales or goal attainment scales. In such studies, the client's perception of the degree of change on the same behaviorally anchored scales should be collected and assessed as well as the coach's perceptions.
As suggested above, a series of qualitative studies of the actual behavior and thoughts of coaches during coaching sessions, such as coding of critical incidents or coding of videos of coaching would help to understand the full range of possible competencies of importance. This could complement the important “micro-moments” studies already completed and those under way from both the coach and client's perspectives.
Studies with larger samples of coaches would be important to capture more variance in possible coach behavior. At the same time, such studies should address coaches from a wide variety of races, genders, age, social economic groups, faiths, and ethnic backgrounds, as well as when coaching clients of variety of races, genders, age, social economic groups, faiths, and ethnic backgrounds (Bozer & Delegach, 2019).
Although it would border on program evaluation, the competencies determining effective client behavior change should be examined as to nature of the coach's background, training, and certifications.
Implications for Practice
If these findings are replicated, the implication for coach certification is to create the “competency models” to be used with more precision. Testing of a coach candidate for their understanding of the job tasks and responsibilities is more valid with a knowledge test. Similarly, testing cultural compatibility of values held and preferred styles can be appropriately done with tests. Such material can be covered in asynchronous videos and tested easily. All of these characteristics should not be labeled competencies, but possibly desired threshold characteristics of a coach. The assessment of actual competencies can be done through reliable coding of audio or video recordings of actual coaching sessions. The list of competencies emerging from effectiveness research is both smaller and more precisely defined than existing models. Both of these aspects would result in more reliable and accurate coding. The above steps would simplify the certification process considerably from present activities and decrease the introduction of “noise” into the assessment process.
For training and retraining of coaches, whether through workshops or on-going supervision, the distinction of the actual competencies helps focus the development on those behaviors that really matter, and minimizes the introduction of inappropriate fads.
Conclusion
Coaches using certain competencies were more effective in helping clients improve their behavior in desired ways. The results are merely the first such study. More are needed. But it provides training and certification programs with evidence of what needs to be developed and screened for in coach's behavior to increase the confidence in their effectiveness and impact on clients.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
