Abstract
In 2008, United Way Toronto (UWT) launched the Creative Institute for Toronto’s Young (CITY) Leaders, a leadership development program for diverse young people working and volunteering in the community and social service sector across Toronto, Canada. The investment in leadership capacity building provided a unique opportunity to explore the short- to medium-term outcomes of the 104 youth who participated in five different program cohorts over 4 years. The UWT set out to evaluate the CITY Leaders Program by focusing primarily on its outcomes and effectiveness. This article provides a retrospective assessment of the impact of CITY Leaders, primarily through program graduates’ reflections on their learning and leadership development experience. It also documents outcomes at the personal, professional, and community levels.
Introduction
The Creative Institute for Toronto’s Young (CITY) Leaders is a leadership development institute for diverse young people working and volunteering in the community and social service sector across Toronto. Launched in 2008 by United Way Toronto (UWT), CITY Leaders was designed and delivered in partnership with the Youth Challenge Fund (YCF) and the University of Toronto’s Factor-Inwentash Faculty of Social Work (FIFSW).
Each year UWT has selected youth from neighborhoods across Toronto to participate in CITY Leaders, which offers a series of participatory educational workshops, applied learning projects, mentorships, and other leadership development opportunities over the course of 8 months. At the time of this evaluation, the program had graduated 104 young leaders from five different program cohorts. By the end of the current funding that was available, seven cohorts or a total of 149 participants had participated in the program.
Resources for Results, an independent evaluation firm, was contracted by UWT in November 2012 to design and implement an evaluation of the CITY Leaders program and to create an evidence base on which to assess the results of the first 5 years of leadership programming. The purpose of the evaluation was to
Examine a range of outcomes for CITY Leaders participants
Understand the contribution of various components to program outcomes
Explore how participants’ characteristics and context affect outcomes
Reflect on the effectiveness of the multi-stakeholder partnership
Increase understanding of what role and value graduates place on an ongoing formal CITY Leaders networking function.
Although leadership development programs are abundant in both the United States and Canada, there is a noticeable absence of evaluations that have assessed the mid- and long-term outcomes and impact of these programs (Russon & Reinelt, 2004), including the lack of appropriate measurement instruments (Black & Earnest, 2009). It is anticipated that this evaluation will contribute important new evidence regarding outcomes and learning about effective practices in leadership development. The evaluation was also well timed to coincide with two strategic reviews at UWT examining the organization’s priorities for youth programming and capacity building into the future.
CITY Leaders History
For well over a decade, UWT has made a sustained investment in leadership development. Guided by strong evidence that organizational effectiveness depends on the availability of highly capable leaders, UWT launched two leadership programs: The Emerging Leaders Program focused on social-sector managers and the Leaders of Today Program focused on Executive Directors. At the time, however, few leadership development programs in Toronto focused on youth, and none was designed to open doors for low-income and racialized youth (Erbstein, 2013). Through its youth development programming, UWT identified the need for youth leadership development to engage diverse youth from across Toronto in the social sector and to provide a “pipeline” of young leaders to support succession planning as baby boomer managers retired.
There was a growing awareness that the social sector must work differently with youth, and extensive efforts were made to engage and consult young people from Toronto’s 13 priority, “marginalized” neighborhoods about their learning needs and program design preferences. A Learning Partners Consortium was organized in 2008 to engage thought leaders in the field of leadership development in the design and development of CITY Leaders. As part of this initial design process, an extensive review of the literature on the design of youth leadership development programs was completed (Klau, 2006; Ritch & Mengel, 2009; Russon & Reinelt, 2004). CITY Leaders was launched in early 2008, and the first two social service-sector cohorts went ahead in quick succession with UWT as sole funder. The core offerings of CITY Leaders included an intensive program of educational sessions, mentorship, group-based applied learning projects, and an online networking platform. This design has evolved over time through formative evaluation, curriculum review, and participant feedback.
After the first cohort went through the program in 2008, CITY Leaders partnered with the FIFSW at the University of Toronto to improve program design, co-deliver, and co-certify the CITY Leaders program. In 2009, the YCF came on board as a funder, promoting a new neighborhood cohort to enhance the important work that UWT was already doing to promote leadership capabilities and opportunities for racialized and African diasporic youth from Toronto’s most marginalized neighborhoods. Additional collaborators and contributors, to both the design and delivery of the program, included the International Center for Studies in Creativity; The Shambhala Institute for Authentic Leadership; The Centre for Social Innovation (CSI); the Schulich School of Business, York University; Laidlaw Foundation; Toronto City Summit Alliance; The Ontario Council of Agencies Serving Immigrants; Maytree Foundation; the Anti-Racism and Cultural Diversity Office, University of Toronto; and the Metcalf Foundation. In 2009, an Advisory Council was also created to ensure ongoing quality and growth of the program. The Council is composed of members from the Faculty of Social Work, graduates of CITY Leaders, private-sector advisors, and UWT staff.
Relevant Literature
Building on the extensive literature review that was previously done during the initial design of the program, a further review was completed to identify core resources on leadership program design and to identify lessons learned from previous efforts to evaluate leadership development programs (Dugan, Turman, & Torrez, 2015; Russon & Reinelt, 2004). This review set the stage for the evaluation, laying out some basic definitions, identifying design elements, noting effective practices, and exploring theories of change that could support the design of the CITY Leaders outcomes evaluation.
The literature revealed consensus that conceptions of leadership in the social sector have shifted away from traditional, top-down ideas of individual leadership, which are now seen as being ineffective in our increasingly diverse and complex social sector (Bolden, 2011; Hubbard, 2006). Leadership is now conceived of as a more flexible, collaborative, democratic process that can come from a range of levels and locations in a community or organization.
Collective leadership is based on the premise that leadership is the product of groups rather than individuals . . . where collective leadership works best is in complex and messy situations that don’t lend themselves to easy answers—precisely the kind of situations the nonprofit sector confronts every day. (Hubbard, 2006, p. 13)
Emotional and social intelligence (Goleman, 2000) and critical self-reflection (Torrez & Rocco, 2015) are also increasingly seen as critical to effective leadership—employers seek employees who have strong social skills, self-discipline in work habits, cooperative abilities, and a motivation for continuous learning on the job (Rosch & Villanueva, 2016). Much of the current leadership literature concludes that the process of leadership development is more important than the actual content (Kress, 2006; Valorose, 2015). Although leadership development necessarily aims to cultivate new kinds of thinking and acting in individuals, leadership cannot be taught in a traditional way (Ritch & Mengel, 2009). Leadership education must create an environment that facilitates collaboration, analysis of issues, self-awareness, professional skills and capacities, and exploration of new leadership approaches and behaviors (Doherty, 2003).
In terms of youth development, Max Klau (2006) addresses two critical questions: “What conceptions of leadership inform the work of youth leadership educators in the field today? What pedagogical techniques are employed by programs to teach the model of leadership they espouse?” (p. 58). These questions are based on insights and research by authors such as Roach et al. (1999), who claim that many youth leadership programs are derived from adult theories of leadership and do not take into account the everyday or natural processes and situations that enable youth, particularly those in at-risk environments, to emerge as leaders. New forms of leadership development emphasize participation, distribution of knowledge, systems thinking, teamwork, and communities of learning and practice (Browne & Campione, 1990; Erbstein, 2013; Shah, 2011). Kress (2006) argues that youth leadership development should emphasize experiential learning and focus on the whole person within his or her context and not simply on one issue or problem or one set of skills.
Grantmakers for Effective Organizations (GEO) in the United States conducted research into community leadership development that remains highly influential and relevant today (Enright, 2006). This study affirmed the direct connection between strong organizational leadership and program results and suggests that leadership development interventions can be intentionally designed to promote long-term community impacts. They argue that leadership program interventions are most effective when they are (a) collective, (b) contextual, and (c) continuous.
Collective
Participatory leadership development strategies that build strong relationships among individuals, organizations, and networks are very effective in producing long-term outcomes. Relationship building is core to leadership development, supporting peer-based observational learning, where participants learn from each other by modeling and imitating new behaviors that they have observed (Bandura, 1986). Effective program practices in this area include activities such as peer-to-peer learning, network development, collaborative problem solving, building sector awareness, and collective advocacy work. Broad-based community involvement is also important in producing community results in social-sector leadership development programs (Dugan et al., 2015; Hubbard, 2006). Active engagement of the target community in needs assessment, identification of core values, planning, implementation, and evaluation ensure that the impacts of the program affect both the youth and the communities in which they live.
Contextual
Leadership development should be relevant and grounded in the context within which its participants are working. It must respond to the day-to-day problems that leaders face. For this reason, it is important to design leadership development processes that are respectful and consultative, authentically addressing youth issues and experience (Enright, 2006). Youth led processes contribute to the relevance and effectiveness of leadership programs. GEO suggests that learning should be customized and personally relevant to participants. Effective practices in this area include building collective knowledge and a process for examining pertinent issues facing participants’ communities, increasing self-awareness, engaging participants in meaningful services to the community, and organizing forums and learning groups focused on practical problem solving based on actual cases from participants’ working lives (Doherty, 2003; Dugan et al., 2015; Enright, 2006).
Continuous
Often leadership programs are not designed to support people to apply new learning and sustain it through their ongoing work with organizations and communities. New perspectives and behaviors are less likely to take hold when a person must apply them alone, without peer support or consultation. Customized, one-to-one coaching is increasingly being used in leadership programs to deepen the self-reflection component of leadership programs and promote behavioral change. Effective practices in this area include individualized coaching as a follow-up strategy to provide support for ongoing development and monitoring of personal and professional change and the use of peer groups to provide support and solve complex problems and issues (Enright, 2006; Ritch & Mengel, 2009).
In terms of insights from the limited number of systematic evaluations that have been conducted on leadership development programs, a number of themes have emerged including the difficulty encountered in contacting the alumni of a program, the importance of collecting both quantitative and qualitative data, the use of multiple methods to triangulate data, the need to focus on capturing process and individual differences, and the need to capture both perceptual and factual evidence related to outcomes and impact (Black & Earnest, 2009; Martineau & Hannum, 2004; Patton, 1990; Valorose, 2015). Of particular relevance to the current study is Pratt, McGuigan, and Katzev’s (2000) assertion that retrospective designs produce a more legitimate assessment of leadership program outcomes than traditional pretest–posttest methodology since many programs evolve over time, a process often described as adaptive implementation (Metz & Bartley, 2012). Pratt et al. (2000) also suggest procedures that can be used to enhance the memories of those completing retrospective questionnaires such as the detailed cohort specific descriptions of the program that were given to respondents in the evaluation of the CITY Leaders program.
Program Overview
CITY Leaders has evolved over the years in its targeting, objectives, program design, and content. Much of this evolution is the result of a consultative, continuous improvement approach that has been implemented through evaluation activities and active engagement of staff and partners in program refinement. In 2009, UWT evaluation staff worked to further consolidate the model, developing a logic model and learning objectives for CITY Leaders, increasing the connections between program themes and content to strengthen program outcomes.
CITY Leaders was designed to focus at the individual level, developing the capacity of young leaders to contribute effectively to the priorities of their organizations and communities. The objectives during the first 3 years of CITY Leaders focused on individual-level outcomes, with more emphasis on personal and professional development. By Cohort 5 in 2011, the objectives had been revised and refocused; they expressed more explicit expectations of improved individual capacity to effect change at the organizational, sectoral, and policy levels.
The Cohorts
CITY Leaders in the sectoral cohorts were diverse young people between the ages of 18 and 32 (and later 25-29) who were already working for social service and other community-based organizations delivering programs and services in Toronto’s inner-city and inner-suburban neighborhoods. Many of these young professionals already had a solid base of capacity in organizational management and program delivery. Many had completed a post-secondary college diploma or university degree. The cohorts in the program from 2008 to 2010 were sectoral. There have been no further sectoral cohorts since then.
CITY Leaders in the neighborhood cohorts were selected from racialized young community volunteers and activists (age = 18-24) who lived in UWT’s 13 priority neighborhoods. These young leaders were already actively engaged in organizing youth and supporting youth programming in their own communities and may already have been working for a community-based agency. This group was on average younger than sectoral cohort participants and had less formal education.
Overview of Program Components
The basic program consisted of a set of educational modules (half, full, and two-day sessions) and other activities such as mentoring and community-based social action projects. The major components of the CITY Leaders program included the following:
Opening retreat
This component has been facilitated over a weekend in a residential learning environment. The emphasis is on personal reflection, team building, introducing the idea of leadership, and creating a strong foundation and culture for peer learning.
A program of educational sessions
Offering core and elective modules in collaboration with guest instructors from a variety of high-profile community, academic, and social service organizations. The format for these modules varied from half-day educational sessions to 2-day intensive workshops.
Certification
By the second cohort (2009), UWT formed a partnership with the University of Toronto’s FIFSW and began granting certificates for successful program completion. As a result, regular attendance, completion of assignments, and make-up work requirements became more important.
Coordinators anchor the program
The UWT program staff played a proactive role in planning and organizing each session, identifying instructors and working together with them to ensure that the proposed session fitted with the identified learning objectives, that the facilitation and materials were well adapted to the learning styles and needs of participants, and that each session was coordinated with the sessions already facilitated. Staff also provided an “anchor” relationship for participants, assisting them to participate and benefit from the program.
Mentoring
The program mobilized senior professionals from a range of sectors and professions to be matched with participants to provide customized, ongoing counseling, career advice, and access to new networks.
Applied, group learning projects
Participants collaborated to identify and complete a practical project using their new knowledge and theory and connecting back to the community and/or the social sector. Financial support of CAD $1,000 was made available to each project team to support the implementation of their project.
Online platform
A proprietary Internet-based social networking platform was developed by UWT as an exclusive space for participants to pursue discussion, share resources, facilitate networking opportunities, and provide an ongoing forum for formative evaluation of the program.
Closing retreat
Added in 2010, this component was designed to provide closure to the CITY Leaders’ program, affirming the learning of participants and promoting the long-term relationship and networking of each cohort into the future.
Graduation
A ceremony organized and attended by funders, peers, and colleagues. It is at this session that participants made a presentation of their community-based social action/advocacy projects and received feedback on their work.
Note. CITY = Creative Institute for Toronto’s Young.
Some of the key changes in the program over the years have included improvements to outreach, recruitment, and selection; revision of program structure and content, resulting in the creation of core and elective modules; selection of more diverse instructors; engagement of alumni in the program; and revision of the mentorship component of the program design.
Evaluation Methodology
An Advisory Group with representatives from the two core program partners worked closely with the evaluator to guide evaluation design and implementation. The partners were interested in documenting the outcomes of CITY Leaders and reviewing the effectiveness of different components of the program in bringing about leadership outcomes. They were also keen to assess the merit of providing further support to leverage the program’s alumni network, to continue to promote leadership development beyond graduation. Although demographic information about participants was available, there was no baseline information benchmarking participants’ leadership knowledge, attitudes, and behavior. It was decided to engage 5 years of participants in a self-reported assessment of outcomes related to learning, personal growth, professional development, and community engagement (Pratt et al., 2000). The evaluation design used multiple methods of inquiry, to optimize triangulation of the data to ensure a higher degree of reliability of the findings (Greene, Caracelli, & Graham, 1989). Specific methodologies included:
an extensive document review of the program’s chronology, reports, plans, selection criteria, participant biographies, demographics, and previous evaluations;
an online survey of CITY Leaders Participant Outcomes—based on the Leadership Program Outcomes Measure (Black & Earnest, 2009);
telephone and in-person interviews with 23 key informants including past participants (graduates), mentors, partners, staff, and other community stakeholders;
two additional online surveys: one for mentors and one for instructors;
a “mini-scan” of leadership development in Toronto, key informant interviews with six managers of other leadership development programs and networks across Toronto.
The online survey of program alumni was the centerpiece of the evaluation and essentially followed the principles and procedures of the total design method developed by Dillman (2000). The team selected an academically validated questionnaire that had been developed by Black (2006) to evaluate the outcomes and impact of executive leadership programs and was thought to be adaptable to the CITY Leaders program model. This questionnaire, the Leadership Program Outcomes Measure (LPOM; Black & Earnest, 2009) was selected as the core instrument to assess the impact of the CITY Leaders Program on participants. Dr. Black granted permission to use and modify the instrument for the evaluation.
The questionnaire was modified only in areas where it was clear that the questions did not fit with our particular population of youth and the nature of the CITY Leaders intervention; the community outcomes section was modified the most. We also included two additional subscales, one to assess the components of the program and one to assess the degree of learning experienced by graduates. As recommended by Dr. Black, open-ended space for participant comments and reflections was added after each of the Likert-type scale questions. The revised questionnaire was pretested with a small group of alumni who were not eligible to participate in the alumni survey. Reliability analysis was conducted using Cronbach’s alpha, and subscale scores had a range from .798 to .881. This verified the internal consistency of the scales used to assess each of the areas of outcomes and justified our reporting of overall scores for each of the scales used in the modified questionnaire.
The online survey was sent out to each cohort with a program overview, to remind them of the process and content of CITY Leaders in their year (Fan & Yan, 2010). Nevertheless, a few respondents noted that they experienced some challenges in remembering the program, the nature of their participation, and how they were affected. They also commented that it was difficult to separate out changes in their lives and attribute them solely to this intervention. Many of these active, engaged young people have participated in multiple other programs and networks during and after their involvement in CITY Leaders.
Persistence was required to motivate past participants to complete the online survey. UWT staff connected personally with CITY Leaders graduates to elicit their participation, sending out a preliminary invitation to participate in the survey, and two email reminders. The completion deadline was extended, and every past participant was telephoned. In the end, a total of 50 graduates responded out of the 114 surveyed for a return rate of 44%. Nulty (2008) argues that online surveys are much less likely to achieve response rates as high as surveys administered on paper. It is critical, however, to minimize non-response error by comparing the profile of survey respondents with the composition of those who had completed the program.
Results
CITY Leaders Participant Profile
Demographic drivers in Toronto have pushed the need for youth leadership strategies that are solidly grounded in a cultural and community context. For many reasons, youth are engaging in civic action, organizing themselves in their own neighborhoods and cultural communities to tackle pressing problems, and to have a say in what happens. Still others are showing their commitment to change by working with organizations in their communities. These are the youth that CITY Leaders sought to engage in its leadership development efforts. CITY Leaders has also intentionally worked with culturally diverse and sometimes marginalized youth from low-income neighborhoods, and as a consequence, has had to develop individual and collective leadership within a context of complex systemic issues that are difficult to address. Participants come to the program with impressive experience. The competitive recruitment, referral, and selection process sought to ensure that participants entered CITY Leaders with work and organizing experience in their own communities. Many sectoral participants had already been involved in paid positions in the social sector, some managing programs and staff.
The sectoral and neighborhood cohorts can be seen as overlapping populations; the outcomes survey revealed that there was little significant variation in their responses. Each cohort blends a range of age, community experience, education, and neighborhood engagement, revealing a continuum of perspectives and experience in the overall participant population.
The Respondents
A good cross-section of participants from all cohorts completed the outcomes survey. The demographics of the respondents were representative of the CITY Leaders graduates.
Gender—Female 31 (62%) and male 16 (32%; n = 47)
M age—28.07 (n = 46)
Cultural background—of those reporting (n = 46), 28 were African diasporic youth (60.9%; 17 African Canadian, 11 Caribbean Canadian), eight were South/South East Asian (17.4%), six Canadian (13%), and six Other (13%).
Outcome Survey Results
At the core of the survey were five multi-item Likert-type scale questions with a range from 1 (not at all) to 5 (a great deal) designed to assess different aspects of program design, participant learning, and three different levels of program outcomes. These scales assessed:
Table 1 provides the overall Likert-type scores, the number of responses, the number of subscale questions, and Cronbach’s alpha for each scale.
Overview of Participant Ratings of Program Components and Outcomes.
Graduates’ Assessment of Program Components
The overall assessment of the extent to which various CITY Leader program components supported participants to learn was 3.54 out of a possible 5 points, meaning that graduates perceived the contribution of program components as having an effect (some/a lot) on promoting learning and leadership. Some of the key themes that emerged in this area include
Relationship building is a key strategy in promoting leadership development;
Retreats were powerful, focused investments in team building and self-knowledge;
Core program modules promoted strong conceptual learning.
The program has been kept largely the same from a structural and content perspective, but the two streams of neighborhood and sectoral participants had important differences in learning styles and interests. These differences shaped the delivery of the program.
Staff noted that it took courage for some neighborhood cohort participants to participate. They had had negative experiences in the classroom prior to the program, so it was critical for CITY Leaders to provide a safe space to learn. The program team worked hard to strike a balance between high expectations of participants’ performance (e.g., the need for accountability in a certificate program) and the critical importance of adapting the program to multiple abilities and learning styles. Other themes that emerged included the following:
Program instructors added great value to the program, promoting specialized knowledge and analysis, but they were not well positioned to develop influential relationships with participants.
The applied learning project appears to have affected participants unevenly and could benefit from more proactive facilitation.
UWT staff “anchor” the program by providing an excellent level of support to participants and promoting their success.
Most CITY Leaders participants were very professional as they moved through the eight month program. Nevertheless, the program can create emotional moments in people’s lives. Some participants come from difficult, sometimes traumatic personal circumstances, and have required the proactive support of the CITY Leaders staff to continue in the program. UWT staff have quietly and faithfully provided support and encouragement to many participants. In some extreme cases, staff have intervened to assist participants with personal crises. The fact that two of the coordinators are program graduates and the current manager has been with the program since its inception adds a great strength to this component of the program.
The most common mentorship activities for program participants were supporting career planning, sharing knowledge about the sector they work in, providing information about the current employment situation, and assisting in accessing new contacts and networks. Mentors, key informants, and graduates reported that the main challenges of the mentorship component were related to negotiating and cultivating a strong working relationship between mentors and mentees. These challenges included the need for clarity regarding the purpose of mentorships and difficulties in communicating and connecting. These respondents had a number of suggestions for the improvement of the experience, including (a) a group meeting at the beginning to establish goals and expectations for mentorship, (b) increased clarity of roles and responsibilities, (c) improved participant screening for readiness and commitment, and (d) regular check-ins to assess the progress of the relationship and the achievement of goals.
It has been difficult to establish a dedicated Internet-based network for CITY Leaders graduates under the umbrella of UWT. At the beginning of the program, UWT had invested in a proprietary online platform for information sharing, networking, and collaboration. This component did not work well, and over time, mainstream social media (particularly LinkedIn) provided excellent low-cost opportunities for professional networking and exchange.
Graduates’ Assessment of Learning
The survey asked graduates to assess what they had learned through their involvement in CITY Leaders. Given that some had been out of the program for 4 or 5 years, it was expected that their answers would provide a more realistic assessment of what had been useful to them, based on their experience and employment since graduation. This section focuses on the areas of learning that graduates assessed as making the greatest contribution to their development as leaders. The overall assessment of learning was M = 3.43.
Respondents (80%) reported that they had learned a lot or a great deal about creative thinking (M = 4.0). Women were more positive about the creative thinking session (females = 4.19; males = 3.63; t = 1.995, df = 45, p = .052). Leadership and leadership styles was important for 78% of respondents (M = 3.98). Graduate interviews suggested that they would like to increase the integration of critical self-reflection and exploration of leadership and leader styles throughout the program.
Diversity and cultural fluency were reported as having a moderate effect (M = 3.48), which is likely an indicator of prior training and life experience in this area, although some respondents commented on the positive effects of working alongside people of different religions and cultures in the City Leaders program. Some instructors felt that sector participants became more knowledgeable and skilled, and neighborhood participants became more self-aware, confident, and networked with colleagues.
The lowest scores for learning were given to social media and social enterprise; yet, some respondents noted how these components had been well-timed and highly influential in their professional lives. The program team was observed as doing an excellent job of adapting the program to the learning needs and styles of participants. Cohorts had different personalities and were demanding in different ways. This was a substantial challenge, since the education, age, and experience levels within and between sectors varied widely. The results raise questions about how much people already knew coming into the program and how much they could benefit from the learning offered. Were some over-qualified or just seeking credentials? Did the selection process target the people who could most benefit from the content offered?
Level 1 Outcomes: Personal Change
This question exploring the personal outcomes of participation in CITY Leaders revealed that the program contributed to significant learning, empowerment, and leadership outcomes, with an aggregate score of M = 3.38. Although the distinction between “personal” outcomes and “professional” outcomes detailed in the next section may be somewhat artificial, this question explored how participants’ self-perception had been changed and also revealed the process by which the program promotes learning.
Almost three quarters of respondents (72%) reported that “Exposure to new ideas and concepts led to my growth” a lot or a great deal (M = 3.92). Women were significantly more likely to say this (females = 4.13; males = 3.50; t = 2.128, df = 45, p = .039). In addition, 60% of all respondents noted “I improved in creative thinking” (M = 3.66). In open-ended comments, they added that they had developed a more in-depth analysis of social issues and a better understanding of how the non-profit sector works. Many felt better able to communicate effectively and articulate thoughts and ideas.
Becoming a part of this network changed my perspective of life and my approach to life; I lost myself in all of the learning and unlearning that CITY Leaders fosters. It took almost two years after graduation for me to truly find myself, I feel fortunate to have had the opportunity to unlearn all that would not serve my ultimate purpose. . . . I didn’t learn all of these lessons while in CITY Leaders, but I would say it was the catalyst for my re-examination of my life. (Graduate, Survey No. 13)
The research also confirmed that observational learning among peers is a powerful way for youth to learn and try out new behaviors. The second most frequent personal outcome reported by 60% of respondents was that they had met “people whose success they could imitate” (M = 3.74).
I’d never had an opportunity to sit down and engage with people doing this work. I thought—“wow this is a 25 year old manager in the social sector without a degree in social work . . . who looks like me!” (Graduate, Interview No. 2) When I came to CITY Leaders I wanted to change the world: I had thousands of ideas. Having the opportunity to meet and mingle with like-minded individuals allowed me to see how I could go about achieving my goal. (Graduate, Interview No. 13)
The exploration of new ideas and behaviors within a supportive community of peers is an important starting point for leadership development. The program brings together influential young people who would not otherwise have met. It facilitates opportunities for peer learning, role modeling, collaboration, and personal growth. Peers are role models and sounding boards for more realistic goals, plans, and attitudes.
The process from beginning to end gave me opportunities to evaluate myself but also to have others give their honest feedback about their perceptions and evaluations of me. (Graduate, Survey No. 36) I have struggled with the balance of personal life and professional life—understanding which part of myself to bring to each role. The CITY Leaders experience connected me to people who have helped me work through this challenge. (Graduate, Survey No. 11)
Team building and developing a cohesive cohort of young leaders have noticeably strengthened the program. Cohort relationships are very important, and a broad diversity of program participants contributes greatly to creativity and learning. Graduates also developed a strong perception of themselves as leaders, with a much clearer idea of their own leadership styles. The program has helped participants to integrate new perceptions and attitudes, including the notion of service to others and the community. Interviewees in particular expressed an understanding that they could now play a range of roles to build leadership and promote action in their communities. CITY Leaders has increased participants’ self-confidence, empowering them to make life-changing decisions and promote community change.
A consistent theme raised by graduates was their perception of an increased ability to promote social change or community action: 60% of respondents agreed that “CITY Leaders helped me to realize that I have the power to make a difference” (M = 3.70). Fifty percent of the respondents reported an “improvement in self-confidence” (M = 3.48).
Confidence and empowerment were a huge aspect of CITY Leaders. As a person of colour in this society it is much harder to gain respect and feel empowered. CITY Leaders helped me realize that I am a leader and can make positive changes. (Graduate, Survey No. 24)
The question, “I feel more respected,” was more likely to be answered positively by the neighborhood participants (M = 3.50) than by the sectoral participants (M = 2.53; t = 2.794, df = 48, p = .007).
One example of life-changing decision making is that graduates are going on to further training, certification, and education that they might not otherwise have pursued: 29 (58%) decided to do so, as a result of their involvement in the program. Graduates have also shifted career directions and, as a result, modified their training/education plans. Half of the respondents noted that “My CITY Leaders experience began a series of life-changing events for me” (M = 3.32). Again, women were also significantly more likely to note this (females = 3.61; males = 2.81; t = 1.883, df = 45, p = .045).
I was lacking inspiration. I was doing community work because I was passionate but with no support, which led to my burn out moment. When I was accepted into CITY Leaders it was a defining day for me because from then I knew it would change my life. (Graduate, Survey No. 34)
Level 2 Outcomes: Professional Change
Graduates reported the greatest level of change in the area of their professional and career development (M = 3.48). Relationships with peers and staff appear to have been the most influential aspect of the program, leveraging professional capacity building and advancement through observational learning and role modeling. “My exposure to other people and ideas helped me develop professionally” was reported by 80.9% of respondents to be important (M = 4.13). Women were more likely to say this (females = 4.39; males = 3.63; t = 3.32, df = 45, p = .002). The program offered participants a range of opportunities to work with peers and mentors to explore career directions and choices. A substantial 44.7% of respondents decided to change their choice of career as a result of their involvement in CITY Leaders. Women were far more likely to say that the program had changed their career direction (females = 54.8%; males = 25%; χ2 = 3.80, df = 1, p = .051). Open-ended responses indicated that many others had clarified their career options and the educational requirements of their career choices. The program shifted many graduates toward more “applied” career choices grounded in a commitment to community development.
A prevalent theme emerging from the survey and interviews was that graduates have become more personally committed to the social sector. Some said they were now interested in pursuing management track positions with social service agencies. This theme is of particular note because this generation has relatively high-salary expectations and talked about the difficulty of choosing between private and not-for-profit employment. Self-employment is increasingly an attractive option for this generation in the face of falling investment in the sector and reduced employment opportunities. About a quarter of respondents commented on their interest in pursuing more entrepreneurial careers in the “public benefit” sector. Participants came to the program with varying degrees of job experience and technical skill. Through CITY Leaders, they improved their professional skills to varying degrees. Lower outcomes scores for technical capacity building appear not so much a negative comment on the quality of the program content and delivery, as much as a reflection of the high degree of skill some participants had when they came into the program. An estimated 20% of participants had more than 3 years of work experience in the sector and/or advanced education coming into the program.
Graduates have been able to deepen their professional capacity and become more sophisticated in the way they work. Respondents confirmed that they had used their new skills and knowledge in their working lives: 54% reported “improving their skills in facilitating group decision making” (M = 3.57) and 47% “became more innovative in their approach to problem solving” (M = 3.47). Graduates also reported that they had experienced a change in perspective and, as a result, a change in their professional behavior. They have become more collaborative and cross-sectoral in their day to day work, and they reported being able to go deeper in their practice, improve the clarity of their objectives, and strengthen the effectiveness of their program management and delivery work.
Graduates improved their analysis of key political, economic, social, and cultural issues in Toronto. They gained a better understanding of poverty and more realistic ideas for advocacy, policy development, and social action. They also spoke of their increased ability to communicate and articulate their ideas and opinions more persuasively. Many reported having a stronger mental map of organizations, funding, and other resources in Toronto’s social sector, and many expressed a greater understanding and awareness of UWT. They have integrated new ways of thinking and working into their daily professional life.
Strong professional networks were the single strongest outcome: 74.5% reported that “CITY Leaders helped me to build a better network of contacts” (51% a great deal and an additional 23.5% a lot; M = 4.11). The research verified that graduates are also extending their connections well beyond the CITY Leaders network.
As a result of my CITY Leaders experience, I have gained the right skills to become a better networker. Specifically, the project we had to conduct, pushed me to make connections with professionals in the health industry, which has now led to me acquiring a board of directors position with a non-profit organization and may open doors for me to obtain employment in my field. (Graduate, Survey No. 14)
CITY Leaders itself has become a strong professional network bringing graduates together as colleagues, collaborators, and allies. The outcomes survey provided us with new insights into the extent and impact of this networking:
23 (50%) network only with participants from their own cohort
23 (50%) network with participants from their own cohort and other cohorts
None had lost touch with their own cohorts
Extensive networking among past and current participants has yielded a range of benefits. Graduates have formed lifelong friendships and collaborations, many have gained critical advice and support to assist them to find and grow in jobs, and still others have formed professional alliances that have served to deepen their ability to affect outcomes through their work. Interestingly, graduates have started to sub-contract and hire each other. Graduates are gaining professional power and creating a youth “voice” in their organizations and communities. Graduate interviewees spoke of becoming more comfortable wielding their power, both in the community and in their organizations. More than two thirds of the survey respondents (68.1%) also reported they are “now more confident and better prepared to assume leadership roles in organizations” (M = 3.89). They have deepened their leadership identities, allowing them to take on leadership roles with more confidence; they are more able to speak up at work, ask for support and do other things that they had previously perceived as risk taking. Yet, more than that, the program appears to have raised the understanding of graduates that there is a professional field of practice within which they can play an important leadership role.
Engaging youth to discuss critical social issues has begun to create a uniquely young and culturally diverse “voice” and perspective to the debate about how to solve the pressing challenges of poverty, social inclusion, accessible transportation, community safety, public health, and food security. Some recognized that as racialized youth, they feel excluded from the sector. This is a critical issue for the field: Opening doors for these young professionals is not just about connecting youth to new people, events, and opportunities. It is also about changing the psychology and perceptions both of the sector and of youth who are seeking to engage in social change. The research identified that CITY Leaders has begun to change the messaging (both as internal dialogue and as external reality) about participants’ right/ability as racialized youth in Canadian society to be a part of the social development sector. CITY Leaders is promoting new perspectives about what is possible, not only among the youth themselves but also among the adults who have connected to the participants over the course of CITY Leaders.
This new, tested confidence has advanced graduates’ careers, supporting them to participate in their work and communities in a more focused and influential way. More than two thirds (68.1%) noted as follows: “I developed the confidence to operate on a different level in my organization and/or career” (M = 3.85). In addition, 46% noted the following: “I have been promoted or have moved up to a level of more responsibility in my work” (M = 3.17). Graduates noted that the support, advice, connections, and opportunities they gained from the program have given them momentum in their careers: some are working for larger, more established organizations, and 46% noted that “I increased my earning power” (M = 3.22).
I was recently approached and offered a new job. I was very unsure about whether or not this was the right role for me. My CITY Leaders mentor guided me through the process and assured me it was a good move. I wouldn’t have been able to see all sides of this decision without the knowledge from CITY Leaders, and also from the direct guidance of my mentor. I recently took the job—it is a big promotion for me and challenging work. I’m certain that my CITY Leaders experience, certification, and relationships helped me to get the job. (Graduate, Survey No. 10)
Some graduates, however, experience barriers as they bring their new knowledge and behaviors back to their organizations. Sixteen graduates (34%) had encountered a lot or a great deal of “barriers to bringing their knowledge back to their workplace.” More women than men reported that they have “shared the knowledge and learning from CITY Leaders with my colleagues and/or management” (females, M = 3.94; males, M = 3.19; t = 2.57, df = 45, p = .014).
CITY Leaders has promoted employability. For an age demographic that has very high levels of unemployment, the CITY Leaders graduates enjoy high-employment levels. Only one of 50 respondents reported currently being unemployed and one under-employed (although four did not answer the question about their current employment, which could mean that they are un/under-employed). The employment information confirms that these young professionals are making a commitment to the social sector: 54.5% are employed in the social services and a significant number (11.4%) are employed in the health sector, which suggests a possible new focus for a CITY Leaders networking session. Respondents reported their current job titles and said that graduates are moving into managerial stream positions as project coordinators and program managers, on par with or more advanced than other social-sector employees of similar age and education. Yet, many of the graduates are employed through time-limited, project-based funding through initiatives funded by the YCF, Action for Neighbourhood Change, and the National Crime Prevention Centre.
Level 3 Outcomes: Community Engagement
Community engagement outcomes have been much more complicated to interpret. There were lower levels of reported change as a result of CITY Leaders (M = 2.92). This is likely related to the fact that youth participants were selected precisely for their prior, active involvement in community-based organizations and in community organizing. It appears that graduates are not necessarily more engaged, but rather that they are more deeply and effectively engaged. In general, neighborhood cohorts were much more likely to say that they had been affected at this level by CITY Leaders. Sectoral cohorts did not necessarily become more connected to communities but did develop a stronger understanding of the issues and needs, developing a stronger ability to empathize, listen, and work more effectively. Sectoral participants did not necessarily connect to neighborhoods in the same way.
A recurring theme in the research has been the increased sophistication of CITY Leaders graduates in their understanding and analysis of complex issues, such as poverty and managing diversity, and their enhanced political awareness. More than half (56.5%) reported as follows: “I increased my appreciation of cultural differences and diversity” (M = 3.39) and 46.8% noted the following: “I have a better understanding of complex issues like poverty, diversity and social exclusion” (M = 3.38).
Graduates are also “now more committed to working for social justice and change in my community/neighborhood” (54.4%), (M = 3.3). Graduates also seem to also have internalized a desire to critique how the social sector currently works and to value innovation, new practices and broader horizons, bringing a stronger new youth perspective to the sector.
The package of new knowledge, experience, and behaviors facilitated by CITY Leaders has enhanced graduates’ ability to work toward their vision for change, with 60.8% reporting “I have a stronger vision for change in my community” (M = 3.70) and an additional 57.5% noting, “I believe that I have new skills, tools and connections to support me to work toward that vision” (M = 3.80). CITY Leaders appears to re-energize and bolster community activists, but this level of involvement may not be sustainable. Thirty graduates (62.5%) reported that they are currently doing volunteer work and are involved in a wide range of formal and informal volunteer positions, including after school programs, youth mentoring, community boards, and youth associations. Some have continued actively to support CITY Leaders, sitting on the Advisory Committee, facilitating workshops and assisting staff. Some have increased their political involvement as they support community organizing and join political parties.
It appears that work and school displaces volunteering to some extent, but given that almost two thirds of CITY Leaders graduates are volunteers, they are well ahead of the average rate of volunteer engagement for their peers across Canada. A 2010 study of volunteering in Canada (Vézina & Crompton, 2012) found that just below half of young people aged 25 to 34 (46%) reported that they volunteer. Greater volunteer engagement, however, received the lowest outcome scores: about one third increased their commitment to local organizations, 23% were more active volunteers, and 24% said that they had focused their commitment by reducing the number of organizations at which they volunteer. More than half (53.2%) of respondents noted that they had increased their responsibility and authority as volunteers. The neighborhood cluster reported a substantially higher outcome in this area (70.6% neighborhood cluster and 43.3% sectoral). Graduates have broadened their reach and pursued social innovation and change through networking and collaboration.
18% have championed a new cause since joining CITY Leaders.
45% became involved in groups and networks that reach across Toronto.
21% have extended their involvement to the provincial or national level.
Graduate Assessment of Overall Outcomes
In retrospect, alumni rated the overall change in their lives as a result of CITY Leaders at M = 6.83 on a 10-point scale. Women reported significantly higher overall scores (females = 7.19; males = 6.13; t = 1.99, df = 45, p = .052). More than two thirds (65%) of alumni felt that it was important or extremely important to continue to offer the CITY Leaders program. Almost all (97.9%) said that they would recommend the program to their peers and colleagues.
I talk excessively about this program because it really did change my life. I had very high expectations about this program and I was blown away that they were exceeded and I really did become a creative young leader. The opportunities that I have been exposed to are invaluable. It’s not just what I learned in the classroom but everything I experienced as a result of my involvement with this amazingly important program. We need more leaders, keep going! The world needs you! (Graduate, Survey No. 3)
Alumni Would Like CITY Leaders to Continue With “Post Graduate” Activities
Previous consultations with graduates had identified a number of ways that CITY Leaders could continue to bring alumni together and support them in their careers. Through the survey, alumni expressed varying levels of interest in a range of activities:
82% were very interested in providing “ongoing professional development” opportunities for graduates.
62% were very interested in creating social opportunities for graduates to reflect, reconnect, and share wisdom.
53% wanted to promote a “strong network presence that supports community action” (very interested): neighborhood (82.4%) and sectoral (40%).
46% want to “use technology creatively and strategically” to support network activities; 44% were very interested in developing a member driven, collaborative structure (very interested): neighborhood (70.6%) and sectoral (33.3%).
Since the completion of this study, two additional cohorts have completed the program for a final total of 149 graduates. The UWT has initiated an Alumni Network, hosted sessions on business start-up, and navigating complex service systems. They are also planning a future cross-cohort retreat.
What Has CITY Leaders Achieved?
City Leaders Has Contributed to Substantial Change in Participants’ Lives
CITY Leaders outcomes are significant: Alumni ratings of change show that the program affected them in a range of ways that have enhanced their ability to lead in organizations and communities. Deeper qualitative analysis revealed that substantial numbers of graduates experienced high degrees of change in their lives as a result of their participation in the program.
Through a retrospective assessment of five program cohorts over the past 5 years, this evaluation found that CITY Leaders also created outcomes that go beyond the immediate changes stimulated by participation to promote “developmental” changes (Black & Earnest, 2009) that persist after the intervention is over, complementing and building on one another. The evaluation also revealed evidence of “transformative” change: fundamental shifts in attitudes and behaviors. There is sufficient evidence to support the belief that these shifts will not only be maintained but will develop further in the future (Black & Earnest, 2009; Enright, 2006; Kress, 2006). Some of these complementary developmental outcomes include the following:
Sustained peer support and collaboration
Strengthened confidence and identity as a leader
Increased personal agency and self-direction
Clarity and realism about educational and career pathways
Enhanced reach and professional connection to the social sector
New approaches, increased politicization, and strengthened commitment to social change
Enhanced professionalism and effectiveness
A Solid Foundation Has Been Built to Promote Broader Change
CITY Leaders has effected change beyond the level of the individuals targeted by the program. In their open-ended responses, alumni identified many ways in which they had contributed to action and change through their volunteer and paid leadership in organizations and communities. Although it was beyond the scope of the evaluation to measure these changes, it was possible to identify three general areas of onward, deepening change: sectoral strengthening, deepened social change practice, and collaborative civic activism (Dugan et al., 2015). Although the evidence suggests that such changes are occurring spontaneously, there is great potential for CITY Leaders to cultivate them more intentionally through work with alumni networks.
Sectoral strengthening
The CITY Leaders program is opening doors for young racialized professionals (Erbstein, 2013). Many CITY Leaders alumni have made educational and career choices that lead them to new ways of engaging in and supporting the social development sector, and they are gaining more responsibility in their careers. There are indications that further targeted intervention could continue to support them to move into executive and civic leadership career pathways.
Deepened social change practice
CITY Leaders alumni are applying new practices in their work. They noted that they are better at their jobs and their community organizing, through having gained a better picture of the root causes of the challenges they are working to resolve. They have greater potential to promote long-term systemic change because they now understand the need for strategic approaches that support policy change (Marks, 2015).
Collaborative civic activism
Alumni involvement in CITY Leaders has promoted their inclusion in the social development sector. Many expressed a strengthened commitment to civic action and social development (Checkoway & Gutierrez, 2006). Their increased reach means that they are now connected to leaders and networks across the Toronto region, the province, and beyond. They are moving out of work contexts that are functionally “siloed,” cultivating new multi-sectoral partnerships and promoting collaborative problem solving. Great potential exists to support CITY Leaders’ alumni network to become a vehicle for collaborative agenda setting, to create a legitimate, persuasive youth “voice” for Toronto.
Participant Outcomes Appear to Be Enhanced in Diverse Groups
Targeting has been an important theme in this evaluation. It has been difficult to draw any reliable conclusions about who can benefit most from participation in the CITY Leader’s program although the evaluation revealed that a diverse mix of youth have benefited from the program. The younger, least educated and experienced participants are perceived to have gained a great deal from the program, but it is hard to differentiate between what they learned as a result of the program and what they gained from the influence of a diverse group of other older, more experienced participants in their cohort. The more experienced, educated participants noted that although the program content was not entirely new to them, they had deepened their professional practice. Most noted that participation in the program had led to enhanced personal empowerment. These identified gains are congruent with what has been observed in the literature (Doherty, 2003; Enright, 2006; Russon & Reinelt, 2004; Valorose, 2015). Future research, however, needs to focus on a more in-depth exploration of the gender differences that were found in the identified impacts of this leadership program (Rhee & Sigler, 2015).
CITY Leaders Has Created an Effective Set of Strategies for Leadership Development
This evaluation has provided a better understanding of the complex mix of interventions that contribute to engaging youth and developing youth leadership. UWT has done a remarkable job of navigating among changing and sometimes competing visions and expectations to create a challenging, open, exciting forum for learning. The evaluation has demonstrated that CITY Leaders creates safe spaces for youth to model and test new leadership behaviors. It offers a set of overlapping and complementary interventions and strategies that operate both on an individual and a collective level (Figure 1). This evaluation identified the following interventions that appear to be mutually supportive in promoting youth leadership outcomes.
Building connections that otherwise would not happen—Promoting relationships between peers, and with mentors, instructors, and new networks.
Facilitating a peer community and observational learning—Promoting a supportive environment of trust and respect that creates new learning opportunities through role modeling and team building.
Introducing new ideas, values, and knowledge—Integrating diversity, leadership, creative problem solving, collaboration, and social justice into new approaches.
Co-creating the big picture—Facilitating a more confident, sophisticated analysis of the systemic forces that produce poverty and exclusion in Toronto and creating an understanding of the structure and resources of the social sector. This builds a more realistic, youth-led, collective vision for social change.
Promoting critical self-assessment—Supporting a better assessment of each participant’s personal assets and aptitudes, providing an understanding of personal decision making and leadership styles, promoting self-direction in pursuing goals and opportunities, and benchmarking progress.
Enhancing professional skills and capacity—Enhancing technical skills, professional knowledge, and access to job and career advice with a view to building professional competence.
Providing access to new opportunities—Opening doors to new contacts and hands-on learning opportunities that allow alumni to try out new ideas, skills, and approaches (e.g., collective learning projects, jobs, conferences, round tables, workshops, “experts,” networks, and political parties).
Key interventions—A package of effective strategies for building youth leadership.
This evaluation offers compelling evidence that UWT has made an important investment in a generation of young leaders who will continue working together and will influence the direction of the social sector in the decades to come. The cohesion and influence of this group is likely to grow significantly in the coming years. Feedback from interviews and surveys, however, suggests that the model currently being delivered may not be the most effective configuration of these strategies. Respondents suggested a number of changes to the program including strengthening the pedagogical anchoring of the program to enhance the links between different components and sessions, changing the structure and timing of the mentorship program, extending CITY Leaders programming to serve graduates, and linking the CITY Leaders participants more strongly to community agencies and other leadership networks. The new insights derived from this in-depth investigation of the CITY Leaders Program will hopefully be instructive to other groups and organizations that are undertaking leadership development initiatives in their own communities.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: The United Way of Greater Toronto funded this study.