Abstract
This study comparatively examined two disability arts partnership projects’ stakeholder perspectives on inclusive practice and social impact. It did so through an innovative abductive research design to visualise the qualitative findings of a comparative social impact assessment of active citizenship. In this paper we examine the inclusive practices of the disability arts partnership projects and an inclusive methodological approach. The approach sought to visualise the social impact footprint, or scope, of disability arts projects on radar diagrams. In developing this approach, we were able to document the enabling outcomes for the lived experience of artists with disability. The research has implications for the inclusion of artists with disability as part of disability specific art projects, ensembles of artists with disability together with nondisabled artists, and the way that creative process outcomes have social impact on the stakeholders and communities where they are performed. For the organisations involved the project demonstrates the wider outcomes of the artistic practice through the social impact of their disability arts programs on their internal and external stakeholders. Further, for arts funders it provides a tool for comparative understanding of social impact across programs.
Introduction
In Rimmer’s (2020, p. 296) study of arts-based organisations in times of austerity (such as post-Global Financial Crisis and the COVID-19 pandemic) he makes the assertion that: …community-based arts organisations operate in a landscape fraught with tensions and contradictions, as they negotiate a course between their principles, the requirements of their funders and the expectations of those with whom they work.
This is primarily because a large proportion of their funding comes from government sources, and government cuts to the arts are now more prevalent globally. A similar challenge is faced by disability service organisations (DSOs) in Western nations where individualised funding has disrupted the traditional block funding business model (David & West, 2017; Green & Mears, 2014). In this new paradigm DSOs must not only seek new sources of organisational funding but must also present value propositions to consumers with disabilities (Darcy et al., 2019; Muir & Salignac, 2017). As Azmat et al. (2023) comment as to the emphasis on social responsibility in the nonprofit sector, this has increased competition between arts organisations for decreasing levels of funding. One of the outcomes has been an increasing level of reporting and accountability to the funding bodies through methods such as social return on investment (SROI). Most nonprofit organisations do not have in house capacity for evaluations such as these and hence use part of their funding to pay for external evaluations (Wearing et al., 2021). The decline in arts funding is a global issue as also evidenced in the US and Europe (Jacobsmeier, 2021; Rex & Campbell, 2022). For this reason, now it is more important than ever for organisations in the disability arts sector to demonstrate the social impact they make.
In this paper we show how nonprofit organisations can measure and demonstrate their inclusive practice, relevance and social impact at the individual and program level to funders through a visual radar diagram. This is in response to calls for researchers ‘to better map their empirical experiences of research as a key way to develop existing inclusive research methodologies’ (Liddiard et al., 2019, p. 164). Our focus is on those organisations with disability arts projects. We do so through the lens of a social relational model of disability (Allan et al., 2020; Darcy et al., 2023), with an emphasis on the lived experience of the participants, in this case, artists with disability (AWD), and our own histories and experiences with disability, inclusion and exclusion. One of the authors identifies as a person with mobility disability with high support who has been active in the disability and arts communities (Darcy, 2019). A second author has lived experience as being a grandmother and carer of a person with cerebral palsy with mobility and intellectual disability who is actively engaged in the performing arts sector.
Matarazzo’s (1997) ground-breaking UK study Use or Ornament? The Social Impact of Participation in the Arts was arguably the first significant empirical investigation of the social impact of the arts. This study highlighted the expansion of the UK’s arts and disability movement and its international significance. It emphasised shifts in attitudes and the movement’s role in legislative and employment practice changes. While Matarazzo’s study was case-based and utilised several types of data to develop a set of indicators, the methodological process was vague and is the subject of much critique. For example, Merli (2002, p. 108) stated that it was “flawed in its design, execution and conceptual basis” and called for future research to develop new approaches to examining the social impact of participatory arts. As a result, new theoretical and empirical approaches to evaluating social impact in the arts have emerged (e.g. Belfiore & Bennett, 2007a, 2009, 2010, Clements, 2007; Collins et al., 2022; Daykin et al., 2008; Galloway, 2009; Matarasso, 2003; Merli, 2002; O'Neill, 2009; Onyx et al., 2018; Reeves, 2002; van den Hoogen, 2014).
The aim of this paper is to extend prior research conducted in the arts and disability spaces, to examine social impact, grounded in social and human capital theory (Halpern 2005; Onyx, 2014; Putnam et al., 1993), to develop a social impact framework of arts and disability that focuses on inclusive practice and creative process. As an international issue, this paper takes direction from the United Nations (2006) Convention on the Rights of Persons with Disabilities (CRPD) work that seeks to improve the social participation of people with disabilities through all areas of citizenship. Some 165 plus nation-state signatories require not only accessibility (Article 9) to the arts (Article 30) but need to report on their progress towards improved social participation and disability citizenship in these areas. The CRPD is underpinned by social approaches to disability that focus on lived experience, barriers to social participation and transformative outcomes (Barnes, 2019; Bunbury, 2019). Therefore, we explore the importance of lived experience and an understanding of AWD as part of the creative process, rather than the barriers to access, participation, representation and empowerment that they face (Collins et al., 2022). This is part of a larger study that investigated the social impact of arts and disability projects, from which three papers have been published (Darcy, et al., 2022; Maxwell et al., 2022; Onyx et al., 2018). We begin with discussions on social impact and how it is measured before presenting the methodological approach, discussion and conclusion.
Social Context and Artistic Practice
In the Australian context, the Australia Council for the Arts acknowledges the significance of arts for people with disabilities (PWD) and their contributions to the arts. In 1995, a crucial research project was commissioned to address the lack of data on AWD (Walsh & London, 1995). The resulting report explored AWD participation, identified barriers, and scrutinised the legal framework, notably the Disability Discrimination Act 1992, advocating for ‘inclusion.’ The 1995 project also scrutinised the state of the Arts and Disability movement in Australia, addressing funding bodies and arts venues as ‘gatekeepers,' international best practices, and strategies for enhancing Australian approaches. This influential work prompted a five-year program by the Australia Council, fostering an arts and disability best practice scheme, a national forum, information and advocacy programs, and advancing research in the field. Notably, it resulted in the creation of fact sheets and guidelines for Access all Areas, an initiative by the Australia Council for the Arts, prepared by Accessible Arts and Arts Access (Accessible Arts & Australia Council, 1999; Australia Council, 1998). In the years following the ‘Access all Areas’ report, there were many new programs focussing on AWD which has expanded the sector. Research has begun to delve into the success and challenges of these programs in relation to internal and external processes (Azmat et al., 2023; Collins et al., 2022).
The arts sector is commonly acknowledged as a challenging employment market (Raynor & Hayward, 2009). Yet, when aligned with personal development, offering a pathway to employment becomes a significant aspect in assessing a project’s potential outcomes and its impact on the social inclusion of AWD (Collins et al., 2022). Social capital emerges when interpersonal relations evolve to enable collective action (Coleman, 1988, 1990), making it equally pertinent to focus on the transformations occurring in nondisabled participants as it is on those with disabilities. In fact, it is the dynamic nexus of this change that creates social inclusion and enhances social impact.
Social inclusion for PWD involves them existing as part of the collective ‘us' rather than as ‘them' or the ‘other' (Hillman, 2005). This integration is shaped by various factors, including structural and environmental elements related to universal design and accessibility, access to information and communication, and support for community engagement (Hadley, 2022). Moreover, social inclusion embodies equity, a fair process enabling both disabled and non-disabled community members to access socially valued opportunities, resources, products, and rewards. The realisation of social inclusion hinges on community members appreciating the diversity and pluralism within their broader community (Azmat et al., 2018; Mor Barak, 2000; Winston et al., 2022). In other words, it is when a community recognises that its identity is created by all its members – with all their diversity and similarities. It is also generated through the allyship of nondisabled artists, producers, and policymakers (Hadley et al., 2022). The arts (e.g., visual or performing) play a dual role in shaping and reflecting identity. Public appearances of people with disabilities (PWD) in performances, films, advertising, or on television reinforce their societal inclusion and value. Moreover, artworks created by AWD enrich the diverse artifacts that collectively represent our society. Conversely, the absence of such artworks contributes to the ‘invisibility' of this significant and important group.
Disability arts projects provide opportunities to make significant gains towards social inclusion: first, through the production of artworks by PWD, which speak of their lives and experiences, and secondly, by sharing such projects with the general community and influencing positive community attitudes (Merrells et al., 2018). Initially, the ‘general community’ is predominantly made up of the families and friends of the AWD – who may vicariously experience artistic achievement, and therefore, increased self-esteem and greater social inclusion. However, as the project or artwork is maintained, the ‘general community’ grows to include a broader audience who may have their views, attitudes and assumptions challenged and changed for the better. Therefore, we are reconceptualising disability from vulnerability to artistic agency where artists are free to express their lived experience in their words, performances and other actions. However, understanding if and how this has occurred requires the measurement of social impact, which is an area of study and development that is receiving considerable attention.
Measuring Social Impact
Social impact refers to “the generation of increased (or decreased) levels of social, cultural and human capital within the constituent communities in which an organisation operates” (Onyx, 2014, p. 12). Of these, the most significant is social capital, which is defined as the “features of [a] social organization, such as trust, norms and networks that can improve the efficiency of society by facilitating coordinated actions” (Putnam et al., 1993, p. 67). Social capital is multidimensional, and often operates in conjunction with cultural and human capital (Schuller, 2001; Woolcock & Narayan, 2000).
The measurement of social impact therefore must also be multidimensional and for this research is conceptualised as part of broader active citizenship. Earlier research by the authors (Edwards et al., 2012, 2015; Onyx et al., 2018) identified several crucial factors of social impact including the essential preconditions of being supported and valued (bonding social capital), the development of skills (human capital), the development of new networks as channels of information and new opportunities (bridging social capital), and supportive connections with powerful allies (linking social capital) (also see Darcy et al., 2014).
The measurement of social impact in research on active citizenship extends well beyond a monetised cost-benefit analysis of SROI. It also goes beyond conventional evaluation measures, which are usually limited to measures of intended output in relation to specific program objectives. However, sustained impact involving significant personal and social change is complex and likely to be the result of multiple causal factors and influences. It is also worth noting that co-design in program development and evaluation is important to ensure that there is potential to improve the lives of those involved (in this case PWD, their families, informal carers and paid attendant support) by empowering them to be involved in framing, making decisions and having control over how they are involved in the research process (Durose et al., 2012). Evaluation should identify which projects work through the inclusivity they offer, for whom, from whose perspective, and how they are best implemented. What does ‘good’ look like, from the perspectives of different stakeholder groups, including PWD? The challenge is to get researchers to change their relationship with PWD from seeing them as subjects to work ‘on’ and ‘for’ to working WITH in an equal co-design and co-production process (Debono et al., 2021). Social impact is dynamic, and involves a wide set of stakeholders, including disabled and nondisabled participants – and their lived experience. There may also be important unintended ‘spill-over’ effects (Veal, 2010) not specified in the statement of organisational objectives. These may be “very important in terms of impact of organisational activities on the wider community, with potential positive and negative wider implications over time” (Onyx, 2014, p. 7).
To capture some of the complexity of social impact, this research developed a framework for identifying and measuring social impact in a community setting, relating to arts projects within a disability program funded by a government arts statutory authority (Onyx, et al., 2018). Central to this work is how PWD were valued as part of the artistic collective and creative process development. Hence, the relative accessibility and inclusive practices within programs is essential to AWD feeling valued in the same way as nondisabled artists. This research in turn built on an earlier project in a different community setting, involving interviews, focus groups and a factor analysis of a questionnaire survey with 71 items which produced a model with 11 factors (Edwards, et al., 2015). This model suggests a ‘rippling’ of social impact at both the individual (micro) and organisational (meso) levels through four distinct stages, from the central core of welcoming and belonging to the impacts felt in the wider community.
Measuring Social Impact Through Radar Frameworks
Traditional methods of measuring social impact of organisations are problematic in cases with marginalised groups (Aldridge, 2014). As the current study focused on AWD (with varying levels of physical, sight, hearing and cognitive ability), some of whom were situated in regional areas, access to these individuals was problematic and a large quantitative study was deemed inappropriate. The ripple model (Edwards et al., 2015) needed adapting and another technique for evaluating social impact using mixed methods was sought; the radar framework was deemed an appropriate technique to include the lived experiences of the research participants, other stakeholders and subjective voices of the researchers.
By its designation, a radar diagram is a web-like model which maps out the score, or importance, of certain factors. Once the factors are connected, the spread displays the overall profile or area of the phenomenon under investigation (see Figure 1). The visualisation of composite indicators (such as the UN Human Development index and the OECD Better Life index), of which social impact is one, is important “in order to facilitate interpretation and enhance understanding of indicator components and their evolution over time” (Albo et al., 2019, p. 1). When composite indicator data is presented visually, it provides policy makers with evidence to make change and also aids in public communication by building an informed citizenry and democracy (Albo et al., 2019). Two reviews conducted by Albo et al. (2016) and Draper et al. (2009) showed that radial visualisations are very common techniques in depicting information; in particular the use of radar charts. This is due to their ability to display complex multidimensional data, their compact layout and aesthetic design. The former review did reveal a difficulty by the user in interpreting radar charts over other radial visualisations such as a flower glyph and circle glyph when layering (i.e. comparing or contrasting) multiple cases in one chart. But “in static reports that focus on gaps or divides (e.g., digital divide) and that focus on a single point in time (i.e., there is no temporal aspect) Radars can be useful” (Albo et al., 2016, p. 577). Example radar diagram.
Researchers in environmental science have used radar diagrams to display the effect of different types of resource or livelihoods management schemes on the five types of capital: human, physical, financial, natural and social (see for example Chen et al., 2013; Zhou et al., 2021). This research has also been adopted in the tourism domain where Qian et al. (2017) have shown that community-based tourism contributes to greater livelihood improvements in rural China, based on the impact to the five types of capital, than a lease-to-operate model. These studies are based on quantifiable indicators ensuring an easy to assign final score for each dimension of capital.
Radar frameworks have emerged over the last two decades in the social sciences area (see Dickson et al., 2011; Fredline et al., 2005; Schmidt et al., 2022). Following a triple bottom line approach, Fredline et al. (2005) map the economic, social and environmental effects of events on a triangular radar diagram. Malena (2003) and Malena and Finn Heinrich (2007) create a ‘civil-society diamond’ radar diagram with four dimensions; structure, environment, impact and values, in order to measure and compare civil societies. This was based on a set of 74 indicators from the CIVICUS Civil Society Index (CSI). Finally, Dickson et al. (2011) use a six-point ‘legacy radar framework’ in their evaluation of the Olympics and Paralympics Legacy outcomes that included disability and accessibility factors.
All of the radar studies mentioned above rely on different types of data (mainly qualitative) by which to quantify the ‘score’ associated with each of the dimensions. In the case of the civil society diamond, it is the scoring of the indicators (for each dimension) which is subject to criticism (Anheier, 2005; Lyons, 2009). That is, there is often subjective bias in the measurement of indicators, particularly if there is an attempt to make cross-cultural comparisons (Howard, 2005). Anheier (2005, p. 242) advocates “taking measures of different facets or civil society dimensions, comparing and analyzing their relationships” rather than combining all of the dimensions to create a single number which defines a civil society because the concept is multifaceted and multi-dimensional. This caution is echoed by Feldman (2013, p. 709) who stated that “researchers should not use filled radar charts to compare social indicators”. These concerns were noted in the analysis phase of the current study and are discussed in the following methodological section.
Methodological Approach
The larger aim of the study was to explore the social impact of arts and disability projects at the individual and program levels. To address this aim, a qualitative abductive case study approach was designed to develop a social impact radar framework. Taylor et al. (2018) advocate for the use of abduction in nonprofit research as a form of creative scientific inquiry to make new discoveries. The qualitative case study methodology facilitates the deconstruction and subsequent reconstruction of diverse phenomena (Baxter & Jack, 2008; Bryman & Hardy, 2004; Stake, 1995; Yin, 2014). Case studies hold significant value in research and learning due to their ability to synthesise and contextualise information from a variety of data sources. They serve as rich and holistic examples that combine qualitative and quantitative data, offering a comprehensive understanding of complex phenomena. By integrating diverse forms of data, such as surveys, interviews, observations, and archival records, case studies provide a nuanced perspective, enabling researchers and learners to explore real-world situations in depth and draw insightful conclusions (Veal & Darcy, 2014; Yin, 2014).
Project Characteristics.
aShort = life of the grant only (1 year); Medium = 2–5 years, and Long = 6+ years or a project with no fixed end.
Two researchers were assigned to each project in the tradition of action research that has been used in disability and arts sectors (see Pettican et al., 2023; Stack & McDonald, 2014) to follow their development over the two years. This embedding of researchers with projects provided a deep and rich understanding of the inclusive and creative processes used. A range of empirical and secondary data was collected from these projects over the duration of two years which the research team compiled together into ten case studies.
Within each of the case studies, primary data was collected in the form of in-depth interviews, focus groups and open-ended surveys with project managers, artists, carers, audience members and other stakeholders invested in the project, and were employed depending on the nature of project and stakeholders. These methods were supplemented with research team observations and reflections of the interaction of AWD with each other and nondisabled artists, together with their overall agency within the disability partnership projects.
The methodological process underwent several steps, which we go on to detail below: 1. Establishing connections and initial interviews with project leads; 2. Developing a schedule of engagement for each of the projects depended on their timeframe and milestones (e.g. at initiation of projects, during the creative process development, at the performance/exhibition and post); 3. Research team members visiting the artistic space or attending performances over multiple times during the course of the project; 4. Observation, interviews, focus groups, surveys with various stakeholders involved with the project; 5. Collecting secondary data; 6. Analysing primary and secondary data; 7. Developing the radar framework (factors, indicators, scoring the projects, radar diagram mapping); and 8. Writing up the case studies – developing the narrative.
A precursor to understanding social impact assessment of the disability partnership projects is understanding the way that programs established inclusive practice to work with AWD. The inclusive practices used in each project aimed to ensure that all artists, whether disabled or nondisabled, were able to fully participate in the program. This meant reasonable adjustments and strategies for engagement were embedded throughout the program design and delivery (Connor et al., 2008) and began with initial discussions with project leads. Project leads then facilitated interviews or focus groups with as many stakeholders as possible connected to the project. Central to the inclusive practice was understanding the dimensions of access of the AWD in the project teams to enhance artistic practice and the creative process used in each partnership project.
Research participants provided written or verbal consent and the project was approved by a university ethics committee and in some cases participating organisations’ own research ethics processes. The data included audio-visual and print material, media reports, social media content, websites, internal organisational documents, and project-related materials. This diverse data collection enabled the ‘triangulation' of information (Bryman & Hardy, 2004; Stake, 1995; Yin, 2014), permitting the exploration of the central research phenomenon (social impact) through multiple lenses, such as the social relational model of disability. This approach enhances data credibility, providing a more reliable and robust foundation for confidently interpreting social impact factors and project indicators.
Developing the Radar Framework
Developing the radar framework was an iterative process, in other words, it continued to be modified as more data was gathered, and the instrument was refined. Each case study was the responsibility of two members of the research team, with the remainder of the team interrogating the interpretation by the assigned members to promote trustworthiness, accountability and reliability in the process. The qualitative data was collected from interviews and focus groups and audio recordings were transcribed. The interview and focus group narrative were thematically analysed (Ryan & Bernard, 2003) by two members of the research team to ensure interrater reliability using NVivo software. The text was coded to a modified set of social impact and active citizenship factors (Edwards, et al., 2015) using the coding process described by Williams and Moser (2019).
The coding process commenced individually within the research team, followed by collaborative discussions to assess the trustworthiness of each code as an indicator of social impact. This dialogue laid the foundation for consensus on coding across all case studies. The two main researchers drew upon their observations of case interactions (notes and debrief meetings), while the rest of the team maintained objectivity, observing projects from a distance through attendance, hearing, and reading accounts of performances/exhibitions. The manual coding approach was applied consistently to audio-visual material, internal documents, and media reports. Recognising the importance of understanding the viewpoint of the participants, we positioned ourselves as ‘insiders' within the community (Gold, 1997), which provided unique insights into their daily lives and challenges. This insider status allowed us to establish trust and rapport with the artists, their family members and project staff more easily, facilitating open and honest discussions during our interviews and observations. However, being insiders also brought potential biases and subjectivity into our research, which we mitigated through our ongoing reflexivity during our project meetings and within our research team during the coding process.
The first stage of open coding resulted in many themes, which, when grouped in the second stage of axial coding, aligned to the Social Impact and Active Citizenship factors that were developed by Edwards et al. (2015). This required a considered understanding of the context of the projects and the populations involved (e.g. disability service organisations, arts and disability organisations, arts organisations, PWD, AWD, nondisabled artists, community arts volunteers and professional arts workers, etc.). However, the indicators of impact were modified to reflect the nuances of individual projects and components as emergent from the coding process. Therefore, there were iterative interpretations of the initial 71 items from the Edwards et al. (2015) study, and the factors were modified based on the emergent coding. A total of 10 factors (five individual and five program) and 33 indicators (each with a definitional description) resulted from the analysis (see Appendix 1). The individual factors were; belonging, social values, networks, skills/creativity and wider social impact (with 17 associated indicators). The program factors were; program activity, welcoming, social values, networks and wider social impact (with associated 16 indicators).
To illustrate, we take the factor of ‘belonging’ which had 5 initial items: 1. I trust people in this organization 2. I feel a sense of belonging to this organization 3. This organisation has been like a large extended family for me 4. I feel everyone is equally welcome in this organization 5. It has been easy to make new friends in this organization
As we show in Figure 2, open coding of the data appeared to align with items 1, 3 and 5 as ‘trust’, ‘family’ and ‘friendship’ were commonly cited as components of the ensemble-like nature of these projects. Yet, several codes were not present and so a second level of axial coding was undertaken to identify a ‘relationship between open codes, for the purpose of developing core codes’ (Strauss, 1998, p. 109). This stage of coding identified a core code that was not connected to the 5 items which is how people belong given their common reason for being part of the project. Once these core codes were organised, the indicator name was developed and definitional description agreed by the research team. Coding process for ‘belonging’ factor.
In the iterative and abductive process of formulating factors, indicators, and definitional descriptions, the subsequent phase involved scoring each partnership project against these indicators. The development of the scoring system adhered to the principles delineated in the legacy radar framework (Dickson et al., 2011). This approach was chosen to ensure a comprehensive ‘360-degree' consideration (Toegel & Conger, 2003) that encompassed all populations, stakeholders, and data sources in evaluating the impact of cultural participation. The scores that were developed were: 1 No evidence of presence 2 Little evidence (i.e. very few examples of presence and/or many examples of absence) 3 To some extent (i.e. generally the same evidence to indicate presence as absence) 4 To a large extent (i.e. more evidence of presence than absence) 5 To the fullest extent (i.e. overwhelming evidence of presence)
Once the scoring system was in place, the research team operationalised the principles developed for the social impact radar framework based on collective and evidence-based scoring described by Malena and Finn Heinrich (2007, p. 348); “Jury members are asked to review the ‘evidence’ or research findings for each indicator, and then to decide by majority which score description best reflects the current state of each indicator”. In this study the following process took place: (1) Two to three researchers individually scored each of the 33 indicators for each of the projects after engaging with all of the case study material and considering the lived experience of the AWD and the inclusive practices present. (2) The researchers met, discussed and settled on a common score for each of the three or four indicators for each factor. (3) The researchers created a total factor rating; an average based on the three or four common indicator scores. (4) The factor ratings were tabulated and plotted on a radar diagram. (5) Each project was then given a total combined factor score out of 132, which was based on 33 indicators multiplied by 4 (the maximum score per indicator). (6) Finally, each project was given a social impact ratio (SIR) out of 1 (total score/132), where scores closest to 1 indicated that a project provided greater evidence of social impact.
Figure 3 provides an example radar diagram from this study. There are two key dimensions required to understand the social impact radar framework — the individuals involved (i.e., skills/creativity, belonging, social values, networks and wider social impact) and the overall program factors (i.e. program activity, welcoming, social values, networks and wider social impact). Each has its own set of indicators and is divided by the red line that separates the two dimensions of impact. The left-hand side of the red line represents the micro/individual factors, where the right-hand side of the line represents the meso/program factors. The radar profile is the area within the thick line which connects the factors. Social impact of the project can be ‘measured’ by the area of the radar, which produces a SIR (which is written below the radar diagrams provided in Figures 4 and 5). Example social impact radar diagram. Radar diagram for project 1. Radar diagram for project 5.
Project Social Impact Ratios and Inclusive Practices.
Using Radar Diagrams
The previous section outlined the methods used to determine the individual and program impact factors and associated indicators of disability arts projects, and to score these on radar diagrams based on the evidence of social impact. This section presents two of the projects in more detail based on the case studies that were undertaken by the research team. We deliberately chose two cases with the highest and lowest social impact footprints for demonstration purposes.
Project 1 – Large Social Impact Profile
Project 1 was run by a Sydney-based organisation, founded in 2000, which offers creative programs for adults with disabilities. As part of their programming the organisation has an art program focused on recreational art. The organisation operates five days a week and caters to approximately 120 PWD per week. In addition to this recreational art program there was an additional program offered which took a group of talented visual artists aside in order to develop their artistic skills. The Art Director and Principal Artist sought additional funding to extend this art program. The aim was to provide 12 months of specialised visual arts and digital media tuition for AWD, helping them acquire skills and build networks to establish professional arts practices. The necessity of introducing AWD to professional artists in their specific medium led the existing program to transform into an effective mentoring initiative, with professional nondisabled artists actively contributing to the development of artistic skills among AWD participants. During the research period, the artist group developed and moved into a gallery space, and they continue to work in this type of space to this day. They have had numerous opportunities to exhibit their work in local galleries in the city and surrounding suburbs and have presented their work to different audiences. The project was found to score highly on both individual and program factors (see Figure 4) and the total score of the indicators was 112/132 giving a SIR of .85.
The organisation established a program that fostered a sense of comfort and belonging among its members/artists, creating a collective atmosphere. The Principal Artist, in articulating their approach to working with PWD, connects the program’s social values to its inclusive and welcoming nature: So we’re trying to treat everybody– equally. I think I notice personally over the years of working at [organisation], even though we’re supportive of people with disabilities and we have to sort of have an understanding of all their little pros and cons I guess, I don’t think we really see them as somebody with a disability ... A lot of them become really good friends of ours and become quite close to you, sort of like a little family. Especially the artist group … everyone’s quite close. So I think … when I meet other people with disabilities it’s not the first thing I think, it’s not the first thing we see, so we really do try to see them for the person first, and for their skills and their interests and everything.
The parents of the artists have also seen how the new program values the development of their children. The following quotes from two of the mothers illustrate this sentiment: I think it’s wonderful because it respects Mark as an individual and it expects Mark to take responsibility for his actions. It’s a small island of independence for Mark in a very highly structured life. He was always on the perimeter of a crowd. He was off standing back. Now he’s in the middle, he’s right there where everything is happening. It’s made such a difference to him and to me and of course he’s turned out to be apparently quite a good artist, very contemporary, which goes over my head a little bit.
The artists’ work in a gallery which is separate from the other activities of the organisation and this has led to the formation of a close-knit group. The parents of the artists have also noticed that their children enjoy being part of a group. In a discussion with a mother and her artist son she asked him: “you’ve got a lot of good friends here” to which he replied, “and they’re my age”. He feels good spending time in the gallery and distinguishes it from the workshop space in which he feels less comfortable.
In addition, the research has shown that much of the professionalism through exhibiting to external audiences has personally affected the artists. In discussing the changes to the artists, the Principal Artist noted: I think there’s been a massive like confidence boost of course … there’s an excitement throughout a lot of the members [artists], so they love – they know what it is to have an exhibition now they know what it means and I think the work involved, like there’s an element of professionalism coming through in the work.
The welcoming and belonging nature of the organisation and its programs have facilitated some strong networks. The networks that the artists form have an influence on their shifted social values where they have been given the tools and support to appreciate the value of art and artistic expression as noted: I always like working with other artists, especially my mentors as I learn new things which makes my art more professional. My mentors always listen to my ideas and what I want to achieve with my stories and illustrations and then they suggest ways to make it better. They do this in a way that I am always included in the creative process and in this way the process always remains fun. (AWD)
Both the increased skill set of the artists and the networks that the organisation has made through the program have been able to challenge perceptions and social values of arts and disability. They have also meant that the program has been allowed to develop beyond its initial scope through the wider social impact in the local and arts communities: I think the project has been really important in allowing the artists to develop skills and things like that, but it’s also been important in enabling us to develop working partnerships with some established artists, and some established arts venues and organisations, and to increasingly have artists linked with cultural producers … they don’t only have to engage with the arts as therapy. (Art Director)
The exhibitions at mainstream galleries and arts spaces have attracted arts-specific audiences and the general community. It was evident that these exhibitions were well received by these audiences. For example, at the end of the funded project, five of the artists exhibited their work at a Sydney gallery. One of the speakers at the event was an arts critic with a top Sydney newspaper who explained that she had been tracking the movement of the artists for some time and was very enthusiastic about the development of their work.
One of the audience members on the night (who attended with the researcher) explained that he did not expect to witness the type of artwork produced by the artists. The 2 things that struck me at the launch were how well the exhibits had been put together and the enthusiasm of the artists. Having the artists there at the launch, so proudly discussing the work they were doing made it a memorable evening.
Furthermore, these workshops have assisted in professional art production, which has meant people in the arts community and beyond have been affected by the program. The nature of the collaborative work with artist and mentors had profound impact on the mentors; allowing them to learn throughout the process. For example, one of the mentors explained: It’s a challenge sometimes ‘cause you never know what’s going to come up because you’ve got particular perceptions and ways of going about something and they have too, and then trying to meet in the middle and you know, find a way to tell stories together… coming here you’re working with all sorts of different artists with different backgrounds and trying to know how far to push and work with and you know, the discoveries.
Project 2 – Small Social Impact Profile
The production of project 2 was driven by a local government council in Sydney, in collaboration with an interactive media arts group and a music therapy group, based at a local university. Project 2 consisted of making and exhibiting mobile, interactive, visual musical instruments to be played by people of all ages, which generate music and animation from small mobile carts. They are designed to be presented in public spaces for an accessible, fun, music and media experience, which offers opportunities for social inclusion and wellbeing.
The first instruments were developed (and thus owned) by the interactive media team, with the input of a music therapy group. We will call this part of the program phase 1. A workshop was conducted, which included music therapists and students at the local vocational training institute, some with disability, who were involved in a training program to support and present the work. The instruments had two public appearances in the local council area. The first engagement was in prototype form. The second appearance, for which project 2 received its funding, was a feature presentation at the council’s celebration of the International Day for People with Disability. This involvement we will call phase 2.
Project 2 scored weakly on individual and program indicators at 51/132 with a SIR of .39. This is reflected in the small radar profile modelled in Figure 5.
Project 2 was found to have created slightly more program than individual social impact. This was due to the creative development of the instruments being undertaken in Phase 1 by a professional interactive media group, that is, there was limited co-design in the initial program and PWD were rarely involved in the development of these instruments. Instead, instruments were displayed at public events and participation, by all members of the public including PWD, was encouraged. As a result of its short-term nature, the project did little to create a welcoming environment in which the participants felt supported by the activities and facilitators. In fact, generally the project had a smaller impact than the council expected. Council’s role in developing the project was as a facilitator and advocate rather than the ‘service provider’ per se. Therefore, they could not control the output created by the instruments, which was up to the professional artists engaged. The director of the interactive media arts group explained that… …the actual level of creative participation was only with the end product, in the form of some performances. [Phase 1] involved much more creative participation during the process of making.
It was clear that the council espoused equity in its drive to include PWD in mainstream art and provide participants with the knowledge, tools and support to appreciate the value of art. We’re also generating – trying to bring it more into mainstream arts and the links with the gallery, and that is part of that, but also trying to expand the opportunities for people to experience this. (Council organiser).
The response from the audience to the performance was very positive and many acknowledged the ingenuity of the technology, which was unexpectedly participatory. One of the audience members, also a DSO worker, said that they found it a “creative experience that dispels the notion of “disability” being an impediment to creative interaction”. Another who was a family member, local resident and policy maker explained that the performance showed “a true respect for PWD that showcases talents instead of objectifies them as recipients of care only”.
Project 2 leaned on the networks that had already been established in previous iterations of the creative process of the work, closely linked to new funding rounds. However, these networks increased to include several DSOs and higher education institutes, which assisted in developing social values across the wider community. There was a group [of volunteers] who will probably learn – they’ve learnt a lot about people with disability but about the technical thing of the mobile carts because they had big input into it. So one of the possibilities, and I don’t know where we’re up to with this, was that they would be trained by [the interactive media group] to do the maintenance of it, a couple of them. Yeah, they were involved at the performances and there was quite a lot of involvement, yeah, and volunteers from other organisations. (Council organiser)
After funding was exhausted, the instruments were taken away from the council by the interactive media arts group and toured in other parts of the country, which meant that project 2 was a one-off event in the community. As a result, those networks that were created to develop and maintain the instruments were no longer needed as the project had been completed with no ongoing plans. Hence, without ongoing artistic activity, the instruments returned to the partner organisation and were not used by individuals involved in the project any longer.
Discussion
This study responds to calls for research to develop new approaches to examine the social impact of participatory arts (Clements, 2007; Collins et al., 2022; Galloway, 2009; Liddiard et al., 2019; Merli, 2002), these also have a strong theoretical grounding. We have done this by basing our work on social impact grounded in social capital theory (Halpern 2005; Onyx, 2014; Putnam et al., 1993), which informed the construction of the factors in the radar framework. This approach has provided a mechanism by which to understand the nature of social impact of active citizenship across the multitude of programs that were offered. It has provided a way to comparatively understand social impact underpinnings of what would otherwise have been individual project case studies. Inclusive practice to support AWD is essential to social impact outcomes.
In addressing the aim of this paper, we found that visualising social impact through a radar diagram helps to both build and support the narrative around contextually specific projects. The narrative which supports the visual depiction amplifies the inclusive practices of projects with high social impact, but at the same time, presents a glaring omission of exclusion of PWD, and subsequently AWD as being central to the creative process in low social impact projects, and their lived experiences. Hence, in low social impact projects AWD were directed by nondisabled program managers who set the artistic creative process without the involvement of AWD.
The findings demonstrate significant implications for arts and disability practice and research. Existing literature on the intersection of social impact, disability, and the arts often emphasises the advantages of arts participation for the AWD as highlighted by Hutchison (2005). This study reinforces the notion that removing barriers to access, participation, representation, and empowerment (Collins et al., 2022) can result in profound and positive social impact for participants, both at the individual and collective levels. The ripple model (Edwards et al., 2015) served as the foundational framework, identifying key factors that shed light on the nature and broad mechanisms of this impact. According to the ripple model, the ultimate impact manifests as ‘contributing to the community.' Then, the radar framework illustrates how this occurs (or not) in terms of the contributing factors leading to this ultimate impact.
The radar diagram offers a form of data visualisation for storytelling. It renders the factors and their relative weight immediately visible. The relative strength of each contributing factor becomes immediately obvious. Project 1 (the project with a larger social impact, illustrated by a wide shaped profile on the radar diagram) showed that a supportive organisation (board and CEO) was crucial to the success of the project. The welcoming and supportive environment, as well as the external funding allowed Project 1 to take risks in their provision of art which assisted in providing a higher calibre of programs for the participants. The ability to be flexible around the needs of external nondisabled artists, the AWD, and other participants with disability facilitated success and removed barriers to access, participation, representation and empowerment (Collins et al., 2022). Importantly, Project 1 benefited heavily from the growth of networks, connections and collaborations with professionals and organisations from the mainstream art world providing conditions for the growth of social capital (Leonard & Onyx, 2003).
In comparison, the radar profile of Project 2 is a much smaller contracted shape. Here social impact factors such as belonging, social values, skills and wider social impact, particularly for individuals, were not well developed because there were barriers to access and participation. Additionally, availability of equipment was an issue and although some networks were developed, a wider impact was not established and the project did not continue after funding was expended.
This study extends previous work on social impact of the arts to include disability. The arts provide an opportunity to create social inclusion (Belfiore, 2002; Belfiore & Bennett, 2007a, 2007b; Collins et al., 2022) which is a long term social impact and a project outcome. This study has also made a significant contribution to social indicators research in communities which are more difficult to access. We found that radar diagrams portray a profile of social impact in a fashion which is easy for the disability and arts communities to use to demonstrate success and impact. This concurs with what Albo et al. (2016) found; that displaying complex data visually in a compact diagrammatic form was aesthetically pleasing from a visual perspective, and Draper et al. (2009) reported that it is an increasingly common technique in information visualisation research as it provides a distinct design metaphor.
Limitations
There were three main limitations in the study. First, the timing and initiation of the research process: Some of the projects had already started rather than being engaged in a participatory action research process from the beginning. This was most notable when accessing AWD and audiences, with regards to the outcomes of cultural participation. Second, the variety of the projects required different methodologies to be employed that reflected the diversity of artistic endeavour, nature of the audience and stage of production. From a research perspective this was both exciting and problematic. Third, the research design was adapted from previous research where the focus groups led to an online questionnaire for a largely homogenous group of staff and volunteers. The population for this study differed in many ways, and the adapted methodology for each project and stakeholder group called for an innovative approach to implementing and understanding social impact on individual and program levels. While the amplification of social impact came through a variety of artistic forms and digital recordings/broadcasts, the success of the project is due to how individual AWD performances resonate with their audiences.
The radar framework was developed with data that relied on qualitative sources and the use of expert assessments (including group consensus) of each of the identified factors of the social impact model previously developed. Although common data sources were used, such as interviews and observations, there were other data sources that varied from project to project such as secondary data, including media releases. Additionally, the research design had no control group or a longitudinal dimension beyond the two-year lifespan of the projects.
Conclusion
The method and the reflexivity are a large part of consciously being aware of and inclusive of lived experience. Tables 1 and 2 demonstrate that of the 10 projects, we can clearly see a typology emerging where the projects which had larger social impact had more inclusive and sustainable practice. Importantly, the study shows the heterogeneity of AWD across the partnership projects together with how they were supported through inclusive practice. There is no solution for inclusive practice simply based on disability type and support needs. As social relational approaches to disability emphasise, each individual’s ‘impairment effects’ (Collins et al., 2022; Connor et al., 2008) need to be incorporated within program design and the creative process.
This study looked for evidence of social impact of arts and disability projects. The study was not an evaluation of projects as it sought to present an alternate method of researching social impact of projects individually and collectively. The methodology and framework carried out in previous research was adapted to identify the underlying factors that contribute to both individual and organisational social impact. In this paper we have reported on a methodological approach we deem appropriate for demonstrating social impact and inclusive practice.
The radar diagram therefore has multiple uses. For the individual project it provides a useful visual management tool to explore the strengths of the program, and more importantly to identify where changes may significantly improve future impact. The results can be used to demonstrate meaningful impact to future funding bodies by nonprofit organisations or the disability arts sector as a collective. This is essential in the current economic climate presenting real challenges to community arts organisations (Azmat et al., 2023; Rimmer, 2020). This impact is more than economic valuation provided by a SROI and touches the lives of artists, their carers, and the wider community in their active citizenship and lived experiences. For the wider policy implications, it provides an evidence based tool to measure social impact in a complex environment. The radar framework, therefore, is a more nuanced evaluation tool, one that does justice to the variation in contributing factors to broad social impact at the individual, program or organisational levels.
Supplemental Material
Supplemental Material - Inclusive Practice and Comparative Social Impact of Disability Arts: A Qualitative and Abductive Approach
Supplemental Material for Inclusive Practice and Comparative Social Impact of Disability Arts: A Qualitative and Abductive Approach by Simone Grabowski, Simon Darcy, Hazel Maxwell and Jenny Onyx in International Journal of Qualitative Methods
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was commissioned and funded under the NSW Arts and Disability Partnership between the NSW Department of Family and Community Services (FACS) and Arts NSW, Australia (grant number 2012001045).
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
