Taking risks to enable participatory data analysis and dissemination: a research note

Abstract

The involvement of all participants within all aspects of the research process is a well-established challenge for participatory research. This is particularly evident in relation to data analysis and dissemination. A novel way of understanding and approaching this challenge emerged through a large-scale international, 3-year participatory research project involving over 200 disabled people. This approach enabled people to be involved at all stages of the research in a manner that was collectively recognised to be participatory and also delivered high-quality findings. At the heart of this emergent approach to participatory research is an engagement with risk. This research note explores the types of risks involved in delivering research that seeks to be authentically participatory.

Keywords

Participatory research data analysis dissemination of findings inclusive practice museum education risk

Introduction

Participatory research might be reasonably expected to enhance the voices of participants and have their needs in mind in every aspect of a project. After all, such research is premised upon ongoing dialogue and consultation, in relationships based on mutuality, understanding and trust (Aldridge, 2016). However, since the early days of participatory research involving disabled people, analysis and dissemination of data undertaken by non-academic participants have very rarely been evident in practice (Cocks and Cockram, 1995; Richardson, 1997). Reviews recognise this as one of the most frequently discussed challenges (Stack and McDonald, 2014) but one that is particularly under-explored and needs to be investigated (Nind, 2008, 2011).

We came to similar conclusions in relation to analysis and dissemination following a wide-ranging systematic review undertaken as part of the ARCHES project (Rix et al., 2020a). This review sought an in-depth analysis of participatory research practice involving people with sensory and intellectual impairments. The review included 54 papers. Involvement in data analysis was evident in just under 35% of studies. Of these, 11 studies used collective analysis in some way, nine used a process of participant verification of findings and one study sought verification from a critical friend. Some papers acknowledged the partial participation evident in their research. Generally, an academic researcher would undertake a first stage data analysis and the participants would then sort the themes, or inversely the participants would undertake an initial thematic sweep and the academic researchers would undertake the next stage of analysis.

Within this systematic review, a few studies moved beyond traditional research analysis, recognising the evolving nature of the ‘messy space’ (Seale et al., 2015). Seale et al. (2015) looked to build upon the strengths that participants already had and to explore the boundaries between groups of participants defined by common objects and shared interests. Nind et al. (2016) suggested that in this process, two models come into play: inclusive immersion, where people learned by being immersed in a research environment, without experts to learn from but with challenges to learn through; and dialogic, where participants’ learning arose from engagement with each other’s contributions to knowledge. Such models respond to theories of empowerment and social justice evident in other participatory research involving particular groupings. This requires not just being open to new socially situated ways of understanding but also ceding control of research into data collection, analysis and distribution (Nicholls, 2009). Participatory research within this messy space therefore calls for us to take a ‘beautiful risk’, similar to that which Biesta (2015) identifies in relation to formal education. Participants are not to be moulded but are to be actively engaged. They are to be responsible agents within the learning situation, the outcomes of which are inherently uncertain.

This research note will explore the nature of data analysis and dissemination that arises when this beautiful risk is embraced within a research context.

A background to our research

Between October 2016 and December 2019, four participatory ‘exploration groups’ were established in London, Madrid, Vienna and Oviedo as part of ARCHES, a Horizon 2020-funded research project. These groups involved over 200 disabled people, working alongside friends, family and other supporters including educators from six museums (some of whom would also identify as disabled). The groups also involved five partners developing a variety of technologies that aimed to enhance access to the space and learning within it. The access preferences and needs of these groups were diverse, but we chose not to be defined by traditional impairment categories.

Membership and numbers attending the groups varied across the project, with between 15 and 25 people regularly in attendance in each group. People came and went, often leaving ideas behind them; ideas which continued to spread and have an influence. The aim was to enhance access to heritage for all, through technology and the development of multisensory activities. These groups met weekly or bi-weekly, undertaking activities of their own design or in response to requests from various participant partners. Across 169 sessions, a whole range of in-museum activities emerged, including access audits, relationship-building exercises, explorations of access preferences, the creation of access proposals, trials of ideas, advice on provision, the testing of software, providing feed back on products, and developing tours and multisensory resources (see Garcia Carrizosa et al., 2019).

Within the project, we were focusing upon data for three distinct purposes.

evaluation of technologies leading to recommendations to technology partners;

evaluation of activities and sites leading to recommendations to museums; and

evaluation of process and method leading to recommendations in European Union reports.

The exploration groups also evaluated their own ways of working, devised ‘rules’, decided how they wished to be represented (for example in demographics) and fed back their views of the project and how it was being run (including presenting on this at conferences).

Coming to recognise our analysis and dissemination as emergent

The ideas for the sessions were initiated by and followed up by the participants, either regular attenders or the less regular. Our findings and our communications emerged while we were involved in the sessions. This was data collection, analysis and dissemination as activity. Following on from the systematic literature review, we came to describe this overall experience of participation as the while of participation (Rix et al., 2020a). As one of the participants explained when discussing the review: ‘It’s simple. Participation happens while you are doing things’.

The notion of the while was a result of the thematic analysis of the 54 studies in the review, which suggested an explanatory underlying theory of participation. The underpinning tensions within these different research projects were consistently framed as issues of power, voice and support. These tensions were evidenced through the meaningful nature of outcomes, which we recognised as projects representing people’s lives, creating moments of learning and being of value to those involved. These outcomes emerged through the ongoing and continuous practicalities of participation as it was experienced, through its component parts (see Figure 1).

Figure 1.

The tensions, outcomes and component parts within the while of participation (Rix et al., 2020a).

This understanding of participation was one that we recognised within our own research experience. The while is the experience of being that emerges from and creates the boundaries in which people find themselves. The underpinning tensions, outcomes and component parts can be recognised within multiple interactions that create and are created by participation. These interactions form around each other; they are moments, but they are both a wave and a particle. Participation therefore emerges as a flow from many directions and is more than a sum of any preceding moments.

This understanding informed our response to data analysis and dissemination (Rix et al., 2020b). In the context of a participatory research group, data emerge within and through the while; and if analysis and dissemination are to be part of the while, then they too must be emergent. If they are not emergent, they are retrospective, situated outside of the particular while. It is within this retrospective space that most research analysis is undertaken. As part of ARCHES, for example, we undertook a retrospective analysis to assess the validity of the participatory process overall, as required by the funders and our ethics protocols. This was regarded as the verification of the while, but did not claim to be participatory. A paper such as this one would also sit within this retrospective space, beyond the while which it seeks to represent.

What does emergent analysis and dissemination look like?

Within ARCHES, we came to understand the emergence of data as a contextual phenomenon involving dissemination of knowledge and learning, first within the project and then beyond. Our emergent approach to data analysis within the exploration groups was based upon ongoing participant verification and participant representation of data. Participant verification and representation of data involved people having an experience, reflecting upon the experience, identifying understandings and insights from that experience, summarising those understandings and insights, recording them and then sharing them with other participants for clarification and verification.

The emergent ongoing analysis typically happened soon after an experience, but at times it also took a longer view. It could look across an extended activity providing snapshots on the way, helping to direct us towards a final output or as a way to represent that experience. Members of one group, for example, were so incensed by their experience at a London Museum, that they spent an afternoon producing a video report where they talked about the access issues that had arisen and then sent it to the museum director. Another group spent a few weeks looking at a museum website and producing a PowerPoint presentation, which they submitted to the management. More widely, across all the groups, members provided ongoing audio, video and written feedback to the technology partners to inform and provide feedback on changes to software they were producing.

Dissemination of ideas was a constant focus for the project, with communication being one of the primary ongoing challenges to access. In attempting to conceptualize this process, three visual metaphors emerged. These arose as part of a training session involving the academic team and the museum coordinators. They emerged as a way to reflect upon the challenges we faced in representing and working with the multiple views and boundaries of participants, within a multifaceted, asymetric, and irregularly patterned venn-diagram of perspectives and experiences. It was suggested that ideas needed to spread through the group like a ripple but, to be true to our emancipatory goals, ripples of knowledge also needed to turn outwards beyond the project. This had echoes of being a pebble in the pond (Skitteral, 2013), in that the idea may be a catalyst for change, but it was also about sharing perspectives without an end goal in sight. Inevitably these ripples were constrained by the context within which and through which they spread, particularly institutional cultures and our relationship with gatekeepers. This created a funnelling effect, which had profound influence on inputs and outputs to and from the group and how they might be represented.

The risk at the heart of participation

Ongoing, emergent analysis and dissemination brought together and shared the multiple views and boundaries of participants. It was an inward process, leading to a point of collective experience – and then an outward process, sharing more widely. All of these emergent processes were underpinned by risk. The risks within this process could be understood in relation to the outcomes and component parts of the while. We took risks around power within our relationships, how we supported each other and enabled voice. These risks were evident in how we represented participants’ lives and ideas, in our moments of learning and how what we were doing was valued by people within and beyond the project. We took risks with the language we used, the roles we asked people to fulfil and by challenging attitudes. We proactively sought to bring together people from diverse cultures and with a wide variety of access preferences. We had to be willing to adapt practices and spaces to enable relationships, to be flexible and take the time for people to enjoy themselves. At any point, these things could go wrong. The space worked for some people but not others; the use of language was only good for some in the room; the speed of activities was not suitable for everyone. As a result, there were genuine fallings-out, people really felt marginalised, activities clearly failed and, on plenty of occasions, intended outcomes did not materialise. However, it was through taking these risks that we came to have a sense of ourselves as a group and as individual groupings. We enabled ourselves to take ownership of the project and move ourselves on unexpected courses. Consequently, completely unexpected relationships emerged, so that at the end people talked about the empathetic power of the project, whilst new ways of working had been established and many unanticipated results had arisen.

We recognised a multitude of potential challenges in seeking to allow direction to emerge. People might drift or feel there was a lack of focus or coherence; there could be a loss of interest or the departure of participants themselves. We knew there would be competing values at play, with a risk to agreement or a danger of some voices coming to dominate. There was also a risk to the credibility of the process, to being believed and trusted, and therefore on delivering to our funders and project partners. From the outset, we recognised that there would be risks to how people viewed the processes and outcomes, the overall project and the participants, and how we felt about ourselves and behaved towards each other. We risked the overall quality of what we could achieve; in particular upon the quality of:

research – the degree to which people can trust the processes and outcomes;

reputation – the way in which people view the overall project and the people involved; and

relations – the way in which two people or groups of people feel and behave towards each other.

Taking a risk with the quality of research

Research is framed by many concepts that seek to ensure the quality of the process, its legitimacy and rigour. In an emergent and participatory frame, it is particularly difficult to assure people that these are being delivered within the analysis. The relational and uncertain nature of participation means the process is inherently chaotic and antithetical to consistency, predictability, replicability and measurement; whilst the authenticity of analysis is dependent upon the participants’ subjective positions being accepted both by internal partners and external audiences.

An evident example of this challenge within ARCHES was the emergent analysis of a multisensory tool being developed by one of the technology partners. This tool was a three-dimensional relief of a painting onto which the image itself was projected; by touching different aspects of this object, various types of aural information about the painting were relayed to the user. The company developing this had conceived it as a tool for people with a visual impairment; however, the participants who were testing it came with a whole range of impairment labels and did not wish to be identified by these but by their access preferences. For the analysis to be valid, the technology company had to open itself to a new way of seeing its own product. Because it was acceptant of this possibility, the company made various significant changes to the design, including adding colour and layers of information, which made it more popular with many people including their original intended audience. Other partners in the project faced similar challenges but not all were able to change their established focus upon a particular ‘type’ of user. For them, the risk was too great. They felt a need to adopt their previous research approaches so as to deliver the product that they had envisaged.

As academic researchers, we had to let go of much of our traditional control and many of our traditional expectations about the research process. So too did many of the other research participants, who anticipated that we were the experts even if we did not position ourselves as such. We were participants alongside each other. Participation was context dependent. It depended upon being involved in a session to be part of what was going on. We recognised that all participants came with skills and experiences, which could lead us in different directions. They may well have resources and motivations of which others were unaware, and which were only revealed within the evolving context of the groups and their activities. We came with skills as academic researchers, we had a voice that was listened to, we could use this position and our knowledge to steer conversations, but it rarely benefited us to do so. If people did not share our understanding, it was fairly obvious. This did not make us passive academic researchers, however; it made us active participatory researchers.

There is little doubt that by seeking to support an open, emergent approach to analysis and dissemination, we lost something in traditional research terms; but we would suggest that we enhanced the overall quality of our experience and what we produced, how we were seen by ourselves and others and the nature our relations. As Aldridge (2016) recognised, approaches that allow experiential data and interpretations to underpin and inform understanding and knowledge production ‘can and do make important contributions’ (p. 146).

Taking a risk with the quality of reputations

Participatory spaces are public. They involve revealing yourself to new people in new contexts and in ways which you frequently have little prior experience of. In presenting the group and its ideas to people beyond its margins, the participants invariably risked presenting the underpinning tensions around power, support and voice that underlaid the group. In the context of ARCHES, where over 200 people attended the four groups across the lifetime of the project, there were many situations in which decisions had to be made about how to present ideas. Above and beyond adhering to the communication rules that the group had established for internal communications, there was also a diverse range of access preferences to be considered. These included people using different spoken and signed national languages, people who preferred simplified language and text supported communication, and people who gained access through audio description, braille and through engaging with multisensory objects. This created a whole range of novel situations for participants, such as a person signing in one language being translated into another spoken language, which was being signed too, alongside an expectation that language would be kept simple and offer audio description and a text output. In such situations, it was all too easy to get things wrong. All those involved were learning in a public arena and at risk of being judged.

There were also clear risks in representing the group in public situations, particularly given the political nature of many issues associated with disability and impairment. For example, it might seem appropriate to use the ‘best’ communicators to present the group’s ideas. This however generally encourages the dominance of speech as the mode of communication, as by ‘best’ we generally mean ‘best at public speaking’. In the context of ARCHES, getting across our message about access preferences required our audience to understand the possible. As a consequence, people who were ‘difficult to understand’ were supported to lead museum visits; signing was undertaken on videos by a range of people including untrained signers; presentations to the press involved participants who liked to talk a great deal, or might focus in quite some detail on a very specific issue, or who were very perfunctory and quiet. This openness was a huge step into the unknown for many of the participants requiring a step of bravura and a willingness to be a representative of others and therefore a target for disagreement. These public sharings almost always paid off, but they had to be argued for, planned for and the groups always had to be open to change, for what they were doing might need to change in that moment.

Taking a risk with the quality of relations

An emergent analysis of experience opens participants up to powerful personal emotions, understandings and beliefs. It is possible that within the participatory context, there is a commonality of experience that can both heighten and reaffirm these feelings and insights, but equally it can reveal competing views. In setting up a project as accessible and emancipatory, there is an inevitable opportunity for disappointment; and, in framing it as emergent research, it provides a platform for emergent frustrations to come to the fore.

Within ARCHES, this risk was evident on a number of occasions in different ways in all of the groups. For example, one week in one city, the entire group of D/deaf participants simply stopped attending. Exactly what had gone wrong was hard to pin down, but at least part of their frustration was due to an evident sense of injustice that the ideas they were sharing were not being acted upon as they hoped. Perhaps the risk they had taken in participating was not seen to be paying off. Over the next few months, however, as institutional changes became evident and an opportunity to design and deliver training emerged, the group returned and were willing to share their involvement publicly. This required taking a risk and maintaining an ongoing respectful dialogue in the intervening period. But it also required a recognition from those who had left that the group itself was changing, and new ideas and ways of understanding were emerging. The participants who had left, should they come back, would be coming back to something that had moved on.

It is tempting to extol the democratic nature of participatory research, but democracy is itself messy. It is a space where compromise is necessary for decisions to be made and where people often get used to seeing themselves as part of the majority or minority. Within a research context that is seeking to be authentic and to serve as a representation of participant’s subjectivities, such democratic compromise can result in the silencing of people’s voices (or a sense that they are being silenced) and the over-projection of other voices.

Within ARCHES, for instance, we undertook a pilot evaluation of people’s participatory experience (Seale et al., 2020), which involved the use of visual storytelling. At the end of this process, those involved shared their understandings of their role within the wider project. One participant, who was used to being marginalised through dominant forms of communication, explained that her artwork represented how she was not listened to within the exploration group and how the group was a space for favouritism. It is not possible to argue against such a statement without in some way proving its point. It was, however, a cause for considerable reflection and ongoing discussion. For example:

Were our attempts to ensure a sense of well-being for those who emphasised their vulnerabilities giving too much weight to some people’s views?

In bringing together and disseminating ideas, were we too easily satisfied by people’s acquiescence in collective situations?

Could we possibly say that our findings were an authentic reflection of collective experience if we could not highlight how issues of access went far beyond a response to impairment and included deeper social biases, exacerbated by shifting and enormously variable subjectivities?

Participation is visceral, but it is also a deeply conscious experience. It is very hard (if not impossible) to practice participation. Even though the majority of participants talked about their experience in a positive way, for a few the abiding memory was negative. It is likely that for others who came and left, for many personal and practical reasons, there was an element of ambivalence about it all. As researchers, however, we need to learn lessons from the success, failure and ambivalence of the participatory experience. We need to ask if we took enough of a risk in how we supported people to analyse and disseminate their ideas and experiences or whether we played it too safe.

Conclusion

Given the wide range of access preferences of the participants within the ARCHES project, it may seem like an extreme example of the challenges offered by participatory research. However, like much associated with disability, reflection upon the challenges faced by this population frequently reveals issues and offers solutions that are relevant to and would be of benefit to a far wider population – to everyone.

Participatory research that relies upon the traditional grammars and processes of research cannot maximise its participatory nature. In particularly, as a consequence of its academic discourse and its field-specific nature, the means by which the research identifies and shares its findings will be exclusionary. To overcome this conundrum, projects mostly adopt the following approaches:

The nature of the sample is restricted to those who can engage in the established ways of working.

The ways of working are dominated by those who have ownership of the valued academic knowledge.

Training is offered to those who can benefit from it.

It is argued that participants are all different and undertaking different but equally valid tasks.

It would be wrong to suggest that all participatory research should abandon these approaches. In particular, the last two seem to be respectful of the roots of participatory practice. However, there are also projects that seek alternative ways of working. These tend towards non-traditional, often artistic, forms of representing data and findings; but, as is evident from ARCHES, they can also adopt the notion and approach of emergent analysis and dissemination. To do so requires a willingness to embrace the risk inherent in being open to the power of participants, in how one conceives of and delivers support and recognises each other’s voice. It requires taking practical risks in challenging people in their language, roles and attitudes, requiring them to open up their practices and spaces to enable relationships that are flexible, enjoyable and generous with time. It requires engaging hopefully with these risks.

The examples given in this research note go a small way to explore the role of risk evident in relation to research, reputation and relations. However, it also clearly evidences the role that risk plays in research, particularly when it seeks to be participatory and seeks to include the participants in analysis and dissemination of data. It is only through embracing these risks that the emancipatory nature of participatory research has the best chance of emerging.

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project leading to this paper received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 693229.

ORCID iD

Jonathan Rix

Author biographies

Jonathan Rix is Professor of Participation and Learning Support at the Open University and the Inland Norway University of Applied Sciences. His research focuses on policies, practices and language that facilitate inclusion within the mainstream; capture diverse perspectives; and develop thinking about the form and function of education.

Helena Garcia Carrizosa was the Research Associate on the ARCHES project and is currently studying for her PhD at the Open University. She is passionate about the educational practices of European heritage sites particularly in relation to accessibility for all.

Kieron Sheehy is a Professor in The Open University’s Faculty of Education and Language Studies specialising inclusion, innovative pedagogies, and fun.

Jane Seale is a Professor of Education. Her research interests lie at the intersections between disability, technology and inclusion, focusing in particular on the role that technologies play in the lives of people with learning disabilities (intellectual impairments) and the factors that influence or sustain their digital exclusion.

Simon Hayhoe is a reader in education at the University of Bath. He is currently working on participatory research analysis with the World Health Organisation Academy to develop learning strategy. He is also co-editor of the series, Qualitative and Visual Methods in Educational Research.

References

Aldridge

. (2016) Participatory Research: Working with Vulnerable Groups in Research and Practice. Bristol: Policy Press.

Biesta

(2013) The Beautiful Risk of Education. London: Paradigm.

Cocks

Cockram

(1995) The participatory research paradigm and intellectual disability. Mental Handicap Research 8(1): 25–37.

Garcia Carrizosa

Diaz

Sisinni

(2019) Towards a participatory museum. Available at: https://www.arches-project.eu/wp-content/uploads/2019/07/EnglishGuide_Hyperlinks_mid.pdf.

Nicholls

(2009) Research and indigenous participation: critical reflexive methods. International Journal of Social Research Methodology 12(2): 117–126.

Nind

(2011) Participatory data analysis: a step too far? Qualitative Research 11(4): 349–363.

Nind

(2008) Conducting Qualitative Research with People with Learning, Communication and Other Disabilities: Methodological Challenges. National Centre for Research Methods NCRM/012

Nind

Chapman

Seale

, et al. (2016) The conundrum of training and capacity building for people with learning disabilities doing research. Journal of Applied Research in Intellectual Disabilities 29(6): 542–551.

Richardson

(2002) Involving people in the analysis: listening, reflecting, discounting nothing. Journal of Learning Disabilities 6(1): 47–60.

10.

Rix

Garcia-Carrisoza

Seale

, et al. (2020a) The while of participation: a systematic review of participatory research involving people with sensory impairments and/or intellectual impairments. Disability & Society 35(7): 1031–1057.

11.

Rix

Garcia-Carrizosa

Hayhoe

, et al. (2020b) Emergent analysis & dissemination within participatory research. International Journal of Research & Method in Education. Epub ahead of print 4 June 2020. DOI: https://doi.org/10.1080/1743727X.2020.1763945

12.

Seale

Garcia-Carrizosa

Rix

, et al. (2020) A participatory approach to the evaluation of participatory museum research projects. International Journal of Research and Method in Education. Epub ahead of print 3 January 2020. DOI: https://doi.org/10.1080/1743727X.2019.1706468

13.

Seale

Nind

Tilley

, et al. (2015) Negotiating a third space for participatory research with people with learning disabilities: an examination of boundaries and spatial practices. Innovation: The European Journal of Social Science Research 28(4): 483–497.

14.

Skitteral

(2013) Transitions? An invitation to think outside Y/our problem box, get fire in your belly and put pebbles in the pond. In: Curran

Runswick-Cole

(eds) Disabled Children’s Childhood Studies. London: Palgrave Macmillan.

15.

Stack

McDonald

(2014) Nothing about us without us: does action research in developmental disabilities research measure up?” Journal of Policy and Practice in Intellectual Disabilities 11(2): 83–91.