Virtual Co-Creation: A Guide to Conducting Online Co-Creation Workshops

Abstract

COVID-19 has impacted research worldwide, with many researchers turning to online methods. While online and co-creation research methods are well established, there are no instructional articles or guides for researchers aiming to conduct online co-creation workshops. Such workshops can provide quality data and are increasingly used. This methods paper addresses this gap by outlining the steps involved in conducting online co-creation workshops. Using the TruSTFood sustainability food labelling study as an example, this paper provides instructions and considerations for researchers. It serves as a guide for those interested in or new to this area and can be used to transfer existing research online owing to social distancing and the pandemic or new research beyond.

Introduction

COVID-19 has brought major life changes worldwide. Lockdowns and restrictions such as social distancing and travel limits have compelled researchers, and particularly those in human research, to adapt processes such as recruitment and data collection. There is an awareness that flexibility in methods is required, and researchers have adopted online methods in an attempt to overcome obstacles (Barroga & Matanguihan, 2020; Burke & Patching, 2020; Karmakar et al., 2020; Lobe et al., 2020).

Online (virtual) research is not new, with interviews, surveys and focus groups having been conducted using the internet. Indeed, synchronous (real-time) and asynchronous (non–real-time) online focus groups have been used from as early as the 1990s (Stewart & Williams, 2005). Online research has many recognised advantages including time and location flexibility, cost-effectiveness, the potential for greater geographic reach and diversity of participants, and greater anonymity (Hesse-Biber & Griffin, 2013; Reisner et al., 2018; Richard et al., 2020). Importantly, able to adhere to public health guidelines such as lockdowns and social distancing (Pocock et al., 2021), online methods offer a safe alternative during a pandemic. However, potential drawbacks have also been noted such as the inability to verify participant’s identities, a lack of non-verbal cues, relatively low response rates, and the exclusion of those who are less technologically savvy, all of which may ultimately impact data quality (Chen & Neo, 2019; Evans & Mathur, 2018; Hesse-Biber & Griffin, 2013). While online methods have been found to elicit lower word counts, the content and ideas produced between online versus offline methods are similar (Richard et al., 2020).

Interviews and focus groups are useful tools for gaining individual feedback and achieving group consensus (Chen & Neo, 2019). However, they have been termed ‘reactive’ (as opposed to ‘proactive’), with participants provided with a set stimulus or question and their response or opinion recorded. This may limit the generation of ideas and spontaneous insights outside of pre-defined questions or topics (Witell et al., 2011). In addition, such methods capture only spoken needs and opinions, with less consideration to latent or subconscious meanings. When it comes to creating new knowledge, products or services, co-creation methods may be more suitable.

Previously, consumers were viewed as external to organisations and businesses, with value and creation taking place from within. However, co-creation is characterised by placing the consumer alongside organisations, with the joint and collaborative defining and solving of problems (Prahalad & Ramaswamy, 2004). It is ‘an open, active, and creative process in which all relevant stakeholders are engaged in an innovation process’ (van Dijk-de Vries et al., 2020, p. 2). It has been suggested that co-creation may have its origins as far back as the early 20^th century (Ind & Coates, 2013). Factors such as the internet and related technologies, the orientation towards services and an open approach to innovation have led to the further development and adoption of co-creation (Ind & Coates, 2013). Co-creation and related methods are situated within the wider area of Participatory Action Research (PAR). Adopted from as early as the 1940s, participatory methods challenge the idea of an objective and knowing organisation or being and instead invite users to actively participate and shape knowledge (Chevalier & Buckles, 2013). The three pillars of PAR are commentaries relating to life in society (participation), engagement with experience (action) and the framing of soundness in thought (research).

In line with the co-operation, participation and equality of all contributors from participatory action research, co-creation workshops provide participants with the flexibility and opportunity to make new discoveries, create new knowledge and develop ideas for new products or services through the use of suitable tasks. Co-creation methods can be beneficial for both consumers and organisations. Both of their needs can be realised, and there is a reduced risk of rejection of new ideas or products (Roberts & Darler, 2017). Consumers may receive cognitive and hedonic benefits such as developing skills and fun and enjoyment in participation (Verleye, 2015). Furthermore, digital tools and technology have led to virtual co-creation with lower costs and prompt interaction with and between consumers (Bettiga et al., 2018).

While co-creation methods may be typically suited to business and marketing, they have also been used in other settings. Within healthcare, shared decision making could be viewed as one type of co-creation. This might involve co-production of treatment programmes or co-learning of information, which can lead to better quality of life for patients (McColl-Kennedy et al., 2012). Specific methods include the use of diaries, where patients write about their experiences and ideas. These diaries can then form the basis of potential improvements. For example, feedback from one patient with regards to the unexpected worse pain of a second hip replacement led to an improvement in how patients are provided with information (Elg et al., 2012). The education sector could also benefit, with co-creation found to increase students’ knowledge, skills and satisfaction (Ribes-Giner et al., 2016)

While virtual ‘reactive’ research methods such as interviews and focus groups are well established, there has been less focus on virtual co-creation methods. Digital technology and the internet have emerged as a platform for co-creation (Chepurna & Criado, 2018). They allow for an open dialogue between consumers or users and the organisation, and the ability to reach a diverse audience. In terms of research, like virtual qualitative methods, this allows for conversation and collaboration with participants across a range of countries and backgrounds.

In addition to the lack of focus on co-creation methods, there has been little emphasis on the use of co-creation methods in the areas of food or labelling (Tardivo et al., 2017). Furthermore, the literature regarding sustainability and co-creation is relatively new, with scope for development (Almeida et al., 2021). There is also a lack of instructional articles on how to conduct the virtual co-creation process. The current paper will outline the practise of conducting virtual co-creation workshops, thus providing insights and suggestions for future researchers. Virtual workshops conducted using these methods to co-create a consumer-friendly holistic sustainability summary label for food (the TruSTFood project) will be used as an example. Specifically, the TruSTFood project was established due to the absence of such a sustainability summary label for food. Given that consumers are the ultimate end-users of food labelling, it is essential that their perceptions, attitudes and preferences relating to sustainability and labelling are considered. The virtual TruSTFood workshops, used as an example in this manuscript, were one element of understanding consumer needs within the project. These workshops were conducted with participants in Denmark, Ireland and the UK.

Methods

This article is based on research conducted as part of the TruSTFood project to co-create a holistic sustainability label for food. The sociodemographic details of the study participants can be seen in Table 1. Results from this research will not be considered in-depth; rather, they will be used as examples and to provide insight into virtual co-creation methods (for results of the project, see Pedersen et al., 2021). Consumer co-creation workshops were considered to be a particularly suitable method as consumers will be the end-user of any label produced.

Table 1.

Sociodemographic Characteristics of TruSTFood Co-Creation Workshops.

	UK and Ireland			Denmark
	Group 1	Group 2	Group 3	Group 4	Group 5	Group 6
Participants (n)	9	8	7	7	7	7
Female/male	5/4	5/3	4/3	3/4	4/3	3/4
Age (mean)	41.8	35.3	39.0	41.4	39.3	45.0
Youngest	18	19	25	26	23	25
Oldest	71	54	62	66	57	63
Occupation
Student	1	1	0	1	1	1
Employed	6	7	7	6	6	5
Retired	1	0	0	0	0	1
Unemployed	1	0	0	0	0	0
Highest level of education achieved
Secondary/high school/vocational/college or less	1	4	2	5	4	5
University undergraduate	6	2	5	1	2	1
University postgraduate	2	2	0	1	1	1

Setup

It is essential that the online platform chosen to host the co-creation is capable for the planned activities. For example, synchronous discussions will require a chatroom with voice and/or video capability. Preferably, these will be located within the platform itself rather than using external software, thereby minimising potential confusion for participants. Asynchronous discussions will require a forum or board with moderation functions such as edit and delete to remove inappropriate content, should this be necessary. If participants will be required to upload media or files in tasks, one should ensure that the platform can handle this and various file types (ppt, pptx, jpg, doc etc.). The inclusion of a messaging function within the platform should also be given consideration, as well as the need for a dedicated technical helpdesk. This will provide participants with direct access to support should it be needed.

Further consideration should also be given to the various formats that might be required to allow all individuals to participate. Will the platform support Windows, Mac, Linux etc.? Will participants be able to complete tasks on their mobile phone? Will more than one language be required? As part of the TruSTFood project, the online qualitative platform Revelation (by FocusVision), fitted all of the above criteria and was well suited to our co-creation needs.

Following setup, rigorous testing should be undertaken to ensure that there are no technical issues, that instructions are readable and user-friendly and that data are captured appropriately. Pilot testing with a few users from the target population will provide useful feedback in terms of functions, user-friendliness and will allow for any issues to be identified early in the study.

Consideration should be given to the establishment of house rules or etiquette. For example, will participants be required to log in to the portal daily? If participants do not log in for several days will this affect participation in the study? The rules should provide clear and simple explanations of what is expected of participants, and the consequences of not adhering to the rules (e.g. warning or full removal from the project). The ability of the platform to send notifications and reminders to participants is also helpful.

Recruitment

In addition to the profile of the individual required for each specific study, consideration should be given to using selection criteria based on technological capabilities. For example, if experience of specific computer programmes or certain skills are required (PowerPoint, Paint etc.). The use of the internet to recruit individuals should mean that participants have basic internet skills at a minimum such as using a browser and messaging/emailing. In addition, researchers should be cognisant that not all individuals may have access to additional software required. If this is to be provided to participants, the cost of doing so must be considered. The use of alternative software programmes is also an option, but this may lead to several alternate sets of participant instructions being required and additional support requests from participants.

Ethics

Additional to standard ethics for human and internet research, it may also be necessary to include information regarding intellectual property and commercialisation. Should consumers co-create an idea or prototype that later becomes an established product or service, will their input be further recognised or rewarded? A statement or agreement clearly outlining possible outcomes and rights or waivers may help to prevent later claims or legal problems.

Facilitation and Engagement

Prior to commencement of the study, moderators should receive training on the use of the platform and instructions for participants. An introduction should outline house rules and what is expected of participants (purpose, time commitment, how to request support etc.). A personal introduction from each moderator allows participants to connect. In the TruSTFood study, the moderator for each group posted a video outlining the purpose and general outline of the workshop and the commitment expected of participants.

Consumer co-creation will typically involve multiple tasks. Researchers will need to decide the length of tasks and how these fit into a schedule. In deciding the length of tasks, consideration should be given to the realistic time required for participants to thoughtfully complete each task, and the complexity of the task.If multiple steps or using a software programme are necessary, will support be required? In creating the schedule, one should be cognisant of the other commitments that participants might have (are they parents, carers, professionals or otherwise busy individuals?). It is also important to think about the ordering of the tasks, with easier and more fun tasks placed at the beginning of the workshops to discourage early dropout. In the TruSTFood study, participation was over a total of 14 days, totalling 3 hours task participation. This included discussions, the provision of information, and creative, ranking and data collection tasks. The schedule (Appendix A1) included free days to provide participants with a break and time to catch up on previously missed tasks. The majority of tasks were released daily and sequentially, to prevent some participants from proceeding further ahead than others. The full schedule and script are available in Appendix A2.

It is advisable that the moderator logs into the platform at least once daily, to ensure that individuals are participating, prompt discussion where necessary, and provide support and help with technical issues. The presence of a moderator can help to foster engagement, encourage discussion and ensure that progress proceeds on time.

Incentives or other rewards are likely to be needed to recompense participants for their time and efforts. In addition, the use of prizes might be considered to garner competition and thus stimulate completion and encourage innovation (Piller & Walcher, 2006). In the TruSTFood project, in addition to a shopping voucher incentive, peer-voting took place and prizes were awarded at the end of the project. These were lower value extra shopping vouchers given to the most active participant, and the users with the most realistic and practical ideas, the most creative ideas and the most thoughtful comments.

Results

This section is not intended to detail the results of the TruSTFood project (for full results see Pedersen et al., 2021); rather, the types of responses received to the tasks in the workshops will be outlined. Feedback on the workshops from participants and moderators will also be explored.

In total, 56 participants were initially recruited across six online workshops. Eleven participants did not complete at least 80% of the tasks and were therefore excluded from analysis and reporting.

Task Participation

Participants engaged well in tasks, although as might be anticipated some participants were more active than others. Similarly anticipated, participation (in terms of number of participants) was greater in earlier tasks compared to later tasks, due to participant dropout.

Ranking tasks allowed for participants general perceptions of sustainability to be assessed. Having chosen their top three images that best reflected sustainability (out of 10 pre-selected and piloted images), participants explained their choices, showing their knowledge relating to sustainability. For example, one participant, after choosing the picture of solar panels to represent sustainability remarked:

‘Renewable energy that does not consume the Earth’s natural resources or result in pollution. We will always have a need for energy, so these methods are sustainable’.

The pictures of wind farms, planet earth and a plant shoot were also frequently chosen. Participants comments on these selections demonstrated an understanding of the different aspects of sustainability. Furthermore, participants mentioned that they found it difficult to choose only a few pictures and commented on more pictures than necessary. This highlighted that participants recognised that sustainability relates to many different, interrelated dimensions.

This multidimensional view of sustainability was also apparent in the sorting task. Participants were given pictures of 10 different foods and asked to sort these according to their own axes (Figure 1). Participants sorted according to a wide range of aspects such as naturalness, food miles, packaging, carbon footprint and health. Some participants used the word sustainability as a general word to mean environmental aspects.

Figure 1.

Examples of sorting foods according to self-selected axes.

For open-ended responses in discussion tasks, those from the UK and Ireland posted an average of 183 words each while those from Denmark posted an average of 87 words each. Participants typically posted one response; however, a small number of participants engaged further and asked questions and interacted with other participants (Figure 2). Indeed, several participants mentioned in their feedback that they liked to see/read others’ opinions on the topic:

Figure 2.

Example of discussion and interaction between participants (names redacted to preserve confidentiality).

‘ I think the workshop was all really interesting. What I liked best was the ability to see other people’s responses and comments via the feed. It was always useful to review what other participants had said once I had submitted my responses’.

Another Participant Mentioned:

‘I think it has been great that you have had the opportunity to read the assignment, then go and think about it, and then return and answer the question. It is interesting to follow other people’s thoughts and creativity, as well as to get inspiration for what sustainability also is’.

Participants most often responded on the day of the discussion task, with only a few participants responding later. Prompts from moderators for further information or regarding missing details were effective around half of the time. Ranking tasks also had good engagement. Participants were forthcoming and most provided full explanations regarding their choices.

Participants also engaged well with the exploration/collection task. This required participants to browse grocery supermarket websites and choose yoghurt and chocolate bars which were least and most sustainable. The majority of participants provided clear screenshots and reasoning for their choices. However, a few participants either failed to provide screenshots of images, provided small thumbnail images or answered only part of the question, for example, provided answers for yoghurt but not chocolate bars. Qualitative feedback from participants indicated they enjoyed this task and found it informative:

‘I thought the tasks were really well thought out. Like organizing the items and researching them at Tesco etc. It created more investment than just answering questions and I feel like I learnt a lot from it, even about my own opinions’.

‘It was rewarding to discover how difficult it actually is to judge whether a food is sustainably produced. There are an incredible number of parameters that are important’.

However, others noted that the task was time consuming, given the number of products in the search results:

‘Looking up individual items on the Tesco website was quite time-consuming...’

Others found the tasks in general time consuming:

‘I find it difficult to find time to both solve one’s own task and to have time to comment on the opinions/solutions of others’.

With regards to the creation task, where participants were required to create their own sustainability label, there was mixed engagement. Some participants provided drawings of labels with multiple details and colours, while others uploaded more basic efforts (Figures 3 and 4). Some participants noted technical issues with uploading their files and viewing others’ files. Others used words and descriptions of their proposed labels, rather than a drawing as instructed.

‘I was not happy about the creative part – that is not where I have my competences’.

Figure 3.

Examples of more engagement on creation task.

Figure 4.

Examples of less engagement on creation task.

Despite technical assistance, a few participants still failed to complete this task. Indeed, some participants commented on the complexity of those tasks using Powerpoint:

‘I found the packaging labelling exercise frustrating because I was not prepared in advance to set up tech for this’

Other Participant Feedback

Participant feedback on the workshops was positive. On a scale from 1 to 7 (1 = not interesting at all, 7 = very interesting), participants rated the workshop as a mean of 6.0 (SD = 0.82). The workshops were also rated as useful (mean = 6.0, SD = 1.12, on a scale from 1 – 7 where 1 = not useful at all and 7 = very useful) and good (mean = 6.0, SD = 1.0, on a scale from 1 – 7 where 1 = bad and 7 = good). Participants also agreed that the workshop was informative (mean = 6.0, SD = 1.61, scale ‘I learned more about sustainability’, 1 = completely disagree, 7 = completely agree), that tasks were neither difficult nor easy (mean = 4, SD = 1.16, scale 1 = difficult, 7 = easy) and that the time for each task was appropriate (mean = 6.0, SD = 1.47, scale ‘the time for each task was appropriate’, 1 = completely disagree, 7 = completely agree). Echoing feedback from the rating scales, open-ended feedback from the participants indicated that they found the workshops enjoyable and interesting.

With regards to possible improvements, several participants mentioned that more time to complete tasks would be appropriate.

‘I can’t think of any specific way the contents of the workshop could be improved but think it should perhaps be a 3 week workshop rather than a 2 week one, especially to aid those who are currently working’.

A number of participants also mentioned that it would be useful to interact with the other participants in a group call or video chat.

‘Online focus group - with the other participants, so we can share our views and ideas. Sometimes it is easier to say something rather than write it down or draw it’.

Moderator Feedback

Feedback from moderators highlighted the need for regular logging in and checking the portal. This allowed for not only the review of content and participation but also allowed for monitoring of messages, technical issues and potential conflicts or offensive content. At the beginning of the study and for the first few days, it was felt that replying to all comments from participants highlighted the support available and fostered a sense of community. This feeling of community was also encouraged by comments and actions not directly related to the tasks, such as thanking participants for their time and wishing them a good weekend.

The asynchronous nature of the discussion tasks provided flexibility and allowed participants to complete their tasks and comment on other’s posts at any time of the day. However, given that there was limited discussion between participants in the TruSTFood study, this may not have been the most optimal format for interactivity. The use of a chat function or weekly meeting may have helped to encourage this.

Given the differing levels of technical expertise of the participants and some tasks being more complicated than others, it is suggested that ‘rest’ or ‘break’ days are positioned after more technical tasks. This will allow those who struggle some extra time and to receive help if necessary, while other participants can use the time to complete any previously missed tasks.

Data Analysis

Depending on tasks used in the workshops, the dataset is likely to contain a mix of formats (Powerpoint files, open-ended responses, numerical data etc.). This will mean that a mix of analysis techniques will be required.

Discussion and open-ended question tasks lend themselves to qualitative analysis such as content analysis or thematic analysis (Braun & Clarke, 2006; Krippendorff, 2018). In the TruSTFood study, such data were read for familiarisation and analysed using an inductive approach. Predominant themes were constructed and further explored (consumers’ general perceptions of sustainability, consumers’ perceptions of sustainability in food, consumers’ assessment of sustainability in food and preferences for a sustainability label for food).

The constructed themes were also used to structure and explore relevant quantitative findings. Ranking questions were analysed by counting the top preferences or assigning a points value to selections, for example, position 1 = 10 points and position 2 = 9 points. Given the relatively small sample size in the TruSTFood study, in-depth quantitative analysis was not possible or suitable.

Discussion

The outbreak of COVID-19 has had an unprecedented effect on research worldwide. While this has presented challenges, there have also been opportunities to conduct research in different ways. Online research, including surveys and focus groups, is not new. However, less is known about conducting co-creation workshops online. Using a completed and successful study as an example, this manuscript outlined the processes involved and best practices in conducting consumer co-creation research online.

Overall, the online co-creation workshops led to interesting and valid results relating to the creation of a consumer-friendly sustainability label for food. The use of various task formats allowed for an understanding of consumer’s attitudes and perceptions. While the subject of the example study was applied market research (sustainability label), co-creation methods have been used in many other areas such as healthcare and education. For example, co-creation methods have been used to develop a complex intervention in a low income setting (Lazo-Porras et al., 2020). Ranking/selection tasks and their associated comments in the TrustFood study indicated that participants were aware of several different aspects of sustainability and indicated their current levels of knowledge. The majority of participants mentioned that sustainability was important in food. Meanwhile, the axes picture sorting task showed that some consumers used the terms sustainability and environmental interchangeably, suggesting that further education on the differences may be beneficial.

The majority of responses received to the given tasks were of high quality. The average number of words (135 words per discussion response) indicates that while participants were not providing overly long answers, they were considering the question and not merely typing a few words to obtain the incentive. This average compares favourably to previous research using Whatsapp as an asynchronous research tool (15–34 words per response) (Chen & Neo, 2019). The scheduling of asynchronous tasks over a 14-day period allowed a diversity of consumers to partake in the workshops. This method meant that regulations and restrictions relating to COVID-19 such as social distancing were respected. Similarly, the method will likely have helped participants in terms of flexibility with COVID-19 related pressures such as childcare and working from home. The ability for participants to consider their responses over a number of days and reflect upon the answers provided by others highlights the relevancy of the method in terms of co-creation. This is echoed by Richard et al. (2021) who state that the online environment is suited to studies which focus on the generation of new ideas. Furthermore, the TrustFood workshops were conducted in three different countries in two different languages, suggesting the method may be used in different settings and cultures.

More complex tasks such as creating a label and displaying pictures on an axis were those which participants remarked were most interesting and enjoyable. However, these were also the tasks most likely to present technical issues and problems. While individuals will vary with regards to their expertise, future online co-creation research may benefit by requiring familiarity or expertise with particular computer programmes which will be used in the study. Technical issues aside, some participants gave up developing their own label further and expressed the complexity of capturing sustainability in one label.

While the use of the internet for co-creation workshops undoubtedly provides benefits, one must also be aware of the ‘digital divide’ and that not all individuals have the physical capability or skills needed to access the internet (Vasilescu et al., 2020). This may not only bias the sample, but may also mean that only those fully skilled in the internet and software packages used are able to properly convey their meaning during these studies.

While feedback was collected from participants at the end of the study, this was limited in terms of quantity and scope. Think aloud interviews with participants during the study or exit interviews may have provided richer data and insights. This would have allowed for further reflection regarding the processes and structure of the workshops, and how to improve particular aspects of the workshops.