Sage Journals: Discover world-class research

Abstract

This study focuses on the development of a reliable coding scheme (CP-CODER) for studying interaction dynamics during the co-production of a public service involving priority communities. CP-CODER addresses the need to involve priority populations in the development of public services, an approach that has been recognized as difficult because of the high possibility of their experiencing negative health, social, and economic outcomes. The coding scheme was designed to capture group dynamics and forms of public engagement by adopting and integrating existing theoretical frameworks in the public service management and qualitative method literature. Coding was conducted on the transcripts of four co-production workshops, which included 26 family caregivers, three local health and social care service provider representatives, and five researchers involved in the co-production of new community-based services. One category was added to the two theoretical frameworks. The kappa ranged from 0.70–1.00 for the eight variables and the 26 individual items. The overall kappa was 0.91, while the overall percentage of agreement was 91.16%. The results of the pilot test showed the importance of integrating and managing three dimensions in co-producing with a priority population: the turn-taking, the content, and the level of abstraction of the discussion. The findings of this study have enhanced and supported both practitioners and researchers in co-producing services, ensuring the equal contribution of all participants, even those whose voices are rarely heard.

Keywords

Focus groups conversation analysis community based research critical realism mixed methods

In the 1970s, participatory research was first introduced to encourage populations to create change and act (Bergold & Stefan, 2012; Macaulay, 2017). The value of this principle is so undeniable that ‘no one is against it’ (Arnstein, 1969). This acknowledgment of the need for public participation has resulted in several research approaches, such as participatory action research and participatory appraisal (Baum et al., 2006; Kemmis & McTaggart, 2005; Pain & Francis, 2003). Despite their relevance, benefits, and participatory approaches may reconfirm unbalanced roles, inequalities, and exclusions of priority populations (Heselton et al., 2021; Rosen, 2021). Participatory approaches are also limited because they can fail to produce social change, which may generate negative consequences among some groups of actors (Bussu et al., 2020; Newitt & Thomas, 2020). Therefore, identifying and managing interaction dynamics during citizens’ involvement practices and settings have become priorities (Olesen, 2018).

Among the several approaches used in participatory research, co-production has become one of the most frequently discussed in the social science literature (Loeffler, 2020). Interaction dynamics that occur during co-production differ from other forms of public engagement because they entail equal collaboration between citizens and providers (or other relevant stakeholders) (Batalden et al., 2016; Bovaird & Loeffler, 2013). Because it allows citizens to provide substantial and active contributions in decision-making processes, co-production promotes and encourages the principles of ethics and democracy (Thomson & Berriman, 2021). In other forms of engagement, citizens express their views (i.e., consultation) and, through dialogue and argument, participate in decision-making regarding the design and delivery of services. However, final decisions are left to providers (Loeffler & Bovaird, 2017). To explore the adoption of co-production, several studies in the social science literature have focused on analyzing interactions at the micro level, such as those among patients and clinicians in the co-production of patient therapies and treatments (Mortari & Silva, 2018; Palumbo, 2016). However, in terms of interaction dynamics, analysis at the meso level, such as citizens, service provider representatives, and other relevant stakeholders in the co-production of public services, is more challenging than analysis at the micro level, where interactions are dyadic (Petrescu, 2019). During the co-production of public services, several participants are involved concurrently in workshops or other forms of public participation, which increases the complexity and unpredictability of their interaction dynamics (Osborne, 2021).

Recent studies have focused on methods, approaches, and tools for generating service ideas during co-production workshops, even with remote collaboration (Archibald et al., 2021; Benson et al., 2021; Hall et al., 2021; Phillipset al., 2021). Other previous studies have offered practical advice to support facilitators in moving from low levels of public engagement to co-production, ensuring equal, active participation during workshops (Gheduzzi et al., 2020; Liabo et al., 2020). However, few studies have analyzed interaction dynamics and models of communication at the interface (Osborne, 2021). Moreover, they have focused only on collecting data on perceptions or encouraging a particular behavior in priority populations (e.g., (Fenge et al., 2011; Moen et al., 2010; Turner et al., 2021; Wilkinson, 1998)). Analyzing interaction dynamics is of paramount importance when priority populations are involved. The implementation of co-production with priority participants has been widely debated and considered urgent (Mulvale et al., 2019; Pacheco-Vega & Parizeau, 2018; Phillips et al., 2021; Rosen, 2021). Participants in co-production workshops are usually citizens with good education, health, and capabilities for communicating and learning in a group (Alonso et al., 2019; Ballantyne & Varey, 2006). Involving priority and marginalized populations in such activities is complex and time-consuming because of differences in power dynamics, competences, skills, and resources (Amann & Sleigh, 2021; Filipe et al., 2017; Mulvale & Robert, 2021). However, the lack of involvement of such populations leads to the risk of generating unbalanced results, thus undermining the democratic principle of co-production (Bussu et al., 2020; Clark, 2018; Fung, 2006; Weaver, 2019). Allowing marginalized populations to have a voice in the service delivery process is a way of creating inclusive and equal public service systems (Beresford et al., 2021; Radonic et al., 2021). A systematic analysis of interactions during co-production workshops for co-producing services would provide empirical evidence of what happens in terms of micro-dynamics and content during interactions involving priority populations, service providers, and other relevant stakeholders.

To contribute to the constructive enhancement of participatory approaches, this study investigated group interaction dynamics during the co-production of services involving a priority population: family caregivers of elderly people living in rural and remote areas. The aim of this study was achieved by developing and applying a reliable coding scheme, CP-CODER. This coding scheme was designed to enable a common understanding of the key factors that influence interaction dynamics at the interface among priority citizens, service providers, and other relevant stakeholders. The following sections describe the design and application of the coding scheme. The theoretical background of the coding scheme is discussed, followed by a case study on the co-production of public services in which CP-CODER was tested and analyzed.

Material and Methods

Conceptualization: Mixing Two Lenses of Analysis

This work adopted a mixed lens of analysis to develop the conceptual coding scheme, which examines the individual interactions in the co-production of public services with a priority population. It borrows (a) the concept of co-production from the forms of public engagement classification (Loeffler & Martin, 2015) that is suggested in the public service management literature, and (b) the analysis of individual interactions from the co-production of interaction approach (Morgan & Hoffman, 2018) that is suggested in the qualitative data analysis literature (Figure 1). This twofold perspective integrates in one context two units of analysis developed in two different streams of literature.

Figure 1.

Process followed for developing the mixed lens of analysis.

To investigate the co-production of public services, the focus of the analysis has been broadened to the four forms of public engagement of Loeffler and Martin (2015) (Loeffler & Martin, 2015). Since several forms of public engagement may coexist during co-production workshop, a mere study of the interactions in co-production would not consent to gather a complete overview of the dynamics among participants (Gheduzzi et al., 2020). Therefore, the interactions in each form of engagement were analyzed, making the results comparable and complete. Loeffler and Martin’s (2015) framework was considered suitable for analyzing the forms of public engagement given its conciseness, maturity, and international acknowledgement.

To study the interaction dynamics during focus group discussions, Morgan and Hoffman’s (2018) coding system for transition between speakers was chosen. This system is peculiar because it analyzes how things are said and what is said. Due to the novelty and originality of analyzing both the content and the micro-dynamics of group discussions, this system was deemed appropriate for analyzing interaction dynamics using different perspectives (Morgan & Hoffman, 2018).

Table 1 synthetizes the content of the two framework chosen for the analysis of interaction dynamics during the co-production of new public services.

Table 1.

Loeffler and Martin (2015) and Morgan and Hoffman’s (2018) Frameworks.

Frameworks	Description
Loeffler and Martin (2015)	Forms of public engagement
Information	One-way communication; service providers inform citizens reporting choices and decisions that they have taken
Consultation	Two-way communication; service providers collect citizens’ feedbacks and opinions before taking decisions
Participation	Two-way communication; service providers invite citizens to joint decision-making processes to ensure representativeness
Co-production	Two-way communication; providers collaborate with citizens as equal partners in prioritizing, designing, delivering and assessing new or existing services
Morgan and Hoffman (2018)	Coding system for transitions between speakers
Questions and answers	Transitions between participants to either questioning or answering questions from the moderator or other participants
Change in topic	Transitions between participants in which the content under discussion shift or remain the same
Interpersonal connection	Negative or positive transitions that disagree or support/ agree with previous statements

On the basis of the above analysis, Loeffler and Martin’s (2015) framework and Morgan and Hoffman’s (2018) coding scheme were selected to form the basis for developing the coding scheme CP-CODER. Four researchers with previous experience in qualitative research agreed on using these two frameworks as guidelines for generating the initial codes on the forms of public engagement and interaction dynamics during group discussions, respectively. By referring to the literature, the researchers established coding rules for distinguishing categories and developing subcategories. For internal clarity, this phase was called conceptualization phase, as the researchers organized theoretical concepts in a usable coding scheme (Figure 2).

Figure 2.

Process Followed for Developing the new Coding Scheme.

Empirical Test: Revising and Enriching Conceptual Coding Scheme

An empirical case was used to test, revise, and enrich the conceptual coding scheme theoretically derived from Martin and Loeffler’s and Morgan and Hoffman’s frameworks. A longitudinal community-based project called Place4Carers was selected, which was intended to co-produce an innovative service for family caregivers of elderly citizens in Vallecamonica, a remote rural area in the north of Italy. The new service aims to support family caregivers in caring for elders and improve their condition (and thus the conditions of their elders). The project was developed by two Italian universities, a local social care service provider, and an institute of neurological diseases, and it was funded by Fondazione Cariplo (Graffigna et al., 2020). This project was considered suitable for this research for at least three reasons. First, it included co-production workshops involving a priority population defined as people with high probability of experiencing negative health, social and economic outcomes for being exposure to an external strain, such as burden and hours of caregiving, over time (Alwang et al., 2001; Bartolomucci et al., 2005; Munari et al., 2021). Second, the workshops involved the priority population, local service providers’ representatives, and researchers, making the interaction dynamics and models of communication interesting and thought-provoking. Third, the direct involvement of researchers in the workshops’ design, implementation, and assessment ensured access to data and transcript interpretation (Haverland & Blatter, 2012).

The coding scheme was applied to four co-production workshops of the Place4Carers project to integrate the abovementioned frameworks and clarify the forms of engagement and interaction dynamics occurring in the workshops. Overall, 26 family caregivers, three local service provider’s representatives, and five researchers were involved. The family caregivers were mostly female (n = 79%), had medium–low levels of education (with high school degree = 41%), had medium–high levels of burden (time dependence burden = 80%; developmental burden = 60%), and had large numbers of hours dedicated to caring activities weekly (caring hours = 78 h/week) (Graffigna et al., 2021). Since this sample of family caregivers had experienced these psychological and social strains for more than 3 months, they were facing high probability of failing into negative health, social and economic outcomes and, thus, they were considered a priority population (Gheduzzi, 2022). During each workshop, one researcher with at least 2 years of experience in qualitative research acted as facilitator. The workshops were held over a period of 4 months. No incentives were provided to the participants.

The qualitative content analysis explained by Elo et al. (2008, 2014) was conducted for coding the four workshops, following an accountable procedure composed by five phases (Figure 2) (Elo et al., 2014; Elo & Kyngäs, 2008). In the preparation phase, the four co-production workshops used for the analysis were transcribed verbatim, and the transcript was transferred to NVivo, the software used for the analysis. Then, the four researchers read the transcripts several times focusing on two units of analysis: forms of public engagement and types of interactions. In the development phase, the four workshops were coded using two unconstructed matrixes that resulted from the conceptualization phase; thus, each transcript was coded twice. The coding process of each categorization matrix followed an accountable process: the codes were grouped on the basis of their similarities and differences into predefined categories suggested by the two theoretical frameworks, the categories were compared to avoid overlapping, and the categories were divided into subcategories while maintaining the same level of abstraction of the reference frameworks (Elo et al., 2014). During the coding process, the latent content was analyzed by including pauses, laughter, and sighs in the analysis (Assarroudi et al., 2018).

The first unconstructed categorization matrix aimed to investigate the forms of public engagement using Loeffler and Martin’s (2015) framework as guideline. Table 2 presents the data structure.

Table 2.

The Complete Data structure of the Unconstructed Matrix, Using Loeffler and Martin’s (2015) Framework.

	Examples of the Coding Scheme
The first lens of analysis
Information
Participants’ information	CG6: Hello everyone, I am CG6, I assist my mother because my mum has been suffering from Parkinson’s for 24 years
Services’ information	CG7: For example, I noticed that they have changed the standard practice for requesting the RSA service: they sent a doctor [at home]!
Project’s information	Facilitator: Thank you for being here. Two Italian universities, a local social care service provider and an institute of neurological diseases, take part in this project, which aims at co-producing a new service to help you in caring for elderly relatives
Consultation
Requests	Facilitator: By reflecting on the issue of being informed [..]: What would you like to know more about? What is the best approach that we can put in place to support you in acquiring such information?
Feedbacks	CG3: Social workers are a reference point but they don’t have all the necessary information
Participation
Agreement	CG4: Absolutely!CG6: You’re completely right!
Disagreement	CG3: no, there isn’t. Every patient is alone and everyone has to make their own search for information
Co-production
Co-commissioning	CG1: I don’t know if it is feasible, but [we can create] some TV commercials to inform caregivers]. There are some commercials about
Co-design	CG1: But it should be a website dedicated to this specific valley, It should contains only the information related to this valley!
Envisioning	What can be done to support you with logistical issues? Just daydream about it

The second unconstructed matrix investigated the interaction dynamics using Morgan and Hoffman’s (2018) coding system as guideline. Table 3 presents its data structure.

Table 3.

The Complete Data Structure of the Unconstructed Matrix, Using Morgan and Hoffman’s (2018) Framework.

	Examples of the Coding Scheme
The second lens of analysis
Questions and answers (always explicit)
Between facilitator and participants	Facilitator: Are you employed?CG5: No, I retired a year ago
Between providers and participants	Provider: Sorry, but.. What do you expect from social workers?CG2: For instance, they should inform us [about the available services]
Among participants	CG2: How old is she?CG3: She’s 95
Change in topic
Introduction of a new topic-Implicit	CG4: Hello everyone, I’m CG4 and I come from Vallecamonica, my mother fell ill 5 years ago
Introduction of a new topic- Explicit	CG2: But I think, sorry [for the interruption], that TV channels are watched by so many people
Expansion-Implicit	Mine [my general practitioner] usually comes to my home to treat my dear one. She is usually very helpful
Expansion-Explicit	CG5: Moreover, it helps us as well, because we can vent our feelings and listen to other people’s experiences
Differentiation-Implicit	CG3: It’s true that we should look for a substitute, but the big challenge is to get him/her accepted
Differentiation-Explicit	CG1: Yes, but I can train my mind also with crossword puzzles!
Interpersonal connection (always explicit)
Agreement	CG2: That’s right!CG5: I agree!
Disagreement	CG2: No, I don’t think the process should work as you have just described
Support	CG3: Based on how I’m feeling now, I would come every day! I like [this idea]!

In the refinement phase, the results of the coding process were discussed by the four researchers to ensure that the categorization matrixes captured how the forms of engagement differed in terms of interactions (Elo et al., 2014).

In the finalization phase, the coding results of the four workshops were compared with each other, and their correspondence with the research question was tested to confirm the validity and accuracy of the analysis. This approach, grounded on the theoretical frameworks, allowed the researchers to support and extend the existing theory while exploring and including new findings.

Ethical Approval

The participants provided written consent before joining the research and the co-production workshops. The ethical committee of the Università Cattolica del Sacro Cuore and the Politecnico di Milano approved the protocol for caregiver involvement.

Inter-Coder Reliability

Cohen’s kappa with a 95% confidence interval was adopted to test the intercoder reliability of the CP-CODER coding scheme. Given the wide use of Cohen’s kappa in assessing interrater reliability, this measure was used to evaluate the agreement among the coders on the coding of the four workshops with the application of CP-CODER (McHugh, 2012). To assess the intercoder reliability, two out of the four trained researchers rated 10% of the workshops’ transcripts independently, checking the agreement on (a) the level of public engagement and on (b) the group interaction dynamics. After evaluating the intercoder reliability, the two coders discussed and resolved disagreements. Then, intercoder reliability was assessed for each category and overall.

Validity

Validity determines “the degree to which researchers claim about knowledge correspond to the reality (or research participants’ constructions of reality) being studied” (Eisner & Peshkin, 1990). Among the different methods of assessing validity in qualitative research, a guiding theory was used to verify findings. Hence, CP-CODER was developed starting from two recognized frameworks in the academic literature for analyzing interaction dynamics during group discussions and distinguishing forms of public engagement (i.e., (Loeffler & Martin, 2015; Morgan & Hoffman, 2018)). The development of CP-CODER followed an accountable procedure that included subcategories, codes, and any integration of the initial coding scheme from the literature of public service management and qualitative data analysis literature (Grayson-Sneed & Smith, 2018). Once the categories and subcategories were established, a structured codebook was created for sorting the data in the coding scheme and started the coding analysis to enhance construct validity (Thyer, 2010).

Results

The following paragraphs report the new conceptual coding framework (CP-CODER) and the results of interrater reliability.

CP-CODER: New Conceptual Coding Framework

This section presents CP-CODER, a new framework developed over 18 months for investigating the forms of public engagement and the interaction dynamics.

The conceptualization phase gave origin to two unconstrained matrixes of main categories, and related subcategories were created deductively from the two theoretical frameworks on the forms of engagement and interaction dynamics. All the forms of public engagement of Loeffler and Martin (2015) (information, consultation, participation, and co-production) and all types of interaction of Morgan and Hoffman’s (2018) coding scheme (questions and answers, change in topic, and interpersonal connections) were retained as the main categories of the two unconstructed preliminary matrixes of CP-CODER. In the refinement phase, the four researchers reflected and agreed on the fact that three factors, i.e. interaction dynamics (how things are said), discussion content (what is said), and the level of abstraction of the content (shared social objects), distinguish co-production from the other forms of engagement (Morgan, 2015). On this basis, the categorization matrix for analyzing interactions was integrated by adding an additional category (shared social object category) that arose from the literature on qualitative methods. Then, the transcripts were reviewed accordingly. Table 4 presents the shared social object category, highlighting the complete data structure and it links the categories with the subcategories identified from the theory.

Table 4.

The Data Structure of the Integrated Category: Shared Social Objects.

	Examples of the Coding Scheme
The second lens of analysis
Shared social objects
Conceptualization-Implicit	CG2: It’s a matter of [service] fragmentation!
Conceptualization-Explicit	So we said: there’s a problem of information
Co-planning-Implicit	CG3: A bus for caregivers that has several seats, which can be booked
Co-planning-Explicit	CG8: Then, as she was saying, a self-help group where everyone can share their time and abilities

The version of CP-CODER that was developed theoretically using two theoretical frameworks was then revised and tested empirically using the four co-production workshops.

The first lens of analysis was used to investigate the adoption of co-production by distinguishing it from the other forms of public engagement. In what follows, the resulting categories and subcategories of the first unconstrained matrix developed from Loeffler and Martin (2015) framework are presented.

Information

It is one-way communication, as defined by Loeffler and Martin (2015) (Loeffler & Martin, 2015). It includes contextual information that clarifies the frame of the discussion, but does not support providers in co-producing public service solutions. The main topics in this dimension are caregivers’ personal life experiences, healthcare services, and project activities. This category is divided into three subcategories:

• Participants’ information .

• Services’ information.

• Project’s information.

Consultation

It is two-way communication established by providers to investigate citizens’ opinions (Loeffler & Martin, 2015). It includes feedbacks, preferences, and suggestions of participants about a specific issue or question highlighted by the facilitators or providers. This category is divided into two subcategories:

• Requests. It are made by the facilitators or providers.

• Feedbacks. They are given by participants.

Participation

It is two-way communication in which providers involve citizens in decision-making processes. It includes open debates among participants in which participants exchange opinions (Loeffler & Martin, 2015). Since this category is characterized by a sequence of different participants’ opinions that agree or disagree with those previously expressed, it is divided into two subcategories:

• Agreement.

• Disagreement .

Co-production

It includes interventions that make substantial contributions to the definition of a new service (Loeffler & Martin, 2015). From the case under investigation, it emerged that co-production involves two main steps: (a) generation of new service ideas for supporting caregivers and (b) development of such ideas, where participants challenge and enrich the emerging ideas by sharing personal opinions on how to develop them practically. This dimension includes also the intervention of facilitators (or providers in some cases) who attempted to rein in the discussion by asking participants to concentrate on co-designing possible service solutions (Gheduzzi et al., 2020). On the basis of these findings, the co-production category is divided into three subcategories:

• Co-commissioning.

• Co-designing.

• Envisioning.

The second lens of analysis was used to analyze how thing were said and what was said during the workshops. In what follows, the resulting categories and subcategories of the second unconstrained matrix developed from Morgan and Hoffman (2018) tool are presented.

Questions and answers

It includes all the requests and responses among participants. This category is organized in the following subcategories:

• Questions and answers among participants.

• Questions and answers between participants and providers.

• Questions and answers between participants and facilitators.

Change in topic

It means any shift in the discussion subject that may change during turn-taking. Morgan and Hoffman’s tool divides this category into three subcategories: introduction of a new topic, expansion, and differentiation. Each subcategory can be explicit or implicit, depending on how the discourse of a previous participant is continued by the following participants. An explicit change in topic means that the participants continue the discourse by joining the discussion and using specific grammatical connections that refer to what has just been said. Several possible statements that the participants used to link their speech with the previous ones, such as “The same situation happened,” “I’ve faced this problem as well,” and “I completely agree with this point”, were coded. By contrast, an implicit continuation of the topic means that the participants do not explicitly refer to previous contents.

• Introduction of a new topic. It includes all statements that shift the topic of the ongoing discussion. The explicit introduction of a new topic is usually highlighted by specific connections, such as “Can I change the subject? I’d like ...,” “I’m sorry to interrupt you, but ...,” or “I’d like to speak about something that has nothing to do with what we are discussing …”

• Expansion. It includes all statements that enrich the ongoing discussion by sharing new information and content. An explicit expansion is usually introduced by transitions such as “Another aspect is …,” “Instead, in my city …,” or “moreover.”

• Differentiation. It occurs when participants compare what has just been said with their beliefs. An explicit differentiation can contain transitions such as “Yes, but ...,” “It could be a good idea, but ...,” or “I completely agree with you on this point, but …”

Interpersonal connections

It includes statements that agree or disagree with other parties’ contributions explicitly. As suggested by Morgan and Hoffman’s tool, this category is divided into three subcategories.

• Agreement. It includes positive codes that state the support of participants for previous contributions by using statements such as “Yes, that’s correct!” or “That’s true.”

• Disagreement. It includes negative codes that highlight the disagreement of participants with previous statements by using sentences such as “No, that’s another point” or “No, I don’t think the process should work as you have just described.”

• Support. It includes positive contributions that not only support previous statements but also show a sense of mutual affection, such as “For me, it’s fine. As he was saying, we should …” or “[Yes,] I would came also every day!” (Morgan & Hoffman, 2018).

Shared social objects

It includes sentences that abstract and discover the main issues and possible service solutions from the personal experiences of each participant. This category is divided into two subcategories, namely, conceptualization and co-planning, which can be either implicit or explicit.

• Conceptualization. It includes abstraction of the issues and topics that arise during the discussion. In this dimension, participants identify the most shared topics and issues that have arisen during the discussion and suggest possible service solutions that can help them deal with these issues. Explicit conceptualization can contain transitions such as “The root problem is …,” “thus,” or “so.”

• Co-planning. It includes statements that detail the shared abstract service solution in concrete and practical terms. Explicit co-planning can contain grammatical conjunctions, such as “then,” “but,” or “and.”

Figure 3 displays the new coding scheme, i.e. CP-CODER, by highlighting how the two lenses of analysis were explicated in categories and how these categories were explicated in subcategories.

Figure 3.

The new coding scheme: CP-CODER.

Intercoder Reliability Check

The kappa for 10% of the transcripts was 0.91, which underlines an almost perfect agreement between the two trained researchers (McHugh, 2012). The overall percentage of agreement for all items was 91.16% (Table 5), which was to be assessed in case Cohen’s kappa could not be evaluated (e.g., both trained researchers used the same code for all the codes of a given category) (Grayson-Sneed et al., 2017).

Table 5.

The Results of the new Coding Scheme, i.e. CP-CODER, Variables and Items.

No.	Item	Percent of Agreement, %
The first lens of analysis
Information (Kappa = 0.92)
1	Participants’ information	100
2	Services’ information	88
3	Project’s information	100
Consultation (Kappa = 0.70)
4	Requests	89
5	Feedbacks	75
Participation (Kappa = 1.00)
6	Agreement	100
7	Disagreement	100
Co-production (Kappa = 0.92)
8	Co-commissioning	86
9	Co-design	100
10	Envisioning	100
The second lens of analysis
Questions and answers (always explicit) (Kappa = 1.00)
11	Between facilitator and participants	100
12	Between providers and participants	100
13	Among participants	100
Change in topic (Kappa = 0.82)
14	Introduction of a new topic-Implicit	100
15	Introduction of a new topic- Explicit	100
16	Expansion-Implicit	64
17	Expansion-Explicit	100
18	Differentiation-Implicit	100
19	Differentiation-Explicit	100
Interpersonal connection (always explicit) (Kappa = 1.00)
20	Agreement	100
21	Disagreement	100
22	Support	100
Shared social objects (Kappa=0.79)
23	Conceptualization-Implicit	100
24	Conceptualization-Explicit	75
25	Co-planning-Implicit	100
26	Co-planning-Explicit	50

Discussion and Conclusion

Discussion

A reliable coding scheme (CP-CODER) was developed to analyze interaction dynamics during co-production involving a priority population: family caregivers of elderly citizens living in rural and remote areas. CP-CODER was developed based on two well-known frameworks of Loeffler and Martin (2015) and Morgan and Hoffman (2018) for analyzing forms of public engagement and interaction dynamics. Then, the coding scheme was integrated with existing theoretical findings, finalized, and applied to four co-production workshops involving family caregivers of elderly people living in remote rural areas. Finally, Cohen’s kappa was adopted to assess the intercoder reliability of the final version of CP-CODER.

This coding scheme addresses the need to analyze interactions at the interface among priority citizens and service providers during co-production workshops. Despite the specific context of its application in the present study, CP-CODER is intended to be applicable not only to family caregivers but also to other priority populations that experience external strain over time and risk negative health, social, and economic outcomes.

Previous studies in the literature have focused on studying interactions during patient–clinician dialogues concerning the co-production of patients’ therapies (e.g., (Realpe et al., 2015; Roter & Larson, 2002; Van Dam et al., 2003)). While these studies have clarified interaction dynamics in shared decision-making at the micro level (i.e., patient–clinician relationships), they have not elucidated interactions at the meso level, at which priority populations and service providers co-produce services during workshop discussions. CP-CODER contributes to filling these gaps in the literature by analyzing interaction dynamics during workshop discussions about service co-production involving priority populations.

The application of this novel coding scheme yielded relevant findings. First, it showed that the co-production of public services in a priority population may require lower forms of public engagement (i.e., information, consultation, and participation). The integration of the technical knowledge of service providers and the experiential knowledge of priority populations requires preliminary phases in which participants become familiar with each other and establish a climate of trust. Equal partnerships between providers and priority citizens, which characterizes co-production, are not achieved immediately because time is required to overcome differences in power dynamics. In planning co-production workshops, both researchers and service providers should include a preliminary phase in which a trusting environment and a common understanding among participants are achieved.

Furthermore, the CP-CODER revealed that interaction dynamics during co-production changed not only in terms of micro-dynamics and subjects, as suggested by Morgan and Hoffman (2018), but also in terms of the level of abstraction of the content. Regarding the focus groups, the co-production workshops required a phase during which participants identified common and recurrent issues that arose during the discussion (Wilkinson, 1998), thus raising the level of abstraction of the content. In focus groups, this phase is the final phase of the collaboration process. Instead, in co-production workshops, it was a necessary step for discussing ideas about possible service solutions. In managing co-production workshops, researchers, and service providers should guide the discussion, encouraging participants to organize and conceptualize the issues, using physical and digital tools when they are deemed effective (Stickdorn & Schneider, 2012).

The findings of this research showed that the micro-dynamics and subjects characterizing co-production are similar to those of the other forms of public engagement (i.e., information, consultation, and participation). However, the level of abstraction of the content was identified as the factor that distinguished co-production from lower forms of public engagement. For this reason, researchers and service providers should support priority populations with ad hoc communication strategies and tools for creating and discussing abstract content. Because of the precarious condition of participants, they are exposed to external strain over time. Thus, they might need support in focusing on discussions, identifying recurrent topics, and transforming the content of such topics from the experiential and personal to the abstract and general.

The development of this new coding scheme offers at least three theoretical and practical contributions. First, as the first coding scheme for analyzing interpersonal interactions during co-production, it enriches the literature on this topic. Co-production has usually been investigated by examining its antecedents or effects, but its mechanism has remained a “black-box” (Gheduzzi et al., 2021; Park, 2020). Despite the high unpredictability of interaction dynamics, the in-depth analysis of the mechanism of co-production is not only pioneering but also fundamental in identifying the real source of value (Osborne, 2021). Indeed, the interactions and exchanges among citizens and service providers are the main sources of value in the co-production process (Osborne et al., 2021). Therefore, an in-depth analysis of the interaction dynamics during co-production is not only pioneering but also fundamental for the successful development of co-production (Dudau et al., 2019).

Second, this study distinguishes co-production from other forms of public engagement in terms of interaction dynamics. This distinction could enable providers and researchers to prevent tokenism or the unbalanced involvement of priority populations in decision-making processes. In some cases, the involvement of priority populations was only feigned because of limited opportunities for their participation or little respect for their comments (Crompton, 2019). Hence, the clarification of interactions during the co-production process encourages the fair involvement of the priority populations (Gheduzzi et al., 2020).

Third, the findings of this research encourage the involvement of priority populations, whose voices are often forgotten despite their willingness to contribute (McColl-Kennedy et al., 2017). The literature on this topic includes a debate on the possibility of involving fragile and marginalized citizens in co-production processes despite their limited motivations and uncertain and stressful conditions (Alonso et al., 2019; Brandsen, 2021). This study contributes to this debate by providing guidelines for interacting with and involving priority populations in their successful inclusion in co-production processes

Conclusion

This study presents a reliable coding scheme that is representative of the interaction dynamics occurring among priority citizens and service providers during co-production workshops. It is recommended as an initial criterion standard for supporting researchers and service providers in conducting co-production with priority populations.

Although the CP-CODER has the potential to be a relevant tool for researchers and practitioners hoping to enhance the involvement of usually excluded citizens and, thus, the creation of more democratic public services, some limitations should to be considered. First, CP-CODER was developed and finalized on a small sample (a single-site project). Second, the validity did not include cross-sectional data or triangulation strategies (Rourke & Anderson, 2004; Thyer, 2010). Third, the analysis was at the meso level (citizens and service providers co-producing a service). However, group dynamics can also occur at the macro level (policy-makers and citizens or voluntary organizations co-producing a new policy or regulation) (Petrescu, 2019). Hence, future research should integrate and test CP-CODER in different contexts and at various levels of co-production to improve its validity and reliability.

CP-CODER will assist providers and researchers in involving priority populations in co-producing new or existing services in a way that ensures equal contribution of all participants and better reflexive practice by researchers. Although CP-CODER was developed on the basis of a specific target group, it can be easily adopted and modified to consider other types of priority populations with high probability of experiencing negative health, social and economic outcomes for being exposure to an external strain over time.

Footnotes

Acknowledgments

The authors are grateful to the carers involved for their time and contributions. We are also grateful to the other research members of the project, including the local home care service provider (Azienda Territoriale per i Servizi alla Persona) and the four nursing homes (Fondazione Angelo Maj, Fondazione Villa Zani, Fondazione Rizzieri, Fondazione Ninj Beccagutti).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Fondazione Cariplo, under Grant call ‘Ricerca sociale - 2017', rif. 2017–0955.

Ethical Approval

The participants provided written consent before joining the research and the co-production workshops. The ethical committee of the Politecnico di Milano and the Università Cattolica del Sacro Cuore approved the protocol for caregiver involvement.

ORCID iD

Eleonora Gheduzzi

References

Alonso

J. M.

Andrews

Clifton

Diaz-Fuentes

(2019). Factors influencing citizens’ co-production of environmental outcomes: A multi-level analysis. Public Management Review, 21(11), 1620–1645. https://doi.org/10.1080/14719037.2019.1619806

Alwang

Siegel

P. B.

Jørgensen

S. L.

Tech

(2001). Vulnerability: A view from different disciplines.Social protection discussion paper. World Bank, 0115, 46.

Amann

Sleigh

(2021). Too vulnerable to involve? Challenges of engaging vulnerable groups in the co-production of public services through research. International Journal of Public Administration, 44(9), 715–727. https://doi.org/10.1080/01900692.2021.1912089

Archibald

Ambagtsheer

Lawless

M. T.

Thompson

M. O.

Shultz

Chehade

M. J.

Whiteway

Sheppard

Plaza

M. P. de

Kitson

A. L.

(2021). Co-Designing evidence-based videos in health care: A case exemplar of developing creative knowledge translation “evidence-experience” resources. International Journal of Qualitative Methods, 20(January 2021), 1–10. https://doi.org/10.1177/16094069211019623

Arnstein

S. R.

(1969). A ladder of citizen participation. Journal of the American Planning Association, 35(4), 216–224. https://doi.org/10.1080/01944366908977225

Assarroudi

Heshmati Nabavi

Armat

M. R.

Ebadi

Vaismoradi

(2018). Directed qualitative content analysis: The description and elaboration of its underpinning methods and data analysis process. Journal of Research in Nursing, 23(1), 42–55. https://doi.org/10.1177/1744987117741667

Ballantyne

Varey

R. J.

(2006). Creating value-in-use through marketing interaction: The exchange logic of relating, communicating and knowing. Marketing Theory, 6(3), 335–348. https://doi.org/10.1177/1470593106066795

Bartolomucci

Palanza

Sacerdote

Panerai

A. E.

Sgoifo

Dantzer

Parmigiani

(2005). Social factors and individual vulnerability to chronic stress exposure. Neuroscience and Biobehavioral Reviews, 29(1), 67–81. https://doi.org/10.1016/j.neubiorev.2004.06.009

Batalden

Margolis

Seid

Armstrong

Opipari-Arrigan

Hartung

(2016). Coproduction of healthcare service. BMJ Quality and Safety, 25(7), 509–517. https://doi.org/10.1136/bmjqs-2015-004315

10.

Baum

MacDougall

Smith

(2006). Participatory action research. Journal of Epidemiology and Community Health, 60(10), 854–857. https://doi.org/10.1136/jech.2004.028662

11.

Benson

Pedersen

Tsalis

Futtrup

Dean

Aschemann-Witzel

(2021). Virtual co-creation: A guide to conducting online co-creation workshops. International Journal of Qualitative Methods, 20(January 2021), 1–15. https://doi.org/10.1177/16094069211053097

12.

Beresford

Kaur

Williams

O. L. I.

Beresford

Farr

Hickey

Kaur

Ocloo

Tembo

(2021). COVID-19 and Co-Production in Health and Social Care Research, Policy and Practice. Bristol University Press, Policy Press.

13.

Bergold

Stefan

(2012). Participatory research methods: A methodological approach in motion. Historical Social Research, 37(4), 191–222.

14.

Bovaird

Loeffler

(2013). The role of co-production for better health and wellbeing: Why we need to change. In Loeffler

Power

Bovaird

Hine-Hughes

(Eds.), Co-Production of Health and Wellbeing in Scotland (pp. 20–27). Governance International.

15.

Brandsen

(2021). Vulnerable citizens: Will co-production make a difference? In Loeffler

Bovaird

(Eds.), The Palgrave Handbook of Co-Production of Public Services and Outcomes (pp. 527–539). Palgrave Macmillan.

16.

Bussu

Mirza

Pattison

Marshall

(2020). Engaging with care: Ethical issues in participatory research. Qualitative Research, 21(5), 667–685. https://doi.org/10.1177/1468794120904883

17.

Clark

J. K.

(2018). Designing public participation: Managing problem settings and social equity. Public Administration Review, 78(3), 362–374. https://doi.org/10.1111/puar.12872

18.

Crompton

(2019). Inside co-production: Stakeholder meaning and situated practice. Social Policy and Administration, 53(2), 219–232. https://doi.org/10.1111/spol.12466

19.

Dudau

Glennon

Verschuere

(2019). Following the yellow brick road? (Dis)enchantment with co-design, co-production and value co-creation in public services. Public Management Review, 21(11), 1577–1594. https://doi.org/10.1080/14719037.2019.1653604

20.

Eisner

E. W.

Peshkin

(1990). Qualitative Inquiry in Education: The Continuing Debate. Teachers College.

21.

Elo

Kääriäinen

Kanste

Pölkki

Utriainen

Kyngäs

(2014). Qualitative content analysis. SAGE Open, 4(1), 1–10. https://doi.org/10.1177/2158244014522633

22.

Elo

Kyngäs

(2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

23.

Fenge

L. A.

Hodges

Cutts

(2011). Seen but seldom heard: Creative participatory methods in a study of youth and risk. International Journal of Qualitative Methods, 10(4), 418–430. https://doi.org/10.1177/160940691101000408

24.

Filipe

Renedo

Marston

(2017). The co-production of what? Knowledge, values, and social relations in health care. PLOS Biology, 15(5), 1–6. https://doi.org/10.1371/journal.pbio.2001403

25.

Fung

(2006). Varieties of participation in complex governance. Public Administration Review, 66(s1), 66–75, https://doi.org/10.1111/j.1540-6210.2006.00667.x

26.

Gheduzzi

(2022). Framing the mechanism of co-production of public services with vulnerabile citizens. [Doctoral Thesis, Politecnico di Milano]. http://hdl.handle.net/10589/181592

27.

Gheduzzi

Masella

Morelli

Graffigna

(2021). How to prevent and avoid barriers in co-production with family carers living in rural and remote area: An Italian case study. Research Involvement and Engagement, 7(1), 1–13. https://doi.org/10.1186/s40900-021-00259-0

28.

Gheduzzi

Morelli

Graffigna

Masella

(2020). Facilitating co-production in public services: Empirical evidence from a co-design experience with family caregivers living in a remote and rural area. Health Services Management Research, 34(1), 21–35. https://doi.org/10.1177/0951484820971452

29.

Graffigna

Barello

Morelli

Gheduzzi

Corbo

Ginex

Ferrari

Lascioli

Feriti

Masella

(2020). Place4Carers: A mixed-method study protocol for engaging family caregivers in meaningful actions for successful ageing in place. BMJ Open, 10(8), Article e037570. https://doi.org/10.1136/bmjopen-2020-037570

30.

Graffigna

Gheduzzi

Morelli

Barello

Corbo

Ginex

Ferrari

Lascioli

Feriti

Masella

(2021). Place4Carers: A multi-method participatory study to co-design, piloting, and transferring a novel psycho-social service for engaging family caregivers in remote rural settings. BMC Health Services Research, 21(1), 1–17. https://doi.org/10.1186/s12913-021-06563-5

31.

Grayson-Sneed

K. A.

Smith

R. C.

(2018). A research coding method to evaluate a smoking cessation model for training residents—a preliminary report. Patient Education and Counseling, 101(3), 541–545. https://doi.org/10.1016/j.pec.2017.09.010

32.

Grayson-Sneed

K. A.

Smith

S. W.

Smith

R. C.

(2017). A research coding method for the basic patient-centered interview. Patient Education and Counseling, 100(3), 518–525. https://doi.org/10.1016/j.pec.2016.10.003

33.

Hall

Gaved

Sargent

(2021). Participatory research approaches in times of Covid-19: A narrative literature review. International Journal of Qualitative Methods, 20(January 2021), 1–15. https://doi.org/10.1177/16094069211010087

34.

Haverland

Blatter

(2012, August 30–September 2). Two or three approaches to explanatory case study research ? [Conference session]. Paper prepared for presentation at the Annual Meeting Of The American Political Science Association, New Orleans, United States. http://www.apsanet.org/content_77049.cfm?navID=988

35.

Heselton

G. A.

Rempel

G. R.

Nicholas

D. B.

(2021). Integrating community participation with interpretative phenomenological analysis: Reflections on engaging the autism community. International Journal of Qualitative Methods, 20(January 2021), 1–10. https://doi.org/10.1177/16094069211055575

36.

Kemmis

McTaggart

(2005). Communicative action and the public sphere. In Denzin

Lincoin

(Eds.), The Sage Handbook of Qualitative Research (3rd ed., pp. 559–603).

37.

Liabo

Boddy

Bortoli

Irvine

Boult

Fredlund

Joseph

Bjornstad

Morris

(2020). Public involvement in health research: What does “good” look like in practice? Research Involvement and Engagement, 6(1), 1–12. https://doi.org/10.1186/s40900-020-0183-x

38.

Loeffler

(2020). Co-Production of Public Services and Outcomes. Palgrave Macmillan.

39.

Loeffler

Bovaird

(2017). From participation to co-production: Widening and deepening the contributions of citizens to public services and outcomes. In van Thiel

Ongaro

(Eds.), The Palgrave Handbook of Public Administration and Management in Europe (pp. 403–423). Palgrave Macmillan.

40.

Loeffler

Martin

(2015). Public management and governance. In Bovaird

Loeffler

(Eds.), Citizen Engagement (pp. 301-318). Routledge.

41.

Macaulay

A. C.

(2017). Participatory research: What is the history? Has the purpose changed? Family Practice, 34(3), 256–258. https://doi.org/10.1093/fampra/cmw117

42.

McColl-Kennedy

J. R.

Snyder

Elg

Witell

Helkkula

Hogan

S. J.

Anderson

(2017). The changing role of the health care customer: Review, synthesis and research agenda. Journal of Service Management, 28(1), 2–33. https://doi.org/10.1108/JOSM-01-2016-0018

43.

McHugh

M. L.

(2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/bm.2012.031

44.

Moen

Antonov

Nilsson

J. L. G.

Ring

(2010). Interaction between participants in focus groups with older patients and general practitioners. Qualitative Health Research, 20(5), 607–616. https://doi.org/10.1177/1049732309354097

45.

Morgan

D. L.

(2015). Focus groups and social interaction. In Gubrium

J. F.

Holstein

J. A.

Marvasti

A. B.

McKinney

K. D.

(Eds.), The SAGE Handbook of Interview Research: The Complexity of the Craft (pp. 161–176). SAGE Publications.

46.

Morgan

D. L.

Hoffman

(2018). A System for coding the interaction in focus groups and dyadic interviews. The Qualitative Report. 23(3), 519–531, https://doi.org/10.46743/2160-3715/2018.2733

47.

Mortari

Silva

(2018). Words faithful to the phenomenon: A discursive analysis method to investigate decision-making processes in the intensive care unit. International Journal of Qualitative Methods, 17(1), 1–14. https://doi.org/10.1177/1609406918797791

48.

Mulvale

Moll

Miatello

Murray-Leung

Rogerson

Sassi

R. B.

(2019). Co-designing services for youth with mental health issues: Novel elicitation approaches. International Journal of Qualitative Methods, 18(January 2019), 1–13. https://doi.org/10.1177/1609406918816244

49.

Mulvale

Robert

(2021). Special issue- engaging vulnerable populations in the co-production of public services. International Journal of Public Administration, 44(9), 711-714. https://doi.org/10.1080/01900692.2021.1921941

50.

Munari

S. C.

Wilson

A. N.

Blow

N. J.

Homer

C. S. E.

Ward

J. E.

(2021). Rethinking the use of ‘vulnerable’. Australian and New Zealand Journal of Public Health, 45(3), 197–199. https://doi.org/10.1111/1753-6405.13098

51.

Newitt

Thomas

N. P.

(2020). Participating in social exclusion: A reflexive account of collaborative research and researcher identities in the field. Action Research, 1–17(2), 105, 121. https://doi.org/10.1177/1476750320954530

52.

Olesen

B. R.

Nordentoft

H. M.

(2018). “ On slippery ground ” – beyond the innocence of collaborative knowledge production. Qualitative Research in Organizations and Management, 13(4), 356–367. https://doi.org/10.1108/QROM-09-2017-1562

53.

Osborne

S. P.

(2021). Public Service Logic: Creating Value for Public Service Users, Citizens, and Society Through Public Service Delivery. Routledge.

54.

Osborne

S. P.

Nasi

Powell

(2021). Beyond co-production: Value creation and public services. Public Administration, 99(4), 1–17. https://doi.org/10.1111/padm.12718

55.

Pacheco-Vega

Parizeau

(2018). Doubly engaged ethnography: Opportunities and challenges when working with vulnerable communities. International Journal of Qualitative Methods, 17(1), 1–13. https://doi.org/10.1177/1609406918790653

56.

Pain

Francis

(2003). Reflections on participatory research. Area, 35(1), 46–54. https://doi.org/10.1111/1475-4762.00109

57.

Palumbo

(2016). Contextualizing co-production of health care: A systematic literature review. International Journal of Public Sector Management, 29(1), 72–90. https://doi.org/10.1108/IJPSM-07-2015-0125

58.

Park

(2020). Beyond patient-centred care: A conceptual framework of co-production mechanisms with vulnerable groups in health and social service settings. Public Management Review, 22(3), 452–474. https://doi.org/10.1080/14719037.2019.1601241

59.

Petrescu

(2019). From marketing to public value: Towards a theory of public service ecosystems. Public Management Review, 21(11), 1733–1752. https://doi.org/10.1080/14719037.2019.1619811

60.

Phillips

Christensen-Strynø

M. B.

Frølunde

(2021). Thinking with autoethnography in collaborative research: A critical, reflexive approach to relational ethics. Qualitative Research, (July 2021), 1–16. https://doi.org/10.1177/14687941211033446

61.

Phillips

Frølunde

Christensen-Strynø

M. B.

(2021). Confronting the complexities of “co-production” in participatory health research: A critical, reflexive approach to power dynamics in a collaborative project on parkinson’s Dance. Qualitative Health Research, 31(7), 1290–1305. https://doi.org/10.1177/10497323211003863

62.

Radonic

Jacob

Kalman

Lewis

E. Y.

(2021). It’s a sprint, not a marathon: A case for building short-term partnerships for community-based participatory research. Qualitative Research, (August 2021), 1–19. 146879412110294. https://doi.org/10.1177/14687941211029477

63.

Realpe

A. X.

Wallace

L. M.

Adams

A. E.

Kidd

J. M.

(2015). The development of a prototype measure of the co-production of health in routine consultations for people with long-term conditions. Patient Education and Counseling, 98(12), 1608–1616. https://doi.org/10.1016/j.pec.2015.07.005

64.

Rosen

(2021). Participatory research in and against time. Qualitative Research, (September 2021), 1–17. 146879412110419. https://doi.org/10.1177/14687941211041940

65.

Roter

Larson

(2002). The roter interaction analysis system (RIAS): Utility and flexibility for analysis of medical interactions. Patient Education and Counseling, 46(4), 243–251. https://doi.org/10.1016/S0738-3991(02)00012-5

66.

Rourke

Anderson

(2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1), 5–18. https://doi.org/10.1007/BF02504769

67.

Stickdorn

Schneider

(2012). This is service design thinking: Basics, tools, cases. BIS Publishers.

68.

Thomson

Berriman

(2021). Starting with the archive: Principles for prospective collaborative research. Qualitative Research, (June 2021), 1–18. https://doi.org/10.1177/14687941211023037

69.

Thyer

(2010). The Handbook of Social Work Research Methods. SAGE Publications.

70.

Turner

Rushby

Gauthier

James

Bahaj

A. B.

Aragon

Sweetnam

Ellis

(2021). An online ‘face to face’ focus group approach for understanding how household energy use and green investment decisions differ by personality traits. International Journal of Qualitative Methods, 20(January 2021), 1–17. https://doi.org/10.1177/16094069211038312

71.

Van Dam

H. A.

Van Der Horst

Van Den Borne

Ryckman

Crebolder

(2003). Provider-patient interaction in diabetes care: Effects on patient self-care and outcomes - a systematic review. Patient Education and Counseling, 51(1), 17–28. https://doi.org/10.1016/S0738-3991(02)00122-2

72.

Weaver

(2019). Co-production, governance and practice: The dynamics and effects of user voice prison councils. Social Policy and Administration, 53(2), 249–264. https://doi.org/10.1111/spol.12442

73.

Wilkinson

(1998). Focus groups in health research. Journal of Health Psychology, 3(3), 329–348. https://doi.org/10.1177/135910539800300304

Development of a Novel Coding Scheme to Explore Interactions in the Co-Production of Public Services with Priority Populations

Abstract

Keywords

Material and Methods

Conceptualization: Mixing Two Lenses of Analysis

Empirical Test: Revising and Enriching Conceptual Coding Scheme

Ethical Approval

Inter-Coder Reliability

Validity

Results

CP-CODER: New Conceptual Coding Framework

Information

Consultation

Participation

Co-production

Questions and answers

Change in topic

Interpersonal connections

Shared social objects

Intercoder Reliability Check

Discussion and Conclusion

Discussion

Conclusion

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

Ethical Approval

ORCID iD

References