Engaging Children in Intervention Development – A Comparison of Four Qualitative Methods and Their Suitability to Elicit Information Relevant for Early Intervention Development

Abstract

Stakeholder involvement in intervention development has become increasingly popular and is understood as crucial for successful implementation. However, methodological difficulties persist regarding the implementation of appropriate participatory methods to engage children and collect relevant information for intervention development. This paper discusses four creative, qualitative methods – a brainstorming task, an individual letter, a story-based task, and a focus group task – and compares their suitability to generate relevant information for early intervention development. These four methods were used in the early stages of a project aiming to develop a peer relationship intervention and were analysed using thematic analysis. Based on theoretical considerations, information categories (a) definition aspects, (b) context factors, and (c) change mechanisms were of interest, while (d) examples and experience and (e) additional information were identified as prevalent categories through inductive thematic analysis. Definition was a prevalent theme generated by the brainstorming task, although the sticky note format limited the depth of individual answers. The secret letter, which allowed children to share ideas privately, was found to be useful for generating more in-depth reflections on definitions and personal experiences. Stories and focus group discussions were found to be useful for generating varied context factors and change mechanisms, although stories also generated fictional elements and external solutions. Providing different qualitative methods and allowing children to choose how they prefer to communicate their thoughts proved important for collecting authentic, in-depth information.

Keywords

arts based methods community based research focus groups narrative methods in qualitative inquiry

Introduction

While health intervention development was theory-based for a long time, recent years have seen a shift to a new mindset focusing on participation (Palmer et al., 2019) as encouraged by funders and governments (Leask et al., 2019). The extent of stakeholder participation in research can be described as a continuum (Leask et al., 2019), which in the case of health care interventions ranges from basing interventions on the target population’s views to involving stakeholders in decision-making (O’Cathain et al., 2019). Any type of stakeholder collaboration has been suggested to identify potential obstacles early on (Hagen et al., 2012), tailor interventions to the target group’s needs (Milton et al., 2021), increase effectiveness of the development process (Leask et al., 2019), and support positive outcomes on community and individual levels (Singh et al., 2017). At the same time, the United Nations Convention on Children’s Rights (UNICEF, 1989) and the “new social studies of childhood” shifted children’s role from passive research subject to active stakeholders (Fargas-Malet et al., 2010). New guidelines and frameworks have been proposed for including children in research (Hawkins et al., 2017; National Institute for Health and Clinical Excellence, 2007). However, less than 1% of empirical adolescent research actually included children in any participatory or advisory roles and when they did it was mainly 12- to 18-year-olds (Sellars et al., 2021). Difficulties of including children in participatory research approaches, which actively involve stakeholder in all decisions (Montreuil et al., 2021), are well-documented, especially regarding children’s voices and power (Bradbury-Jones et al., 2018). However, a first step would be to reflect on appropriate participatory methods, which aim to engage stakeholders in a particular data collection process (Montreuil et al., 2021), but could be used on the full continuum of stakeholder collaboration from collecting feedback to placing all control in stakeholders’ hands (Leask et al., 2019).

Finding appropriate participatory methods to engage younger children and collect their perspectives while minimising bias poses substantial difficulties. Observations and measurements are ruled out as they frame children as the research object rather than active stakeholders (Barker & Weller, 2003). Additionally, children have difficulties with the abstract nature of questionnaires aiming to assess their true level of knowledge (Mellor & Moore, 2014). Traditional interview methods are prone to researcher inflicted bias due to power imbalances between child and adult researcher (Barker & Weller, 2003; Spyrou, 2011). Even for older age groups concerns have been raised about participants’ ability to share their actual feelings (Bahn & Barratt-Pugh, 2013). To adequately involve children, a safe environment and trusting relationship needs to be established and the researcher’s role needs to be shifted from leading the research to empowering the child and facilitating them to leading the research (Benninger & Savahl, 2017). More varied, open, experimental, and arts-based methods are suggested (Fails et al., 2012), allowing children to choose methods to introduce their lines of thoughts and lead the discussion (Fargas-Malet et al., 2010). Research efforts aiming to involve children as active stakeholders thus need to embrace qualitative methodology and go beyond traditional forms of interview and focus groups.

Methods suitable to inform early intervention development need to generate data to define how the problem occurs in children’s lives (i.e., definition) and clarify how to achieve successful outcomes (i.e., intervention strategies) (Hagen et al., 2012). Thus, in-depth stakeholder collaboration is advised to (a) define the problem, (b) understand which malleable context factors promote positive changes, and (c) identify change mechanisms (i.e., possible intervention strategies) (Wight et al., 2015). However, there is currently a lack of guidance on suitable participatory methods to collect data relevant to these specific information categories in early intervention development stages. Thus, researchers regularly describe their participatory development process as learning experience (Dykstra Steinbrenner et al., 2015), instead of being able to follow existing guidelines. To address this gap, the present study compared four qualitative methods on their suitability to engage children and elicit data relevant to information categories needed for early intervention development (i.e., definition of the problem, context factors and change mechanisms).

The Present Study

The present study was embedded in a wider research project with a focus on peer relationships in children aged 9 to 12. The long-term goal was to develop an intervention facilitating positive peer relationships in the context of primary-secondary school transition. In the first stage of this research, a qualitative research workshop was implemented in schools to understand children’s perspectives regarding peer relationships and school transition. The unique set-up of this study, which engaged a large sample (n = 506) of children in different qualitative methods on the same topic, allowed for a novel analysis and discussion of the nature of collected data. Specifically, this study aimed to assess the potential of four qualitative research methods - a brainstorming task, an individual letter, a story-based task, and a focus-group task – to engage children aged 9 to 12 and elicit information relevant for early intervention development. Methods were compared to address the research questions (a) how much detail and depth are generated for information categories relevant in intervention development (i.e., definition, context, change mechanism, experience) by each of the methods and (b) which advantages and challenges are posed by each method. Based on the theoretical guidelines presented above (Hagen et al., 2012; Wight et al., 2015), three information categories were of interest: definition aspects (i.e., core aspects of friendship), context factors (i.e., environmental aspects, circumstances or other people facilitating friendships), change mechanisms (i.e., strategies and sequences of action that lead to building and navigating friendships). As an additional category of interest, examples and experiences were added, as these were prevalent in the data and they are likely to describe how context factors and change mechanisms come into effect in the target group’s lives. This analysis aimed to guide researchers when choosing and implementing qualitative participatory methods with children in early stages of intervention development.

Study Design

Participants

In order to reach a large and diverse sample and allow all children to participate in this research, a two-hour research workshop was implemented in schools between October and December 2018. Ethical approval was obtained from the ethics committee at the University of Music and Performing Arts, Vienna (EK-Nr. 10/2018). To reach the target age group of 9- to 12-year-old children, fourth-year primary school and first-year secondary school classes were recruited via the Lower Austrian Department of Education, networking events, and social media. The research workshop was conducted in 54 classes from 29 schools across Lower Austria. In some schools, multiple classes of the same year (e.g., two first-year secondary school classes) participated. Schools were selected to represent diverse backgrounds, with urban and rural areas, large and very small schools, and different socio-economic backgrounds. The full sample comprised 916 students from 22% primary schools, 71% secondary schools (including comprehensive secondary schools and grammar schools), and 7% special needs schools. Due to this study’s focus on the detail and depth of collected information, it was deemed more appropriate to focus the analysis on a coherent subset of the sample to minimise age- or school-related biases and make the data more comparable. Thus, the presented analysis focused exclusively on data collected from 28 first-year classrooms of state-funded, comprehensive secondary schools. The included sample comprised of 506 students with a mean age of 10.59 (SD = .69) and a gender distribution of 50% boys and 45.8% girls (see Table 1).

Table 1.

Socio-Demographic Information of Included Sample.

Variable	N = 506	Percentage, %
School area
Urban area	228	45
Rural area	278	55
Gender
Female	232	45.8
Male	253	50
Missing	21	4.2
Age
9	1	1.7
10	249	49.2
11	181	35.7
12	54	10.7
Missing	21	4.2

Note. To make the data more comparable, only first-year classrooms of state-funded secondary schools were included in the sample. Information on age and gender were collected on individual questionnaires, which were not completed by all children, thus there is 4.2% of missing data in these categories.

The School-Based Research Workshop

The research workshop was developed by an international team of psychologists, psychiatrists, teachers, and arts practitioners and iteratively refined with stakeholders. This workshop employed participatory methods to work collaboratively with children as active stakeholders and establish a shared understanding of the importance and difficulties of peer relationships. Interactive activities such as brainstorming with sticky notes, elaboration with stimuli, cooperative storytelling, group discussions, and individual letters were adapted from the literature (Fails et al., 2012; Halskov & Dalsgård, 2006; Walsh et al., 2011) Creative methods foster a comfortable atmosphere, dissolve communication barriers, and elicit children’s authentic voices (Jackson Foster et al., 2018). Providing methods featuring different expression modes (e.g., verbal discussion, writing, drawing) was essential to give children control over their communication channel (Fargas-Malet et al., 2010). The workshop was pilot tested with two classes a month before the start of the study. Activities were adjusted to the research aims while being minimally distressing (e.g., talking about negative experiences). The workshop comprised seven research activities and additional game-based activities, used as transitions and activation (see Figure 1). To establish rapport and trust between researchers and children (Spyrou, 2011), an extended introduction with games and activities establishing the workshop’s aims (creating a “Happy Zone”) was incorporated. Throughout the implementation phase, coherence between research aims, implementation of activities (wording, data collection procedure), and quality of data was monitored and adjusted as needed (Morse et al., 2002).

Figure 1.

Flowchart workshop structure. Note. Activities marked with asterisk were included in the presented analysis.

Parents (or legal caregivers) received information sheets as well as child-friendly information sheets to be read to their child. If both the child and the parent were happy for the child to participate, parents were asked to provide written consent. At the beginning of the workshop, children were again informed about the aims and process of this research and were asked to provide additional verbal assent. Workshops were implemented by two workshop leaders (members of the research group) and up to four facilitators (students in psychology, music therapy, and pedagogy). To achieve maximum implementation fidelity, a written workshop manual was developed providing detailed activity descriptions and instructions (materials are available in a data repository on the open science framework platform OSF at https://osf.io/2u7r4/) and facilitators participated in training before and during the study period. This paper discusses four methods implemented in the workshop which were similar regarding research questions addressed and thus allow for a comparison of the collected data.

Brainstorming With Sticky Notes

Brainstorming activities are typically used in early development stages to collaboratively collect a variety of ideas (Fails et al., 2012). Open tasks were found to elicit more ideas, while written and verbal activities elicited more specific criteria (Sluis-Thiescheffer et al., 2007). Thus, two related but slightly different brainstorming activities were implemented. First, children were asked to think of different ways their peers or classmates could support them. This open research question was posed to the whole group and children were encouraged to discuss each other’s ideas (e.g., “Ok, you want to cheer up your friend, what could you do to cheer them up?”). Ideas were collected by the workshop leaders on a poster, and after ten minutes of discussion, children were handed sticky notes and pens to add further ideas themselves. Next, children received green and red sticky notes in small groups and were asked to collect positive and negative characteristics of peer relationships as a group and individually (green “What makes a friendship?”, red “What stops you being friends with someone?”). In this analysis, data of both brainstorming tasks (i.e., Peer Relationships, Helps & Stops Peer Relationships) was combined, since collected data was similar and generated themes overlapped. To make the data more comparable, this analysis focused on positive aspects of friendships only.

The Secret Letter

An individual “secret letter” activity was included to allow children to (a) anonymously express thoughts and (b) give more introvert children, who might prefer not to discuss personal experiences in a group setting, an outlet (Barker & Weller, 2003). At this point, children were reminded of the previous questions around positive and negative aspects of peer relationships. A deliberately open question “Is there anything else you want us to know regarding your peer experiences?” was posed, allowing children to structure their replies themselves (Bagnoli, 2009). To make this activity exciting, children were instructed to look for a quiet place in the classroom and let nobody else have a look at their secret letter. The finished letters were folded and returned to a crafted post box.

Finishing the Story

A set of cartoon pictures of children was presented, and a volunteer was asked to draw a card. After agreeing on a name for the fictional character, the workshop leader explained that this child was about to transition to secondary school and was quite anxious. Participants were now asked to brainstorm why this fictional character might feel nervous about their school transition. A scenario, which engaged participants and evoked comments (Bahn & Barratt-Pugh, 2013), was created aiming to instil a sense of ownership over the story in the children (Walsh et al., 2011) and provide a background for the following task. Children were then asked to gather in small groups and continue the character’s story. They were encouraged to explore how fears could be overcome, what could happen to change the situation and who could help the character. The use of a fictional character allowed children to distance themselves from this potentially emotional situation and explore and discuss “extreme” feelings of a fictional, very anxious child (Blythe & Wright, 2006).

Focus Group Collecting Advice

The data collection process was continuously monitored and research activities were adapted as needed (Morse et al., 2002). For reasons presented in the discussion section, the Finishing the Story task was adapted and turned into an open focus group, collecting advice for the fictional character. Out of the presented workshop data, 40% of workshops included the story task, while 60% of workshops included the adapted focus group task. To create a focus group setting, facilitators asked children to gather in small groups and jointly write a letter to the fictional character, giving advice regarding strategies to overcome fears and navigate the school transition. To facilitate discussion by shifting the focus from the child-researcher conversation to a conversation between the child and a character, using puppets has been suggested (Epstein et al., 2008). As puppets seemed too immature for the age group of children above the age of 10, they were asked to write a letter to a fictional character. Additionally, listing ideas and strategies can facilitate more elaborate discussion around strategies and possible intervention efforts (Colucci, 2007).

Data Analysis

Data were collected during the workshop on post-it notes, posters, and the blackboard. Additionally, workshop facilitators took notes on structured observation sheets. The data are available on the open science framework platform OSF at https://osf.io/avtw2/. While this paper will focus on research methods and discuss information categories, three separate reflexive thematic analyses or content analyses were conducted to analyse the data regarding (a) peer relationships (Krammer et al., 2023), (b) school transition (Stiehl et al., 2023) and (c) children’s use of online devices (Mittmann et al., 2022).

Analytic Approach

Guided by this paper’s aim of comparing four qualitative methods regarding their suitability to elicit information relevant for early intervention development, a bespoke thematic analysis was conducted, including elements from deductive framework approaches (Goldsmith, 2021) and inductive thematic analysis (Braun & Clarke, 2006). Data collected on hand-written notes was digitalised and electronic documents were analysed in the qualitative data analysis software QSR International's NVivo 12. Imported data took the form of clearly separated word chunks, short sentences, or short cohesive narration or exchanges. After the first author familiarised herself with the data, a framework of pre-defined themes was created, based on goals of early intervention development stages defined in the literature (i.e., definition, context, change mechanisms) (Hagen et al., 2012; Wight et al., 2015) and dominant themes generated from the data (i.e., examples and experience, additional information) (Goldsmith, 2021). Data was indexed using line-by-line coding on a semantic level to apply the framework to all the data (Goldsmith, 2021). Thus, each data point (word chunks, sentences) was linked (i.e., coded) to at least one of the pre-defined themes. Some data points might be linked to multiple pre-defined themes simultaneously, as information might refer to multiple themes (e.g., asking a teacher for help, who then encourages classmates to be kinder, would classify as change mechanism, but the teacher would simultaneously be a context factor). The first author created the framework and lead the indexing process but discussed each analysis step with the other authors.

The final framework included the themes (a) definition aspects, (b) context factors, and (c) change mechanisms, which were a-priori defined based on literature, and (d) examples and experience and (e) additional information, as generated from the data. Definition aspects (a) were defined as aspects important to build and maintain friendships, that is, pertaining to all accounts of values, benefits, expectations regarding friendships (e.g., “when they are there for you”). Context factors (b) were defined as external factors facilitating friendships, that is, environmental aspects, circumstances, or other people (e.g., school trips). Change mechanisms (c) were defined as strategies and sequences of action aiming to build and maintain friendships (e.g., daily phone calls to keep in touch). Examples and experiences (d) were defined as personal accounts using first person and mostly past tense to describe any situations relevant to friendship building and management (e.g., “I make friends when I join them in a game”). Additional information (e) was added to the framework to create a comprehensive picture of the generated data, including information which might not be relevant for a friendship analysis (e.g., feedback on the workshop).

To further describe the data, a data-driven inductive approach was used (Braun & Clarke, 2006) to create sub-themes for each of the pre-defined themes. The first author completed a preliminary analysis and then discussed coding decisions, themes, and definitions with the other authors. An iterative process of identifying, structuring, and discussing sub-themes with the team of authors, following collaborative analysis procedures (Braun & Clarke, 2019), was implemented until a coherent structure of consistent, clearly defined and mutually exclusive sub-themes was created (Braun & Clarke, 2006). These inductive sub-themes aimed to further describe data within each of the pre-defined themes (e.g., context factors were further differentiated into events and places, personal circumstances, assets, and characteristics of other people) and thus had to be mutually exclusive. Sub-themes were numbered consistently (e.g., A1) across all four qualitative methods to facilitate their comparison across methods. For definition of sub-themes, see analysis Tables 1-4 in the appendix, column “interpretation” of the codes and “examples”. In order to make comparisons across methods, Table 2 was created to summarise the data and variations in the pre-defined themes and inductively generated subthemes were examined (Goldsmith, 2021).

Table 2.

Overview of Themes Generated for all Four Methods.

Method	A) definition of friendship	B) Context factors	C) Change mechanisms	D) Examples/experience	E) Additional information
Brainstorming with sticky notes	A1) Characteristics of friendships (Getting along well)	B1) Events and places (School trips)	C1) General Behaviours (Being nice/helping)		E1) Other sources of happiness (cuddling with my cat/horse back riding)
	A2) Values and expected benefits of friendship (Being who you are)	B2) Personal circumstances (Living close to each other)	C2) Shared activities (cycling/ice cream)
		B2) Personal circumstances (Living close to each other)	C3) Identity aspects (Being honest/shy)
Secret letter	A2) Values and expected benefits of friendship (everyone should be respected by others)		C1) General Behaviours (strategies, e.g. First: approach them, second: Play or behaviours, e.g. say hello)	D1) Self-disclosure about feelings and own friendships (feelings and emotional experiences, e.g. I recently lost my best friend)	E2) Direct messages
					- To classmates (e.g. you are the best)
				D2) personal strategy use (I make friends when I start playing with them)	- To researchers (e.g. Children ages 10-15 are easily embarrassed)
	A3) Philosophical reflections and questions (It takes time to become friends)		C3) Identity aspects (always being happy)		E3) workshop feedback (What you are doing is great)
			C3) Identity aspects (always being happy)	D3) Self-Disclosure About worries and problems unrelated to friendships (stealing money)	E4) unrelated comments (How do you get white teeth – by eating apples)
Finishing the Story	A2) Values and expected benefits of friendship (The more friends you have, the safer you feel)	B1) events and Places (School trips, birthday parties)	C1) General behaviours and activities (cheering someone up/have fun and talk to them)	D4) personal experience of making new friends (I met someone through a Playstation game, I followed them, we became friends.)	E5) external solutions
	A4) Long-term outcomes (They stay friends forever)	B2) personal circumstance (They have known each other forever)	C2) Shared activities (Building a fort in the woods)		- Wishful thinking (A new girl joins class and defends him.)
	A5) Implicit expectations of why friendships form (Because new friend is the prettiest girl in class, everyone wants to be friends with her too)	B3) assets (PS4, new computer)	C3) Identity aspects (she is funny/good in sports)		- Fictional elements (He travels back in time to the summer break)
		B4) characteristics and behaviours of Other People (Parents, teachers, Old friends, classmates)	C4) Approach others (Asking do you want to be my friend)	D5) putting Oneself in Character’s situation (I would go crazy if I was bullied. I’d ask my dad for help.)	- Unrelated Solution (santa clause comes to school)
			C5) Social Support and help seeking (Talk to the teacher)		E6) Discussion between participants
			C6) cognitive emotion Regulation strategies (Tells himself “I’m the strongest”)		E6) Discussion between participants
Focus Group collecting Advice	A2) Values and expected benefits of friendship (You need to stick together, than things will change)	B1) events and Places (Making friends in fortnite)	C1) General Behaviours (Helping)	D2) personal Strategy use (I had three points written down: saying hello, asking about similarities, inviting to a play date. It does not always work 100%.)	E5) external solutions
			C3) Identity aspects (be brave)		E5) external solutions
	A3) philosophical reflections (The longer you are in the same class, the better you get along usually)		C4) Approach others (Join in with some children and stay with them)		- Wishful thinking (changing school)
			C5) Social Support and help seeking (Tell a teacher)		- Wishful thinking (changing school)
			C6) cognitive strategies (Tell yourself you are brave)	D4) personal experience of making new friends (I was also worried about the new school)	- Fictional elements (poison bread)
			C7) Avoiding others (Don’t speak to the really mean children, that’s too dangerous.)		- Fictional elements (poison bread)
	A5) Implicit expectations of why friendships form (Wearing football shoes, everyone will like him, because it looks cool)	B4) characteristics and behaviours of Other people (Parents, teachers, Old friends, classmates)	C8) confrontation (You fight back.)		E6) Discussion between participants

Epistemology and Researcher Positionality

The pre-defined research aims (as required for funding proposals) and predetermined methodological set-up of this research workshop did not allow children to set their own agenda. This is not fully in line with the specific participatory epistemology (Pincock & Jones, 2020) of research approaches aiming to empower and emancipate children (Kellett, 2010). However, to strike an appropriate balance between participatory research approaches and the important research questions under consideration here, the researchers aimed to reduce power imbalances between children and adults as much as possible and perceived participants as active contributors. A critical realist viewpoint was adopted by the researchers. Researchers believe that the presented findings relate to a reality of friendship problems, trajectories, and strategies in the context of school transition. At the same time, researchers are aware that the researchers themselves had a role in constructing this knowledge (Coyle, 2016; Maxwell, 2022). As this knowledge was socially constructed in the interactions between researchers and children, the findings cannot be understood as absolutely objective (Willig, 2008). Researchers were mindful about their role and actively reflected on their interactions with children and their inevitable influences in an understanding that it cannot be avoided that their own biography shaped the research process (Denzin & Lincoln, 2011).

All authors and workshop facilitators are white, middle-class researchers with higher education degrees. The authors’ backgrounds are in psychology, education, and psychiatry and all senior authors are experienced with qualitative research and intervention development. The first author, a female early career researcher with a psychology degree, was involved in workshop development and implementation, oversaw data collection, and conducted the data analysis. She has conducted a review of peer relationship interventions (Pollak et al., 2022) which has impacted deductively defined themes of the presented analysis. Inductively generated sub-themes are likely to have been influenced by the first author’s experience of collecting data through discussion and interaction with participants. To achieve reflexivity of the research process, the authors reflected on their own input to the data generation process and discussed analysis and interpretation with authors, who were not involved in the workshop development and delivery (second and fourth author), to achieve a balanced perspective on the data. However, the authors’ personal experience in implementing the described methods and their reflections on children’s behaviour and feedback to the methods, is considered a strength and adding to a comprehensive discussion of methodological aspects.

Results

While all of the pre-defined themes were generated for the story and focus group task, only four out of five themes were generated for the brainstorming and secret letter task. Regarding the inductively generated subthemes, there was some overlap between methods (see Table 2), emphasising similarities in generated data.

Brainstorming With Sticky Notes

Data collected on sticky notes mostly comprised of one to three keywords, mostly nouns or verbs. As the aim of this task was to collect ideas around the concept of friendship, most data points concerned behaviours, activities, and states that participants associated with an established friendship (see Table 2). The most common theme was (C) change mechanisms aiming to establish or maintain friendships. Three distinct sub-themes were generated including (C1) general behaviours (e.g., “being nice”), (C2) shared activities (e.g., “cycling”), and (C3) identity aspects (e.g., “being shy”). However, they could equally be described as (A) definition aspects of friendships (i.e., how does a friend behave towards you), as the keyword format of the data makes it impossible to know whether data points were concerning behaviours to make new friends (i.e., change mechanisms) or behaviours characterising established friendships (i.e., definition aspects). For ease of data presentation, they will be discussed as change mechanisms throughout. These three sub-themes were most common within brainstorming data. The theme of (A) definition aspects was less common and was further differentiated into sub-themes (A1) characteristics of friendships (e.g., “getting along well”) and (A2) values and expected benefits of friendship (e.g., “be who you are – there is no point in pretending”). Additionally, some (B) context factors were mentioned, which contributed to the establishment and/or maintenance of friendships. Participants named (B1) events, such as school trips, and (B2) personal circumstances, such as having money or living close to each other. The theme of (D) experience was not linked to any data collected with brainstorming. However, for (E) additional information, the sub-theme (E1) other sources of happiness was generated (e.g., “cuddling with the cat”).

The Secret Letter

Data collected with the Secret Letter ranged from few keywords or short sentences to longer accounts of specific thoughts or experiences. Due to this task’s deliberately open instruction to write down “anything else you would like to tell us about your friendships”, data covered strategies to make new friends, personal experiences of friendship, and definitions of friendship (see Table 2). The theme of (A) definition aspects was further differentiated into (A2) values and expected benefits of friendship and (A3) philosophical reflections and questions, such as “it takes time to make friends”. The theme of (C) change mechanisms included sub-themes of (C1) general behaviours and (C3) identity aspects, similar to data collected in the brainstorming task. Experiences (D) were further differentiated into (D1) self-disclosure about own friendships (i.e., feelings, worries and difficult situations inflicted by friends), (D2) personal strategy use (i.e. participants reporting how they make friends), and (D3) self-disclosure about worries and problems unrelated to friendships (e.g. stealing money). Additionally, this task also elicited (E) additional information in the form of (E2) messages to classmates (e.g., “you are the best”) and the researchers (e.g., “secret: Children ages 10-15 are easily embarrassed”) or (E3) feedback regarding the workshop and its activities. Due to the task’s open format, another sub-theme of (E4) unrelated comments was identified (e.g., “how do you get white teeth – by eating apples”). The open nature of this task also led to a lack of specific context factors (B).

Finishing the Story

Data collected in the Finishing the Story task took the form of short, full sentences describing how the fictional character navigates the school transition and tries to make new friends. All pre-defined themes were present, and various sub-themes were created, mirroring the variety of data collected with this task (see Table 2). Definition aspects (A) were differentiated into sub-themes (A2) values and expected benefits of friendship, (A4) long-term outcomes, and (A5) implicit expectations of why friendships form. The theme of (B) context factors was multi-facetted, including sub-themes of (B1) events and places, (B2) personal circumstances, (B3) assets (e.g., PS4), and (B4) characteristics and behaviours of other people (e.g., parents, teachers, old friends, classmates). The theme of (C) change mechanisms was more common than in other tasks and more sub-themes were generated. Participants mentioned (C1) general behaviours, (C2) shared activities, and (C3) identity factors like they did in other tasks, but they also mentioned (C4) approaching others (e.g., “he asked if they want to be friends with him”), (C5) social support and help seeking behaviours (e.g., “talking to the teacher”), and (C6) cognitive (emotion regulation) strategies (e.g., “he tells himself ‘I’m the strongest’“). The theme of (D) experience was less common but took two forms; (D4) personal experience of making new friends and (D2) participants putting themselves in the character’s situation (e.g., “I’d go crazy, I’d ask my dad for help”).

For additional information (E), two sub-themes were created. External solutions (E1), which might be seen as bridging context factors and strategies in their focus on external factors as solutions to the emotional or friendship problems. Accounts categorised as external solutions could be described as wishful thinking or easy solutions (e.g., lost friend returns to school, bullies are bullied themselves and apologise), and sometimes included fictional elements (e.g., lottery win, ninja fighters, time travel). Some events and situations were completely unrelated to friendships (e.g., Santa Clause coming to school), but made the character happy thus lifting stress and problems but are unlikely to resolve the friendship problems in the long-run. Although these accounts are solutions to the character’s problems and provide insight into children’s perceptions, none of these events can be strategically planned or controlled and were thus not categorised as change mechanisms relevant for intervention planning. The task also elicited some form of (E2) discussion between participants regarding their preferred strategies and their beliefs.

Focus Group Collecting Advice

As this task aimed at eliciting group discussion on useful advice for a fictional character, collected data took the form of a list of suggestions presented in full sentences. As this task shared the same background story with the Finish the Story task, identified themes were similar, although differing in their main focal points (see Table 2).

The theme of (A) definition aspects concerned (A2) values and expected benefits of friendships (A3) philosophical reflections, and (A5) implicit expectations about why friendships form. Context factors (B) were differentiated into the very common sub-theme (B4) characteristics and behaviours of other people and the less common theme of (B1) events and places to make friends. The theme of (C) change mechanisms was the most multi-facetted in this task. Sub-themes included (C1) general behaviours, (C2) identity factors, (C4) approaching others, (C5) social support and help seeking, (C6) cognitive strategies, (C7) avoiding others, and (C8) confrontation. Accounts of (D) experience concerned (D2) personal strategy use and (D4) personal experience of making new friends. Within the theme of (E) additional information, (E6) discussion was a very common sub-theme, as the task allowed for more follow-up questions by researchers. External solutions (E5), including easy solutions/wishful thinking and fictional elements were less common.

Discussion

This paper aimed to compare four qualitative methods regarding their suitability to engage children aged 9 to 12 and produce specific, pre-defined information relevant for early intervention development. Although all four methods were found to elicit data relevant to (almost) all pre-defined information categories, substantial differences emerged regarding the depth of the data as evident in data-driven sub-themes. In order to contrast results from each method, the following discussion will focus on each of the pre-defined information categories (see Table 3 for an overview of advantages and disadvantages of methods).

Table 3.

Advantages and Disadvantages of Methods.

Method	Aims	Data format	Useful for	Advantages	Challenges
Brainstorming with Sticky notes	Collecting as many ideas and thoughts as possible	Each reply usually consists of 1–3 short keywords	Definition aspects, context if explicitly stated in instructions	Generating many ideas, enabling all children to participate without speaking out in front of a group	Eliciting short keywords, hardly any context; especially younger children might be put off by writing tasks
Secret Letter - “Is there anything else you’d like to tell us about this topic?”	Providing space to elaborate on topics; useful to express ideas, opinions or experiences participants might not want to share with the whole group	Eliciting a wide range of different replies about experience, thoughts, worries, questions, statements	Personal experience and examples, any data that is explicitly asked for in instructions	Individual writing allows participants to focus on what is most important to them; secrecy ensures participants can express thoughts freely	Young children struggle with open questions and might be put off by writing tasks
Finishing a Story	Proving context in form of a specific situation promotes discussion of specific questions; facilitates children's imagination of abstract scenarios; allows to discuss “extreme” scenarios without personal background	Coherent stories about a given scenario featuring strategies and context factors	A variety of context factors and change mechanisms, implicit expectations, and definitions of the problem	Discussion between participants about possible strategies and influential context factors is elicited, researcher is present to ask further questions and note down ideas	Stories elicit fantastical elements that deviate from children’s lived reality; by agreeing on one storyline, varying opinions might get lost
Focus Group - Giving Advice to a fictional character	Creating a background for an open group discussion on a specific question; discussing a fictional character allows participants to distance themselves	List of different strategies and experiences	Asking follow-up questions on the real-life importance of context factors and change mechanisms	Different strategies and influential context factors are collected; researcher is present to ask further questions and note down ideas	Some advice/comments are very general, thus researchers need to be present to ask questions and stir participant’s discussion in desired direction

Definition

Discussing definition aspects is crucial in early intervention development to understand how the target group identifies the problem in their lives (Hagen et al., 2012). The sub-theme (A2) “values and expected benefits”, referring the importance of friendships, was generated by all four methods, suggesting comments on why and how a topic is relevant can be elicited with a variety of methods. Apart from this, data on definition aspects differed widely.

The brainstorming task was intended to elicit (A1) “friendship characteristics” and mostly succeeded in doing so. However, the keyword-style of answers (due to sticky-note format) made it difficult to clearly differentiate between definitions of established friendships and strategies (i.e., change mechanisms) to build new friends. This link between definitions and strategies, however, is likely to hold true for other topics. Asking children about health problems, for example, is likely to produce characteristics of feeling unwell and behaviours to become better (e.g., staying in bed, drinking tea). However, this difference might not be profound in children’s minds. Children seem more likely to describe actions and feelings they relate to a topic, rather than discussing abstract definitions. Indeed, abilities linked to abstract thinking, such as stimulus-independent thoughts, first develop around the age of 11–13 (Dumontheil, 2014).

The secret letter and focus group provided more room for lengthier (A3) “philosophical reflections”, informing definitions by reflecting broadly on a topic and its context. At the same time, the focus group and the story elicited information about children’s “implicit expectations” (A5) by focusing on a fictional character. Children talked about why friendships evolve, which equally expresses what is important in friendships. This process of using concrete examples and stimuli to elicit and discuss implicit believes and create a joint understanding of a certain concept was previously used successfully with visual methods such as photography (Alexander et al., 2014).

Thus, researchers aiming to create joint definitions need to be aware of limitations of young children’s abstract thinking abilities and anticipate discussing more concrete actions instead. Specific stimuli, such as a story, help eliciting associations and implicit expectations. If researchers aim to elicit individual reflections, the secret letter seems ideally suited, while a focus group task would allow for reflections and discussion between participants. The differences in definition aspects elicited by various methods (i.e., superficial characteristics vs. implicit beliefs) suggest that a combination of methods is needed to gather a rich, in-depth understanding of children’s perceptions.

Context

Context factors are relevant for intervention development purposes as researchers need to identify which factors are malleable and provide options to bring about change (Wight et al., 2015). While it might be expected that context factors would always be addressed, this analysis made evident that methods for children need to explicitly ask for context factors and provide enough room for their discussion.

Context was not a common theme generated for the brainstorming task or the secret letter. Since both tasks posed questions on friendship characteristics, children focused on this and did not reflect on context. Indeed, children’s ability to explicitly remember factors (i.e., explicitly state factors) is still developing throughout adolescence, even though they perform well on implicit memory tasks (i.e., knowing they have encountered something before) (Billingsley et al., 2002; Dumontheil, 2014). Thus, there seems to be a need to explicitly address certain categories of interest in brainstorming instructions. However, since brainstorming categories are typically defined a-priori (Fails et al., 2012), this task might be well suited to collect different types of context factors, if explicitly stated in instructions.

Both the story-based task and the focus group produced a range of different context factors, including (B1) “events”, (B3) “assets”, and (B4) “characteristics of other people”. Naturally, telling a story requires a certain amount of context, such as places, events, and supporting characters. However, stories were also found to elicit a certain level of fictionality. When asked to expand on the real-life implications of certain context factors (e.g., other people) during the focus group, children would sometimes contradict earlier statements or say these factors are not important in their own lives. Increased discussion during the focus group facilitated a more nuanced picture of children’s perceptions regarding context factors. However, the focus group did not elicit as many or as detailed accounts of context factors as did the story task.

Researchers aiming to understand important and malleable context factors, need to explicitly ask for context factors in open brainstorming or writing tasks. Stories, however, seem ideal to elicit rich data about children’s perceptions of important context, without explicitly prompting (and potentially priming) certain factors. In order to understand their real-life importance for intervention efforts, combining a story-based task with open discussion seems useful.

Change Mechanisms

Change mechanisms provide information on how issues can be resolved from the target groups’ perspective and ideally translate into intervention strategies. The themes (C1) “general behaviours”, (C2) “activities”, and (C3) “identity factors” were generated from almost all methods. This illustrates that children of this age group are very focused on basic behavioural strategies (Stiehl et al., 2023). The use of cognitive regulation strategies is only fully developed between the ages of 9–11 (Pons et al., 2004) and metacognitive abilities to plan, execute, and reflect on strategies develop between the age of 8 and 10 (Veenman et al., 2004). Nonetheless, the story-based task and the focus group elicited more specific strategies (e.g., cognitive strategies, help seeking, confrontation). Specific scenarios and rich context make it easier for children to imagine themselves in a situation (Chiasson & Gutwin, 2005) and discuss abstract topics, such as emotions (Bahman & Maffini, 2008). Combining a story-based task with a focus group seems promising to discuss change mechanisms, since children’s metacognitive capacities are facilitated through discussions with adults (Garrison & Akyol, 2015; Lewis, 2017; Robson, 2016).

However, stories and focus groups also generated (E5) “external solutions” (i.e., external events and external people’s behaviours), which cannot be targeted by an intervention. Storytelling seemed to prompt fictional storylines, similar to the ones on TV or in books, focusing on plot twists instead of realistic strategies to overcome real-life issues. While this illustrates children’s enjoyment of creating a story (Fails et al., 2014), it emphasises the importance of linking stories with discussion about strategies they use themselves or perceive as realistic. It became clear during the focus group that children would sometimes suggest strategies for the fictional character but admit to not finding them helpful themselves. A focus group also allows for follow-up questions on personality or previous experience, allowing researchers to explore different perspectives and influencing factors. However, for all group tasks, it has to be kept in mind, that discussed ideas are products of the group dynamic, which should be considered in the analysis (Colucci, 2007).

For researchers aiming to discuss change mechanisms, a story-task providing situational context and a fictional character seems ideal. Additionally, focus groups allow researchers to support children’s reflective abilities and explore real-life strategies beyond fictional story elements and “external solutions”.

Experience and Examples

Exploring deeper layers of reflection about experiences sometimes is the primary aim of participatory studies. However this process might make children feel vulnerable, especially when they are in their peer groups (Pincock & Jones, 2020). Thus, the selection process of qualitative methods to discuss (sensitive) intervention topics with children, should carefully consider to what degree the discussion of personal experiences is intended and which safeguarding techniques can be implemented.

The brainstorming task did not produce any personal experiences or elaborate examples since there was no direct prompt. This makes it a safe task when avoiding personal accounts on sensitive topics. The secret letter elicited many personal accounts, differing in detail and volume as found for similar experience-focused writing tasks (e.g., dairies) (Barker & Weller, 2003). The opportunity to individually and anonymously express thoughts, which children might not want to share with a group or with the researchers directly (Barker & Weller, 2003), presents a chance to collect rich and impactful data. In this study for example, it was vital to include individual tasks, as it is difficult to express honest opinions about peer relationships or negative peer experiences within one’s peer group. Additionally, children feeling less comfortable to participate in peer group activities were the potential target group of the intervention, thus their voices were in some ways the most important. Additionally, while leaving children alone with their reflections might be a safeguarding issue with some topics, collecting written notes on children’s well-being can be used for safeguarding purposes. While the nature of a secret note does not allow researchers to directly identify individual children, it allows children to mention problems, which can then be dealt with on a group level. Especially in the school context, researchers are even obliged to share any concerning messages with teachers, who know their students well and can decide to follow-up through classroom discussion or individual discussion with students who might be affected.

The story-based task elicited more specific accounts, such as (D4) “experience of making new friends” or examples of (D5) “putting oneself in the character’s situation”. As found when offering children to speak through puppets (Epstein et al., 2008), allowing children to express themselves through a fictional character, elicited specific experiences without having to speak in the first person. Thus, using game-based story tasks has been suggested previously if recounting personal stories might be too traumatic (Fargas-Malet et al., 2010). Additionally, a fictional character can be described with specific traits and problems, thus engaging participants in a discussion of possible solutions for a very specific target group (Blythe & Wright, 2006). Contrary, the focus group produced mostly personal information on (D4) “personal experience” and (D2) “personal strategy use”. This is likely to be an effect of the researcher’s presence and follow-up questions, as children are more likely to tell their own stories, when encouraged or prompted by open-ended questions and question-requests (Ponizovsky-Bergelson et al., 2019). However, the researcher’s presence allows for guidance to steer away from sensitive issues and providing active support when needed.

If researchers aim to collect personal experiences, individual writing tasks are vital to include the voices of shy or withdrawn children, which might even be the target group of some interventions. For sensitive topics, a fictional character is useful to allow children to distance themselves while discussing examples through the character.

Implementation Factors and Limitations of This Research

This analysis aimed to assess the suitability of four qualitative methods to elicit information relevant for early intervention development. The presented methods, however, were deliberately chosen and adapted in an intervention development project with the purpose of generating data in pre-defined information categories. Thus, the authors had to continuously reflect on the potential impact of this study design on presented findings. To minimise potential bias and maximise transparency, credibility, reflexivity, and transferability (Williams et al., 2020), further reflections on the findings and interlinked implementation factors will be discussed.

Conducting a research workshop in schools was considered a strength of this research as it allowed children from all socioeconomic backgrounds to participate. However, research spaces evoke certain communication cultures (Barker & Weller, 2003), which was noticeable in the school context. Children struggled with open tasks as they seemed used to very precise instructions and are rarely asked to provide unprompted thoughts and reflections in written form. Therefore, the open question for the secret letter often had to be followed-up by more specific instructions and data regularly mirrored data of previous tasks. Children also seemed eager to provide “correct” answers, although follow-up questions in the focus group often revealed that they had not experienced suggested strategies as helpful. As emphasised in literature on research with children, expectation bias, social desirability, and acquiescence bias need to be considered (Fargas-Malet et al., 2010). This research workshop was started by stressing that the children themselves are the experts and efforts were made to ask follow-up questions about children’s real experiences. Still, this bias seemed to persist. However, a change in children’s behaviour and outspokenness with their “group facilitator” was noticed towards the end of the workshop, indicating a prolonged period of rapport building with later data collection might increase authenticity of results and minimise biases (Spyrou, 2011).

Additionally, it proved important to let children choose their communication outlet (Fargas-Malet et al., 2010). While individual writing tasks allow shy children to express thoughts privately, some children struggled with language barriers and spelling insecurities and preferred researchers to write for them. Thus, group discussion was favoured by many children and the “individual” brainstorming resembled a group task. To ensure children were comfortable voicing their opinions, they were allowed to choose their peer group, as a trusted peer group facilitates children’s outspokenness (Pincock & Jones, 2020). However, ideas and opinions elicited by group tasks – which the brainstorming, the story and the focus group were – are products of group dynamics rather than reflections of individual opinions (Colucci, 2007). In this study, this became evident as initial responses were regularly priming following responses. In the group brainstorming, the first answer provided by a child usually set the tone for the following answers by classmates. Some priming effects were even found across tasks. Some sub-themes kept reappearing in all methods and while these themes might just reflect the most prominent thoughts, patterns were likely influenced by priming effects, order of tasks, and wording. Thus, the order of activities is vital. While the secret letter generated more experiences than the previous tasks (brainstorming), it also generated a lot of repetitions of general behaviours which were the dominant data category of the previous task. While switching the order of tasks is sometimes a useful way around this issue, it might as well be beneficial to wisely plan for priming effects when developing a research study. Using a secret letter task after a focus group, for example, could allow children to voice any disagreements with the group consensus and offer researchers a chance to compare group norms and individual thoughts.

As these methods were chosen and implemented with adult perspectives of what constitutes “child-friendly”, researchers continuously reflected upon methods and adapted them to avoid restricting children’s answers (Barker & Weller, 2003) – either due to implementation issues (e.g., written vs. verbal) or the nature of the question (e.g., leading or too abstract questions). Researchers tried to reduce power balances by reminding children they are the experts and encouraging them to lead the direction of discussion. However, all methods were presented within a frame of research questions, thus constituting a participatory data collection approach. Presented methods can, however, be used on the continuum of participatory research, depending on the desired degree of participation. Implementation can be adapted to increase children’s power by including them in decisions about research questions (e.g., not pre-defining the story background but letting children identify difficult situations themselves) and analysis (e.g., letting children order or rank the most important aspects generated in the brainstorming).

For future research, it would be interesting to control for child variables (e.g., gender or trait introversion) and analyse if certain methods are particularly suitable or engaging for a subgroup of participants. In the presented research, it was neither possible to control for a range of socio-demographic or psychological variables (due to the classroom format), nor was it intended as the workshop aimed to gather input from a group of diverse children. While projects on the development of universal preventive approaches will likely work with similarly diverse groups, intervention development for a particular target group would likely benefit from studies discussing the use of qualitative methods in particular subgroups (e.g., by socio-economic background or history of problem behaviours).

Lastly, it should be noted that transferability of presented results concerning the nature of data collected with specific methods is likely to be limited to similar intervention development efforts. Thus, this analysis should be understood as guide for other intervention developers rather than a generalised assessment of qualitative methods’ characteristics, as qualitative methods such as focus groups or brainstorming can take various forms and be adapted for a multitude of purposes.

Conclusion

This paper aimed to discuss and compare four qualitative methods regarding their suitability to engage children and collect relevant information in early intervention development stages. A unique study design with a large sample of children involved in four participatory data collection methods on similar research questions, allowed for an innovative comparison of collected data. The presented analysis aims to guide researchers and practitioners on how to choose methods appropriate for child stakeholders and intervention development aims. For creating joint definitions with children, a brainstorming task supports rapid generation of general associations, while an individual writing activity provides room for lengthy reflections. A context-specific story task facilitates children’s imagination and generates implicit expectations. When aiming to explore context factors, this should be explicitly stated in the instructions, as brainstorming and writing tasks are otherwise unlikely to address context. Although stories produce a range of context factors, pairing them with focus group discussions helps to clarify which factors children actually perceive as important in their lives. Similarly, for change mechanisms, stories help children to immerse themselves in hypothetical scenarios and generate a range of answers. Linking stories with focus group discussion is however beneficial as stories tend to facilitate fantastical elements or external solutions. As discussing personal experience might be sensitive, individual writing tasks are ideal to privately express thoughts. Stories can be used to allow children to distance themselves or explore difficult situations by speaking about characters. For implementation, implications of the setting and priming effects between methods need to be considered. When increasing children’s involvement in the analysis and decision-making process, these participatory data collection methods are adaptable for full participatory research projects to engage children while simultaneously generating relevant information.

Supplemental Material

Supplemental Material - Engaging Children in Intervention Development – A Comparison of Four Qualitative Methods and Their Suitability to Elicit Information Relevant for Early Intervention Development

Supplemental Material for Engaging Children in Intervention Development – A Comparison of Four Qualitative Methods and Their Suitability to Elicit Information Relevant for Early Intervention Development by Isabella Pollak, Katharina A.M. Stiehl, Beate Schrank, James Birchwood, Ina Krammer, Marija Mitic, Jack C. Rogers, and Kate A. Woodcock in International Journal of Qualitative Methods

Footnotes

Acknowledgements

This study was conceived as part of the interdisciplinary D.O.T. (Die Offene Tür - The Open Door) research project. We want to acknowledge Adam Barnard (playwright and theatre director) and João Dias (computer scientist) for their contribution as co-investigators throughout the D.O.T. project.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was part of the D.O.T. (Die Offene Tür - The Open Door) research project, which was funded by Ludwig Boltzmann Society Open Innovation for Science, Karl Landsteiner University of Health Sciences, and Lower Austrian Research and Education Association (NFB). The D.O.T. project grew during a sandpit event organised by the Open Innovation Center of Ludwig Boltzmann Society and facilitated by Know Innovation. The authors would like to thank these organisations for their role in the formation of the research group.

Ethical Statement

ORCID iDs

Isabella Pollak

Katharina A.M. Stiehl

Ina Krammer

Marija Mitic

Kate A. Woodcock

Supplemental Material

Supplemental material for this article is available online.

References

Alexander

S. A.

Frohlich

K. L.

Fusco

(2014). Problematizing “play-for-health” discourses through children’s photo-elicited narratives. Qualitative Health Research, 24(10), 1329–1341. https://doi.org/10.1177/1049732314546753

Bagnoli

(2009). Beyond the standard interview: The use of graphic elicitation and arts-based methods. Qualitative Research, 9(5), 547–570. https://doi.org/10.1177/1468794109343625

Bahman

Maffini

(2008). Developing children’s emotional intelligence. Bloomsbury Publishing.

Bahn

Barratt-Pugh

(2013). Getting reticent young male participants to talk: Using artefact-mediated interviews to promote discursive interaction. Qualitative Social Work, 12(2), 186–199. https://doi.org/10.1177/1473325011420501

Barker

Weller

(2003). Is it fun? Developing children centred research methods. International Journal of Sociology and Social Policy, 23(1), 33–58. https://doi.org/10.1108/01443330310790435

Benninger

Savahl

(2017). The children’s Delphi: Considerations for developing a programme for promoting children’s self-concept and well-being. Child and Family Social Work, 22(2), 1094–1103. https://doi.org/10.1111/cfs.12329

Billingsley

R. L.

Smith

M. L.

McAndrews

M. P.

(2002). Developmental patterns in priming and familiarity in explicit recollection. Journal of Experimental Child Psychology, 82(3), 251–277. https://doi.org/10.1016/S0022-0965(02)00007-3

Blythe

M. A.

Wright

P. C.

(2006). Pastiche scenarios: Fiction as a resource for user centred design. Interacting with Computers, 18(5), 1139–1164. https://doi.org/10.1016/j.intcom.2006.02.001

Bradbury-Jones

Isham

Taylor

(2018). The complexities and contradictions in participatory research with vulnerable children and young people: A qualitative systematic review. Social Science and Medicine, 215(August), 80–91. https://doi.org/10.1016/j.socscimed.2018.08.038

10.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

11.

Braun

Clarke

(2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806

12.

Chiasson

Gutwin

(2005). Design principles for children’ s technology. Interfaces, 7(28), 1–9.

13.

Colucci

(2007). “Focus groups can be fun”: The use of activity-oriented questions in focus group discussions. Qualitative Health Research, 17(10), 1422–1433. https://doi.org/10.1177/1049732307308129

14.

Coyle

(2016). Introduction to qualitative psychological research. In Lyons

Coyle

(Eds.), Analysing qualitative data in psychology (2nd ed.). Sage Publications.

15.

Denzin

N. K.

Lincoln

Y. S.

(Eds.). (2011). The Sage handbook of qualitative research. Sage.

16.

Dumontheil

(2014). Development of abstract thinking during childhood and adolescence: The role of rostrolateral prefrontal cortex. Developmental Cognitive Neuroscience, 10, 57–76. https://doi.org/10.1016/j.dcn.2014.07.009

17.

Dykstra Steinbrenner

J. R.

Watson

L. R.

Boyd

B. A.

Wilson

K. P.

Crais

E. R.

Baranek

G. T.

Flippin

Flagler

(2015). Developing feasible and effective school-based interventions for children with ASD: A case study of the iterative development process. Journal of Early Intervention, 37(1), 23–43. https://doi.org/10.1177/1053815115588827

18.

Epstein

Stevens

McKeever

Baruchel

Jones

(2008). Using puppetry to elicit children’s talk for research: Feature. Nursing Inquiry, 15(1), 49–56. https://doi.org/10.1111/j.1440-1800.2008.00395.x

19.

Fails

J. A.

Druin

Guha

M. L.

(2014). Interactive storytelling: Interacting with people, environment, and technology. International Journal of Arts and Technology, 7(1), 112–124. https://doi.org/10.1504/IJART.2014.058946

20.

Fails

J. A.

Guha

M. L.

Druin

(2012). Methods and techniques for involving children in the design of new technology for children. Foundations and Trends in Human-Computer Interaction, 6(2), 85–166. https://doi.org/10.1561/1100000018

21.

Fargas-Malet

McSherry

Larkin

Robinson

(2010). Research with children: Methodological issues and innovative techniques. Journal of Early Childhood Research, 8(2), 175–192. https://doi.org/10.1177/1476718X09345412

22.

Garrison

D. R.

Akyol

(2015). Toward the development of a metacognition construct for communities of inquiry. Internet and Higher Education, 24, 66–71. https://doi.org/10.1016/j.iheduc.2014.10.001

23.

Goldsmith

L. J.

(2021). Using framework analysis in applied qualitative research. Qualitative Report, 26(6), 2061–2076. https://doi.org/10.46743/2160-3715/2021.5011

24.

Hagen

Collins

Metcalf

Nicholas

Rahilly

Swainston

(2012). Participatory design of evidence-based online youth mental health promotion, intervention and treatment. In A young and well cooperative research Centre innovative methodologies guide. http://www.academia.edu/download/29275573/Hagen_PD_Guide_12.pdf

25.

Halskov

Dalsgård

(2006). Inspiration card workshops. In Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS, 2006, 2–11. https://doi.org/10.1145/1142405.1142409

26.

Hawkins

Madden

Fletcher

Midgley

Grant

Cox

Moore

Campbell

Murphy

Bonell

White

(2017). Development of a framework for the co-production and prototyping of public health interventions. BMC Public Health, 17(1), 1–11. https://doi.org/10.1186/s12889-017-4695-8

27.

Jackson Foster

L. J.

Deafenbaugh

Miller

(2018). Group metaphor map making: Application to integrated arts-based focus groups. Qualitative Social Work, 17(2), 305–322. https://doi.org/10.1177/1473325016667475

28.

Kellett

(2010). Small shoes, big steps! Empowering children as active researchers. American Journal of Community Psychology, 46(1), 195–203. https://doi.org/10.1007/s10464-010-9324-y

29.

Krammer

Schrank

Pollak

Stiehl

K. A. M.

Nater

U. M.

Woodcock

K. A.

(2023). Early adolescents’ perspectives on factors that facilitate and hinder friendship development with peers at the time of school transition. Journal of School Psychology, 98(March), 113–132. https://doi.org/10.1016/j.jsp.2023.03.001

30.

Leask

Sandlund

Skelton

Altenburg

Cardon

Chin A Paw

De Bourdeaudhuij

V. M.

Chastin

(2019). Principles and recommendations for the application and reporting of participatory methodologies in the development and evaluation of public health interventions. Research Involvement and Engagement, 5(2), 1–16. https://link.springer.com/epdf/10.1186/s40900-018-0136-9?author_access_token=0ZGYas7wm5JAnAdLOPOB3W_BpE1tBhCbnbw3BuzI2RNmxm3lxZTVih0kzWDJAzV7mP0CKPn42bLwVCrmADpnsHIcNxWkNsUc8rsTIBNSY29vQpEqKnU6Ov2pQZ0YCcriU7YyTx1ikt95DqmiFlN_zw%3D%3D

31.

Lewis

(2017). Supporting the development of young children’s metacognition through the use of video-stimulated reflective dialogue. Early Child Development and Care, 0(0), 1–17. https://doi.org/10.1080/03004430.2017.1417273

32.

Maxwell

J. A.

(2022). Critical realism as a stance for designing qualitative research. The Sage Handbook of Qualitative Research Design, 142–154. https://doi.org/10.4135/9781529770278.n10

33.

Mellor

Moore

K. A.

(2014). The use of likert scales with children. Journal of Pediatric Psychology, 39(3), 369–379. https://doi.org/10.1093/jpepsy/jst079

34.

Milton

A. C.

Stewart

Ospina-Pinillos

Davenport

Hickie

I. B.

(2021). Participatory design of an activities-based collective mentoring program in after-school care settings: Connect, promote, and protect program. JMIR Pediatrics and Parenting, 4(2). https://doi.org/10.2196/22822

35.

Mittmann

Woodcock

Dörfler

Krammer

Pollak

Schrank

(2022). “TikTok is my life and snapchat is my ventricle”: A mixed-methods study on the role of online communication tools for friendships in early adolescents. In Journal of Early Adolescence, 42(Issue 2). https://doi.org/10.1177/02724316211020368

36.

Montreuil

Bogossian

Laberge-Perrault

Racine

(2021). A review of approaches, strategies and ethical considerations in participatory research with children. International Journal of Qualitative Methods, 20, 1–15. https://doi.org/10.1177/1609406920987962

37.

Morse

J. M.

Barrett

Mayan

Olson

Spiers

(2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 13–22. https://doi.org/10.1177/160940690200100202

38.

National Institute for Health and Care Excellence (NICE) . (2007). Behaviour change: The principles for effective interventions. National Institute for Health and Care Excellence, October 2007. http://www.nice.org.uk/nicemedia/live/11868/37987/37987.pdf

39.

O’Cathain

Croot

Sworn

Duncan

Rousseau

Turner

Yardley

Hoddinott

(2019). Taxonomy of approaches to developing interventions to improve health: A systematic methods overview. Pilot and Feasibility Studies, 5(1), 1–27. https://doi.org/10.1186/s40814-019-0425-6

40.

Palmer

V. J.

Weavell

Callander

Piper

Richard

Maher

Boyd

Herrman

Furler

Gunn

Iedema

Robert

(2019). The Participatory Zeitgeist: An explanatory theoretical model of change in an era of coproduction and codesign in healthcare improvement. Medical Humanities, 45(3), 247–257. https://doi.org/10.1136/medhum-2017-011398

41.

Pincock

Jones

(2020). Challenging power dynamics and eliciting marginalized adolescent voices through qualitative methods. International Journal of Qualitative Methods, 19, 1–11. https://doi.org/10.1177/1609406920958895

42.

Pollak

Mitic

Birchwood

Dörfler

Krammer

Rogers

J. C.

Schek

E. J.

Schrank

Stiehl

K. A. M.

Woodcock

K. A.

(2022). A systematic review of intervention programs promoting peer relationships among children and adolescents: Methods and targets used in effective programs. Adolescent Research Review, 8(3), 297–321. https://doi.org/10.1007/s40894-022-00195-4

43.

Ponizovsky-Bergelson

Dayan

Wahle

Roer-Strier

(2019). A qualitative interview with young children: What encourages or inhibits young children’s participation? International Journal of Qualitative Methods, 18, 1–9. https://doi.org/10.1177/1609406919840516

44.

Pons

Harris

P. L.

de Rosnay

(2004). Emotion comprehension between 3 and 11 years: Developmental periods and hierarchical organization. European Journal of Developmental Psychology, 1(2), 127–152. https://doi.org/10.1080/17405620344000022

45.

Robson

(2016). Self-regulation and metacognition in young children: Does it matter if adults are present or not? British Educational Research Journal, 42(2), 185–206. https://doi.org/10.1002/berj.3205

46.

Sellars

Pavarini

Michelson

Creswell

Fazel

(2021). Young people’s advisory groups in health research: Scoping review and mapping of practices. Archives of Disease in Childhood, 106(7), 698–704. https://doi.org/10.1136/archdischild-2020-320452

47.

Singh

Srivastava

Haldane

Chuah

Koh

Seng Chia

Legido-Quigley

(2017). Community participation in health services development: A systematic review on outcomes. European Journal of Public Health, 27(suppl_3), 1–25. https://doi.org/10.1093/eurpub/ckx187.429

48.

Sluis-Thiescheffer

Bekker

Eggen

(2007). Comparing early design methods for children. In Proceedings of the 6th International Conference on Interaction Design And Children, IDC, 2007, 17–24. https://doi.org/10.1145/1297277.1297281

49.

Spyrou

(2011). The limits of children’s voices: From authenticity to critical, reflexive representation. Childhood, 18(2), 151–165. https://doi.org/10.1177/0907568210387834

50.

Stiehl

K. A. M.

Krammer

Schrank

Pollak

Silani

Woodcock

K. A.

(2023). Children’s perspective on fears connected to school transition and intended coping strategies. In Social psychology of education (issue 0123456789). Springer Netherlands. https://doi.org/10.1007/s11218-023-09759-1

51.

UNICEF . (1989). United Nations convention on the Rights of the child. In “International Journal of Law, Policy and the family. https://downloads.unicef.org.uk/wp-content/uploads/2010/05/UNCRC_united_nations_convention_on_the_rights_of_the_child.pdf?_ga=2.136460409.857185160.1512610390-650986988.1512610390. https://doi.org/10.1093/lawfam/5.2.132

52.

Veenman

M. V. J.

Wilhelm

Beishuizen

J. J.

(2004). The relation between intellectual and metacognitive skills from a developmental perspective. Learning and Instruction, 14(1), 89–109. https://doi.org/10.1016/j.learninstruc.2003.10.004

53.

Walsh

Druin

Guha

M. L.

Foss

Golub

Hatley

Bonsignore

Franckel

(2011). Layered elaboration: A new technique for co-design with children. In CHI 2010: Communicating. https://doi.org/10.1145/1979742.1979556

54.

Wight

Wimbush

Jepson

Doi

(2015). Six steps in quality intervention development (6SQuID). Journal of Epidemiology and Community Health, 70(5), 520–525. https://doi.org/10.1136/jech-2015-205952

55.

Williams

Boylan

A. M.

Nunan

(2020). Critical appraisal of qualitative research: Necessity, partialities and the issue of bias. BMJ Evidence-Based Medicine, 25(1), 9–11. https://doi.org/10.1136/bmjebm-2018-111132

56.

Willig . (2008). Introducing qualitative research in psychology: Adventures in theory and method (2. Open Univ. Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.60 MB