The Impact of Young Children’s Persistent Challenging Behavior on Child Care Providers: “It’s Like Floating in a Sea With No Paddle”

Abstract

The purpose of this descriptive study was to highlight the experiences of 11 child care providers whose classrooms included preschoolers who engaged in persistent challenging behavior, thus expanding the limited literature on this topic. This qualitative study included two interviews with each provider. The results center around the effect of challenging behavior on the classroom experiences of providers. Findings indicate that children’s challenging behavior impacts the physical and social environment, as well as relationships among providers, children, families, and program administrators. Implications for research and practice are discussed.

Keywords

challenging behavior child care early childhood education suspension expulsion

Enrollment in child care is the norm for many young children in the United States. According to Child Care Aware of America (2022), in 2019 over 12 million children under the age of five were in some form of care outside the home. Options for care that families typically choose from include center-based care, family child care, relative care, or a care provider in the family home. The National Survey of Early Care and Education (NSECE, 2021) revealed that 7.15 million children under five were in center-based care. These demographics highlight the fact that many children in the U.S. are growing up in center-based care, including child care environments.

Educators, researchers, and policy makers have become aware that many children begin school without the emotional, social, behavioral, and academic skills that are necessary for success (Bierman & Motamedia, 2015). Additionally, families of children who engage in challenging behavior report difficulties in finding affordable child care that will provide an engaging and appropriate environment for their children (Worcester et al., 2008). Challenging behavior in preschoolers (ages 3–5) has been defined as a repeated pattern of behavior that interferes with optimal learning or engagement in pro-social interactions with peers and adults (Smith & Fox, 2003) and puts a child at high risk for later social problems or school failure (Kaiser & Rasminsky, 2007). It is important to consider physical descriptions of problematic behaviors (i.e., biting, screaming), as well as the intensity, frequency, and duration of the behavior.

Many early childhood teachers report feeling overwhelmed and stressed by challenging behavior (Friedman-Krauss et al., 2014; Zulauf-McCurdy & Zinsser, 2020), thus leaving them feeling unprepared to meet the needs of children who engage in these behaviors (Hemmeter et al., 2021). As a result, child care providers in communities across the country report that children are being suspended or expelled from early childhood programs at an alarming rate, due to their challenging behavior (Children’s Equity Project, 2020). The National Center for Pyramid Model Innovations defines suspension as “the child is temporarily removed from the classroom and/or class peers” and expulsion as “the permanent dismissal of the child from the program in response to problem behavior” (NCPMI, n.d., paras. 1–2).

In Gilliam (2005), reported that preschoolers with challenging behaviors were three times more likely to be expelled from early childhood settings than children in Grades K 12 combined. He also noted that expulsion rates for children in state-funded prekindergarten programs were highest in faith-affiliated centers and for-profit child care programs. Also, expulsion rates in child care settings were 11.93 expulsions/1,000 children compared to 6.16 expulsions/1,000 children in public prekindergarten classrooms and 6.59/1,000 in Head Start programs. High rates of expulsion from some early childhood settings have perpetuated across the years, and differences in rates of expulsion have become more apparent (Children’s Equity Project, 2020). For instance, children who have experienced trauma are more likely to be expelled (Zeng et al., 2019) as are children with disabilities (Zeng et al., 2021). Expulsion also disproportionately impacts African American children and contributes to systemic racism (National Black Child Development Institute, 2018). It should be noted that extreme responses to challenging behavior, such as suspension and expulsion, are adult-mediated decisions, contingent on teacher and administrator perceptions of challenging behavior, and therefore vulnerable to bias (Gilliam et al., 2016).

A qualitative study conducted by Doubet and Ostrosky (2014, 2016) included interviews with seven families of preschoolers who engaged in persistent challenging behavior, resulting in suspension and expulsion. When asked about their experiences as they sought support for their children, families reported asking child care staff for help, doubting their parenting skills, feeling stressed, and wanting child care staff to offer suggestions to help their children. These findings informed the current study which focused on understanding providers’ experiences in child care environments when presented with children who the providers perceived as engaging in challenging behavior. Thus, the purpose of this study was to expand the limited literature on child care providers’ experiences when children engage in behavior perceived as challenging. While it is recognized that children in child care are at higher risk to be suspended or expelled, most of the current literature on suspension and expulsion has focused on children in public preschool programs (Children’s Equity Project, 2020) or the perspectives of administrators (Clayback & Hemmeter, 2021). Additionally, survey methodology has often been used to gather data on teacher perceptions (c.f., Zulauf-McCurdy & Zinsser, 2020). The current study was designed to extend the literature by listening to the voices of child care providers as we addressed the following research question: What is the impact of challenging behavior on the classroom experiences of child care providers?

Method

This study was approved by the university Institutional Review Board. Data for this study were collected by interviewing 11 child care providers who reported that they worked with children ages 3 to 5 who engaged in persistent challenging behavior. These participants were recruited from 29 licensed child care centers in a small urban community in the Midwest; 83 providers worked with preschoolers in these 29 centers. Recruitment materials were either hand-delivered or placed in the office mailboxes of all potential participants. Flyers also were posted on staff bulletin boards in break rooms and offices within these centers.

The criteria for participation included: being a lead provider of 3- to 5-year-old children in a child care setting and having worked with children with perceived challenging behaviors within the last 3 years (children who the provider described as being at-risk for expulsion or being expelled due to behavior seen as challenging). For the purpose of this study, challenging behavior was defined as behaviors that cause a child in a child care setting to be expelled (asked to leave or is disenrolled) or to be at-risk of being expelled (family was told that the behavior must improve or the child would be disenrolled). After two rounds of recruiting participants, 21 child care providers met the inclusion criteria and expressed an interest in participating in the current study (25% of the 83 providers in the 29 child care centers).

To interview individuals representing a diverse set of voices, purposeful sampling was used to select a pool of participants from the 21 providers who responded to our recruitment efforts. Palinkas et al. (2016) state, “Purposeful sampling is widely used in qualitative research for the identification and selection of information-rich cases related to the phenomenon of interest” (p. 1). Demographic information was gathered using a Provider Information Form which included questions about the child care center and the provider’s role, child care experiences, level of education, gender, age, and race. A systematic process was conducted to review this information and select participants who represented a diverse group in terms of size and type of child care centers and the race, age, education level, and child care experiences of the providers. Also, limited research support guided the decision to select a smaller number of participants. The first two authors reviewed the Provider Information Forms submitted by the 21 interested participants and then selected 11 individuals (purposeful sampling) who mirrored the characteristics of the 21 potential participants. For example, the 21 potential participants included 67% Caucasian and 33% African American providers; individuals from other races did not respond to the recruitment flyer. These data were used to select the 11 interviewees, 55% who identified as Caucasian and 45% who identified as African American.

Participants

The 11 participants taught in child care centers located in a Midwestern U.S. community with a population of approximately 130,000 residents; all participants were female (see demographics in Table 1). Thirty-six percent of the participants reported they had a high school diploma and some college credits, 36% had associate’s degrees, and 28% held bachelor’s degrees. Their average time in the child care profession was 13 years and the average number of years in their current job was 5 years.

Table 1.

Child-care Participants in the Study on Impact of Children’s Persistent Challenging Behaviors.

Name*	Age	Ethnicity/race	Education	Years in current job	Years in child care	Type of child care center
Bernita	44	Caucasian	AA	5	23	Non-profit
Cheryl	33	African American	AA	<1	10	For-profit
Denise	56	Caucasian	BA	8	25	For-profit
Eliza	29	African American	AA	9	9	Non-profit
Inez	53	Caucasian	AA	5	15	For-profit
Kathy	43	African American	Some college	3	8	Non-profit
Melinda	27	Caucasian	BA	4	4	Faith-based
Nina	54	African American	Some college	8	18	For-profit
Sally	25	Caucasian	BA	<1	5	For-profit
Tenesha	38	Caucasian	Some college	3	12	For-profit
Tori	37	African American	Some college, CDA	4	15	Non-profit

Pseudonyms were assigned.

Data reported by the 11 interviewees about the children they worked with are displayed in Table 2. The average number of children in their classrooms was 20 (range = 13–29). The providers reported working with an average of 8 children who engaged in challenging behavior over the last 3 years (range = 3–15). When asked to report the number of children in the last 3 years who had left or were at–risk of leaving their classrooms due to perceived challenging behavior, the average was 3 (range = 1–7). The racial diversity of the students in their classes, as reported by providers, was 53% African American, 41% Caucasian, 4% Asian, and 2% Hispanic. Providers also reported that 54% of the families in their classrooms lived in low resource areas.

Table 2.

Children in the Classrooms Where Child-care Participants Taught.

Demographics	Bernita	Cheryl	Denise	Eliza	Inez	Kathy	Melinda	Nina	Sally	Tenesha	Tori
Children in the classroom	19	16	29	15	25	20	13	21	22	20	20
Children with challenging behavior in the last 3 years	15	15	3	4	6	10	6	6	4	10	7
Expelled or at risk, due to challenging behavior	4	7	1	2	2	3	4	3	3	3	2
Ethnic/racial diversity of students
African American	15	14	2	13	4	12	8	11	10	14	13
Caucasian	2	1	23	2	15	8	4	10	11	6	7
Hispanic	1	1	0	0	1	0	1	0	1	0	0
Asian American	0	0	4	0	5	0	0	0	0	0	0
Families from low resourced areas	18	8	1	14	3	17	11	17	5	12	12
Individualized Education Plan (IEP)	2	1	1	0	3	3	1	2	1	3	3

Setting

Information about providers’ experiences with children who engaged in behavior perceived as persistently challenging was gathered through a semi-structured interview, member checking, and a follow up discussion. Interviews were conducted in locations agreed upon by each participant and the first author, including private areas in child care centers, an office at a community college, and private meeting rooms in restaurants. The interviews were conducted in quiet, comfortable spaces that allowed for the protection of confidentiality and for audio recording. Children were not present during the interviews.

Interview Protocol

The semi-structured nature of the interviews was shaped by the questions, but also allowed participants to tell their stories and the researcher to probe beyond the interview protocol (Berg, 2001; Bogdan & Biklen, 1998). The first author conducted all interviews, which enhanced trust and allowed her to probe issues that emerged across participants. For all 11 participants, the initial interview, member checking, and the follow up discussion took place within a 42-day time period. The interview questions were developed following a review of the literature and findings from previous studies with families of young children who engaged in challenging behavior (c.f., Doubet & Ostrosky, 2014, 2016). All 11 providers participated in both the initial and follow up interviews and answered all of the questions. Sample questions included: Can you give me some details about your work experiences in the child care profession? Thinking about your last 3 years as a child care provider, how have children with challenging behavior affected your classroom experiences? (See Supplemental Materials for interview questions.)

The questions used in the follow up discussions were developed following member checking and a review of the first set of interview transcripts. These questions were designed to gain clarity on topics that participants discussed during the initial interviews (e.g., define terms, add details, provide descriptions). Follow up discussions also focused on delving further into topics that some providers mentioned in the first interview (e.g., some providers discussed their philosophy of teaching while others did not mention this during the initial interview). By reviewing the 11 initial transcripts, the first two authors identified topics in need of in-depth probing.

Procedures

All semi-structured interviews and follow up discussions were conducted by the first author and took place in the same settings. The initial interviews lasted approximately 1 hr (range = 45–75 min). The follow up discussions lasted approximately 30 min each, were conducted to gain clarity on topics that participants discussed during the first interviews and enabled the interviewer to probe deeper into topics that were mentioned in the first interview. All interviews were audio recorded and transcribed by a professional transcriber. To ensure accuracy in transcription, the first author reviewed all transcripts while listening to the audio recordings, editing the transcripts for errors.

Member checking was used to confirm the findings from the interviews (Brantlinger et al., 2005). Each participant was asked to read their transcript for accuracy; four of the participants edited their transcriptions, while the other seven individuals reported that no changes were necessary. After making these minor changes to the transcriptions, the first two authors reviewed the first set of interviews, follow-up questions were developed, and the second interviews were conducted and transcribed.

The recruitment materials informed potential participants about remuneration gifts. Following the first interview, participants received a children’s book and a $10 gift certificate. After the second interview, participants received information about community supports for children who engaged in challenging behavior and another $10 gift certificate.

Data Interpretation/Analysis

Interview data were coded following Braun and Clarke’s (2006) six-phase thematic analysis process: (a) becoming familiar with the data, (b) generating initial codes, (c) searching for themes, (d) reviewing themes, (e) defining and naming themes, and (f) presenting the data. Coding began with the first two authors reviewing each transcript a minimum of three times to become familiar with the content and looking for references to topics, non-answers, and repetitions as they became immersed in the data. Initial codes were developed as the researchers paid attention to phrases or key ideas, relationships between content, and patterns. The two authors independently coded one interview at a time and met to discuss their codes and reach consensus for each interview. A codebook was developed and used to code subsequent interviews. The two researchers continued to meet after independently coding each transcript, to refine the codebook and definitions and to reach consensus on their coding.

Matrices were developed to visually document the placement of data into categories then themes and sub-themes. After all the interviews were coded, the researchers reviewed each theme and sub-theme to ensure that they were distinct. A table was created with supporting quotes under each sub-theme to confirm that the data accurately represented them. Paying attention to the number of participants who contributed to each theme and sub-theme allowed the authors to determine whether they represented (a) a shared experience for many or all participants, (b) the unique perspectives of one or two people, or (c) characteristics of a small subset of individuals.

Trustworthiness and Credibility

The researchers sought to ensure trustworthiness and credibility by considering the standards of high-quality qualitative research in special education, which included the use of triangulation, collaborative work, member checks, and thick descriptions (Brantlinger et al., 2005). Data triangulation occurred by using multiple data sources including the Provider Information Form, interviews, and member checks. A collaborative data analysis process was used, potential instances of bias were discussed, and disconfirming evidence was considered. Also, a consensus process was used to interpret the data. Finally, thick descriptions were gathered from each transcript to use as evidence for each theme.

Positionality Statement

The researchers reflected on their positionality throughout the course of the study and met regularly to discuss the potential impact of bias during data analysis (Brantlinger et al., 2005). All three authors are white females who have taught in early childhood/early childhood special education classrooms, and they have worked as teacher educators, researchers, and professional development providers. The researchers had no prior relationships with the participants, nor did they have any connection to the child care settings from which participants were recruited. The authors support the promotion of social emotional learning for all young children and recognize that not all child care providers feel adequately prepared to prevent and address challenging behavior. Also, the authors are uniquely positioned to examine the issue of suspension and expulsion given their research and teaching focus on the promotion of social emotional competence and the prevention of challenging behavior.

Findings

Participants were asked to reflect on how children who they perceived as engaging in challenging behavior affected the provider’s typical day. This question was followed by prompts regarding the children and adults in the classroom, as well as the impact of challenging behavior on the entire child care center. As mentioned previously, follow-up interviews enabled the researchers to gain clarity and additional information on some topics. Four themes emerged from the data: (a) the impact of challenging behavior on the physical environment and classroom activities, (b) the effects of challenging behavior on the social environment in the classroom and the child care center, (c) the impact of challenging behavior on relationships with children, and (d) family relationships, communication, and advice. Each theme is described in detail with supporting quotes that emerged from the two sets of interviews. Pseudonyms are used for the study participants.

Impact on the Physical Environment and Classroom Activities

When participants shared their perceptions of how challenging behavior impacted the classroom environment, their thoughts were categorized into three sub-themes: positioning furniture, safety concerns, and adaptations to the classroom materials for children with challenging behavior. Child care providers talked about the need to make physical changes in their classrooms when their enrollment included children who engaged in persistent challenging behavior. Most of the providers mentioned how they rearranged classroom furniture to have a better view of the entire room. For example, Sally explained how she moved furniture to see the far end of her room “because someone was hitting with the blocks. And I don’t feel comfortable with one of my children being out of my sight at any moment.” Tenesha echoed Sally’s concern when she said, “It does affect how your room is set up. I’ve rearranged this room at least three times, and I have found that the more open the centers are, the easier it is to see and control what’s going on.”

Most of the providers also discussed how safety issues impacted the physical environment. For instance, Kathy talked about children who would “tear up the classroom,” and she would “put things away so that when a particular child would go on a rampage, there wouldn’t be chairs or anything around.” Denise explained her fear regarding other children getting hurt, and her sense of responsibility in keeping everyone in her room safe. In the following quote Melinda explained her approach to ensure a safe environment:

If I see there is a certain thing that they are drawn to, to take out their physical behavior on, or it’s something that’s easy for them to pick up and throw when they are angry, I’ll either try to move it out of the way, or I will completely remove it.

The third topic related to the physical environment focused on materials. Three providers mentioned that they added materials to their classroom in response to children’s challenging behavior. For example, when Tori noticed children having a difficult day, she added some fine motor activities (e.g., play dough, bubble wrap to pop) to center choices. Nina reported that she brought a chair into the classroom where children “can go to think” when they engage in challenging behavior. Finally, Inez described the addition of a large box in her classroom as a place where children could go and be alone:

We have implemented something else called Safe Place, which is part of the Conscious Discipline program, where the children—when they feel the need, they can go there voluntarily and sit. It’s an area away from the children, other children. There is a box that they can sit [in] and play with soft, cushy things. It’s an area for them to calm themselves, get control. It’s been very effective.

All 11 providers described different ways that a child whose behavior was seen as challenging impacts the physical environment of their classrooms. Rearranging furniture, safety concerns, and material selection emerged as important considerations.

When participants shared their perceptions of how challenging behavior impacted classroom activities, their thoughts were categorized into two sub-themes: changing activities and lesson plans, and decreased time spent with individual students. Most participants reflected on their inability to accomplish their intended goals for the day and the need to constantly change their plans. For instance, regarding interacting with children with challenging behavior, Denise shared:

It takes up a lot of time, and it takes away from my [lesson] plans for the day. We don’t get through the plans. . .you don’t plan as much or as hard or maybe as challenging to others [students], because you can’t get through it. You can’t do it.

Providers also talked about the time and energy it takes when a child who engages in persistent challenging behavior interrupts large group activities. Kathy reported that she often cannot follow through with her lesson plans because children are fighting, causing her to stop an activity like reading a story, address the problem, and then get the rest of the students re-engaged in the activity. Tori also referred to this “stop and start over again” process, and Nina reported, “It throws me off balance. I try really hard to stay focused, but then if I have to put all of my attention into one child, it really kind of throws things off.” A similar concern was echoed by Bernita who noted that the other students “are not going to get to do what they normally would. That one child is taking up a lot of my time.” Also, Eliza feared that by focusing so much on a child who was having difficulties, the other children would not learn the appropriate way to gain her attention.

A few participants shared their concerns about not getting to spend time with some children due to the amount of time spent with the children who engage in challenging behavior. For instance, Sally explained such concerns when she said:

I oftentimes feel a little bit sensitive to, or a little bit sad that challenging behavior interferes with other kids, or that they have to wait or that—sometimes the best listeners in my class—I feel like I hardly get to tell them how great they are . . . I want to play with them more.

Children who engage in challenging behavior clearly influenced what participants accomplished in a given day and how they arranged their classroom environments, as they all noted the impact that disruptive behavior had on their day-to-day classroom life.

Effects on the Social Environment

To gather data on the social environment in the classroom and the center, the researchers asked the providers about the influence of a child with challenging behavior on peers and adults. Responses were categorized into four sub-themes: (a) children in the classroom, (b) other adults in the classroom, (c) families of other children, and (d) the broader context of the child care center.

Children in the Classroom

All providers commented on how children who engage in behavior deemed as challenging influence their peers, with the most frequent comments focusing on how other children sometimes imitate the challenging behavior. For example, Eliza felt that her students imitated what they saw other children do, while Denise stated that children who are “typically good” will misbehave when they are “left alone without my attention. So, then they become noisy or rowdy. Maybe not always doing what they’re supposed to be doing.” Other providers spoke about the rest of the class “falling apart” and how chaotic the room became during this wait time and how frustrating that was for both children and adults.

Three providers described children’s reactions to a child who engages in challenging behavior as not wanting to play with that child or referring to that child as a “bad kid.” Sally shared the following incident from when she started as the lead provider in her classroom:

When I first came to the room, [the children said] “Luke did this. Luke did that.” All the other kids blamed it on Luke. And I thought this kid doesn’t really have a chance if he’s in this role. And he fills it well when he’s in there. One day he was not there, and they said “Luke did it. Luke is a bad kid.” I heard that a lot.

Melinda described the actions of some of her students in relation to a child who engaged in challenging behavior:

A lot of times they [classmates] are scared. They are afraid of that child. Kids do not want to play with that child. If they see that child get upset, their response is to back away. They just don’t know how to express what they feel, and so they generally will reject the child. I try to talk with the children who are affected by it and try to help them understand that we are doing the best that we can to keep everyone safe. That is what my job is, keeping them safe.

Other Adults in the Classroom

Most participants reflected on the impact children who engage in challenging behavior had on their teaching assistant (TA). Additionally, they discussed the relationship between teamwork and their work with children who engage in challenging behavior. Tenesha shared her insights in both areas when she said:

My assistant is very stressed out; it’s very tense in here some days. Some days it’s great. Some days we don’t have any problems. Some days it’s just like, oh! You want to just pull your hair out. It’s frustrating some days. My assistant gets very, very frustrated very easily sometimes. Actually, we have been trying to stick beside each other, we’ve been trying to do the exact same thing that the other one would do when they’re not here.

Melinda also shared her thoughts regarding teamwork: “[My TA] prefers that I handle the situation, which is ok, generally her response and responsibility is to help me with the other children.” She also noted that she and her TA planned how they, as a team, would respond to a tantrum or other behaviors seen as challenging.

Providing another view of teamwork, Eliza noted that her TA “does not have as much patience as I do. I have to step in to try to take over, and then she is stressed, she is irritated, she loses her focus.” Nina also commented on teaming issues when she shared:

Sometimes I might feel like she gets stressed, or he gets stressed, or they don’t approve of the way that I’m handling things. She tells me by the look on her face and body language that she doesn’t approve of the way sometimes that I talk to a child or when I lead the child by the arm to where I want them to be. Sometimes I don’t feel like those are things that I can discuss with them.

Families of Other Children

The third topic that emerged under the social environment category focused on the families of children in the classroom. Providers talked about families asking them about the situation and safety concerns. Participants reported that families often came to them with questions and concerns when there was a child who engaged in challenging behavior in the same classroom as their child. These providers said that typically families began a conversation by saying that their child mentioned getting hit or kicked by another child, or that another child was “always” in trouble or was a “bad kid,” and the family member wanted more information about the situation. One provider emphasized that she tells families that no child is a “bad kid,” and that all children develop at different rates. This provider explained to families that she and her staff are working with individual children to reduce problem behavior.

Melinda said that usually families want to make sure she is aware that their children are talking at home about another child’s behavior. When this happened, Melinda assured them that “I am doing the best that I can for each child.” Not surprisingly, safety motivated many families to talk to providers about their concerns. Tori remembered a time when a mother reported that her child did not want to come to the center anymore because of a child with behavior issues. Tori said, “She [the child] was scared of him and the mom told me about this.”

Some participants also spoke of families who told their children not to play with a child who engaged in challenging behavior. In fact, a few families asked providers not to allow their children to play with other children whom they felt engaged in persistent challenging behavior. Eliza spoke of such an interaction:

Some of the families see the way they are acting and behaving, and they say “I don’t want my child to play with him. I don’t want my child to pick up some of the bad behaviors that he has, and if they play together, they are going to start acting like him.” So, they wanted me to keep him away from their child.

Kathy shared that several of her students’ families believed, “If you’re hit, hit back”; she believed that this attitude created difficulties in her classroom. Two providers talked about families threatening to move their children to a different center if “something did not change” regarding a child who engaged in persistent challenging behavior.

Impact on the Center Environment

According to the participants, a child who engages in challenging behavior can affect the social environment outside of the child’s classroom. Most providers commented on ways they felt that such a child affects other staff and children in their center. Some providers specifically spoke about the added stress and distractions felt by people in other classrooms when a child is screaming or throwing objects. Bernita talked about her experience when other providers came to her room after they heard a child screaming and “generally causing a commotion.” She reported that comments and input from other providers at an “already difficult time” drew unhealthy attention to the behavior. This also caused disruptions in the classrooms the providers left to come to her room.

Providers also discussed the amount of time that other staff (director, assistant director, other providers, substitutes, secretaries, kitchen staff) expended trying to support a child who engaged in challenging behavior. Tenesha referred to the center’s director and assistant director when she shared: “That affects them so they can’t do their jobs. They can’t do what they’re supposed to be doing ’cause they’re dealing with this child.” Kathy, who worked in a center that included nine other classrooms, talked about people who got involved when a child “acted out”:

Actually, everybody focuses on that particular child, everybody in the center . . . whether they are in the child care field or they may work in the kitchen area . . . everybody comes in trying to calm that child down or interact with that child, asking him what was wrong. And sometimes it works and other times, it doesn’t.

Interestingly, Sally was the only provider who mentioned how a child with persistent challenging behavior might hurt the reputation of the child care center. Her explanation of this impact painted a picture of the domino effect that can occur:

I think it affects everyone. I think it affects the whole system. I think it affects the other children in the room. So, then it affects the providers. And then, in turn, I’m so stressed that I affect the directors. And my stress comes up and up and up, and the kids go home and tell their parents, “I got punched and I got kicked.” And I think it does affect the center, because I think it makes the center look bad too.

Relationships With Children

Providers talked about their special bonds with some children who engaged in challenging behavior. For example, Denise shared that “you need to build relationships with children every day.” Participants spoke of slowing down, taking tiny steps, talking individually with each child daily, and having one-to-one time with children.

Almost half of the providers expressed feelings of fondness and talked about becoming attached to students who engaged in behavior perceived as challenging. For example, Melinda described her relationship with a student who struggled with behavior: “Because we have gone through this together, he has gotten very attached. He knows that I am going to be there for him.” Also, Tori talked about feeling sad when a child’s behavior caused other children to not want to play with him, and Cheryl shared the regret she felt when children left the center due to their persistent challenging behavior, “I like those kids, they pull at my heart.”

Tenesha expanded on her feelings about working with a child who engaged in persistent challenging behavior and the joys and challenges of that experience. Before Nathan was in Tenesha’s classroom, she noted that he had “earned a reputation as a difficult kid.” His tantrums could be heard throughout the center, and his delayed expressive language concerned his providers. Initially, Tenesha was unsure about having Nathan in her room:

At first, I was like, I do not want this kid in my room. It’s not that I don’t care. But I’m like, “Holy cow, how am I supposed to communicate with this kid?” I felt like I was floating in a sea and no paddle. That I was just going around in circles. And then, one day out of the clear blue, he said something! And I just looked at him, “Wow! You actually got that.” I just can’t explain it. You can be working with them for weeks and weeks, and they don’t get it. And then you say something and it’s like the whole room just lights up in gold. You’re like, “Wow! They got it!”

Families: Relationships, Communication, and Advice

Participants were asked about their relationships and communication with the families of children who engaged in persistent challenging behavior. Tenesha felt that when she and a family member worked together to support a child, their relationship became “tighter,” and they grew closer to one another. The majority of providers reported that they felt comfortable in their relationships with families and had even approached families to talk about their concerns regarding children’s behavior. However, these same providers reported that some families acknowledged a child’s difficult behavior, while some families did not. Sally shared, “I feel like it [parent’s response] depends on my tact. How the parent reacts also depends on my timing, what kind of day they have had, and frankly, what kind of day I have had.” Inez added that she felt that “Parents are a big part of the problem.” Finally, two providers noted that they tried to have resources available when they talked to families about concerns in the classroom.

Several participants spoke about providing written communication to families through a daily report sent home with children. They described a section on this report which addressed what type of day the child had. Most of the providers reported they chose between marking a “happy face or a sad face.” Cheryl commented that she sometimes reminded children throughout the day of the importance of “getting a happy face” at the end of the day. When Nina described this type of communication with families, she explained how she used the daily report as a springboard for private conversations with families:

We do our daily report that we have to fill out, and I will put on there what type of day they had, if they were happy, if they were active. And then, there’s the other, they did like hitting or kicking, I will put that in there also, like hitting another child. And then, if I want the families to talk to me, I’ll just leave it [the daily report] blank, and as well what kind of day that they have [had]. And then, I will tell them, I will pull them to the side, away from the other kids, or even their own child, and we’ll step out in the hall, and we’ll talk about it. “He was hitting today, or he did this today.”

Two providers reflected on their fears regarding the possibility of families using corporal punishment with their children because of the provider talking about behavioral concerns. Elaborating on this point, Melinda stated:

If I know that telling them [families] that their child has difficult behavior is going to result in something really bad at home, maybe a spanking, I might be a little bit more careful with how I word it. . .it’s a very sensitive issue.

Another issue brought up by a few providers was their concern that if they “make a big deal” about a child’s behaviors, the family might remove the child from the center rather than “dealing with the problem.” For example, Cheryl said that right after she and the director met with a family member to discuss ways to support a child who engaged in behavior perceived as challenging, the family member told them “I will be taking him out of the center. I want to try a different atmosphere in a different center.” Cheryl felt bad about losing this child without getting the chance to try some of her ideas. Similarly, Bernita spoke about a mother who appeared embarrassed that her daughter was having a difficult time at the center. Bernita felt that the family member was on the verge of removing her because she did not want people to think she did not know how to raise her child, which in turn caused Bernita to avoid discussing the behavior with the mother.

Based on their experiences with children who engaged in challenging behavior, providers were asked to share their advice for families. The majority of participants wanted more families to talk to them about their concerns. Getting help and information from the child’s provider and from others at the center was cited as an important place to start. Many providers recommended that families get involved with their children, such as supporting and listening to their children, visiting the classroom, and playing with their children. Other suggestions were for families to be patient, to start the day in a positive way, and to have a consistent routine at home. A few providers also recommended that families implement strategies at home that have proven successful in the classroom such as using positive language with children and teaching expectations.

Discussion

Overall, in this study, we found that children who engage in challenging behavior can influence the physical, academic, and social environment in child care classrooms and centers. In addition to the negative impact on the children who have been suspended or expelled or are at risk for these punitive disciplinary actions (i.e., decreased self-esteem, stigma, poor peer relationships) and their families (i.e., financial implications, stigma, lower self-confidence in parenting), child care providers in the current study reported negative social effects on peers, assistant teachers, administrators, and other adults in their centers as a result of a child engaging in challenging behavior. These 11 participants shared many stories about their experiences, and they offered some advice based on the lessons they had learned. Their self-refection and ability to talk through some emotionally charged and painful experiences helped answer the research question addressed in this study. Several child care providers noted that they wanted to participate in this study with the hope that their stories might help support children with challenging behavior and the professionals who care for them. It is important to point out that these children often do not have IEPs nor the support of special education services, leaving providers feeling isolated and unsure of what strategies to implement or what next steps to follow.

Findings from this study indicate that children who engage in persistent challenging behavior have a noteworthy impact on classroom activities and child care experiences for both children and adults. This study extends the literature by including the voices of child care providers, an underrepresented group in the literature on challenging behavior. During their interviews, all participants talked about ways that having a child with behaviors perceived as challenging impacted classroom activities and everyone’s experiences. The providers reported a reduction in opportunities for all children, those with challenging behavior and their classmates, to become engaged in classroom experiences as a result of frequent disruptions caused by some children’s difficult behavior. A similar interruption to activities, schedules, and daily plans was reported by families of children with challenging behavior in a study conducted by Doubet and Ostrosky (2014). The participants in that study described family activities that were altered, reduced, or omitted due to the behavior of a child with persistent challenging behavior.

Child care providers also discussed the negative effects a child with perceived challenging behavior can have on the social interactions of all children in the room. The providers described situations where peers would not play with a child who engaged in persistent challenging behavior, were afraid of that child, and used negative words when speaking about the child. The importance of peer-related positive social interactions and competence is well documented in the literature (c.f., Favazza et al., 2022; Meyer & Ostrosky, 2014; Meyer et al., 2021). Persistent peer rejection is a common experience for children with challenging behavior, and when not addressed early it is a contributing factor to school failure (Dunlap et al., 2006). Teaching children problem solving skills (Golden et al., 2021), fostering emotional literacy (Joseph et al., 2021), and using positive descriptive feedback (Conroy & Sutherland, 2021) can help children learn new skills and result in improved social emotional competence.

Providers also shared that children with perceived challenging behavior can impact the relationships with and among families at a center. Navigating difficult conversations with families about children harming each other highlights differences in behavioral expectations between a child’s home and early childhood setting. Likewise, statements made by teachers such as “parents are a big part of the problem” are concerning and can negatively impact family/professional partnerships. Comments like these, reflecting a provider’s bias, can impact how a child is supported in the classroom. This aligns with research conducted by Zulauf-McCurdy and Zinsser (2020) who found that children in classrooms where teachers had stronger and more positive relationships with families were less likely to be expelled as a result of challenging behavior. Likewise, Freedle and Wahman (2021) found that families with young children who exhibited challenging behavior reported a relation between family functioning and well-being and family members’ experiences with professionals.

Finally, the importance of quality pre-service and in-service training cannot be minimized, especially related to promoting social emotional competence, preventing challenging behavior, intervening using a team-based approach when young children engage in challenging behavior, and partnering with families. As noted earlier, many early childhood professionals do not feel well-prepared to address behaviors considered challenging (Friedman-Krauss et al., 2014; Zulauf-McCurdy & Zinsser, 2020). This also was a recurring theme mentioned by participants in the current study. While some researchers have studied professional development related to preparing future and current early childhood professionals on these topics, more work is needed in this area (Flower et al., 2017; Garrity, 2019; Hemmeter et al., 2008; Nenonene et al., 2019). Professional development and support from program leadership at community colleges and universities and within early childhood programs themselves must prioritize social emotional competence and place promotion and prevention at the forefront.

Limitations

While this study provides a significant amount of insight into child care providers’ experiences when working with children with behavior that they perceive as challenging, it would have been helpful to have a more diverse group of participants. All participants were female; the voices of male providers would have added breadth to this study. The 11 participants worked in their current jobs for an average of 5 years and had been employed in child care settings for an average of 13 years. While this longevity in the profession was important to the study, the voices of novice providers are missing. They might have provided different insights especially as the topic of social emotional development has taken center stage in more recent years (c.f., https://www.nytimes.com/2019/01/23/learning/empathy-and-resilience-responsibility-and-self-care-resources-for-social-and-emotional-learning-from-the-new-york-times.html). Another consideration is that the average age of providers in this study was 40 years; younger providers might have had different types of experiences and knowledge to reflect upon (e.g., their relationships and comfort in interacting with families and administrators, their understanding of the critical role of social emotional competence in children’s success).

Finally, conducting interviews with administrators and families, in addition to child care providers, might have resulted in more depth and the ability to triangulate data. While the study design included member checks and second interviews, conducting interviews with other people might have provided other viewpoints about children with challenging behavior. Likewise, observational data on children’s behavior were not collected to validate providers’ perceptions. Self-reported data can be prone to bias; thus, additional measures to corroborate providers’ insights would have proven useful.

Implications for Research

As noted in the Limitations section, this study was designed to focus on the voices of child care providers. Future research should highlight the impact of children’s challenging behavior on a diverse group of participants including administrators, novice providers, assistants, and male providers. There also is a need to listen to the perceptions of individuals who work for agencies that provide support to early childhood providers and children with behaviors seen as challenging. By listening to the voices of these important colleagues, a more comprehensive perspective on this topic should emerge.

In addition, further investigation is necessary to study relationships between factors that may contribute to providers’ success or lack of success in supporting children with challenging behavior. For example, research could examine levels of support available, strategies implemented, personal philosophy, behavioral expectations of young children, providers’ abilities to teach social emotional skills, the quality of the classroom environment and interactions, current or past experiences with behavior support, and partnering with families to address persistent challenging behavior. Comparing the insights of providers who currently have a child in their classroom who engages in behavior perceived as challenging with those of providers who discuss previous experiences with a child with perceived challenging behavior also might render new insights on this topic. Finally, future research should include intervention studies measuring change in participants’ feelings of confidence and levels of stress when working with young children who engage in persistent challenging behavior. Provider and family education sessions designed and conducted to increase participants’ knowledge of children’s social and emotional development, shared understandings about behavior, and proactive strategies could be offered. Pre and post measures could be used to document the impact of such interventions.

Implications for Practice

The results of this study have implications for future practice. Child care providers in this study commented on how children’s challenging behavior impacted the classroom environment and activities and relationships with children, families, and other professionals. When designing pre- and in-service professional development opportunities for early childhood providers, planners should refer to research-based models of collaborative personnel development systems, which include the following components: (a) awareness of the importance of the issues and evidence-based practices, (b) training (pre- and in-service), and (c) individualized, follow-up support including coaching and mentoring (Dunst, 2015).

One model that brings together all of the components discussed above in an effort to promote social emotional competence and address behaviors deemed as challenging and supports the adults who work with young children is the Pyramid Model Framework (Hemmeter et al., 2006, 2021). This framework recognizes the importance of developing the skills of adults in both universal and individualized supports for children’s social and emotional development. This model has proven successful in child care programs and provides the needed components to help early childhood professionals, such as the participants in this study, promote appropriate behavior, prevent problem behavior, use data to understand issues related to problem behavior, and adopt evidence-based intervention practices (Hemmeter et al., 2022).

From participants’ comments in the current study, it is evident that child care center administrators are on the front line when providers first seek support. Therefore, administrators need to be knowledgeable about children’s social emotional development and evidence-based strategies to support children who engage in persistent challenging behavior (Kupersmidt et al., 2000). Administrators must become skillful supporters and coaches, so their staff is willing to implement new strategies with young children who engage in challenging behavior (Clayback & Hemmeter, 2021; Snyder et al., 2022).

All 11 child care providers discussed their eagerness for families to talk with them regarding children’s behavior seen as challenging. Open, two-way communication and family/provider partnerships could be improved by seeking input from families about effective strategies used in home settings, as well as including helpful tips for communication in family handbooks. Joint workshops with families and providers could be offered on topics related to social and emotional development and communication among staff and family members. Also, early childhood professionals and administrators might benefit from professional development that focuses on discussing difficult topics with families.

Several providers expressed a desire to learn more about community agencies that offer services for young children who engage in persistent challenging behavior. Early childhood organizations could plan events where representatives from various agencies and direct service providers share program specific information with child care providers and administrators. Additionally, pamphlets or databases could be developed and disseminated to staff in child care centers that describe the community agencies along with pertinent contact information.

Conclusion

There is a fairly vast and growing evidence base revealing that early childhood providers need more information, training, and support for teaching social, emotional, and behavioral skills to the children in their care. This study provides evidence from child care providers about their experiences and perceptions that can be used to inform future education and training opportunities. The findings also add to the existing literature by expanding our knowledge and understanding of the impact of a child with perceived challenging behavior in child care settings. These 11 participants shared their perceptions and experiences, offered suggestions for improvement, and described their frustrations and challenges. It is important to include the perspectives of all of the adults who are available to support children with challenging behaviors and their families, for when we engage in a collective effort to ameliorate challenging behaviors, we set children up for lifelong success. It is only when we listen to the voices of all early childhood professionals and consider their thoughts and ideas that high quality services and supports can be provided to each and every child.

Supplemental Material

sj-docx-1-tec-10.1177_02711214231180871 – Supplemental material for The Impact of Young Children’s Persistent Challenging Behavior on Child Care Providers: “It’s Like Floating in a Sea With No Paddle”

Supplemental material, sj-docx-1-tec-10.1177_02711214231180871 for The Impact of Young Children’s Persistent Challenging Behavior on Child Care Providers: “It’s Like Floating in a Sea With No Paddle” by Sharon Doubet, Michaelene M. Ostrosky and Amanda C. Quesenberry in Topics in Early Childhood Special Education

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available on the Topics in Early Childhood Special Education website with the online version of this article.

References

Berg

B. L

. (2001). Qualitative research methods for the social sciences. Allyn & Bacon.

Bierman

K. L.

Motamedia

. (2015). SEL programs for preschool children. In Durlak

J. A.

Domitrovich

C. E.

Weissberg

R. P.

Gullato

T. P.

(Eds.), Handbook of social and emotional learning (pp. 135–150). The Guilford Press.

Bogdan

R. C.

Biklen

S. K

. (1998). Qualitative research for education: An introduction to theory and methods (3rd ed.). Allyn & Bacon.

Brantlinger

Jimenez

Klinger

Pugach

Richardson

. (2005). Qualitative studies in special education. Exceptional Children, 71, 195–207. https://doi.org/10.1177/001440290507100205

Braun

Clarke

. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Child Care Aware of America. (2022). The US and the high price of child care: 2019. https://www.childcareaware.org/our-issues/research/the-us-and-the-high-price-of-child-care-2019/

Children’s Equity Project. (2020). Start with equity from the early years to the early grades: Data, research, and an actionable child equity policy agenda. https://childandfamilysuccess.asu.edu/cep/start-with-equity

Clayback

K. A.

Hemmeter

M. L

. (2021). Exclusionary discipline practices in early childhood settings: A survey of child care directors. Early Childhood Research Quarterly, 55, 129–136.

Conroy

M. A.

Sutherland

K. S

. (2021). Using positive descriptive feedback to improve children’s behavior. In Hemmeter

M. L.

Ostrosky

M. M.

Fox

(Eds.), Unpacking the pyramid model: A practical guide for preschool teachers (pp. 101–108). Brookes.

10.

Doubet

S. L.

Ostrosky

M. M

. (2014). The impact of challenging behavior on families: I don’t know what to do. Topics in Early Childhood Special Education, 34, 223–233. https://doi.org/10.1177/0271121414539019

11.

Doubet

S. L.

Ostrosky

M. M

. (2016). Parents’ experiences when seeking assistance for their children with challenging behaviors. Topics in Early Childhood Special Education, 36(3), 176–185. https://doi.org/10.1177/0271121416660293

12.

Dunlap

Strain

Fox

Carta

Conroy

Smith

Kern

Hemmeter

M. L.

Timm

M. A.

Mccart

Sailor

Markey

D. J.

Lardieri

Sowell

. (2006). Prevention and intervention with young children’s challenging behavior: Perspectives regarding current knowledge. Behavioral Disorders, 32, 29–45. https://doi.org/10.1177/019874290603200103

13.

Dunst

C. J

. (2015). Improving the design and implementation of in-service professional development in early childhood intervention. Infants & Young Children, 28(2), 210–219.

14.

Favazza

P. C.

Ostrosky

M. M.

de Boer

A. A.

Rademaker

F. C

. (2022). How do we support the peer acceptance of children with disabilities? In Jones

M. H.

Estell

(Eds.), Peer relationships in classroom management: Evidence and interventions for teaching (pp. 77–94). Routledge.

15.

Flower

McKenna

J. W.

Haring

C. D

. (2017). Behavior and classroom management: Are teacher preparation programs really preparing our teachers? Preventing School Failure: Alternative Education for Children and Youth, 61(2), 163–169.

16.

Freedle

Wahman

C. L

. (2021). Understanding families’ experiences raising young children with challenging behavior through an ecological lens: A qualitative meta-synthesis. Topics in Early Childhood Special Education, 42(3), 269–281.

17.

Friedman-Krauss

A. H.

Raver

C. C.

Morris

P. A.

Jones

S. M

. (2014). The role of classroom-level child behavior problems in predicting preschool teacher stress and classroom emotional climate. Early Education and Development, 25(4), 530–552. https://doi.org/10.1080/10409289.2013.817030

18.

Garrity

S. M.

Longstreth

S. L.

Linder

L. K.

Salcedo Potter

. (2019). Early childhood education centre director perceptions of challenging behaviour: Promising practices and implications for professional development. Children & Society, 33(2), 168–184.

19.

Gilliam

W. S

. (2005). Pre-kindergarteners left behind: Expulsion rates in state prekindergarten systems. Yale University Child Study Center. Retrieved May 23, 2005, from http://info.med.yale.edu/chldstdy/

20.

Gilliam

W. S.

Maupin

A. N.

Reyes

C. R.

Accavitti

Shic

. (2016). Do early educators’ implicit biases regarding sex and race relate to behavior expectations and recommendations of preschool expulsions and suspensions? Yale Child Study Center.

21.

Golden

A. K.

Taylor

A. L.

Korba

Hemmeter

M. L

. (2021). Teaching social problem solving. In Hemmeter

M. L.

Ostrosky

M. M.

Fox

(Eds.), Unpacking the pyramid model: A practical guide for preschool teachers (pp. 119–130). Brookes.

22.

Hemmeter

M. L.

Barton

Fox

Vatland

Henry

Pham

Horth

Taylor

Binder

von der Embse

Veguilla

. (2022). Program-wide implementation of the pyramid model: Supporting fidelity at the program and classroom levels. Early Childhood Research Quarterly, 59, 56–73.

23.

Hemmeter

M. L.

Ostrosky

M. M.

Fox

. (2006). Social and emotional foundations for early learning: A conceptual model for intervention. School Psychology Review, 35(4), 583–601.

24.

Hemmeter

M. L.

Ostrosky

M. M.

Fox

. (2021). Unpacking the pyramid model: A practical guide for preschool teachers. Brookes.

25.

Hemmeter

M. L.

Santos

R. M.

Ostrosky

M. M

. (2008). Preparing early childhood educators to address young children’s social-emotional development and challenging behavior: A survey of higher education programs in nine states. Journal of Early Intervention, 30(4), 321–340. https://doi.org/10.1177/1053815108320900

26.

Joseph

G. E.

Yates

Ostrosky

M. M

. (2021). Strategies to foster emotional literacy. In Hemmeter

M. L.

Ostrosky

M. M.

Fox

(Eds.), Unpacking the pyramid model: A practical guide for preschool teachers (pp. 131–139). Brookes.

27.

Kaiser

Rasminsky

. (2007). Challenging behavior: Understanding, preventing, and responding effectively (2nd ed.). Pearson Education Inc.

28.

Kupersmidt

J. B.

Bryant

Willoughby

. (2000). Prevalence of aggressive behaviors among preschoolers in head start and community child care programs. Behavioral Disorders, 26(1), 42–52. https://doi.org/10.1177/019874290002600105

29.

Meyer

L. E.

Milam

M. E.

Ostrosky

M. M

. (2021). Friendship skills and strategies for teaching them. In Hemmeter

M. L.

Ostrosky

M. M.

Fox

(Eds.), Unpacking the pyramid model: A practical guide for preschool teachers (pp. 109–118). Brookes.

30.

Meyer

L. E.

Ostrosky

M. M

. (2014). Measuring the friendships of young children with disabilities: A review of the literature. Topics in Early Childhood Special Education, 34(3), 186–196. https://doi.org/10.1177/0271121413513038

31.

National Black Child Development Institute. (2018). Delivering on the promise of effective early childhood education. https://www.nbcdi.org/sites/default/files/import_files/nbcdi-releases-delivering-promise-effective-early-childhood-education-white-paper-examines-policy.pdf

32.

National Center for Pyramid Model Innovations (NCPMI). (n.d.). Suspension and expulsion in early childhood. https://challengingbehavior.org/pyramid-model/suspension-expulsion/

33.

Nenonene

R. L.

Gallagher

C. E.

Kelly

M. K.

Collopy

R. M

. (2019). Challenges and opportunities of infusing social, emotional, and cultural competencies into teacher preparation. Teacher Education Quarterly, 46(4), 92–115.

34.

NSECE (National Survey of Early Care and Education). (2021). Chartbook: Center-based early care and education providers in 2012 and 2019: Counts and characteristics. www.acf.hhs.gov/sites/default/files/documents/opre/cb-counts-and-characteristics-chartbook_508_2.pdf

35.

Palinkas

L. A.

Horwitz

S. M.

Green

C. A.

Wisdom

J. P.

Duan

Hoagwood

. (2016). Purposeful sampling for qualitative data collection and analysis in mixed method research. Administration and Policy in Mental Health and Mental Health Services, 42(5), 533–544.

36.

Smith

B. J.

Fox

. (2003). Systems of service delivery: A synthesis of evidence relevant to young children at risk of or who have challenging behavior. University of South Florida, Center for Evidence-Based Practice, Young Children with Challenging Behavior.

37.

Snyder

Hemmeter

M. L.

Fox

L. K

. (2022). Essentials of practice-based coaching: Supporting effective practices in early childhood. Brookes.

38.

Worcester

J. A.

Nesman

T. M.

Raffaele Mendez

L. M.

Keller

H. R

. (2008). Giving voice to parents of young children with challenging behavior. Exceptional Children, 74, 509–525. https://doi.org/10.1177/001440290807400406

39.

Zeng

Corr

O’Grady

Guan

. (2019). Adverse childhood experiences and preschool suspension expulsion: A population study. Child Abuse and Neglect, 19, 104–149. https://doi.org/10.1016/j.chiabu.2019.104149

40.

Zeng

Pereiera

Larson

Corr

O’Grady

MacDonald

. (2021). Preschool suspension and expulsion for young children with disabilities. Exceptional Children, 87(2), 199–216. https://doi.org/10.1177/0014402920949832

41.

Zulauf-McCurdy

C. A.

Zinsser

K. M

. (2020). How teachers’ perceptions of the parent-child relationship affects children’s risk for early childhood expulsion. Psychology in the Schools, 58(1), 69–88.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB