Sage Journals: Discover world-class research

Abstract

Community-based methods for autism evaluation may be one solution for ameliorating delays in diagnosis, which are exacerbated for children from minoritized backgrounds. However, limited research has examined caregiver satisfaction with community-based models of autism evaluation. Thus, our objective was to use a mixed-methods approach to investigate caregiver satisfaction with their child's autism evaluation conducted across a statewide system of primary care autism diagnosis. Results indicated overall high satisfaction and no significant differences were found between satisfaction total scores nor caregiver stress and any child/family demographic variables. Satisfaction and stress were also not related to autism diagnostic outcome, clinician diagnostic certainty, or diagnostic accuracy. Qualitative suggestions for evaluation improvement include more thorough explanation of diagnosis and service recommendations. Overall, our findings indicate high caregiver satisfaction with multiple dimensions of community-based autism evaluation in the primary care setting, suggesting this may be a feasible and sustainable model that caregivers find acceptable.

Keywords

autism satisfaction primary care developmental delay caregiver stress screening

Autism is a neurodevelopmental condition with a current United States (US) prevalence of 1 in 36 children.¹ Despite observable symptoms in the first year of life² and diagnostic stability in the second year of life,³ the average age of diagnosis in the United States remains after 4 years^1,4 with further delays for children from diverse racial and ethnic,^5,6 socioeconomic,⁷ and geographic backgrounds.^8,9 Given the individual and societal impact (eg, increased lifetime care costs)^10,11 of diagnostic delays there has been a push to test novel methods that promote equitable access. Community-based tiered models of autism evaluation, which equip primary care physicians and other clinicians (PCPs) with skills to conduct autism evaluations, may be one solution for ameliorating diagnostic delays.^12-14 These models have been shown to be feasible, accurate, and reduce diagnostic delays.^15-19 However, family satisfaction with this type of streamlined evaluation is not yet known.

Patient satisfaction is a growing area of interest for health organizations striving to identify the critical standards and values held by consumers and is one component of the Institute for Healthcare's Quadruple Aim for achieving value-based healthcare.²⁰ Research has identified communication, access, clinician interpersonal skills, and care coordination as strongly related to patient satisfaction.²¹ Satisfaction ratings are predictive of compliance with care and clinical outcomes,²² and influences the likelihood patients would recommend their clinician.²³ Within primary care, there has been an emphasis placed on patient satisfaction as an outcome measure alongside health status and quality of life evaluations.²⁴

Limited research has been conducted on satisfaction with autism diagnostic services. However, findings suggest that caregiver satisfaction varies with demographic background of the patient and factors related to the diagnostic evaluation.^25,26 Specifically, age of child at diagnosis, quality of information received at diagnosis, clinician time spent on the evaluation, perceived clinician verbal and nonverbal skills, support offered following diagnosis, and parental stress during the diagnostic process have all been shown to be related to satisfaction.²⁶ For example, caregivers of younger children who receive more information surrounding the diagnosis and treatment resources demonstrate increased satisfaction, particularly when the process is streamlined (ie, visiting fewer providers to obtain diagnosis).²⁷ Further, satisfaction is associated with lower stress in caregivers of children with disabilities.²⁸ Given that caregivers of children with disabilities experience high levels of stress, ensuring satisfaction during primary care evaluations is of great importance.²⁹

To date, only one study of family satisfaction with primary care autism evaluations has been published.³⁰ In this study, 90% of caregivers endorsed feeling either “extremely” or “moderately” satisfied with the autism evaluation process.³⁰ Understanding caregiver satisfaction is essential to building and implementing successful systems of autism diagnostic care. Thus, the objective of the current study was to use a mixed-methods approach to investigate caregiver satisfaction with their child's autism evaluation conducted within a statewide system of primary care autism diagnosis. Specifically, we examined (1) quantitative ratings of caregiver satisfaction and the association between satisfaction and caregiver stress and child and clinician diagnostic factors (ie, autism outcome, clinician diagnostic certainty, and diagnostic accuracy), and (2) qualitative outcomes from open-ended responses on a caregiver satisfaction survey.

Methods

Data for the present study were collected as a part of a larger study focused on primary care autism diagnostic accuracy. The study was approved by the Indiana University Institutional Review Board (see McNally Keehn et al³¹), and written informed consent was obtained from legal guardians of all participants.

Setting

Early Autism Evaluation (EAE) Hub System

The EAE Hub system is a statewide network of community primary care and general pediatric clinicians trained to provide streamlined diagnostic evaluations for young children ages 14 to 48 months implemented in the United States³¹ (see Supplemental Material for detailed information). Following training, EAE Hub clinicians participate in a monthly learning collaborative to maintain skills. EAE Hub clinicians serve as regional consultants who receive evaluation referrals from a child's primary care clinician when the child is determined to be at increased likelihood for autism.³¹ EAE Hub clinicians follow a standard clinical evaluation protocol, conducted in a 90-minute office visit, including administration/review of standard developmental and autism screening tools, a Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) focused developmental history and clinical interview, physical examination, and administration and scoring of the Screening Tool for Autism in Toddlers.³² The EAE Hub clinician synthesizes all assessment data to make a best-estimate clinical diagnosis and provides the family with diagnostic counseling and a written summary with prioritized next step clinical recommendations. The full clinical report is disseminated to the family and referring PCP following the evaluation. Children with complex diagnostic profiles (ie, for whom EAE Hub clinicians cannot make a diagnostic decision) are referred for comprehensive autism evaluation at a specialty diagnostic center.

Participants

Participants included 98 children, ages 14 to 18 months (M = 2.61 years; SD = 0.6; White 69%, Black 8%, Latine 13%; see Table 1). Sixty-five percent (n = 64) of children received a diagnosis of autism at the EAE Hub evaluation. Demographics of children and their caregivers (M = 28 years at birth; SD = 6; 46% household income < $50 000/year; 33% ≤ high school education) are presented in Table 1.

Table 1.
Participant Demographics.

N = 98

Age (years), M (SD) 2.61 (0.6)

Child race and ethnicity, n (%)

White 68 (72)

Black or African American 8 (9)

Asian 1 (1)

Latine (any race) 13 (14)

Two or more races 4 (4)

Caregiver relationship, n (%)

Biological parent 91 (96)

Legal guardian 4 (4)

Primary caregiver age at child's birth, M (SD) 28 (6)

Household annual income, n (%)

< $50 000 41 (46)

≥ $50 000-$99 000 34 (37)

≥ $100 000 15 (17)

Caregiver education, n (%)

≤HS diploma 32 (33)

Some college-associate degree 34 (35)

Bachelor's degree 23 (24)

Graduate degree 9 (9)

Autism diagnosis, n (%) 64 (65)

Global developmental delay diagnosis, n (%) 28 (29)

Language delay/disorder diagnosis, n (%) 18 (18)

EAE Hub Clinician Diagnostic Certainty, n (%)

Certain 85 (88)

Not certain 11 (12)

All data presented as n (%) unless otherwise noted. Percentages are calculated based on those children for which data are available.

Measures

Demographic Survey. Caregivers completed a brief questionnaire with items about child race and ethnicity, family annual household income, and caregiver education level.

EAE Hub Satisfaction Survey. The EAE Hub Satisfaction Survey was adapted from survey items published in 3 studies on patient and family satisfaction with autism diagnostic services.^26,30,33 Survey items from across studies were selected by the EAE Hub team, consisting of a medicine-pediatrics physician, clinical psychologist, and 2 parents of adult children with disabilities, including autism. Item language was modified to fit the EAE Hub context. Caregivers completed this brief 11-item questionnaire regarding satisfaction with their child's EAE Hub evaluation (see Table 2). Items are rated on a 5-point Likert scale. A total score was calculated from satisfaction items 1 to 7, and 11 (range 8-40; higher scores indicate greater satisfaction). One item evaluated overall stress with the evaluation; 2 additional questions evaluated caregiver perception of their child's behavior during the evaluation. Finally, 2 open-ended items asked caregivers to provide suggestions to improve the EAE Hub diagnostic process.

Table 2.
Caregiver Satisfaction With EAE Hub Evaluations: N = 98.

Items 1-9 rated on Likert scale (5 = strongly agree; 1 = strongly disagree); n (%)

Mean Strongly agree (5) Agree (4) Neither agree nor disagree (3) Disagree (2) Strongly disagree (1)

1. The provider(s) at the clinic spent enough time with my child and me. 4.4 54 (55%) 31 (32%) 9 (9%) 4 (4%) 0 (0%)

2. The provider(s) at the clinic listened carefully to me. 4.6 68 (69%) 22 (22%) 4 (4%) 4 (4%) 0 (0%)

3. The provider(s) at the clinic were sensitive to my family's values and customs. 4.6 67 (68%) 26 (27%) 4 (4%) 1 (1%) 0 (0%)

4. The provider(s) at the clinic made me feel like a partner in my child's care. 4.5 63 (64%) 26 (27%) 5 (5%) 4 (4%) 0 (0%)

5. The provider(s) at the clinic discussed with me the range of options to consider for my child's treatment. 4.4 56 (57%) 27 (28%) 8 (8%) 7 (7%) 0 (0%)

6. The provider(s) at the clinic made it easy for me to ask questions or raise concerns. 4.5 61 (63%) 26 (27%) 9 (9%) 2 (2%) 0 (0%)

7. I am satisfied with the amount of information about my child's diagnosis that was given to me by the clinic. 4.1 45 (46%) 31 (32%) 11 (11%) 9 (9%) 2 (2%)

8. My child behaved in his/her usual way during the evaluation, showing his/her usual behaviors and skills. 3.9 39 (40%) 31 (32%) 16 (16%) 7 (7%) 5 (5%)

9. My child's daily routine was typical on the day of the evaluation (eg, no unusual changes in sleep or eating, etc). 4.2 52 (53%) 30 (31%) 5 (4%) 9 (9%) 2 (2%)

Item 10 rated on Likert scale (5 = not stressed at all; 1 = extremely stressed); n (%)

Mean Not at all stressed (1) Slightly stressed (2) Moderately stressed (3) Very stressed (4) Extremely stressed (5)

10. Rate how much stress you felt overall during your experience at the clinic. 1.88 47 (48%) 28 (29%) 12 (12%) 10 (10%) 1 (1%)

Item 11 rated on Likert scale (5 = completely satisfied at all; 1 = not at all satisfied); n (%)

Mean Completely satisfied (5) Very satisfied (4) Moderately satisfied (3) Slightly satisfied (2) Not at all satisfied (1)

11. Rate how satisfied you were overall with the clinic diagnostic process. 4.1 44 (45%) 23 (24%) 22 (22%) 6 (6%) 3 (3%)

EAE Hub Diagnostic Certainty Form. EAE Hub clinicians submitted data on results of the child's autism evaluation, including categorical diagnosis (autism; non-autism) and clinician diagnostic certainty (rated on a 5-point Likert scale; Not Certain at all–Completely certain).

Autism Specialist Diagnostic Certainty Form. The autism specialist submitted data on results of the child's research evaluation, including categorical diagnosis and specialist diagnostic certainty (ie, as described above).

Procedure

Seven EAE Hubs set within primary care practices across Indiana participated in a larger prospective diagnostic study from June 2019 through August 2022.³¹ At study enrollment, which typically occurred within 1 to 2 weeks of the EAE Hub evaluation, caregivers were sent an electronic link with (1) demographic survey and (2) EAE Hub satisfaction survey. This allowed for caregivers to rate satisfaction with the EAE Hub evaluation while the appointment was salient and to ensure there was no bias with recall related to the research evaluation. The follow-up research evaluation was completed within 12 weeks of the EAE Hub evaluation. EAE Hub evaluation data (ie, EAE Hub diagnosis and diagnostic certainty ratings) were collected by a member of the study team. To determine EAE Hub diagnostic accuracy, the study evaluation team, consisting of a licensed clinical psychologist with expertise in evaluation of autism in young children and a clinical research technician, traveled to the EAE Hub to conduct a follow-up gold-standard autism diagnostic assessment (including measures of child developmental and adaptive functioning and autism symptom severity) within 16 weeks of the initial EAE Hub evaluation. Study personnel involved in data collection, including expert clinicians who conducted research evaluations EAE Hub diagnosis to reduce bias in diagnostic decision making. Participants were compensated with a gift card for study completion.

Statistical Analyses

A mixed methods approach was utilized for the present study. Descriptive statistics were calculated for satisfaction survey items and total score (sum of items 1-7 and 11). Pearson correlations, chi-squares tests, and independent-samples t-tests were performed to examine associations between satisfaction total scores, caregiver stress, child and family demographic characteristics, and EAE Hub diagnostic outcome and clinician certainty. Additionally, associations between total satisfaction and caregiver stress and diagnostic agreement between EAE Hub and autism specialist evaluation were examined. Version 28 of SPSS was used to conduct statistical analyses.

Analysis of 2 open-ended questions on the satisfaction survey were conducted using a constructivist grounded theory approach.³⁴ During initial coding, inductive codes were created by 2 members of the research team by organizing responses into categories and subcategories. During focused coding, the most salient codes (also called focused themes) were identified, and then the rest of the data was coded. Three meetings were held with a third research member to reach consensus on focused themes.

Results

Quantitative Findings

See Table 2 for item-level satisfaction survey data. Overall, results revealed high caregiver satisfaction (total score M = 35.03, SD = 5.6) with the EAE Hub evaluation process. Mean individual satisfaction item scores ranged from 4.1 to 4.6 (range 1-5; higher scores representing higher satisfaction). Highest ratings of satisfaction were on items related to caregivers’ perceptions of clinician listening skills (mean rating: 4.6; 91% agree/strongly agree) and sensitivity to family values and customs (mean rating: 4.6; 91% agree/strongly agree). Items with the lowest satisfaction ratings were related to the amount of information provided about the child's diagnosis (mean rating: 4.1; 78% agree/strongly agree), discussion about treatment options (mean rating: 4.4; 85% agree/strongly agree), and satisfaction with the overall diagnostic process (mean rating: 4.1; 69% agree/strongly agree).

Seventy-two percent of caregivers strongly agreed/agreed with the statement, “My child behaved in his/her usual way during the evaluation, showing his/her usual behaviors and skills” (mean rating: 3.9) and 84% of caregivers strongly agreed/agreed with the statement, “My child's daily routine was typical on the day of the evaluation (eg, no unusual changes in sleep or eating)” (mean rating: 4.2). Regarding stress, 77% of caregivers reported feeling “not at all stressed/slightly stressed” during their child's evaluation.

No significant associations were found between satisfaction total scores or caregiver stress and child/family demographic characteristics, including parental education, household annual income, and child race or ethnicity (all Ps > .08). Overall satisfaction and stress did not differ across autism diagnostic outcome (ie, autism; nonautism; Ps > .43) or by EAE Hub clinician diagnostic certainty (Ps > .07). Similarly, there was no significant association between total satisfaction or stress with accuracy of the EAE Hub diagnosis (ie, agreement between EAE Hub clinician and expert specialist diagnosis; Ps > .57).

Qualitative Findings

In total, 41% (n = 40) of caregivers provided a response to the 2 open-ended questions. Three main themes emerged during data analysis, which are detailed below and summarized in Table 3.

Table 3.
Satisfaction Thematic Analysis.

Domain Element N = 40 Example quote

Evaluation process Prior to evaluation 2 “Tell families how long to expect to be there and give families tips to better prepare child for change in routine.”

“Get better people on the phones and at the check-in counter.”

Evaluation environment 4 "A bigger room to play around, the space was not enough and a child typically a 2 year old will feel the sense of clinic and not a place he would usually play.”

“We were in the same small medical room for almost 3 hours - a bigger area that less “clinical” would be helpful for evaluations.”

Building rapport 9 “Dr x did a wonderful job answering any and all questions and concerns, explaining the situation and findings to me while maintaining a relaxed professional relationship. She also was very patient with my son.”

“You could spend a little more time getting to know the child all the way.”

Diagnosis/feedback 10 “How severe the diagnosis is and to talk more about it.”

“Explain what it means to be on the spectrum. Tell me what I need to do to help my kid develop - not just therapy to send him to, but how do I need to interact with him differently on a personal level. What should I expect to be different in his behavior from kids who aren't on the spectrum and what is the best way to teach him? For example: my pediatrician (prior to diagnosis) said that if he throws a tantrum, I need to completely ignore him. Is that still true for a kid on the spectrum? How do I need to change my parenting?”

Postevaluation Recommendations for next steps 2 “Maybe have handouts printed for parents to take home to do more research on their own once they know what to look for. This might mean you print different packets and have them ready.”

“More discussion about next steps after the diagnosis. And treatment options.”

Follow-up 2 “Follow-up (1-2 weeks post) with nurse or doc to provide additional guidance after some time to digest all that was handed to us that day. “

“Somebody was supposed to follow up with me with additional information on developmental preschool and that never happened.”

Overall satisfaction Positive 11 “Thinking about the possibility of your child being autistic is very emotional, which makes the process of the diagnosis very stressful. The staff at the clinic made it as comfortable and consoling as they could.”

“To be honest, in a difficult situation, they made it a very pleasant experience. Was very helpful, very kind and informative.”

“Dr x did a wonderful job answering any and all questions and concerns, explaining situation and findings to me while maintaining a relaxed professional relationship. She also was very patient with my son. Dr x covered all options and went out of her way to provide extra resource help for our family. Thank you Dr X.”

Theme 1: Evaluation Process. The first theme, Evaluation Process, encompassed 4 subthemes: Prior to evaluation; evaluation environment; building rapport; and diagnosis/feedback. Recommendations (n = 2) emerged regarding improvements to guide families on what to expect in the weeks prior to and the day of their child's evaluation.
Tell families how long to expect to be there and give families tips to better prepare child for a change in routine. (Caregiver 16)

Several caregivers (n = 4) provided feedback about the clinical space in which autism evaluations took place and recommendations for creating a more conducive setting for young children navigating novel environments.
We were in the same small medical room for almost 3 hours—a bigger area that [is] less “clinical” would be helpful for evaluations. (Caregiver 20)

Caregivers (n = 9) commented around the process of clinician rapport and the importance of their child feeling comfortable around the clinician.
You could spend a little more time getting to know the child all the way. (Caregiver 34)

Finally, (n = 10) caregivers responded to the clinician's delivery of the diagnosis and made suggestions for improvement in how diagnostic feedback and counseling is delivered.
Explain what it means to be on the spectrum. Tell me what I need to do to help my kid develop—not just therapy to send him to, but how do I need to interact with him differently on a personal level … How do I need to change my parenting? (Caregiver 19)

Theme 2: Postevaluation. The second theme, postevaluation, yielded 2 subthemes: Recommendations for next steps and follow-up. Recommendations for next steps (n = 2) included requests for more caregiver education regarding what to expect in terms of modifying interactions with their autistic children, appropriate next steps following diagnosis, and more thorough review of treatment options following an autism diagnosis.
Maybe have handouts printed for parents to take home to do more research on their own once they know what to look for. This might mean you print different packets and have them ready. (Caregiver 17)

The subtheme of follow-up (n = 2) referred to caregivers’ desire to have a clinic staff member follow up with their family after the evaluation to provide additional resources or answer any lingering questions that may have arisen after caregivers have had time to adjust to the diagnosis.
Follow-up (1-2 weeks post) with nurse or doc to provide additional guidance after some time to digest all that was handed to us that day. (Caregiver 31)

Theme 3: Overall Satisfaction. One subtheme, positive, emerged from the overall satisfaction theme and included general compliments and other positive feedback regarding the evaluation process as a whole.
To be honest, in a difficult situation, they made it a very pleasant experience. Was very helpful, very kind and informative. (Caregiver 22)

Finally, there was no association between EAE site or clinician with the number and types of qualitative themes represented in the data (all Ps > .44).

Discussion

Patient satisfaction is a critical component of the Institute for Healthcare's Quadruple Aim of improving population health, reducing care costs, enhancing patient experience, and improving provider satisfaction.²⁰ Limited research to date has examined caregiver satisfaction with tiered community-based primary care models of autism diagnosis. Thus, the aim of the present study was to build on our existing knowledge about patient satisfaction with autism diagnosis by employing a mixed-methods approach to examine caregiver satisfaction with evaluations conducted across the EAE Hub system, a statewide network of community primary care and general pediatric clinicians trained to provide streamlined diagnostic evaluations for young children ages 14 to 48 months.

Caregivers reported high satisfaction with multiple dimensions of EAE Hub evaluation. Satisfaction was highest with caregivers’ perceptions of clinician listening skills and sensitivity to family values and customs, suggesting that caregivers felt heard even within the context of a 90-minute primary care evaluation. Satisfaction ratings were lower, but still moderate, for domains related to the amount of information provided during the visit, discussion about treatment options, and satisfaction with the overall diagnostic process. Despite some variability in satisfaction, it appears that caregiver satisfaction with EAE Hub evaluation is comparable to traditional evaluations for autism conducted in specialty diagnostic centers.³⁰ Considering that long wait times negatively impact family satisfaction,⁴ and traditional specialty diagnostic centers often have waitlist that exceed a year,³⁵ autism evaluation models embedded within the primary care setting offer a feasible and sustainable alternative that is acceptable to families.

Satisfaction was not associated with diagnostic outcome, clinician diagnostic confidence, or diagnostic accuracy (ie, compared to expert evaluation) indicating that satisfaction is not contingent upon receiving a specific outcome during the evaluation. Caregiver stress was also not related to these diagnostic factors. Although other studies have found high levels of caregiver stress prior to an autism diagnostic assessment,³⁶ our findings suggest that EAE Hub clinicians and their teams can be trained and supported to address the emotional needs of caregivers during an otherwise stressful process. Further, research on association between demographic factors and caregiver satisfaction with autism evaluation has been mixed.^25,26,37 In our sample of diverse children and families, child race/ethnicity, family income, and caregiver education level were not associated with caregiver satisfaction. These findings indicate that other factors, such as the amount of diagnostic information received and the communication skills of the clinician²⁶ may have a stronger influence on satisfaction than demographic characteristics. As such, diagnostic feedback and service recommendations for autistic children may consider individual caregiver characteristics and needs, rather than rely on general demographic characteristics.

Several takeaways for improving EAE Hub evaluations are notable from our qualitative analysis. Caregivers suggested that, when scheduling families for their evaluation, care should be taken to explain evaluation procedures and expectations. Similar research has found that most parents arrive at their evaluation appointments with limited understanding of the assessment process, creating challenges in sufficiently preparing themselves and their child.³⁸ Additionally, clinical teams should ensure that a child-friendly space has been prepared for the evaluation and enough time has been spent building rapport prior to beginning the assessment.

Caregivers reported a desire for more comprehensive information about autism and how they can help support their child. Previous research has found that parents appreciate feedback that is clear, direct, honest, and thorough.³⁹ Some families also provided feedback on improvements to follow-up care and coordination of services. This is in line with previous findings demonstrating increased caregiver satisfaction with service models that include coordination of care with community early childhood intervention providers.⁴⁰ By coordinating care across traditionally siloed service systems (eg, medical, early intervention, and educational), clinicians can more equitably support families who may struggle to navigate service provision, such as those with limited health literacy or experiencing social determinants of health risk factors.³⁹ Taken together, by incorporating skill building across these areas into PCP training programs, we can improve family satisfaction.

Limitations

Satisfaction ratings were generally high across items and total scores; as such it is possible that our analyses were limited by the statistical issue of range restriction. Further, it is possible that our findings may have been biased by participant and caregiver self-selection into a larger research study. Due to limited samples sizes across EAE Hub clinician and site, we cannot rule out potential site- and clinician-specific differences in satisfaction ratings. In addition, a large majority of evaluations and thus data collection occurred during the COVID-19 pandemic, potentially influencing caregiver-reported satisfaction. For example, it may be that caregivers experienced higher levels of satisfaction due to receiving the evaluation during a period in which healthcare access was very challenging. As such, our findings should be interpreted with caution and warrant replication in independent studies. Future research should include larger comparative studies of caregiver satisfaction across various autism evaluation models, as well as qualitative data on caregiver perspectives, in order to inform quality improvement efforts and refinement of evaluation models and care planning.⁸ In addition, the current paper focused on caregiver satisfaction and did not address caregiver experience. Future studies also warrant investigating caregiver experience with the autism evaluation models, including how patient satisfaction factors into the overall experience as well as how patients weigh the service received against their expectations.⁴¹

Conclusion

This study evaluated associations between caregiver satisfaction across a statewide system of autism evaluation in the primary care setting. Overall satisfaction was high with positive qualitative feedback and recommendations shared by caregivers. Improvements to communication about evaluation procedures, appropriate evaluation environments, clear and thorough diagnostic feedback, and care coordination and navigation of postevaluation recommendations are all ways in which PCPs can make targeted changes to enhance caregiver satisfaction. Although head-to-head evaluation of diverse evaluation models is needed, our findings suggest that streamlined primary care models of autism diagnosis are well accepted by caregivers.

Supplemental Material

sj-docx-1-jpx-10.1177_23743735241305531 - Supplemental material for Mixed Methods Analysis of Caregiver Satisfaction With the Early Autism Evaluation Hub System

Supplemental material, sj-docx-1-jpx-10.1177_23743735241305531 for Mixed Methods Analysis of Caregiver Satisfaction With the Early Autism Evaluation Hub System by Ann Marie Martin, Jordan Huskins, Angela Paxton, Amira Nafiseh, Mary R. Ciccarelli, Brandon Keehn and Rebecca McNally Keehn in Journal of Patient Experience

	N = 98
Age (years), M (SD)	2.61 (0.6)
Child race and ethnicity, n (%)
White	68 (72)
Black or African American	8 (9)
Asian	1 (1)
Latine (any race)	13 (14)
Two or more races	4 (4)
Caregiver relationship, n (%)
Biological parent	91 (96)
Legal guardian	4 (4)
Primary caregiver age at child's birth, M (SD)	28 (6)
Household annual income, n (%)
< $50 000	41 (46)
≥ $50 000-$99 000	34 (37)
≥ $100 000	15 (17)
Caregiver education, n (%)
≤HS diploma	32 (33)
Some college-associate degree	34 (35)
Bachelor's degree	23 (24)
Graduate degree	9 (9)
Autism diagnosis, n (%)	64 (65)
Global developmental delay diagnosis, n (%)	28 (29)
Language delay/disorder diagnosis, n (%)	18 (18)
EAE Hub Clinician Diagnostic Certainty, n (%)
Certain	85 (88)
Not certain	11 (12)

Items 1-9 rated on Likert scale (5 = strongly agree; 1 = strongly disagree); n (%)
1. The provider(s) at the clinic spent enough time with my child and me.	4.4	54 (55%)	31 (32%)	9 (9%)	4 (4%)	0 (0%)
2. The provider(s) at the clinic listened carefully to me.	4.6	68 (69%)	22 (22%)	4 (4%)	4 (4%)	0 (0%)
3. The provider(s) at the clinic were sensitive to my family's values and customs.	4.6	67 (68%)	26 (27%)	4 (4%)	1 (1%)	0 (0%)
4. The provider(s) at the clinic made me feel like a partner in my child's care.	4.5	63 (64%)	26 (27%)	5 (5%)	4 (4%)	0 (0%)
5. The provider(s) at the clinic discussed with me the range of options to consider for my child's treatment.	4.4	56 (57%)	27 (28%)	8 (8%)	7 (7%)	0 (0%)
6. The provider(s) at the clinic made it easy for me to ask questions or raise concerns.	4.5	61 (63%)	26 (27%)	9 (9%)	2 (2%)	0 (0%)
7. I am satisfied with the amount of information about my child's diagnosis that was given to me by the clinic.	4.1	45 (46%)	31 (32%)	11 (11%)	9 (9%)	2 (2%)
8. My child behaved in his/her usual way during the evaluation, showing his/her usual behaviors and skills.	3.9	39 (40%)	31 (32%)	16 (16%)	7 (7%)	5 (5%)
9. My child's daily routine was typical on the day of the evaluation (eg, no unusual changes in sleep or eating, etc).	4.2	52 (53%)	30 (31%)	5 (4%)	9 (9%)	2 (2%)

Item 10 rated on Likert scale (5 = not stressed at all; 1 = extremely stressed); n (%)
10. Rate how much stress you felt overall during your experience at the clinic.	1.88	47 (48%)	28 (29%)	12 (12%)	10 (10%)	1 (1%)

Item 11 rated on Likert scale (5 = completely satisfied at all; 1 = not at all satisfied); n (%)
11. Rate how satisfied you were overall with the clinic diagnostic process.	4.1	44 (45%)	23 (24%)	22 (22%)	6 (6%)	3 (3%)

Domain	Element	N = 40	Example quote
Evaluation process	Prior to evaluation	2	“Tell families how long to expect to be there and give families tips to better prepare child for change in routine.”
			“Get better people on the phones and at the check-in counter.”
	Evaluation environment	4	"A bigger room to play around, the space was not enough and a child typically a 2 year old will feel the sense of clinic and not a place he would usually play.”
			“We were in the same small medical room for almost 3 hours - a bigger area that less “clinical” would be helpful for evaluations.”
	Building rapport	9	“Dr x did a wonderful job answering any and all questions and concerns, explaining the situation and findings to me while maintaining a relaxed professional relationship. She also was very patient with my son.”
			“You could spend a little more time getting to know the child all the way.”
	Diagnosis/feedback	10	“How severe the diagnosis is and to talk more about it.”
			“Explain what it means to be on the spectrum. Tell me what I need to do to help my kid develop - not just therapy to send him to, but how do I need to interact with him differently on a personal level. What should I expect to be different in his behavior from kids who aren't on the spectrum and what is the best way to teach him? For example: my pediatrician (prior to diagnosis) said that if he throws a tantrum, I need to completely ignore him. Is that still true for a kid on the spectrum? How do I need to change my parenting?”
Postevaluation	Recommendations for next steps	2	“Maybe have handouts printed for parents to take home to do more research on their own once they know what to look for. This might mean you print different packets and have them ready.”
			“More discussion about next steps after the diagnosis. And treatment options.”
	Follow-up	2	“Follow-up (1-2 weeks post) with nurse or doc to provide additional guidance after some time to digest all that was handed to us that day. “
			“Somebody was supposed to follow up with me with additional information on developmental preschool and that never happened.”
Overall satisfaction	Positive	11	“Thinking about the possibility of your child being autistic is very emotional, which makes the process of the diagnosis very stressful. The staff at the clinic made it as comfortable and consoling as they could.”
			“To be honest, in a difficult situation, they made it a very pleasant experience. Was very helpful, very kind and informative.”
			“Dr x did a wonderful job answering any and all questions and concerns, explaining situation and findings to me while maintaining a relaxed professional relationship. She also was very patient with my son. Dr x covered all options and went out of her way to provide extra resource help for our family. Thank you Dr X.”

Footnotes

Abbreviations

Acknowledgments

We are grateful to Mary Delaney for her work as a practice liaison for the Indiana University School of Medicine Early Autism Evaluation Hub leadership team. We are indebted to the Early Autism Evaluation Hub clinicians for their commitment to the children and families of Indiana, our learning collaborative network, and their participation in this research. Finally, and most importantly, we are grateful to the children and caregivers who participated in this study.

Authors’ Contributions

All authors have made substantial intellectual contributions to this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval

The University Ethics Review Committee at Indiana University approved our study (approval: 1806262614) on August 19, 2020. Caregiver participants gave written consent for review and signature before starting the research evaluation.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by NIMH R21MH121747 (McNally Keehn, Keehn, Swigonski); UL1TR002529-Indiana Clinical and Translational Sciences, and K12TR004415 (RMK). Additional support was provided by Riley Children's Foundation and HRSA T73MC00015 (AMM).

ORCID iD

Ann Marie Martin

Statement of Human and Animal Rights

Not applicable.

Statement of Informed Consent

Not applicable.

Supplemental Material

Supplemental material for this article is available online.

References

Maenner

Warren

Williams

, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2020. MMWR Surveill Summ. 2023;72(2):1‐14.

Tanner

Dounavi

. The emergence of autism symptoms prior to 18 months of age: a systematic literature review. J Autism Dev Disord. 2021;51(3):973‐993.

Pierce

Gazestani

Bacon

, et al. Evaluation of the diagnostic stability of the early autism spectrum disorder phenotype in the general population starting at 12 months. JAMA Pediatr. 2019;173(6):578‐587.

Zwaigenbaum

Penner

. Autism spectrum disorder: advances in diagnosis and evaluation. Br Med J. 2018;361.

Liptak

Benzoni

Mruzek

, et al. Disparities in diagnosis and access to health services for children with autism: data from the National Survey of Children's Health. J Dev Behav Pediatr. 2008;29(3):152‐160.

Mandell

Wiggins

Carpenter

, et al. Racial/ethnic disparities in the identification of children with autism spectrum disorders. Am J Public Health. 2009;99(3):493‐498.

Durkin

Maenner

Meaney

, et al. Socioeconomic inequality in the prevalence of autism spectrum disorder: evidence from a US cross-sectional study. PLoS One. 2010;5(7):e11551.

Mandell

Novak

Zubritsky

. Factors associated with age of diagnosis among children with autism spectrum disorders. Pediatrics. 2005;116(6):1480‐1486.

Murphy

Ruble

. A comparative study of rurality and urbanicity on access to and satisfaction with services for children with autism spectrum disorders. Rural Spec Educ Q. 2012;31(3):3‐11.

10.

Cashin

Buckley

Trollor

Lennox

. A scoping review of what is known of the physical health of adults with autism spectrum disorder. J Intellect Disabil. 2018;22(1):96‐108.

11.

Penner

Rayar

Bashir

Roberts

Hancock-Howard

Coyte

. Cost-effectiveness analysis comparing pre-diagnosis autism spectrum disorder (ASD)-targeted intervention with Ontario’s autism intervention program. J Autism Dev Disord. 2015;45(9):2833‐2847.

12.

Sohl

Rynkiewicz

Nanclares-Nogués

, et al. Project extension for community health outcomes (ECHO) autism: a successful model to increase capacity in community-based care. Brain Sci. 2022;12(3):327.

13.

Miller

Gabrielsen

Villalobos

, et al. The each child study: systematic screening for autism spectrum disorders in a pediatric setting. Pediatrics. 2011;127(5):866‐871.

14.

Wieckowski

Zuckerman

Broder-Fingert

Robins

. Addressing current barriers to autism diagnoses through a tiered diagnostic approach involving pediatric primary care providers. Autism Res. 2022;15(12):2216‐2222. doi:https://doi.org/10.1002/aur.2832

15.

Hine

Herrington

Rothman

, et al. Embedding autism spectrum disorder diagnosis within the medical home: decreasing wait times through streamlined assessment. J Autism Dev Disord. 2018;48(8):2846‐2853. doi:https://doi.org/10.1007/s10803-018-3548-3

16.

Mazurek

Parker

Chan

Kuhlthau

Sohl

. Effectiveness of the extension for community health outcomes model as applied to primary care for autism: a partial stepped-wedge randomized clinical trial. JAMA Pediatr. 2020;174(5):e196306.

17.

McNally Keehn

Ciccarelli

Szczepaniak

Tomlin

Lock

Swigonski

. A statewide tiered system for screening and diagnosis of autism spectrum disorder. Pediatrics. 2020;146(2):e20193876.

18.

Swanson

Warren

Stone

Vehorn

Dohrmann

Humberd

. The diagnosis of autism in community pediatric settings: does advanced training facilitate practice change? Autism. 2014;18(5):555‐561. doi:https://doi.org/10.1177/1362361313481507

19.

Warren

Stone

Humberd

. A training model for the diagnosis of autism in community pediatric practice. J Dev Behav Pediatr. 2009;30(5):442‐446. doi:https://doi.org/10.1097/DBP.0b013e3181ba0e4e

20.

Bodenheimer

Sinsky

. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573‐576.

21.

Anderson

Barbara

Feldman

. What patients want: a content analysis of key qualities that influence patient satisfaction. J Med Pract Manag. 2007;22(5):255‐261.

22.

Luk

. Patient satisfaction: concept analysis in the healthcare context. Patient Educ Couns. 2019;102(4):790‐796.

23.

Tung

Chang

. Patient satisfaction with and recommendation of a primary care provider: associations of perceived quality and patient education. Int J Qual Health Care. 2009;21(3):206‐213.

24.

Al-Abri

Al-Balushi

. Patient satisfaction survey as a tool towards quality improvement. Oman Med J. 2014;29(1):3‐7.

25.

Nik Adib

Ibrahim

Ab Rahman

, et al. Predictors of caregivers’ satisfaction with the management of children with autism spectrum disorder: a study at multiple levels of health care. Int J Environ Res Public Health. 2019;16(10):1684.

26.

Crane

Chester

Goddard

Henry

Hill

. Experiences of autism diagnosis: a survey of over 1000 parents in the United Kingdom. Autism. 2016;20(2):153‐62.

27.

Goin-Kochel

Mackintosh

Myers

. How many doctors does it take to make an autism spectrum diagnosis? Autism. 2006;10(5):439‐451.

28.

King

Rosenbaum

Goffin

. Family-centered caregiving and well-being of parents of children with disabilities: linking process with outcome. J Pediatr Psychol. 1999;24(1):41‐53.

29.

Liptak

Orlando

Yingling

, et al. Satisfaction with primary health care received by families of children with developmental disabilities. J Pediatr Health Care. 2006;20(4):245‐252.

30.

Ahlers

Gabrielsen

Ellzey

, et al. A pilot project using pediatricians as initial diagnosticians in multidisciplinary autism evaluations for young children. J Dev Behav Pediatr. 2019;40(1):1‐11.

31.

McNally Keehn

Swigonski

Enneking

, et al. Diagnostic accuracy of primary care clinicians across a statewide system of autism evaluation. Pediatrics. 2023;152(2):e2023061188.

32.

Stone

Ousley

. STAT User’s Manual. Treatment and Research Institute for Autism Spectrum Disorders (TRIAD). 2008.

33.

Juárez

Weitlauf

Nicholson

, et al. Early identification of ASD through telemedicine: potential value for underserved populations. J Autism Dev Disord. 2018;48(8):2601‐2610.

34.

Charmaz

. Constructing grounded theory: a practical guide through qualitative analysis. Sage; 2006.

35.

Zwaigenbaum

Warren

. Commentary: embracing innovation is necessary to improve assessment and care for individuals with ASD: a reflection on Kanne and Bishop (2020). J Child Psychol Psychiatry. 2021;62(2):143‐145.

36.

DesChamps

Ibañez

Edmunds

Dick

Stone

. Parenting stress in caregivers of young children with ASD concerns prior to a formal diagnosis. Autism Res. 2020;13(1):82‐92.

37.

Fisher

Lynch

. Differences between black and white caregivers in the association between autism diagnostic process satisfaction and service use. J Autism Dev Disord. 2024:1‐3.

38.

Burden

. “You’ve actually got two patients, you haven’t got one”: parent-informed criteria for evaluating the autism assessment process. Adv Autism. 2024;10(1):12‐24.

39.

Jashar

Fein

Berry

, et al. Parental perceptions of a comprehensive diagnostic evaluation for toddlers at risk for autism spectrum disorder. J Autism Dev Disord. 2019;49:1763‐1777.

40.

Brinster

Brukilacchio

Fikki-Urbanovsky

Shahidullah

Ravenscroft

. Improving efficiency and equity in early autism evaluations: the (S) TAAR model. J Autism Dev Disord. 2023;53(1):275‐284.

41.

Berkowitz

. The patient experience and patient satisfaction: measurement of a complex dynamic. Online J Issues Nurs. 2016;21(1):E1-E8.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

Items 1-9 rated on Likert scale (5 = strongly agree; 1 = strongly disagree); n (%)
	Mean	Strongly agree (5)	Agree (4)	Neither agree nor disagree (3)	Disagree (2)	Strongly disagree (1)
1. The provider(s) at the clinic spent enough time with my child and me.	4.4	54 (55%)	31 (32%)	9 (9%)	4 (4%)	0 (0%)
2. The provider(s) at the clinic listened carefully to me.	4.6	68 (69%)	22 (22%)	4 (4%)	4 (4%)	0 (0%)
3. The provider(s) at the clinic were sensitive to my family's values and customs.	4.6	67 (68%)	26 (27%)	4 (4%)	1 (1%)	0 (0%)
4. The provider(s) at the clinic made me feel like a partner in my child's care.	4.5	63 (64%)	26 (27%)	5 (5%)	4 (4%)	0 (0%)
5. The provider(s) at the clinic discussed with me the range of options to consider for my child's treatment.	4.4	56 (57%)	27 (28%)	8 (8%)	7 (7%)	0 (0%)
6. The provider(s) at the clinic made it easy for me to ask questions or raise concerns.	4.5	61 (63%)	26 (27%)	9 (9%)	2 (2%)	0 (0%)
7. I am satisfied with the amount of information about my child's diagnosis that was given to me by the clinic.	4.1	45 (46%)	31 (32%)	11 (11%)	9 (9%)	2 (2%)
8. My child behaved in his/her usual way during the evaluation, showing his/her usual behaviors and skills.	3.9	39 (40%)	31 (32%)	16 (16%)	7 (7%)	5 (5%)
9. My child's daily routine was typical on the day of the evaluation (eg, no unusual changes in sleep or eating, etc).	4.2	52 (53%)	30 (31%)	5 (4%)	9 (9%)	2 (2%)

Item 10 rated on Likert scale (5 = not stressed at all; 1 = extremely stressed); n (%)
	Mean	Not at all stressed (1)	Slightly stressed (2)	Moderately stressed (3)	Very stressed (4)	Extremely stressed (5)
10. Rate how much stress you felt overall during your experience at the clinic.	1.88	47 (48%)	28 (29%)	12 (12%)	10 (10%)	1 (1%)

Item 11 rated on Likert scale (5 = completely satisfied at all; 1 = not at all satisfied); n (%)
	Mean	Completely satisfied (5)	Very satisfied (4)	Moderately satisfied (3)	Slightly satisfied (2)	Not at all satisfied (1)
11. Rate how satisfied you were overall with the clinic diagnostic process.	4.1	44 (45%)	23 (24%)	22 (22%)	6 (6%)	3 (3%)