Self-report questionnaires based on Harter’s response format (“Some kids . . . but other kids . . . ”) are commonly used in developmental and clinical research settings, but the reliability and validity of this format in middle childhood are still under debate. The current study aimed to test the psychometric proprieties of Harter versus Likert response formats as applied to two attachment questionnaires in a sample of 410 Italian children aged 8 to 10 years. Participants completed the Experiences in Close Relationships–Revised Child version (n = 102, 4-point Likert-type scale; n = 104, adapted Harter version) and the Security Scale (n = 95, Harter’s format; n = 109 adapted 4-point Likert version). Results of multigroup confirmatory factor analyses indicated that the two response formats exhibited comparable reliability and factorial validity, although a slight superiority of Harter’s format emerged for the Security Scale. External validity was supported for both answer formats. Implications for developmental theory and practice are discussed.
AchenbachT. M.McConaughyS. H.HowellC. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232. doi:10.1037/0033-2909.101.2.213
2.
BeaujeanA. A. (2014). Latent variable modeling using R: A step-by-step guide. New York, NY: Routledge.
3.
BorgersN.HoxJ. J. (2000, October). Reliability of responses in questionnaire research with children. Paper presented at the Fifth International Conference on Logic and Methodology, Cologne, Germany.
4.
BorgersN.HoxJ. J. (2001). Item non-response in questionnaires with children. Journal of Official Statistics, 17, 321-335.
5.
BorgersN.HoxJ. J.SikkelD. (2004). Response effects in surveys on children and adolescents: The effect of number of response options, negative wording, and neutral midpoint. Quality & Quantity, 38, 17-33.
6.
BorgersN.LeeuwE. D.HoxJ. J. (1999). Surveying children: Cognitive development and response quality in questionnaire research. In ChristiansonA.GustafsonJ. R.KlevmarkenA.RosénB.HanssonK.-G.GranquistL. (Eds.), Official statistics in a changing world (pp. 133-140). Stockholm, Sweden: SCB.
7.
BorgersN.LeeuwE. D.HoxJ. J. (2000). Children as respondents in survey research: Cognitive development and response quality. Bulletin de Méthodologie Sociologique, 66, 60-75. doi:10.1177/075910630006600106
8.
BosmansG.KernsK. A. (2015). Attachment in middle childhood: Progress and prospects. In BosmansG.KernsK. A. (Eds.), Attachment in middle childhood: Theoretical advances and new directions in an emerging field: New directions for child and adolescent development (pp. 1-14). Hoboken, NJ: Wiley.
9.
BoyceW.TorsheimT.CurrieC.ZambonA. (2006). The Family Affluence Scale as a measure of national wealth: Validation of an adolescent self-report measure. Social Indicators Research, 78, 473-487. doi:10.1007/s11205-005-1607-6
10.
BrennanK. A.ClarkC. L.ShaverP. R. (1998). Self-report measurement of adult attachment: An integrative overview. In SimpsonJ. A.RholesW. S. (Eds.), Attachment theory and close relationships (pp. 46-76). New York, NY: Guilford Press.
11.
BrenningK.SoenensB.BraetC.BosmansG. (2011). An adaptation of the experiences in Close Relationships Scale-Revised for use with children and adolescents. Journal of Social and Personal Relationships, 28, 1048-1072. doi:10.1177/0265407511402418
12.
BrenningK.Van PetegemS.VanhalstJ.SoenensB. (2014). The psychometric qualities of a short version of the Experiences in Close Relationships Scale–Revised Child version. Personality and Individual Differences, 68, 118-123. doi:10.1016/j.paid.2014.04.005
13.
BrethertonI. (1987). New perspective on attachment relations: Security, communication and internal working models. In OsofskyJ. (Ed.), Handbook of Infant Development (pp. 1061-1100). New York: Wiley.
14.
BrownT. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
15.
BrumariuL. E.MadiganS.GiusepponeK. R.AbtahiM. M.KernsK. A. (2018): The Security Scale as a measure of attachment: Meta-analytic evidence of validity. Attachment & Human Development, 20, 600-625. doi:10.1080/14616734.2018.1433217
16.
Bureau of Labor Statistics. (2005). National Longitudinal Survey of Youth 1997 Cohort, 1997–2003. Retrieved from https://www.bls.gov/nls/nlsy97.htm
17.
BusoneraA.San MartiniP.ZavattiniG. C.SantonaA. (2014). Psychometric properties of an Italian version of the Experiences in Close Relationships-Revised (ECR–R) Scale. Psychological Reports, 114, 785-801. doi:10.2466/03.21.PR0.114k23w9
18.
CattellR. B.CattellA. K. S. (1981). Misurare l’intelligenza con i test “Culture Fair.”Florence, Italy: Giunti O.S.
19.
ChambersC. T.CraigK. D. (1998). An intrusive impact of anchors in children’s faces pain scales. Pain, 78, 27-37. doi:10.1016/S0304-3959(98)00112-2
20.
ChambersC. T.JohnstonC. (2002). Developmental differences in children’s use of rating scales. Journal of Pediatric Psychology, 27, 27-36. doi:10.1093/jpepsy/27.1.27
21.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233-255. doi:10.1207/S15328007SEM0902_5
22.
ColeC.LoftusE. (1987). The memory of children. In CeciS.TogliaM.RossD. (Eds.), Children’s eyewitness memory (pp. 178-208). New York, NY: Springer.
23.
CrandallV. C.CrandallV. J.KatkovskyW. (1965). A children’s social desirability questionnaire. Journal of Consulting Psychology, 29, 27-36.
24.
DanielsonC. K.PhelpsC. R. (2003). The assessment of children’s social skills through self-report: A potential screening instrument for classroom use. Measurement and Evaluation in Counseling and Development, 35, 218-229.
25.
De LeeuwE.BorgersN.SmitsA. (2004). Pretesting questionnaires for children and adolescents. In PresserS.RothgebJ. M.CouperM. P.LesslerJ. T.MartinE.MartinJ.SingerE. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 409-429). Hoboken, NJ: Wiley.
26.
De LeeuwE.OtterM. (1995). The reliability of children’s responses to questionnaire items: Question effects in children’s questionnaire data. In HoxJ.van der MeulenB.KanssensJ.ter LaakJ.TavecchioL. (Eds.), Advances in family research (pp. 251-257). Amsterdam, Netherlands: Thesis Publishers.
27.
FinneganR. A.HodgesE. V. E.PerryD. G. (1996). Preoccupied and avoidant coping during middle childhood. Child Development, 67, 1318-1328. doi:10.1111/j.1467-8624.1996.tb01798.x
28.
FraleyR. C.WallerN. G.BrennanK. A. (2000). An item-response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350-365. doi:10.1037/0022-3514.78.2.350
29.
FuchsM. (2005). Children and adolescents as respondents: Experiments on question order, response order, scale effects and the effect of numeric values associated with response options. Journal of Official Statistics, 21, 701-725.
30.
FuchsM. (2009). The reliability of children’s survey responses: The impact of cognitive functioning on respondent behavior. In Statistics Canada (Ed.), Symposium 2008: Data collection: Challenges, achievements and new directions (pp. 1-8). Ottawa, Ontario, Canada: Statistics Canada.
31.
GadermannA. M.GuhnM.ZumboB. D. (2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation, 17, 1-13. https://pareonline.net/pdf/v17n3.pdf
32.
GelmanR.BaillargeonR. (1983). A review of some Piagetian concepts. In FlavellJ. H.MarkmanM. (Eds.), Handbook of child psychology (Vol. 3): Cognitive development (pp. 167-230). New York, NY: Wiley.
33.
GoodmanR. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581-586.
34.
GrohA. M.FearonR. P.Bakermans-KranenburgM. Jvan IjzendoornM. H.SteeleR. D.RoismanG. I. (2014). The significance of attachment security for children’s social competence with peers: A meta-analytic study. Attachment & Human Development, 16, 103-136. doi:10.1080/14616734.2014.883636
35.
GrossJ. T.SternJ. A.BrettB. E.CassidyJ. (2017). The multifaceted nature of prosocial behavior in children: Links with attachment theory and research. Social Development, 26, 661-678. doi:10.1111/sode.12242
36.
GulloneE.TaffeJ. (2012). The Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA): A psychometric evaluation. Psychological Assessment, 24, 409-417.
HarterS. (2012). Manual for the Self-Perception Profile for Children, Denver, CO: University of Denver Press. (Original work published 1985)
39.
HartleyJ.BettsL. (2010). Four layouts and a finding: The effects of changes in the order of the verbal labels and the numerical values on Likert-type scale scores. International Journal of Social Research Methodology, 13, 17-27. doi:10.1080/13645570802648077
40.
HoladayB.Turner-HensonA. (1989). Response effects in surveys with school-age children. Nursing Research, 38, 248-250. doi:10.1097/00006199-198907000-00019
41.
InhelderB.PiagetJ. (1958). The growth of logical thinking from childhood to adolescence. London, England: Routledge.
KernsK. A.KlepacL.ColeA. (1996). Peer relationships and preadolescents’ perceptions of security in the child-mother relationship. Developmental Psychology, 32, 457-466. doi:10.1037/0012-1649.32.3.457
44.
LagattutaK. H.SayfanL.BamfordC. (2012). Do you know how I feel? Parents underestimate worry and overestimate optimism compared to child self-report. Journal of Experimental Child Psychology, 113, 211-232. doi:10.1016/j.jecp.2012.04.001
45.
LauK.-L.LeeJ. C. K. (2008). Validation of a Chinese goal orientation questionnaire. British Journal of Educational Psychology, 78, 331-353. doi:10.1111/j.2044-8279.2008.tb00486.x
46.
LernerR. M.von EyeA.LernerJ.Lewin-BizanS.BowersE. (2010). Special issue introduction: The meaning and measurement of thriving: A view of the issues. Journal of Youth and Adolescence, 39, 707-719. doi:10.1007/s10964-010-9531-8
47.
LikertR. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, 1-55.
48.
LissitzR. W.GreenS. B. (1975). Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 60, 10-13. doi:10.1037/h0076268
49.
MadiganS.BrumariuL. E.VillaniV.AtkinsonL.Lyons-RuthK. (2016). Representational and questionnaire measures of attachment: A meta-analysis of relations to child internalizing and externalizing problems. Psychological Bulletin, 142, 367-399.
50.
MarciT.LionettiF.MoscardinoU.PastoreM.CalvoV.AltoéG. (2018). Measuring attachment security via the Security Scale: Latent structure, invariance across mothers and fathers and convergent validity. European Journal of Developmental Psychology, 15, 481-492. doi:10.1080/17405629.2017.1317632
51.
MarciT.MoscardinoU.AltoéG. (2019). The brief Experience in Close Relationships Scale-Revised Child Version (ECR-RC): Factor structure and invariance across middle childhood and early adolescence. International Journal of Behavioral Development, 43, 409-423. doi:10.1177/0165025418785975
52.
MarshH. W. (1986). Negative item bias in rating scales for pre-adolescent children. Developmental Psychology, 22, 37-49. doi:10.1037/0012-1649.22.1.37
53.
MuthénL. K.MuthénB. O. (2001). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén.
54.
PantellR. H.LewisC. C. (1987). Measuring the impact of medical care on children. Journal of Chronic Diseases, 40(Suppl. 1), 99S-108S. doi:10.1016/S0021-9681(87)80039-5
55.
R Development Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
56.
RaikesH. A.ThompsonR. A. (2005). Relationships past, present, and future: Reflections on attachment in middle childhood. In KernsK. A.RichardsonR. A. (Eds.), Attachment in middle childhood (pp. 255-282). New York, NY: Guilford Press.
57.
RhemtullaM.Brosseau-LiardP. É.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354-373. doi:10.1037/a0029315
58.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. doi:10.18637/jss.v048.i02
59.
SaywitzK. (1987). Children’s testimony: Age-related patterns of memory errors. In CeciS.TogliaM.RossD. (Eds.), Children’s eyewitness memory (pp. 36-52). New York, NY: Springer.
60.
Schermelleh-EngelK.MoosbruggerH.MüllerH. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8, 23-74.
61.
ScottJ. (1997). Children as respondents: Methods for improving data quality. In LybergL. (Ed.), Survey measurements and process quality (pp. 331-352). New York, NY: Wiley.
62.
SelmanR. (1980). The growth of interpersonal understanding. San Diego, CA: Academic Press.
63.
ShulrufB.HattieJ.DixonR. (2008). Factors affecting responses to Likert type questionnaires: Introduction of the ImpExp, a new comprehensive model. Social Psychology of Education, 11, 59-78.
64.
SijtsmaK. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107-120. doi:10.1007/S11336-008-9101-0
65.
SkoczenI.GłogowskaK.KamzaA.WłodarczykA. (2019). The Experiences in Close Relationships Scale-Revised Child Version: Reliability and structural validity of the Polish adaptation. European Journal of Developmental Psychology, 16, 488-500. doi:10.1080/17405629.2017.1422719
66.
SroufeL. A. (2005). Attachment and development: A prospective, longitudinal study from birth to adulthood. Attachment & Human Development, 7, 349-367. doi:10.1080/14616730500365928
67.
SturgessJ.RodgerS.OzanneA. (2002). A Review of the Use of Self-Report Assessment with Young Children. British Journal of Occupational Therapy, 65, 108-116. doi:10.1177/030802260206500302
68.
SturgessJ.ZivianiJ. (1996). A self-report play skills questionnaire: Technical development. Australian Occupational Therapy Journal, 43, 142-154. doi:10.1111/j.1440-1630.1996.tb01850.x
69.
TobiaV.MarzocchiG. M. (2017). The Strengths and Difficulties Questionnaire-Parents for Italian school-aged children: Psychometric properties and norms. Child Psychiatry & Human Development, 49(1), 1-8. doi:10.1007/s10578-017-0723-2
70.
VienoA.SantinelloM.LenziM.BaldassariD.MirandolaM. (2009). Health status in immigrants and native early adolescents in Italy. Journal of Community Health, 34, 181-187. doi:10.1007/s10900-008-9144-2
71.
WatersH. S.WatersE. (2006). The attachment working models concept: Among other things, we build script-like representations of secure base experiences. Attachment & Human Development, 8, 185-197, doi:10.1080/14616730600856016
72.
WrightK. D.AsmundsonG. J. G. (2003). Health anxiety in children: Development and psychometric properties of the Childhood Illness Attitude Scales. Cognitive Behaviour Therapy, 34, 194-201. doi:10.1080/16506070310014691
73.
YeagerD. S.KrosnickJ. A. (2011). Does mentioning “some people” and “other people” in a survey question increase the accuracy of adolescents’ self-reports?Developmental Psychology, 47, 1674-1679. doi:10.1037/a0025440
74.
YeagerD. S.KrosnickJ. A. (2012). Does mentioning “some people” and “other people” in an opinion question improve measurement quality?Public Opinion Quarterly, 76, 131-141. doi:10.1093/poq/nfr066
75.
ZinbargR. E.RevelleW.YovelI.LiW. (2005). Cronbach’s α, Revelle’s β, and Mcdonald’s ωh: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. doi:10.1007/s11336-003-0974-7
76.
ZumboB. D.GadermannA. M.ZeisserC. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6, 21-29. doi:10.22237/jmasm/1177992180
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
