The article examines two important aspects of data quality in self-completion surveys of young people, taking advantage of a unique data source: Understanding Society: the United Kingdom Household Longitudinal Study. Young persons aged 10–15 are asked to complete a self-administered paper questionnaire at annual intervals. The number of completed interviews varies over waves from 4,049 to 5,020. Data are also collected from parents, providing important explanatory covariates for our analysis. Stronger parent–child relationship and higher mother’s involvement in education were associated with lower item nonresponse rate and lower inconsistency throughout waves. We also found some evidence for a negative panel conditioning effect with an increase of social desirability bias and measurement errors in the subsequent waves. There was a higher level of inconsistent responses and a higher probability of social desirability bias throughout waves in more sensitive items.
Get full access to this article
View all access options for this article.
References
1.
AlwinD. F.2007. Margins of error: A study of reliability in survey measurement. Hoboken, NJ: Wiley Blackwell.
2.
AmatoP. R.OchiltreeG.. 1997. Interviewing children about their families: A note on data quality. Journal of Marriage and the Family49:669–75.
3.
BailarB. A.1989. Information needs, surveys, and measurement errors. In Panel surveys, eds. KasprzykD.DuncanG.KaltonG.SinghM. P., 1–24. New York: Wiley.
4.
BartelsL. M.1999. Panel effects in the American national election studies. Political Analysis8:1–20.
5.
BeebeT. J.HarrisonP. A.McraeJ. A.JrAndersonR. E.FulkersonJ. A.1998. An evaluation of computer-assisted self-interviews in a school setting. Public Opinion Quarterly62:623–32.
6.
BorgersN.de LeeuwE.HoxJ. P.. 2000. Children as respondents in survey research: Cognitive development and response quality. Bulletin of Sociological Methodology66:60–75.
7.
BorgersN.HoxJ. P.. 2001. Item nonresponse in questionnaire research with children. Journal of Official Statistics17:321–35.
8.
BorgersN.HoxJ. P.SikkelD.. 2003. Response quality in survey research with children and adolescents: The effect of labeled response options and vague quantifiers. International Journal of Public Opinion Research15:83–94.
9.
BorgersN.HoxJ. P.SikkelD.. 2004. Response effects in surveys on children and adolescents: The effect of number of response options, negative wording, and neutral mid-point. Quality & Quantity38:17–33.
10.
BridgeR. G.ReederL. G.KanouseD.KinderD. R.NagyV. T.JuddC. M.. 1977. Interviewing changes attitudes—Sometimes. Public Opinion Quarterly41:56–64.
11.
BuckN.McFallS.. 2012. Understanding society: Design overview. Longitudinal and Life Course Studies3:5–17.
12.
CantorD.1989. Substantive implications of selected operational longitudinal design features: The National Crime Survey as a case study. In Panel surveys, eds. KasprzykD.DuncanG.KaltonG.SinghM. P., 25–51. New York: Wiley.
13.
CantorD.2008. A review and summary of studies on panel conditioning. In Handbook of longitudinal research: Design, measurement and analysis, ed. MenardS., 123–38. Burlington, VT: Academic Press.
14.
CernatA.2015. The impact of mixing modes on reliability in longitudinal studies. Sociological Methods & Research44:427–57.
15.
ChevalierA.HarmonC.SullivanV. O’WalkerI.. 2013. The impact of parental income and education on the schooling of their children. IZA Journal of Labor Economics2:8.
16.
ClausenA.1968. Response validity: Vote report. Public Opinion Quarterly41:56–64.
17.
CohenS. B.BurtV. L.. 1985. Data collection frequency effect in the national medical care expenditure survey. Journal of Economic and Social Measurement13:125–51.
18.
CorderL. S.HorvitzD. G.. 1989. Panel effects in the national medical care utilization and expenditure survey. In Panel surveys, eds. KasprzykD.DuncanG.KaltonG.SinghM. P., 304–18. New York: Wiley.
19.
CrossleyT. F.de BresserJ.DelaneyL.WinterJ.. 2017. Can survey participation alter household saving behaviour?The Economic Journal127:2332–57.
20.
DahlG. B.LochnerL.. 2012. The impact of family income on child achievement: Evidence from the earned income tax credit. American Economic Review102:1927–56.
21.
Davis-KeanP. E.2005. The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment. Journal of Family Psychology19:294–304.
22.
DuanD.AlegriaM.CaninoG.McGuireT.TakeuchiD.. 2007. Survey conditioning in self-reported mental health service use: Randomized comparison of alternative instrument formats. Health Services Research42:890–907.
23.
FendrichM.KimJ. Y. S.. 2001. Multiwave analysis of retest artifact in the national longitudinal survey of youth drug use. Drug and Alcohol Dependence62:239–53.
24.
FendrichM.RosenbaumD. P.. 2003. Recanting of substance use reports in a longitudinal prevention study. Drug and Alcohol Dependence70:241–53.
25.
FisherP.2016. Does repeated measurement improve income data quality?ISER Working Paper No. 2016-1, University of Essex, Colchester, UK.
26.
FuchsM.2005. Children and adolescents as respondents. Experiments on question order, response order, scale effects and the effect of numeric values associated with response options. Journal of Official Statistics21:701–25.
27.
GoswamiH.FoxC.PollockG.. 2016. The current evidence base and future needs in improving children’s well-being across Europe: Is there a case for a comparative longitudinal survey?Child Indicators Research9:371–88.
28.
Halpern-MannersA.WarrenJ. R.TorcheF.. 2014. Panel conditioning in a longitudinal study of illicit behaviours. Public Opinion Quarterly78:565–90.
29.
Halpern-MannersA.WarrenJ. R.TorcheF.. 2017. Panel conditioning in the general social survey. Sociological Methods & Research46:103–24.
30.
HaunbergerS.2014. Item nonresponse in face-to-face interviews with children. Journal of Official Statistics30:459–77.
31.
HavemanR.WolfeB.. 1995. The determinants of children’s attainments: A review of methods and findings. Journal of Economic Literature33:1829–78.
32.
KönigS.2011. Online-Forschung mit Kindern. Berlin, Germany: Budrich UniPress.
33.
KrautR. E.McConahayJ. B.. 1973. How being interviewed affects voting: An experiment. The Public Opinion Quarterly37:398–406.
34.
LynnP.2009. Methods for longitudinal surveys. In Methodology of longitudinal surveys, ed. LynnP., 1–19. Chichester, UK: Wiley.
35.
MannC. B.2005. Unintentional voter mobilization: Does participation in preelection surveys increase voter turnout?Annals of the American Academy of Political and Social Science601:155–68.
36.
MavletovaA.2015a. A gamification effect in longitudinal web surveys among children and adolescents. International Journal of Market Research57:413–38.
37.
MavletovaA.2015b. Web surveys among children and adolescents: Is there a gamification effect?Social Science Computer Review33:372–98.
38.
PiagetJ.1929. Introduction to the child’s conception of the world. New York: Harcourt.
39.
PollockJ. OzanGoswamiH.ReesG.StasulaneA.. 2018. Measuring youth well-being: How a pan-European longitudinal survey can improve policy. Berlin, Germany: Springer.
40.
RendtelU.NordbergL.JänttiM.HanishJ.BasicE.. 2004. Report of quality of income data. Wiesbaden, The Federal Statistical Office: Chintex, Working Paper No. 21.
41.
SchräplerJ.-P.2003. Respondent behaviour in panel studies: A case study for income-nonresponse by means of the British Household Panel Study. Colchester, University of Essex: ISER Working Paper Series No. 2003-08.
42.
ScottJ.2008. Children as respondents: The challenge for quantitative methods. In Research with children: Perspectives and practices, eds. ChristensenP.JamesA., 87–108. New York: Routledge.
43.
SeginerR.1983. Parents’ educational expectations and children’s academic achievements: A literature review. Merrill-Palmer Quarterly29:1–23.
44.
SmithJ. K.GerberA. S.OrlichA.. 2003. Self-prophecy effects and voter turnout: An experimental replication. Political Psychology24:593–604.
45.
SmithK.PlattL.. 2013. How do children answer questions about frequencies and quantities? Evidence from a large-scale field test. CLS Working paper 2013/12, Centre for Longitudinal Studies, London.
46.
SpangenbergE. R.ObermillerC.. 1996. To cheat or not to cheat: Reducing cheating by requesting self-prophecy. Marketing Education Review6:95–103.
47.
SturgisP.AllumN.Brunton-SmithI.. 2009. Attitudes over time: The psychology of panel conditioning. In Methodology of longitudinal surveys, ed. LynnP., 113–26. Chichester, UK: John Wiley & Sons.
48.
ToepoelV.DasM.van SoestA.. 2009. Relating question type to panel conditioning: Comparing trained and fresh respondents. Survey Research Methods3:73–80.
49.
TorcheF.WarrenJ. R.Halpern-MannersA.. 2012. Panel conditioning in a longitudinal study of Chilean adolescents’ substance use: Evidence from an experiment. Social Forces90:891–918.
50.
TourangeauR.RipsL.RasinskiK.. 2004. The psychology of survey response. Cambridge, MA: Cambridge University Press.
51.
VaillancourtP. M.1977. Stability of children’s survey responses. Public Opinion Quarterly37:373–87.
52.
VarniJ. W.LimbersC. A.BurwinkleT. M.. 2007. How young can children reliably and validly self-report their health-related quality of life? An analysis of 8,591 children across age subgroups with the PedsQL™ 4.0 Generic Core Scales. Health and Quality of Life Outcomes5:43. doi: 10.1186/1477-7525-5-1.
53.
VoglS.2012. Children between the age of 5 and 11: What “don’t know” answers tell us. Quality & Quantity46:993–1011.
54.
VoglS.2013. Telephone versus face-to-face interviews: A mode comparison of semi-structured interviews with children. Sociological Methodology43:138–82.
55.
WarrenJ. R.Halpern-MannersA.. 2012. Panel conditioning in longitudinal social science surveys. 2012. Sociological Methods and Research41(4): 491–534.
56.
WatertonJ.LievesleyD.. 1989. Evidence of conditioning effects in the British social attitudes panel survey. In Panel surveys, eds. KasprzykD.DuncanG.KaltonG.SinghM. P., 319–39. New York: Wiley.
57.
WhiteK. R.1982. The relation between socioeconomic status and academic achievement. Psychological Bulletin91:461–81.
58.
YiY.LeeB.. 2016. Adolescents’ characteristics associated with item nonresponse on BMI questions. Health Behavior and Policy Review3:597–606.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
