This study investigated inconsistent responding to survey items by participants involved in longitudinal, web-based substance use research. We also examined cross-sectional and prospective predictors of inconsistent responding. Middle school (N = 1,023) and college students (N = 995) from multiple sites in the United States responded to online surveys assessing substance use and related variables in three waves of data collection. We applied a procedure for creating an index of inconsistent responding at each wave that involved identifying pairs of items with considerable redundancy and calculating discrepancies in responses to these items. Inconsistent responding was generally low in the Middle School sample and moderate in the College sample, with individuals showing only modest stability in inconsistent responding over time. Multiple regression analyses identified several baseline variables—including demographic, personality, and behavioral variables—that were uniquely associated with inconsistent responding both cross-sectionally and prospectively. Alcohol and substance involvement showed some bivariate associations with inconsistent responding, but these associations largely were accounted for by other factors. The results suggest that high levels of carelessness or inconsistency do not appear to characterize participants’ responses to longitudinal web-based surveys of substance use and support the use of inconsistency indices as a tool for identifying potentially problematic responders.
AunolaK.StattinH.NurmiJ.-E. (2000). Adolescents’ achievement strategies, school adjustment, and externalizing and internalizing problem behaviors. Journal of Youth and Adolescence, 29, 289-306.
2.
AytaclarS.TarterR. E.KirisciL.LuS. (1999). Association between hyperactivity and executive cognitive functioning in childhood and substance use in early adolescence. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 172-178.
3.
BachmanJ. G.JohnstonL. D.O’MalleyP. M. (2005). Monitoring the future: Questionnaire responses from the nation’s high school seniors, 2004. Ann Arbor, MI: Institute for Social Research.
4.
BarnesG. M.HoffmanJ. H.WelteJ. W.FarrellM. P.DintcheffB. A. (2006). Effects of parental monitoring and peer deviance on substance use and delinquency. Journal of Marriage and Family, 68, 1084-1104.
5.
BlanchardE. B.Jones-AlexanderJ.BuckleyT. C.FornerisC. A. (1996). Psychometric properties of the PTSD Checklist (PCL). Behaviour Research and Therapy, 34, 669-673.
6.
BoboJ. K.GilchristL. D.ElmerJ. F.SnowW. H.SchinkeS. P. (1986). Hassles, role strain, and peer relations in young adolescents. Journal of Early Adolescence, 6, 339-352.
7.
BryantA. L.SchulenbergJ. E.O’MalleyP. M.BachmanJ. G.JohnstonL. D. (2003). How academic achievement, attitudes, and behaviors relate to the course of substance use during adolescence: A 6-year, multiwave national longitudinal study. Journal of Research on Adolescence, 13, 361-397.
8.
BuchananT.JohnsonJ. A.GoldbergL. R. (2005). Implementing a five-factor personality inventory for use on the internet. European Journal of Psychological Assessment, 21, 115-127.
9.
ButcherJ.DahlstromW.GrahamJ.TellegenA.KaemmerB. (1989). Manual for administration and scoring: MMPI-2. Minneapolis: University of Minnesota Press.
10.
ChassinL.PressonC. C.ToddM.RoseJ. S.ShermanS. J. (1998). Maternal socialization of adolescent smoking: The intergenerational transmission of parenting and smoking. Developmental Psychology, 34, 1189-1201.
11.
ColderC. R.ChassinL. (1993). The stress and negative affect model of adolescent alcohol use and the moderating effects of behavioral undercontrol. Journal of Studies on Alcohol and Drugs, 54, 326-333.
12.
CollinsR. L.ParksG. A.MarlattG. A. (1985). Social determinants of alcohol consumption: The effects of social interaction and model status on the self-administration of alcohol. Journal of Consulting and Clinical Psychology, 53, 189-200.
13.
FarrellA. D.DanishS. J.HowardC. W. (1992). Relationship between drug use and other problem behaviors in urban adolescents. Journal of Consulting and Clinical Psychology, 60, 705-712.
14.
GiancolaP. R.MezzichA. C.TarterR. E. (1998). Disruptive, delinquent and aggressive behavior in female adolescents with a psychoactive substance use disorder: Relation to executive cognitive functioning. Journal of Studies on Alcohol and Drugs, 59, 560-567.
15.
GoslingS. D.VazireS.SrivastavaS.JohnO. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59, 93-104.
16.
HuangJ. L.CurranP. G.KeeneyJ.PoposkiE. M.DeShonR. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27, 99-114.
17.
JohnO. P.SrivastavaS. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In PervinL. A.JohnO. P. (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 102-138). New York, NY: Guilford Press.
18.
JohnsonJ. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39, 103-129.
19.
JohnstonL. D.O’MalleyP. M.BachmanJ. G.SchulenbergJ. E. (2012). Monitoring the Future national results on adolescent drug use: Overview of key findings, 2011. Ann Arbor, MI: University of Michigan.
20.
KerrM.StattinH. (2000). What parents know, how they know it, and several forms of adolescent adjustment: Further support for a reinterpretation of monitoring. Developmental Psychology, 36, 366-380.
21.
KurtzJ. E.ParrishC. L. (2001). Semantic response consistency and protocol validity in structured personality assessment: The case of the NEO-PI-R. Journal of Personality Assessment, 76, 315-332.
22.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
23.
LaurentJ.CatanzaroS. J.JoinerT. E.Jr.RudolphK. D.PotterK. I.LambertS.. . . GathrightT. (1999). A measure of positive and negative affect for children: Scale development and preliminary validation. Psychological Assessment, 11, 326-338.
24.
LinkM. W.MokdadA. H. (2005). Effects of survey mode on self-reports of adult alcohol consumption: A comparison of mail, web and telephone approaches. Journal of Studies on Alcohol and Drugs, 66, 239-245.
25.
LynamD.SmithG.WhitesideS.CydersM. (2006). The UPPS-P: Assessing five personality pathways to impulsive behavior (Technical Report). Retrieved from http://www1.psych.purdue.edu/~dlynam/uppspage.htm
26.
MarchJ. S.ParkerJ. D.SullivanK.StallingsP.ConnersC. K. (1997). The Multidimensional Anxiety Scale for Children (MASC): Factor structure, reliability, and validity. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 554-565.
27.
McGrathR. E.MitchellM.KimB. H.HoughL. (2010). Evidence for response bias as a source of error variance in applied assessment. Psychological Bulletin, 136, 450-470.
28.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17, 437-455.
29.
MillerE. T.NealD. J.RobertsL. J.BaerJ. S.CresslerS. O.MetrikJ.MarlattG. A. (2002). Test-retest reliability of alcohol measures: Is there a difference between Internet-based assessment and traditional methods?Psychology of Addictive Behaviors, 16, 56-63.
30.
MillerP. G.SønderlundA. L. (2010). Using the internet to research hidden populations of illicit drug users: A review. Addiction, 105, 1557-1567.
31.
NesselroadeJ. R.BaltesP. B. (1979). Longitudinal research in the study of behavior and development. New York, NY: Academic Press.
ReadJ. P.ColderC. R.MerrillJ. E.OuimetteP.WhiteJ.SwartoutA. (2012). Trauma and posttraumatic stress symptoms influence alcohol and other drug problem trajectories in the first year of college. Journal of Consulting and Clinical Psychology, 80, 426-439.
35.
ReadJ. P.FarrowS. M.JaanimägiU.OuimetteP. (2009). Assessing trauma and traumatic stress via the Internet: Measurement equivalence and participant reactions. Traumatology, 15(1), 94-102.
36.
ReadJ. P.KahlerC. W.StrongD. R.ColderC. R. (2006). Development and preliminary validation of the Young Adult Alcohol Consequences Questionnaire. Journal of Studies on Alcohol, 67, 169-177.
37.
ReipsU.-D. (2000). The web experiment method: Advantages, disadvantages, and solutions. In BirnbaumM. H. (Ed.), Psychological experiments on the Internet (pp. 89-117). San Diego, CA: Academic Press.
38.
SaitzR.HelmuthE. D.AromaaS. E.GuardA.BelangerM.RosenbloomD. L. (2004). Web-based screening and brief intervention for the spectrum of alcohol problems. Preventive Medicine, 39, 969-975.
39.
SchellT. L.MartinoS. C.EllicksonP. L.CollinsR. L.McCaffreyD. (2005). Measuring developmental changes in alcohol expectancies. Psychology of Addictive Behaviors, 19, 217-220.
40.
SchinkaJ. A.KinderB. N.KremerT. (1997). Research validity scales for the NEO-PI-R: Development and initial validation. Journal of Personality Assessment, 68, 127-138.
41.
SchmidtW. C. (1997). World-wide web survey research: Benefits, potential problems, and solutions. Behavior Research Methods, 29, 274-279.
42.
SherK. J.RutledgeP. C. (2007). Heavy drinking across the transition to college: Predicting first-semester heavy drinking from precollege variables. Addictive Behaviors, 32, 819-835.
43.
TurnerC. F.KuL.RogersS. M.LindbergL. D.PleckJ. H.SonensteinF. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867-873.
44.
WatsonD.ClarkL. A.TellegenA. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063-1070.
45.
WrightD. L.AquilinoW. S.SuppleA. J. (1998). A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly, 62, 331-353.
46.
WuY.NewfieldS. A. (2007). Comparing data collected by computerized and written surveys for adolescent health research. Journal of School Health, 77, 23-28.
