Online platforms represent a cost-effective option for data collection; however, it is unclear whether online administration of certain kinds of tasks (e.g., behavioral measures of aggression) poses validity threats. The present study provided a preliminary examination of effort (as indexed by total number of presses), differential drop-out, and believability of an online version of the single-session point subtraction aggression paradigm (PSAP). Two subsamples of participants were recruited; a sample recruited through Amazon’s Mechanical Turk (n = 758) and an in-person undergraduate sample (n = 88). All participants completed the PSAP, along with measures of trait hostility and state anger. The online sample did not differ from the in-person sample on effort (i.e., total number of presses), and did not find the task less believable. Higher scores on state anger were associated with lower likelihood of beginning the online PSAP, but were not associated with prematurely closing the task. State anger was related to aggressive responding on the PSAP. Limitations of the online PSAP and considerations for future research are discussed.
AndersonC. A.BushmanB. J. (1997). External validity of “trivial” experiments: The case of laboratory aggression. Review of General Psychology, 1(1), 19-41. https://doi.org/10.1037/1089-2680.1.1.19
2.
BlackG.ForresterA.WilksM.RiazM.MaguireH.CarlinP. (2011). Using initiative to provide clinical intervention groups in prison: A process evaluation. International Review of Psychiatry, 23(1), 70-76. https://doi.org/10.3109/09540261.2010.544293
3.
BuhrmesterM.KwangT.GoslingS. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science, 6(1), 3-5. https://doi.org/10.1037/e527772014-223
4.
CherekD. R.MoellerF. G.SchnappW.DoughertyD. M. (1997). Studies of violent and nonviolent male parolees: I. Laboratory and psychometric measurements of aggression. Biological Psychiatry, 41(5), 514-522. https://doi.org/10.1016/S0006-3223(96)00059-5
5.
Cota-McKinleyA. L.WoodyW. D.BellP. A. (2001). Vengeance: Effects of gender, age, and religious background. Aggressive Behavior, 27(5), 343-350. https://doi.org/10.1002/ab.1019
6.
DennisS. A.GoodsonB. M.PearsonC. (2018). MTurk workers’ use of low-cost “virtual private servers” to circumvent screening methods: A research note. Behavioral Research in Accounting, 2(3). https://doi.org/10.2139/ssrn.3233954
7.
DeWallC. N.FinkelE. J.LambertN. M.SlotterE. B.BodenhausenG. V.PondR. S.Jr.RenzettiC. M.FinchamF. D. (2013). The voodoo doll task: Introducing and validating a novel method for studying aggressive inclinations. Aggressive behavior, 39(6), 419-439. https://doi.org/10.1002/ab.21496
8.
GenioleS. N.MacDonellE. T.McCormickC. M. (2017). The Point Subtraction Aggression Paradigm as a laboratory tool for investigating the neuroendocrinology of aggression and competition. Hormones and Behavior, 92, 103-116. https://doi.org/10.1016/j.yhbeh.2016.04.006
9.
GiancolaP. R.ChermackS. T. (1998). Construct validity of laboratory aggression paradigms: A response to Tedeschi and Quigley (1996). Aggression and Violent Behavior, 3(3), 237-253. https://doi.org/10.1016/S1359-1789(97)00004-9
GolombB. A.Cortez-PerezM.JaworskiB. A.MednickS.DimsdaleJ. (2007). Point subtraction aggression paradigm: Validity of a brief schedule of use. Violence and Victims, 22(1), 95-103. https://doi.org/10.1891/vv-v22i1a006
12.
GreitemeyerT.SagioglouC. (2016). Subjective socioeconomic status causes aggression: A test of the theory of social deprivation. Journal of Personality and Social Psychology, 111(2), 178-194. https://doi.org/10.1037/pspi0000058
13.
HauserD. J.SchwarzN. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400-407. https://doi.org/10.3758/s13428-015-0578-z
14.
JASP Team. (2019). JASP (Version 0.11.1) [Computer software]. Author.
15.
KennedyR.CliffordS.BurleighT.WaggonerP.JewellR. (2020). The shape of and solutions to the MTurk quality crisis. Political Science Research and Method, 8(4), 614-629. https://doi.org/10.1017/psrm.2020.6
16.
KruegerR. F.DerringerJ.MarkonK. E.WatsonD.SkodolA. E. (2012). Initial construction of a maladaptive personality trait model and inventory forDSM-5. Psychological Medicine, 42(9), 1879-1890. https://doi.org/10.1017/S0033291711002674
17.
LiebermanJ. D.SolomonS.GreenbergJ.McGregorH. A. (1999). A hot new way to measure aggression: Hot sauce allocation. Aggressive Behavior: Official Journal of the International Society for Research on Aggression, 25(5), 331-348. https://doi.org/10.1002/(SICI)1098-2337(1999)25:5<331::AID-AB2>3.0.CO;2-1
18.
LutzJ. (2016). The validity of crowdsourcing data in studying anger and aggressive behavior. Social Psychology, 41(1), 38-51. https://doi.org/10.1027/1864-9335/a000256
19.
McKeeM.RoringS.WinterowdC.PorrasC. (2012). The relationship of negative self-schemas and insecure partner attachment styles with anger experience and expression among male batterers. Journal of Interpersonal Violence, 27(13), 2685-2702. https://doi.org/10.1177/0886260512436395
20.
NewA. S.HazlettE. A.NewmarkR. E.ZhangJ.TriebwasserJ.MeyersonD.LazarusS.TrisdorferR.GoldsteinK. E.GoodmanM.KoenigsbergH. W.FloryJ. D.SieverL. J.KoenigsbergH. W. (2009). Laboratory induced aggression: A positron emission tomography study of aggressive individuals with borderline personality disorder. Biological Psychiatry, 66(12), 1107-1114. https://doi.org/10.1016/j.biopsych.2009.07.015
21.
PeduzziP.ConcatoJ.KemperE.HolfordT. R.FeinsteinA. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49(12), 1373-1379. https://doi.org/10.1016/S0895-4356(96)00236-3
22.
Perez-RodriguezM. M.HazlettE. A.RichE. L.RipollL. H.WeinerD. M.SpenceN.GoodmanM.KoenigsbergaH. W.SieverL. J.NewA. S. (2012). Striatal activity in borderline personality disorder with comorbid intermittent explosive disorder: Sex differences. Journal of Psychiatric Research, 46(6), 797-804. https://doi.org/10.1016/j.jpsychires.2012.02.014
23.
SpielbergerC. D. (1999). State-Trait Anger Expression Inventory-II (STAXI-II). PAR.
TaylorS. P. (1967). Aggressive behavior and physiological arousal as a function of provocation and the tendency to inhibit aggression 1. Journal of Personality, 35(2), 297-310. https://doi.org/10.1111/j.1467-6494.1967.tb01430.x
26.
VaidyaJ. G.GrayE. K.HaigJ.WatsonD. (2002). On the temporal stability of personality: Evidence for differential stability and the role of life experiences. Journal of Personality and Social Psychology, 83(6), 1469-1484. https://doi.org/10.1037/0022-3514.83.6.1469
27.
WagenmakersE. J.MoreyR. D.LeeM. D. (2016). Bayesian benefits for the pragmatic researcher. Current Directions in Psychological Science, 25(3), 169-176. https://doi.org/10.1177/0963721416643289
ZeichnerA.FreyF. C.ParrottD. J.ButrynM. F. (1999). Measurement of laboratory aggression: A new response-choice paradigm. Psychological Reports, 85(3_suppl.), 1229-1237. https://doi.org/10.2466/pr0.1999.85.3f.1229
30.
ZhouH.FishbachA. (2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4), 493-504. https://doi.org/10.1037/pspa0000056
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
