Using an Online Platform to Administer the Single-Session Point Subtraction Aggression Paradigm: An Initial Examination of Feasibility and Validity

Abstract

Online platforms represent a cost-effective option for data collection; however, it is unclear whether online administration of certain kinds of tasks (e.g., behavioral measures of aggression) poses validity threats. The present study provided a preliminary examination of effort (as indexed by total number of presses), differential drop-out, and believability of an online version of the single-session point subtraction aggression paradigm (PSAP). Two subsamples of participants were recruited; a sample recruited through Amazon’s Mechanical Turk (n = 758) and an in-person undergraduate sample (n = 88). All participants completed the PSAP, along with measures of trait hostility and state anger. The online sample did not differ from the in-person sample on effort (i.e., total number of presses), and did not find the task less believable. Higher scores on state anger were associated with lower likelihood of beginning the online PSAP, but were not associated with prematurely closing the task. State anger was related to aggressive responding on the PSAP. Limitations of the online PSAP and considerations for future research are discussed.

Keywords

Mechanical Turk MTurk point subtraction aggression paradigm validity aggression

Get full access to this article

View all access options for this article.

References

Anderson

C. A.

Bushman

B. J.

(1997). External validity of “trivial” experiments: The case of laboratory aggression. Review of General Psychology, 1(1), 19-41. https://doi.org/10.1037/1089-2680.1.1.19

Black

Forrester

Wilks

Riaz

Maguire

Carlin

(2011). Using initiative to provide clinical intervention groups in prison: A process evaluation. International Review of Psychiatry, 23(1), 70-76. https://doi.org/10.3109/09540261.2010.544293

Buhrmester

Kwang

Gosling

S. D.

(2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3-5. https://doi.org/10.1037/e527772014-223

Cherek

D. R.

Moeller

F. G.

Schnapp

Dougherty

D. M.

(1997). Studies of violent and nonviolent male parolees: I. Laboratory and psychometric measurements of aggression. Biological Psychiatry, 41(5), 514-522. https://doi.org/10.1016/S0006-3223(96)00059-5

Cota-McKinley

A. L.

Woody

W. D.

Bell

P. A.

(2001). Vengeance: Effects of gender, age, and religious background. Aggressive Behavior, 27(5), 343-350. https://doi.org/10.1002/ab.1019

Dennis

S. A.

Goodson

B. M.

Pearson

(2018). MTurk workers’ use of low-cost “virtual private servers” to circumvent screening methods: A research note. Behavioral Research in Accounting, 2(3). https://doi.org/10.2139/ssrn.3233954

DeWall

C. N.

Finkel

E. J.

Lambert

N. M.

Slotter

E. B.

Bodenhausen

G. V.

Pond

R. S.

Jr. Renzetti

C. M.

Fincham

F. D.

(2013). The voodoo doll task: Introducing and validating a novel method for studying aggressive inclinations. Aggressive behavior, 39(6), 419-439. https://doi.org/10.1002/ab.21496

Geniole

S. N.

MacDonell

E. T.

McCormick

C. M.

(2017). The Point Subtraction Aggression Paradigm as a laboratory tool for investigating the neuroendocrinology of aggression and competition. Hormones and Behavior, 92, 103-116. https://doi.org/10.1016/j.yhbeh.2016.04.006

Giancola

P. R.

Chermack

S. T.

(1998). Construct validity of laboratory aggression paradigms: A response to Tedeschi and Quigley (1996). Aggression and Violent Behavior, 3(3), 237-253. https://doi.org/10.1016/S1359-1789(97)00004-9

10.

Glickman

M. E.

van Dyk

D. A.

(2007). Basic Bayesian methods. Methods in Molecular Biology, 404, 319-338. https://doi.org/10.1007/978-1-59745-530-5_16

11.

Golomb

B. A.

Cortez-Perez

Jaworski

B. A.

Mednick

Dimsdale

(2007). Point subtraction aggression paradigm: Validity of a brief schedule of use. Violence and Victims, 22(1), 95-103. https://doi.org/10.1891/vv-v22i1a006

12.

Greitemeyer

Sagioglou

(2016). Subjective socioeconomic status causes aggression: A test of the theory of social deprivation. Journal of Personality and Social Psychology, 111(2), 178-194. https://doi.org/10.1037/pspi0000058

13.

Hauser

D. J.

Schwarz

(2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400-407. https://doi.org/10.3758/s13428-015-0578-z

14.

JASP Team. (2019). JASP (Version 0.11.1) [Computer software]. Author.

15.

Kennedy

Clifford

Burleigh

Waggoner

Jewell

(2020). The shape of and solutions to the MTurk quality crisis. Political Science Research and Method, 8(4), 614-629. https://doi.org/10.1017/psrm.2020.6

16.

Krueger

R. F.

Derringer

Markon

K. E.

Watson

Skodol

A. E.

(2012). Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychological Medicine, 42(9), 1879-1890. https://doi.org/10.1017/S0033291711002674

17.

Lieberman

J. D.

Solomon

Greenberg

McGregor

H. A.

(1999). A hot new way to measure aggression: Hot sauce allocation. Aggressive Behavior: Official Journal of the International Society for Research on Aggression, 25(5), 331-348. https://doi.org/10.1002/(SICI)1098-2337(1999)25:5<331::AID-AB2>3.0.CO;2-1

18.

Lutz

(2016). The validity of crowdsourcing data in studying anger and aggressive behavior. Social Psychology, 41(1), 38-51. https://doi.org/10.1027/1864-9335/a000256

19.

McKee

Roring

Winterowd

Porras

(2012). The relationship of negative self-schemas and insecure partner attachment styles with anger experience and expression among male batterers. Journal of Interpersonal Violence, 27(13), 2685-2702. https://doi.org/10.1177/0886260512436395

20.

New

A. S.

Hazlett

E. A.

Newmark

R. E.

Zhang

Triebwasser

Meyerson

Lazarus

Trisdorfer

Goldstein

K. E.

Goodman

Koenigsberg

H. W.

Flory

J. D.

Siever

L. J.

Koenigsberg

H. W.

(2009). Laboratory induced aggression: A positron emission tomography study of aggressive individuals with borderline personality disorder. Biological Psychiatry, 66(12), 1107-1114. https://doi.org/10.1016/j.biopsych.2009.07.015

21.

Peduzzi

Concato

Kemper

Holford

T. R.

Feinstein

A. R.

(1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49(12), 1373-1379. https://doi.org/10.1016/S0895-4356(96)00236-3

22.

Perez-Rodriguez

M. M.

Hazlett

E. A.

Rich

E. L.

Ripoll

L. H.

Weiner

D. M.

Spence

Goodman

Koenigsberga

H. W.

Siever

L. J.

New

A. S.

(2012). Striatal activity in borderline personality disorder with comorbid intermittent explosive disorder: Sex differences. Journal of Psychiatric Research, 46(6), 797-804. https://doi.org/10.1016/j.jpsychires.2012.02.014

23.

Spielberger

C. D.

(1999). State-Trait Anger Expression Inventory-II (STAXI-II). PAR.

24.

Stokel-Walker

(2018). Bots on Amazon’s Mechanical Turk are ruining psychology studies. https://www.newscientist.com/article/2176436-bots-on-amazons-mechanical-turk-are-ruining-psychology-studies/

25.

Taylor

S. P.

(1967). Aggressive behavior and physiological arousal as a function of provocation and the tendency to inhibit aggression 1. Journal of Personality, 35(2), 297-310. https://doi.org/10.1111/j.1467-6494.1967.tb01430.x

26.

Vaidya

J. G.

Gray

E. K.

Haig

Watson

(2002). On the temporal stability of personality: Evidence for differential stability and the role of life experiences. Journal of Personality and Social Psychology, 83(6), 1469-1484. https://doi.org/10.1037/0022-3514.83.6.1469

27.

Wagenmakers

E. J.

Morey

R. D.

Lee

M. D.

(2016). Bayesian benefits for the pragmatic researcher. Current Directions in Psychological Science, 25(3), 169-176. https://doi.org/10.1177/0963721416643289

28.

Williamson

(2016). On the ethics of crowdsourced research. PS: Political Science & Politics, 49(1), 77-81. https://doi.org/10.1017/S104909651500116X

29.

Zeichner

Frey

F. C.

Parrott

D. J.

Butryn

M. F.

(1999). Measurement of laboratory aggression: A new response-choice paradigm. Psychological Reports, 85(3_suppl.), 1229-1237. https://doi.org/10.2466/pr0.1999.85.3f.1229

30.

Zhou

Fishbach

(2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4), 493-504. https://doi.org/10.1037/pspa0000056

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB