Screening Amazon’s Mechanical Turk for Adults With ADHD

Abstract

Objective: Researchers are increasingly using Amazon’s Mechanical Turk (MTurk; www.mturk.com) to recruit study participants. However, the utility of MTurk for investigations of ADHD in adulthood is unknown. Method: A total of 6,526 MTurk workers (median age range = 26-35 years) completed an online screening survey assessing their diagnostic histories and symptoms of ADHD, as well as relevant demographic correlates and diagnostic comorbidity. Results: The prevalence of MTurk workers who reported being diagnosed as children and as adults, and the percentage of workers whose ADHD appeared to persist from childhood into adulthood, are consistent with those observed in “offline” samples. Relative to MTurk workers diagnosed with ADHD as adults, workers diagnosed with ADHD as children were more likely to be male and without college degrees, as well as less likely to have comorbid depression or anxiety disorders. Conclusion: Amazon’s MTurk holds promise as a recruitment tool for studying adults with ADHD.

Keywords

adult ADHD online sample assessment

Get full access to this article

View all access options for this article.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Arch

J. J.

(2014). Cognitive behavior therapy and pharmacotherapy for anxiety: Treatment preferences and credibility among pregnant and non-pregnant women. Behavior Research and Therapy, 52, 53-60.

Bardeen

J. R.

Fergus

T. A.

Orcutt

H. K.

(2013). Experiential avoidance as a moderator of the relationship between anxiety sensitivity and perceived stress. Behavior Therapy, 44, 459-469.

Barkley

R. A.

(2006). Attention-deficit hyperactivity disorder (3rd ed.). New York, NY: Guilford Press.

Barkley

R. A.

Murphy

K. R.

Fischer

(2008). ADHD in adults: What the science says. New York, NY: Guilford Press.

Beck

A. T.

Epstein

Brown

Steer

R. A.

(1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893-897.

Beck

A. T.

Ward

C. H.

Mendelson

Mock

Erbaugh

(1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561-571.

Behrend

T. S.

Sharek

D. J.

Meade

A. W.

Wiebe

E. N.

(2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43, 800-813.

10.

Biederman

Faraone

S. V.

Spencer

T. J.

Mick

Monuteaux

M. C.

Aleardi

(2006). Functional impairments in adults with self-reports of diagnosed ADHD: A controlled study of 1001 adults in the community. Journal of Clinical Psychiatry, 67, 524-540.

11.

Brochu

P. M.

Pearl

R. L.

Puhl

R. M.

Brownell

K. D.

(2014). Do media portrayals of obesity influence support for weight-related medical policy? Health Psychology, 33, 197-200.

12.

Buhrmester

Kwang

Gosling

S. D.

(2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5.

13.

Chandler

Mueller

Paolacci

(2014). Nonnaivete among Amazon Mechanical Turk workers: Consequences and solutions for behavior researchers. Behavior Research Methods, 46, 112-130.

14.

Gardner

R. M.

Brown

D. L.

Boice

(2012). Using Amazon’s Mechanical Turk website to measure accuracy of body size estimation and body dissatisfaction. Body Image, 9, 532-534.

15.

Harris

P. A.

Taylor

Thielke

Payne

Gonzalez

Conde

J. G.

(2009). Research electronic data capture (REDCap): A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42, 377-381.

16.

Henshaw

E. J.

(2014). Too sick, not sick enough? Effects of treatment type and timing on depression stigma. Journal of Nervous and Mental Disease, 202, 292-299.

17.

Kessler

R. C.

Adler

Ames

Demler

Faraone

Hiripi

. . . Walters

E. E.

(2005). The World Health Organization adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35, 245-256.

18.

Kessler

R. C.

Adler

Barkley

Biederman

Conners

C. K.

Demler

. . . Zaslavsky

A. M.

(2006). The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey replication. The American Journal of Psychiatry, 163, 716-723.

19.

Kessler

R. C.

Adler

Gruber

M. J.

Sarawate

C. A.

Spencer

Van Brunt

D. L.

(2007). Validity of the World Health Organization adult ADHD Self-Report Scale (ASRS) screener in a representative sample of health plan members. International Journal of Methods in Psychiatric Research, 16, 52-65.

20.

Kessler

R. C.

Berglund

Demler

Jin

Merikangas

K. R.

Walters

E. E.

(2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey replication. Archives of General Psychiatry, 62, 593-602.

21.

Kessler

R. C.

Chiu

W. T.

Demler

Walters

E. E.

(2005). Prevalence, severity, and comorbidity of twelve-month DSM-IV disorders in the National Comorbidity Survey Replication (NCSR). Archives of General Psychiatry, 62, 617-627.

22.

Klein

R. G.

Mannuzza

Olazagasti

M. A. R.

Roizen

Hutchison

J. A.

Lashua

E. C.

Castellanos

(2012). Clinical and functional outcome of childhood attention-deficit/hyperactive disorder 33 years later. Archives of General Psychiatry, 69, 1295-1303.

23.

Knouse

L. E.

Bagwell

C. L.

Barkley

R. A.

Murphy

K. R.

(2005). Accuracy of self-evaluation in adults with attention-deficit hyperactivity disorder. Journal of Attention Disorders, 8, 221-234.

24.

Kuriyan

A. B.

Pelham

W. E.

Molina

B. S. G.

Waschbusch

D. A.

Gnagy

E. M.

Sibley

M. H.

. . . Kent

K. M.

(2013). Young adult educational and vocational outcomes of children diagnosed with ADHD. Journal of Abnormal Child Psychology, 41, 27-41.

25.

Lebowitz

M. S.

Ahn

Nolen-Hoeksema

(2013). Fixable or fate? Perceptions of the biology of depression. Journal of Consulting and Clinical Psychology, 81, 518-527.

26.

Litman

Robinson

Rosenzweig

(2014). The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behavior Research Methods, 47, 519-528.

27.

Loney

Ledolter

Kramer

J. R.

Volpe

R. J.

(2007). Retrospective ratings of ADHD symptoms made at young adulthood by clinic-referred boys with ADHD-related problems, their brothers without ADHD, and control participants. Psychological Assessment, 19, 269-280.

28.

Mason

Suri

(2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1-23.

29.

Owens

J. S.

Goldfine

M. E.

Evangelista

N. M.

Hoza

Kaiser

N. M.

(2007). A critical review of self-perceptions and the positive illusory bias in children with ADHD. Clinical Child and Family Psychology Review, 10, 335-351.

30.

Paolacci

Chandler

Ipeirotis

P. G.

(2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411-419.

31.

Papa

Lancaster

N. G.

Kahler

(2014). Commonalities in grief responding across bereavement and non-bereavement losses. Journal of Affective Disorders, 161, 136-143.

32.

Shapiro

D. N.

Chandler

Mueller

P. A.

(2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1, 213-220.

33.

Solanto

M. V.

Marks

D. J.

Wasserstain

Mitchell

Abikoff

Alvir

J. M. J.

Kofman

M. D.

(2010). Efficacy of meta-cognitive therapy for adult ADHD. American Journal of Psychiatry, 167, 958-968.

34.

Substance Abuse and Mental Health Services Administration. (2013). Results from the 2012 National Survey on Drug Use and Health: Mental health findings. Rockville, MD: Author.

35.

U.S. Census Bureau. (2014). Current Population Survey, 2014 annual social and economic supplement. Retrieved from https://www.census.gov/hhes/socdemo/education/data/cps/2014/tables.html

36.

Victor

S. E.

Klonsky

E. D.

(2014). Daily emotion in non-suicidal self-injury. Journal of Clinical Psychology, 70, 364-375.

37.

Visser

S. N.

Danielson

M. L.

Bitsko

R. H.

Holbrook

J. R.

Kogan

M. D.

Ghandour

R. M.

. . . Blummberg

S. J.

(2014). Trends in the parent-report of health care provider-diagnosis and medication treatment for ADHD disorder: United States, 2003–2011. Journal of the American Academy of Child & Adolescent Psychiatry, 53, 34-46.

38.

Wilens

T. E.

Biederman

Faraone

S. V.

Martelon

Westerberg

Spencer

T. J.

(2009). Presenting ADHD symptoms, subtypes, and comorbid disorders in clinically referred adults with ADHD. Journal of Clinical Psychiatry, 70, 1557-1562.