Online purposive samples have unknown biases and may not strictly be used to make inferences about wider populations, yet such inferences continue to occur. We compared the demographic and drug use characteristics of Australian ecstasy users from a probability (National Drug Strategy Household Survey, n = 726) and purposive sample (online survey conducted as part of a mixed-methods study of online drug discussion, n = 753) using nonparametric (bootstrap) and meta-analysis techniques. We found significant differences in demographics and drug use prevalence. Ideally, online purposive samples of hidden populations should be interpreted in conjunction with probability samples and ethnographic fieldwork.
AdèrH. J.MellenberghG. J.HandD. J.. 2008. Advising on research methods: A consultant’s companion. Huizen, the Netherlands: Johannes van Kessel Publishing.
2.
AIHW (Australian Institute of Health and Welfare). 2002. 2001 National drug strategy household survey: First results. Canberra, Australia: AIHW.
3.
AIHW (Australian Institute of Health and Welfare). 2005. 2004 National drug strategy household survey: First results. Canberra, Australia: AIHW.
4.
AIHW (Australian Institute of Health and Welfare). 2008. 2007 National drug strategy household survey: First results. Canberra, Australia: AIHW.
5.
BarrattM. J.2012. The efficacy of interviewing young drug users through online chat. Drug and Alcohol Review31:566–72.
6.
BarrattM. J.LentonS.. 2010. Beyond recruitment? Participatory online research with people who use drugs. International Journal of Internet Research Ethics3:69–86.
7.
BauermeisterJ. A.ZimmermanM. A.JohnsM. M.GlowackiP.StoddardS.VolzE.. 2012. Innovative recruitment using online networks: Lessons learned from an online study of alcohol and other drug use utilizing a web-based, respondent-driven sampling (webRDS) strategy. Journal of Studies on Alcohol and Drugs73:834–38.
8.
BockT.SargeantJ.. 2002. Small sample market research. International Journal of Market Research44:235–44.
9.
BowenA.DanielC.WilliamsM.BairdG.. 2008. Identifying multiple submissions in Internet research: Preserving data integrity. AIDS and Behavior12:964–73.
10.
BraunsteinM. S.1993. Sampling a hidden population: Noninstitutionalized drug users. AIDS Education and Prevention5:131–40.
11.
CaetanoR.2001. Non-response in alcohol and drug surveys: A research topic in need of further attention. Addiction96:1541–45.
12.
CouperM. P.2000. Web surveys: A review of issues and approaches. Public Opinion Quarterly64:464–94.
13.
CrosbyR. A.SalazarL. F.DiClementeR. J.LangD. L.. 2010. Balancing rigor against the inherent limitations of investigating hard-to-reach populations. Health Education Research25:1–5.
14.
DillmanD. A.2007. Mail and Internet surveys. The tailored design method. Hoboken, NJ: Wiley.
15.
FalzonM.-A.2009. Introduction. In Multi-sited ethnography: Theory, praxis and locality in contemporary research, ed. FalzonM.-A., 1–23. Surrey, UK: Ashgate.
16.
FrickerR. D.2008. Sampling methods for web and e-mail surveys. In The Sage handbook of online research methods, eds. FieldingN.LeeR. M.BlankG., 195–216. London: Sage.
17.
GileK. J.HandcockM. S.. 2010. Respondent-driven sampling: An assessment of current methodology. Sociological Methodology40:285–327.
18.
GrovesR. M.2006. Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly70:646–75.
19.
GrovesR. M.2011. Three eras of survey research. Public Opinion Quarterly75:861–71.
20.
GuarteJ. M.BarriosE. B.. 2006. Estimation under purposive sampling. Communications in Statistics—Simulation and Computation35:277–84.
21.
HathawayA. D.2004. Cannabis careers reconsidered: Transitions and trajectories of committed long-term users. Contemporary Drug Problems31:401–23.
22.
HeckathornD. D.2011. Snowball versus respondent-driven sampling. Sociological Methodology41:355–66.
23.
HineC.2008. Virtual ethnography: Modes, varieties, affordances. In The Sage handbook of online research methods, eds. FieldingN.LeeR. M.BlankG., 257–70. London: Sage.
24.
KakinamiL.ConnerK. R.. 2010. Sampling strategies for addiction research. In Addiction research methods, eds. MillerP. G.StrangJ.MillerP. M., 27–42. Chichester, UK: Wiley-Blackwell.
25.
LivingstonM.DietzeP.FerrisJ.PennayD.HayesL.LentonS.. 2013. Surveying alcohol and other drug use through telephone sampling: A comparison of landline and mobile phone samples. BMC Medical Research Methods13:Article 41.
26.
LohrS. L.2010. Sampling: Design and analysis. Pacific Grove, CA: Brooks/Cole.
27.
LucasS. R.2014. Beyond the existence proof: Ontological conditions, epistemological implications, and in-depth interview research. Quality and Quantity48:387–408.
28.
MillerP. G.JohnstonJ.DunnM.FryC. L.DegenhardtL.. 2010. Comparing probability and non-probability sampling methods in ecstasy research: Implications for the Internet as a research tool. Substance Use and Misuse45:437–50.
29.
MillerP. G.JohnstonJ.McElweeP. R.NobleR.. 2007. A pilot study using the Internet to study patterns of party drug use: Processes, findings and limitations. Drug and Alcohol Review26:169–74.
30.
MillerP. G.SønderlundA. L.. 2010. Using the Internet to research hidden populations of illicit drug users: A review. Addiction105:1557–67.
31.
MooreD.2005. Key moments in the ethnography of drug-related harm: Reality checks for policy-makers?” In Preventing harmful substance use. The evidence base for policy and practice, eds.
StockwellT.GruenewaldP. J.ToumbourouJ. W.LoxleyW., 433–42. New York: John Wiley.
32.
PetersonJ. A.Schacht ReisingerH.SchwartzR. P.MitchellS. G.. 2008. Targeted sampling in drug abuse research: A review and case study. Field Methods20:155–70.
33.
PotterG. R.ChatwinC.. 2011. Researching cannabis markets online: Some lessons from the virtual field. In Markets, methods and messages. Dynamics in European drug research, eds. FountainJ.FrankV. A.KorfD. J., 69–84. Lengerich, Germany: Pabst Science.
RodgersJ. L.1999. The bootstrap, the jackknife, and the randomization test: A sampling taxonomy. Multivariate Behavioral Research34:441–56.
36.
SalganikM.2006. Variance estimation, design effects, and sample size calculations for respondent-driven sampling. Journal of Urban Health83:98–112.
37.
SalganikM. J.HeckathornD. D.. 2004. Sampling and estimation in hidden populations using respondent-driven sampling. Sociological Methodology34:193–240.
38.
SifaneckS. J.NeaigusA.. 2001. The ethnographic accessing, sampling and screening of hidden populations: Heroin sniffers in New York City. Addiction Research and Theory9:519–43.
39.
SmithS. M.AlbaumG. S.. 2005. Fundamentals of marketing research. London: Sage.
SterneJ. A. C.2009. Meta-analysis in stata: An updated collection from the stata journal. College Station, TX: Stata Press.
42.
ToppL.BarkerB.DegenhardtL.. 2004. The external validity of results derived from ecstasy users recruited using purposive sampling strategies. Drug and Alcohol Dependence73:33–40.
43.
ToppL.BreenC.KayeS.DarkeS.. 2002. NSW party drug trends 2001. Findings of the Illicit Drug Reporting System (IDRS) party drugs module. Sydney, Australia: National Drug and Alcohol Research Centre, University of New South Wales.
44.
WangJ.CarlsonR. G.FalckR. S.SiegalH. A.RahmanA.LiL.. 2005. Respondent-driven sampling to recruit MDMA users: A methodological assessment. Drug and Alcohol Dependence78:147–57.
45.
WejnertC.HeckathornD. D.. 2008. Web-based network sampling: Efficiency and efficacy of respondent-driven sampling for online research. Sociological Methods and Research37:105–34.
46.
WilliamsonG. R.2003. Misrepresenting random sampling? A systematic review of research papers in the Journal of Advanced Nursing. Journal of Advanced Nursing44:278–88.
47.
ZhaoJ.StockwellT. I. M.MacdonaldS.. 2009. Non-response bias in alcohol and drug population surveys. Drug and Alcohol Review28:648–57.
