Twitter data are widely used in the social sciences. The Twitter Application Programming Interface (API) allows researchers to build large databases of user activity efficiently. Despite the potential of Twitter as a data source, less attention has been paid to issues of sampling, and in particular, the implications of different sampling strategies on overall data quality. This research proposes a set of conceptual distinctions between four types of populations that emerge when analyzing Twitter data and suggests sampling strategies that facilitate more comprehensive data collection from the Twitter API. Using three applications drawn from large databases of Twitter activity, this research also compares the results from the proposed sampling strategies, which provide defensible representations of the population of activity, to those collected with more frequently used hashtag samples. The results suggest that hashtag samples misrepresent important aspects of Twitter activity and may lead researchers to erroneous conclusions.
AlwinD. F. (2007). Margins of error: A study of reliability in survey measurement. Hoboken, NJ: John Wiley.
2.
BastosM. T.RaimundoR. L. G.TravitzkiR. (2013). Gatekeeping Twitter: Message diffusion in political hashtags. Media, Culture & Society, 35, 260–270.
3.
BennettW. L.SegerbergA. (2013). The logic of connective action: Digital media and the personalization of contentious politics. New York, NY: Cambridge University Press.
4.
BerinskyA. J.HuberG. A.LenzG. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20, 351–368. doi:10.1093/pan/mpr057
5.
boydd.CrawfordK. (2012). Critical questions for big data. Information, Communication & Society, 15, 662–679.
6.
boydd. m.EllisonN. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13, 210–230. doi:10.1111/j.1083-6101.2007.00393.x
BrunsA.HighfieldT.BurgessJ. (2013). The Arab spring and social media audiences English and Arabic Twitter users and their networks. American Behavioral Scientist, 57, 871–898.
9.
BurgessJ.BrunsA. (2012). Twitter archives and the challenges of “big social data” for media and communication research. M/C Journal, 15, 1–7.
10.
ChristensonD. P.GlickD. M. (2013). Crowdsourcing panel studies and real-time experiments in MTurk. The Political Methodologist, 20, 27–32.
11.
ConoverM. D.DavisC.FerraraE.McKelveyK.MenczerF.FlamminiA. (2013). The geospatial characteristics of a social movement communication network. PLoS One, 8, e55957.
12.
ConoverM. D.FerraraE.MenczerF.FlamminiA. (2013). The digital evolution of Occupy Wall Street. PLoS One, 8, e64679.
13.
DiMaggioP.NagM.BleiD. (2013). Exploiting affinities between topic modeling and the sociological perspective on culture: Application to newspaper coverage of U.S. Government arts funding. Poetics, 41, 570–606.
14.
DriscollK.WalkerS. (2014). Working within a Black Box: Transparency in the collection and production of Big Twitter data. International Journal of Communication, 8, 1745–1764.
15.
EarlJ.HurwitzH. M.MesinasA. M.TolanM.ArlottiA. (2013). This protest will be tweeted: Twitter and protest policing during the Pittsburgh G20. Information, Communication & Society, 16, 459–478.
16.
GainousJ.WagnerK. M. (2013). Tweeting to power: The social media revolution in American politics. New York, NY: Oxford University Press.
17.
Gayo-AvelloD. (2013). A meta-analysis of state-of-the-art electoral prediction from Twitter data. Social Science Computer Review, 31, 649–679.
GleasonB. (2013). #Occupy Wall Street exploring informal learning about a social movement on Twitter. American Behavioral Scientist, 57, 966–982.
20.
González-BailónS.WangN.RiveroA.Borge-HolthoeferJ.MorenoY. (2012). Assessing the bias in communication networks sampled from Twitter. arXiv. Retrieved fromhttp://arxiv.org/abs/1212.1684
21.
JonesJ. (2014). Switching in Twitter’s hashtagged exchanges. Journal of Business and Technical Communication, 28, 83–108.
22.
KwonE. S.SungY. (2011). Follow me! Global marketers’ Twitter use. Journal of Interactive Advertising, 12, 4–16.
23.
MorstatterF.PfefferJ.LiuH. (2014). When is it biased? Assessing the representativeness of Twitter’s streaming API. In Proceedings of the 23rd International Conference on World Wide Web (pp. 555–556). New York, NY: ACM. doi:10.1145/2567948.2576952
24.
MorstatterF.PfefferJ.LiuH.CarleyK. M. (2013). Is the sample good enough? Comparing data from Twitter’s streaming API with Twitter’s Firehose. arXiv. Retrieved fromhttp://arxiv.org/abs/1306.5204
25.
MurphyK. P. (2012). Machine learning: A probabilistic perspective. Cambridge, MA: MIT Press.
26.
O’BrienD. T.SampsonR. J.WinshipC. (2015). Ecometrics in the age of big data: Measuring and assessing “broken windows” using large-scale administrative records. Sociological Methodology, 45, 101–147.
RiederB. (2012). The refraction chamber: Twitter as sphere and network. First Monday, 17, 1–17.
29.
TanL.PonnamS.GillhamP.EdwardsB.JohnsonE. (2013). Analyzing the impact of social media on social movements: A computational study on Twitter and the Occupy Wall Street movement. In 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (pp. 1259–1266). Niagara Falls, ON: IEEE.
30.
TinatiR.HalfordS.CarrL.PopeC. (2014). Big data: Methodological challenges and approaches for sociological analysis. Sociology, 48, 663–681.
31.
TremayneM. (2014). Anatomy of protest in the digital era: A network analysis of Twitter and Occupy Wall Street. Social Movement Studies, 13, 110–126. doi:10.1080/14742837.2013.830969
32.
VisF. (2013). Twitter as a reporting tool for breaking news: Journalists tweeting the 2011 UK Riots. Digital Journalism, 1, 27–47.
33.
WatkinsB.LewisR. (2014). Initiating dialogue on social media: An investigation of athletes’ use of dialogic principles and structural features of Twitter. Public Relations Review, 40, 853–855.
34.
WuS.HofmanJ. M.MasonW. A.WattsD. J. (2011). Who says what to whom on Twitter. In Proceedings of the 20th International Conference on World Wide Web (pp. 705–714). New York, NY: ACM.
