Official statistics are fundamental to democracy. With increasing demands for more relevant, frequent and rich statistical information, and declining resources, National Statistical Offices are continually looking for more cost effective ways in the production of official statistics. With the advent of the Internet of Things, they are increasingly exploring opportunities to harness Big Data as a source for official statistics. Use of Big Data, however, raises a number of ethical and statistical challenges for official statisticians, which are explored in this paper. This paper also proposes methods to adjust for self-selection bias, or coverage bias, normally associated with Big Data, by utilising random samples generally available from National Statistical Offices. We conclude that National Statistical Offices are generally well equipped to address these challenges.
Holt,T. (2007). Official statistics, public policy and public trust. Journal of Royal Statistical Society, A171, 1-20. Presidential Address.
2.
United Nations (2013). Fundamental principles of official statistics. https://unstats.un.org/unsd/dnss/gp/fp-english.pdf.
3.
Daas,PPuts,MBuelens,Bvan den Hurk,P. (2015). Big Data as a Source for Official Statistics : Journal of Official Statistics, 31, 249-262.
4.
Nordholt,E. (2005). The Dutch virtual Census 2001: A new approach by combining different sources – IOS Press Statistical Journal of the United Nations Economic Commission for Europe, 22, 25-37.
5.
Australian Bureau of Statistics (2016).
6.
Husek,N. (2017). Telematics data for official statistics – an experience with Big Data. Submitted for publication.
7.
Marley,JDefina,RTraeger,KElazar,DAmarasinghe,ABiggs,GTam.S-M. (2016). Investigative Pilot Report (unpublished).
8.
Tam,S-M. (1987). Analysis of a repeated survey using a dynamic linear model. International Statistical Review, 55, 63-73.
9.
Tam,S-M. (2015). A Statistical Framework for Analysing Big Data. Survey Statistician, 72, 36-51.
10.
Choi,HVarian,R. (2009). Predicting initial claims for unemployment insurance using Google Trends. Google Technical Report. https://static.googleusercontent.com/media/research.google.com/en/archive/papers/initialclaimsUS.pdf.
11.
Tam,S-MClarke,F. (2015). Big Data, official statistics and some experience of the Australian Bureau of Statistics. International Statistical Review, 83, 436-448.
12.
Elliott,MValliant,R. (2017). Inference for non-probability samples. Statistical Science, 32, 249-264.
13.
International Statistical Institute (1986). ISI declaration of professional ethics. ISI Declaration-isi-web.org.
14.
American Statistical Association (2016). Ethical guidelines for statistical practice. Ethical Guidelines for Statistical Practice.
15.
Royal Statistical Society (1993). Code of coduct. http://www.rss.org.uk/favicon.ico.
16.
Sydney Morning Hearlad (2017). Federal Court rejects application for Telstra to supply ‘personal’ metadata. http://www.smh.com.au/technology/technology-news/federal-court-rejects-application-for-telstra-to-supply-personal-metadata-20170120-gtvc85.html.
17.
Felix,R. (2013). International access to restricted data – a principle based approach. Journal of the International Association of Official Statistics, 29, 289-300. International access to restricted data: A principles-based standards approach – IOS Press.
18.
Tam,S-MFarley-Larmour,KGare,M. (2010). Supporting research and protecting confidentiality – ABS microdata access: current strategies and future directions. Journal of the International Association of Official Statistics, 26, 65-74. Supporting research and protecting confidentiality. ABS microdata access: Current strategies and future directions – IOS Press.
19.
Couper,M. (2013). Is the sky falling? Survey Research Methods, 7, 145-156. Is the Sky Falling? New Technology, Changing Media, and the Future of Surveys Couper Survey Research Methods.
20.
Meng,X. (2016). Statistical paradises and paradoxes in Big Data. Talk to the 2016 Royal Statistical Society Conference. https://www.youtube.com/subscribe_embed?usegapi=1&card=1&channelid=UC83oOOF9lg-g1XMT_UK1tUw&origin=https%3A%2F%2Fapis.google.com&gsrc=3p&jsh=m%3B%2F_%2Fscs%2Fabc-static%2F_%2Fjs%2Fk%3Dgapi.gapi.en.ellQXbSf-LI.O%2Fm%3D__features__%2Fam%3DAAg%2Frt%3Dj%2Fd%3D1%2Frs%3DAHpOoo9jm0At0b0B7I7G3MSvlepU00mZfA.
21.
Puza,BO’Neill,T. (2006). Selection bias in binary data from voluntary surveys. Mathematical Scientist, 31, 85-94. selection bias in binary data from voluntary surveys – Google Scholar.
22.
Raghunathan,T. (2015). Statistical challenges in combining information from big and small data sources. Paper presented to the Expert Panel meeting at the National Academy of Science. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/120417/NAS-Paper.pdf?sequence=1&isAllowed=y.
23.
Tam,S-M. (2008). Informing the nation – open access to statistical information in Australia. Journal of the International Association of Official Statistics, 24, 145-153. Informing the nation – open access to statistical information in Australia – IOS Press.
24.
Australian Bureau of Statistics (2009). ABS data quality framework. 1520.0 – ABS Data Quality Framework, May 2009.
25.
Tam,S-MKraayenbrink,R. (2006). Data communication – emerging international trends and practice of the Australian Bureau of Statistics. Journal of the United Nations Economic Commission for Europe, 23, 229-247. Data communication – Emerging international trends and practices of the Australian Bureau of Statistics – IOS Press.
