Can Smart City Data be Used to Create New Official Statistics?

Abstract

In this article we evaluate the viability of using big data produced by smart city systems for creating new official statistics. We assess sixteen sources of urban transportation and environmental big data that are published as open data or were made available to the project for Dublin, Ireland. These data were systematically explored through a process of data checking and wrangling, building tools to display and analyse the data, and evaluating them with respect to 16 measures of their suitability: access, sustainability and reliability, transparency and interpretability, privacy, fidelity, cleanliness, completeness, spatial granularity, temporal granularity, spatial coverage, coherence, metadata availability, changes over time, standardisation, methodological transparency, and relevance. We assessed how the data could be used to produce key performance indicators and potential new official statistics. Our analysis reveals that, at present, a limited set of smart city data is suitable for creating new official statistics, though others could potentially be made suitable with changes to data management. If these new official statistics are to be realised then National Statistical Institutions need to work closely with those organisations generating the data to try and implement a robust set of procedures and standards that will produce consistent, long-term data sets.

Keywords

Big data transport environment data quality key performance indicators

References

ANSI 2016. Directory of Smart and Sustainable Cities Standardization Initiatives and Related Activities. American National Standards Institute Network on Smart and Sustainable Cities (ANSSC). Available at: https://www.ansi.org/standards_activities/-standards_boards_panels/anssc/overview#Standards (accessed February 2021).

Bradshaw, R. 2019. “Instrumentalization in the Public Smart Bikeshare Sector.” PhD Doctoral thesis, Maynooth University. Available at: http://mural.maynoothuniversity.ie/10509/ (accessed February 2021).

Celikoglu, H.B. 2013. “An approach to dynamic classification of traffic flow patterns.” Computer-Aided Civil and Infrastructure Engineering; 28, 4: 273–288. DOI: doi.org/10.1111/j.1467-8667.2012.00792.x.

ECOSOC. 2015. Report of the Global Working Group on Big data for official statistics. United National Economic and Social Council. 46th Statistical Commission. Available at: http://unstats.un.org/unsd/statcom/doc15/2015-4-BigData.pdf (accessed February 2021).

ESSC. 2013. The Scheveningen Memorandum on Big Data and Official Statistics. Scheveningen: Directors General of the National Statistical Institutes (DGINS). Available at: https://ec.europa.eu/eurostat/cros/system/files/SCHEVENINGEN_MEMORANDUM%20Final%20version.pdf. (accessed February 2021).

ESSC. 2014. “ESS Big Data Action Plan and Roadmap 1.0.” Riga: European Statistical System Committee. 26th September 2014, Riga. Latvia. Available at: https://ec.europa.eu/eurostat/cros/content/ess-big-data-action-plan-and-roadmap-10_en (accessed February 2021)

ESSC. 2018. “The Bucharest Memorandum on Official Statistics in a Datafied Society (Trusted Smart Statistics).” Bucharest: Directors General of the National Statistics Institutes (DGINS), 12th October 2018, Bucharest, Bulgaria. Available at: https://ec.europa.eu/eurostat/documents/7330775/7339482/The þ Bucharest þ Memorandum þ on þ Trusted þ Smart þ Statistics þ FINAL.pdf (accessed February 2021).

European Union. 2008. “Directive 2008/50/EC of the European Parliament and of the Council of 21 May 2008 on ambient air quality and cleaner air for Europe.” Official Journal of the European Union 51: 1–44. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32008L0050&from=EN (accessed February 2021).

Eurostat. 2014a. Accreditation procedure for statistical data from non-official sources. European Commission. Available at: https://ec.europa.eu/eurostat/cros/system/files/-D5_Accreditation%20procedure%20for%20statistical%20data%20from%20nonofficial%20sources_20140206_0.pdf (accessed February 2021).

10.

Eurostat. 2014b. “Big data – an opportunity or a threat to official statistics?” Paper presented at the Conference of European Statisticians, 62nd plenary session, Paris, 9 – 11 April, 2014. Available at: http://www.unece.org/fileadmin/DAM/stats/documents/ece/ces/2014/32-Eurostat-Big_Data.pdf (accessed February 2021).

11.

Eurostat. 2018. Report describing the quality aspects of Big Data for Official Statistics. ESSnet Big Data, Work Package 8, Deliverable 8.2. Available at: https://ec.europa.eu/eurostat/cros/sites/crosportal/files/WP8_Deliverable_8.2_Quality_aspects.pdf (accessed February 2021).

12.

Florescu, D., M. Karlberg, F. Reis, P.R. Del Castillo, M. Skaliotis, and A. Wirthmann. 2014. “Will ‘big data’ transform official statistics?” European Conference on the Quality of Official Statistics. Available at: http://www.q2014.at/fileadmin/user_upload/ESTAT-Q2014-BigDataOS-v1a.pdf (accessed February 2021).

13.

Karvonen, A., F. Cugurullo, and F. Caprotti. 2018. Inside Smart Cities: Place, Politics and Urban Innovation. London: Routledge.

14.

Kahn, J., M. Ketzel, K. Kakosimos M. Sorensen, and S.S. Jensen. 2018. “Road traffic air and noise pollution exposure assessment – A review of tools and techniques.” Science of The Total Environment 634 no. 1: 661–676. DOI: https://doi.org/10.1016/j.scitotenv.2018.03.374.

15.

Kitchin, R. 2015. “The Opportunities, Challenges and Risks of Big Data for Official Statistics.” Statistical Journal of the International Association of Official Statistics 31. 3: 471–481. DOI: https://doi.org/10.3233/SJI-150906.

16.

Kitchin, R. 2016. Getting smarter about smart cities: Improving data privacy and data security. Data Protection Unit, Department of the Taoiseach, Dublin, Ireland. Available at: http://mural.maynoothuniversity.ie/7242/1/Smart (accessed February 2021).

17.

Kitchin, R. 2017. “Data-Driven Urbanism”. In: Data and the City. Kitchin R., G. McArdle, T. Lauriault, eds.: 44–56. Routledge: London.

18.

Kitchin, R, T. Lauriault, and G. McArdle. 2015. “Knowing and governing cities through urban indicators, city benchmarking and real-time dashboards.” Regional Studies, Regional Science 2: 1–28. DOI: https://doi.org/10.1080/21681376.2014.983149.

19.

Kitchin, R., and G. McArdle. 2016. “What makes Big Data, Big Data? Exploring the ontological characteristics of 26 datasets.” Big Data and Society 3: 1–10. DOI: https://doi.org/10.1177/2053951716631130.

20.

Landefeld, S. 2014. “Uses of Big Data for Official Statistics: Privacy, Incentives, Statistical Challenges, and Other Issues.” Paper presented at International Conference on Big Data for Official Statistics, 28–30 October 2014, Beijing, China. Available at: https://unstats.un.org/unsd/trade/events/2014/beijing/Steve%20Landefeld%20-%20Uses%20of%20Big%20Data%20for%20official%20statistics.pdf (accessed February 2021).

21.

MacFeely, S. 2018. “The privacy dilemma for official statistics in a big data world.” Open Access Biostatistics and Bioinformatics 2. 1: 1:3. DOI: https://doi.org/10.31031/OABB.2018.02.000526.

22.

McArdle, G, and R. Kitchin. 2016. “Improving the Veracity of Open and Real-Time Urban Data.” Built Environment 42, 3: 446–462. DOI: https://doi.org/10.2148/benv.42.3.457.

23.

OAG. 2019. Defining Late: is 15 minutes the right measure? Official Aviation Guide, Worldwide Aviation Limited. Available at: https://www.oag.com/hubfs/Defining_Late/Defining-Late-Report.pdf?hsLang=en-gb (accessed February 2021).

24.

OECD. 2011. Quality Framework and Guidelines for OECD Statistical Activities. Organisation for Economic Co-operation and Development, Statistics Directorate. Available at: https://www.oecd.org/sdd/qualityframeworkforoecdstatisticalactivi-ties.htm (accessed February 2021).

25.

Reboot. 2017. Understanding the Users of Open Data. Reboot Inc. for NYC Open Data. Available at: https://opendata.cityofnewyork.us/wp-content/uploads/2017/07/Understanding-the-Users-of-Open-Data_Reboot.pdf (accessed February 2021).

26.

Riley, J. 2017. Understanding Metadata: What is Metadata, and What is it For?: A Primer. Baltimore: National Information Standards Organization. Available at: http://www.niso.org/publications/press/UnderstandingMetadata.pdf (accessed February 2021).

27.

Severo, M, A. Feredj, and A. Romele. 2016. “Soft Data and Public Policy: Can Social Media Offer Alternatives to Official Statistics in Urban Policymaking.” Policy and Internet 8, 3: 354–372. DOI: https://doi.org/10.1002/poi3.127.

28.

Solove, D.J. 2006. “A Taxonomy of Privacy.” University of Pennsylvania Law Review 154. 3: 477–560. Available at: https://scholarship.law.upenn.edu/penn_law_review/-vol154/iss3/1 (accessed February 2021).

29.

Struijs, P., B. Braaksma, and P.J.H. Daas. 2014. “Official statistics and Big Data.” Big Data and Society 1, 1: 1–6. DOI: https://doi.org/10.1177/2053951714538417.

30.

Townsend, A. 2013. Smart Cities: Big data, Civic Hackers, and the Quest for a New Utopia. New York: W.W. Norton & Co.

31.

UN Big Data Working Group. 2017. Bogota Declaration. United Nations 4th International Conference on Big Data for Official Statistics. 8–10 November 2017, Bogotá, Colombia. Available at: https://unstats.un.org/unsd/bigdata/conferences/2017/Bogota%20declaration%20-%20Final%20version.pdf (accessed February 2021).

32.

UNECE. 1992. Fundamental Principles of Official Statistics. United Nations Economic Commission for Europe. Available at: https://unece.org/statistics/fundamental-principles-official-statistics (accessed February 2021).

33.

UNECE. 2014. A Suggested Framework for the Quality of Big Data. United Nations Economic Commission for Europe. Available at: https://statswiki.unece.org/download/attachments/108102944/Big%20Data%20Quality%20Framework%20-%20final-%20Jan08-2015.pdf (accessed February 2021).

34.

White, J.M. 2019. “Standardising the city: A material-discursive genealogy of CPA-I_001, ISO 37120 and BSI PAS 181.” Doctoral thesis, Maynooth University. Available at: http://mural.maynoothuniversity.ie/10848/ (accessed February 2021).

35.

WHO. 2006. WHO Air quality guidelines Global update 2005: particulate matter, ozone, nitrogen dioxide and sulfur dioxide. World Health Organization Regional Office for Europe. Copenhagen. Available at: https://apps.who.int/iris/handle/10665/107823 (accessed February 2021).

36.

Zein, A. 2020. “Short-term effects of the coronoavirus outbreak: what does the shipping data say?” UNCTAD Transport and Trade Facilitation Newsletter N885 – First Quarter 2020. 48.