Statistical open source software for official statistics: State of play and future directions

Abstract

Statistical organizations worldwide are increasingly adopting open source technologies for producing official statistics. This shift is motivated by the potential of open source tools to increase transparency, improve efficiency, and enhance reproducibility. Moreover, young professionals in statistics and data science enter the labour market with strong skills in open source tools. The adoption of open source software signifies a change in how statistical organizations operate and collaborate.

This paper provides an overview of the state of open source adoption in official statistics. It details the open source movement among statistical organizations, the experiences of Statistics Netherlands with open source adoption and the creation of R-packages implementing common statistical methods. It also describes the development and use of the “awesome list of official statistics software” and discusses a set of principles for open source in official statistics, derived from best practices across various organizations. These principles have been endorsed (June 2025) by the Conference of European Statisticians (CES).

Furthermore, it explores future directions for maturing this community, including metrics for assessing maturity, such as true independence of software modules, support for uncertainty propagation, and privacy by design. Moreover it presents ideas on redesigning the statistical open source landscape.

Keywords

Open-source methodology architecture community principles strategy collaboration

Get full access to this article

View all access options for this article.

References

The Journey to Using R – Experience of Central Statistics Office (Ireland), Conference of European Statistics (CES), Geneva, 2023.

Sandberg

Thindberg

. DAPLA – a cloud based statistical production system and its implications for Statistics Norway. In: Conference of European Statistics (CES), Geneva, 2023.

Statistics Poland. Open Source, Conference of European Statistics (CES), Geneva, 2023.

Kowarik

van der Loo

. Using R in the Statistical Office: the experience of Statistics Netherlands and Statistics Austria. Romanian Statistical Review 2018; 66: 15–29.

Publishing Open Source Code at HICC (Statistics Canada). https://canada-ca.github.io/open-source-logiciel-libre/en/open-source-standards.html

Ralphs

. Transforming statistical workflows to use open-source technology at the UK Office for National Statistics. In: Conference of European Statistics (CES), Geneva, 2023.

Van der Loo

Ten Bosch

. Free and open source software at Statistics Netherlands. In: Conference of European Statistics (CES), Geneva, 2023.

Rahman

(executive ed.) The future of national statistics offices: A call to action. United Nations Economic Commission for Europe (UNECE), 2025 May.

Statistical Open Source Software Charter and Report, 2025. https://unece.github.io/OSS/HLG-MOS%20OSS%20Charter.pdf

10.

van der Loo

MPJ

de Jonge

. Data validation infrastructure for R. J Stat Softw 2021; 97: 1–31.

11.

van der Loo

MPJ

de Jonge

. Data validation. In: Wiley StatsRef: statistics reference online. American Cancer Society, 2020, pp. 1–7. doi:10.1002/9781118445112.stat08255

12.

van der Loo

de Jonge

. dcmodify: Modify data using externally defined modification rules. 2024. 10.32614/CRAN.package.dcmodify, R package version 0.9.0, https://CRAN.R-project.org/package=dcmodify.

13.

de Jonge

van der Loo

. errorlocate: Locate errors with validation rules. 2023. 10.32614/CRAN.package.errorlocate, R package version 1.1.1, https://CRAN.R-project.org/package=errorlocate.

14.

van der Loo

. simputation: Simple imputation. 2024. 10.32614/CRAN.package.simputation, R package version 0.2.9, https://CRAN.R-project.org/package=simputation.

15.

van der Loo

de Jonge

.rspa: Adapt numerical records to fit (in)equality restrictions. 2022. 10.32614/CRAN.package.rspa, R package version 0.2.8, https://cran.r-project.org/package=rspa.

16.

van der Loo

de Jonge

. deductive: Data correction and imputation using deductive methods. 2025. 10.32614/CRAN.package.deductive, R package version 1.0.1, https://CRAN.R-project.org/package=deductive.

17.

de Jonge

van der Loo

. validatetools: Checking and simplifying validation rule sets. 2023. 10.32614/CRAN.package.validatetools, R package version 0.5.2, https://CRAN.R-project.org/package=validatetools.

18.

van der Loo

MPJ

. Split-apply-combine with dynamic grouping. J Stat Softw 2025; 112: 1–21.

19.

van der Loo

MPJ

. Monitoring data in R with the lumberjack package. J Stat Softw 2021; 98: 1–13.

20.

van der Laan

. Reclin2: a toolkit for record linkage and deduplication. R J 2022; 14: 320–328. ISSN: 2073-4859.

21.

Bogaart

van der Loo

Pannekoek

. _rtrim: Trends and indices for monitoring data_. 2024. 10.32614/CRAN.package.rtrim, R package version 2.3.0, https://CRAN.R-project.org/package=rtrim.

22.

Tennekes

. Tmap: thematic maps in R. J Stat Softw 2018; 84: 1–39.

23.

Boonstra

. hbsae: Hierarchical Bayesian small area estimation. 2022. 10.32614/CRAN.package.hbsae, R package version 1.2, https://CRAN.R-project.org/package=hbsae.

24.

https://siscc.org/

25.

ten Bosch

van der Loo

Kowarik

. The awesome list of official statistical software: 100…and counting. In: The Use of R in Official Statistics - uRos202, 2020.

26.

European Statistics Code of Practice - Quality - Eurostat

27.

https://ec.europa.eu/info/departments/informatics/open-source-software-strategy_en

28.

CSPA 2.0 (2023) common statistical production architecture: https://statswiki.unece.org/spaces/CSPA/overview (accessed 08 April 2025).

29.

van der Loo

MPJ

Pannekoek

Rijnveld

. Computational Estimates of data editing related variance. UNECE Work Session on Statistical Data Editing (The Hague, Netherlands), 2017.

30.

ten Bosch

de Jonge

Laloli

. To be FAIR, what is missing in Official Statistics?. In: Conference On Smart Metadata for Official Statistics (COSMOS 2024), Paris, 2024.

31.

Umm-e-Laila

Zahoor Mehboob

, et al. Comparison of open source maturity models. Procedia Comput Sci 2017; 111: 348–354.

32.

Adewumi

Misra

Omoregbe

. A review of models for evaluating quality in open source software. IERI Proc 2013; 4: 88–92.

33.

Van der Loo

MPJ

. Computing in the statistical office. Stat J IAOS 2021; 37: 1023–1036.