Abstract
This article outlines a framework for formal description, justification and evaluation in development of architectures for large-scale statistical production systems. Following an introduction of the main components of the framework, we consider four related issues: (1) Use of some simple schematic models for survey quality, cost, risk, and stakeholder utility to outline several groups of questions that may inform decisions on system design and architecture. (2) Integration of system architecture with models for total survey quality (TSQ) and adaptive total design (ATD). (3) Possible use of concepts from the Generic Statistical Business Process Model (GSBPM) and the Generic Statistical Information Model (GSIM). (4) The role of governance processes in the practical implementation of these ideas.
Keywords
References
1.
Australian Bureau of Statistics (2012). International Collaboration Effort - Statistical Network Progress Report. Paper presented by Brian Pink at the meeting of Directors General of participating countries Australia, New Zealand, Norway, Sweden, Canada, UK in New York, February 2012.
2.
Biemer, P.P. (2010). Total Survey Error: Design, Implementation and Evaluation. Public Opinion Quarterly, 74, 817-848.
3.
Bisgaard, S. and Pinho, A. (2003). Follow-Up Experiments to Verify Dispersion Effects: Taguchi’s Welding Experiment. Quality Engineering, 16, 335-343.
4.
Borowik, J., Henden, M., and Fraser, B. (2012). Riding the Big Data Wave to Streamline Acquiring and Managing Data. Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
5.
Box, George E.P. (1957). Evolutionary Operation: A Method for Increasing Industrial Productivity, Journal of the Royal Statistical Society. Series C (Applied Statistics). Vol. 6, No. 2 (June 1957), 81-101.
6.
Box, G.E.P. and Draper, N.R. (1969). Evolutionary Operation. New York: Wiley.
7.
Braaksma, B. (2009). Redesigning a Statistical Institution: the Dutch Case. In Proceedings of Modernisation of Statistics Production. International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/H_2_business_models_Braaksma_final.pdf. (accessed February 12, 2013).
8.
Brackstone, G. (1999). Managing Data Quality in a Statistical Agency. Survey Methodology, 25, 139-149.
9.
Brenneman, W.A. and Nair, V.N. (2001). Methods for Identifying Dispersion Effects in Unreplicated Factorial Experiments: A Critical Analysis and Proposed Strategies. Technometrics, 43, 388-405.
10.
Calinescua, M., Bhulaia, S., and Schouten, B. (2013). Optimal Resource Allocation in Survey Designs. European Journal of Operational Research, 21, 115-121.
11.
Camstra, A. and Renssen, R. (2011). Standard Process Steps Based on Standard Methods as a Part of the Business Architecture. 58th Congress of the International Statistical Institute. Dublin, 21-26 August 2011.
12.
Carly-Baxter, L., Mitchell, S., and Peytchev, A. (2011). Developing and Implementing Adaptive Total Design. Presented at the 2011 International Field Directors and Technologies Conference (IFD&TC), Scottsdale, AZ. Available at: http://www.ifdtc.org/PC2011/presentation_2011_files/3A-Lisa%20Carley-Baxter.pdf. (accessed February 12, 2013).
13.
Chief Information Officers Council CIOC (1999). Federal Enterprise Architecture Framework. Version 1.1, Chief Information Officers Council, September 1999.
14.
Chief Information Officers Council CIOC (2001). A Practical Guide to Federal Enterprise Architecture. Version 1.0, Chief Information Officers Council, February 2001.
15.
Couper, M. (1998). Measuring Survey Quality in a CASIC Environment. In: Proceedings of the Section on Survey Research Methods, American Statistical Association.
16.
Cunningham-Hunter, D., Mitchell, S., Carley-Baxter, L., and Keating, M. (2012). Using Paradata to Monitor Survey Production, Cost, and Quality within an Adaptive Total Design Framework. Paper presented at the 2012 Federal Conference on Survey Methodology, Washington, DC. Available at: http://www.fcsm.gov/12papers/Hunter_2012FCSM_V-C.pdf. (accessed February 12, 2013).
17.
Dunnet, G. (2007). The BmTS: Creating a new business model for a national statistical office of the 21st century. Invited paper at the UNECE/Eurostat /OECD Meeting on the Management of Statistical Information Systems (MSIS). Geneva, 8-10 May 2007. Available at: http://www.unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.50/2007/mtg1/wp.11.e.pdf. (accessed February 13, 2013).
18.
Eltinge, J.L. (2012). Doing More with Less: Balancing Survey Cost, Quality, Risk and Stakeholder Utility. Presentation at the Fourth International Conference on Establishment Surveys, Montreal, Canada.
19.
Eltinge, J.L. and Phipps, P.A. (2009). Characterization, Evaluation and Management of Prospective Benefits, Costs and Risks in the Development of New Statistical Programs for Energy. Proceedings of the Section on Survey Research Methods, Joint Statistical Meetings.
20.
European Commission (2009). Communication from the Commission to the European Parliament and the Council on the production method of EU statistics: a vision for the next decade, Document COM(2009)404. Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uriCOM:2009:0404:FIN:EN:PDF. (accessed February 12, 2013).
21.
Field, S. (2009). From Red to Green: The role of Enterprise Architecture in the ONS Corporate IT Strategy. UNECE/Eurostat/OECD Meeting on the Management of Statistical Information Systems (MSIS 2009). Oslo, Norway, 18-20 May 2009.
22.
Finselbach, H. (2011). Implementing Standardised Systems, Processes, Tools and Methods with the ONS Design Charter. 58th Congress of the International Statistical Institute. Dublin, 21-26 August 2011.
23.
Gloersen, R. and Saeboe, V. (2009). Standardisation for Improvements in Statistics Norway. In Proceedings of Modernisation of Statistics Production International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/H_1_business_models_Gloersen.pdf. (accessed February 12, 2013).
24.
Groves, R.M. (1989). Survey Errors and Survey Costs. New York: John Wiley & Sons.
25.
Groves, R.M. and Heeringa, S. (2006). Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs. Journal of the Royal Statistical Society Series A: Statistics in Society, 169(Part 3), 439-457.
26.
Groves, R.M. and Lyberg, L. (2010). Total Survey Error: Past, Present and Future. Public Opinion Quarterly, 74, 849-879.
27.
Hahn, G.J. (1982). Statistical Assessment of a Process Change. Journal of Quality Technology, 14, 1-9.
28.
Hall, J.M. and Johnson, M.E. (2009). When Should a Process be Art, not Science. Harvard Business Review, March 2009 issue, 58-64. Hamilton, A. and Tam, S.-M. (2012a). Toward GSIM V1.0 as a Cornerstone for Common Reference Architecture. Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
29.
Hamilton, A. and Tam, S.-M. (2012b). Platform for International Collaboration - Preliminary Thoughts From the ABS. Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
30.
Jug, M. (2009). Uptake of Service Oriented Architecture in Statistics - Are We Really So Different? In Proceedings of Modernisation of Statistics Production International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/C_1_SOA_Jug_final.pdf. (accessed February 12, 2013).
31.
Kreuter, F., Olson, K., Wagner, J., Yan, T., Ezzati-Rice, T.M., Casas-Cordero, C., Lemay, M., Peytchev, A., Groves, R.M., and Raghunathan, T.E. (2010). Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Nonresponse: Examples from Multiple Surveys. Journal of the Royal Statistical Society, Series A, 173, 389-407.
32.
Laflamme, F., Pasture, T., Talon, J., Maydan, M., and Miller, A. (2008). Using Paradata to Actively Manage Data Collection, Proceedings of the ASA Survey Methods Research Section, Denver, Colorado.
33.
Linacre, S. (2011). Standardising Methods and Tools: A Look Back Over 30-Plus Years of Experience. Paper presented at Statistics Canada Symposium 2011.
34.
Lyberg, L. (2012). Survey Quality. Waksberg Memorial Lecture delivered at the Fourth International Conference on Establishment Surveys, 14 June 2012.
35.
MIT (2004). Engineering Systems Division Architecture Committee: The Influence of Architecture in Engineering Systems. Paper presented at the Engineering Systems Symposium, MIT, Cambridge MA, USA, 29-31 March 2004. Available at: http://esd.mit.edu/symposium/monograph/default.htm. (accessed February 12, 2013).
36.
Munoz-Lopez, J. (2012). Value of Systems Architecture for Statistical Organizations.
37.
Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
38.
OMB (2009). Improving Agency Performance Using Information and Information Technology EA Assessment Framework v3.1, Office of Management and Budget. June 2009.
39.
Penneck, S. (2009). The Office for National Statistics (ONS) Statistical Modernisation Programme: What Went Right? What Went Wrong? In Proceedings of Modernisation of Statistics Production, International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/D_1_management_ Penneck_final.pdf. (accessed February 12, 2013).
40.
Pink, B., Borowik, J., and Lee, G. (2010). The Case for an International Statistical Innovation Program -Transforming National and International Statistics Systems. Statistical Journal of the International Association for Official Statistics (IAOS), 26, Number 3-4. Available at: http://iospress.metapress.com/content/2h5764574t6318r4/fulltext.html. (accessed February 12, 2013).
41.
Renssen, R. and Van Delden, A. (2008). Standardisation of Design and Production of Statistics; A Service Oriented Approach at Statistics Netherlands. Paper presented at the IAOS Congress Reshaping Official Statistics, Shanghai, October 2008.
42.
Renssen, R., Wings, J., and Paulussen, R. (2009). Processes, Methods and Tools. Statistics Netherlands internal report. Available at: http://www.cbs.nl/NR/rdonlyres/8EFDD386-0FD9-4567-90BC-6ADAB5C67B3D/0/2009Processesmethodsandtoolsart.pdf. (accessed February 12, 2013.).
43.
Rozanski, N. and Woods, E. (2005). Software Systems Architecture. New Jersey, United States of America: Addison Wesley.
44.
Schouten, B., Calinescu, M., and Luiten, A. (2011). Optimizing Quality of Response Through Adaptive Survey Designs, Statistics Netherlands, Available at: http://www.cbs.nl/NR/rdonlyres/2D62BF4A-6783-4AC4-8E45-12EF20C6675C/0/2011x1018.pdf. (accessed February 12, 2013).
45.
Sowa, J.F. and Zachman, J. (1992). Extending and Formalizing the Framework for Information Systems Architecture, IBM Systems Journal, 31.
46.
Statistics Canada (2012). Corporate Business Plan - Statistics Canada - 2012-13 to 2014-15. Available at: http://www.statcan.gc.ca/about-apercu/business-plan-affaires_2012-2015-eng.htm. (accessed February 12, 2013).
47.
Studman, B. (2010). A Collaborative Development Approach to Agile Statistical Processing Architecture - Australian Bureau of Statistics (ABS) Experience and Aspirations. UNECE/Eurostat/OECD Meeting on the Management of Statistical Information Systems (MSIS 2010). Daejeon, Republic of Korea, 26-29 April 2010.
48.
Sundgren, B. (2007). Process Reengineering at Statistics Sweden. From UNECE/Eurostat /OECD Meeting on the Management of Statistical Information Systems (MSIS). Geneva, 8-10 May 2007. Available at: http://www.unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.50/2007/mtg1/wp.22.e.pdf. (accessed February 13, 2013).
49.
Sundgren, B. (2010). The Systems Approach to Official Statistics. In Official Statistics Methodology and Applications in Honour of Daniel Thorburn. Stockholm University, Stockholm. 225-260. Available at: http://officialstatistics.files.wordpress.com/2010/05/bok18.pdf. (accessed February 12, 2013).
50.
Sundgren, B. (2011). Towards a System of Official Statistics Based on a Coherent Combination of Data Sources, Including Surveys and Administrative Data. Available at: https://sites.google.com/site/bosundgren/my-life/Towardsasystemofofficialstatisticsbasedonacoherentcombinationofdatasources.doc?attredirects¼0. (accessed February 12, 2013).
51.
The Open Group (2009). The Open Group Architecture Framework (TOGAF), Version 9.
52.
The Open Group: San Francisco, CA. Available from: http://www.togaf.info/. (accessed February 12, 2013).
53.
Todorov, V. (2011). Cooperation Models for Software Development. Meeting on the Management of Statistical Information Systems (MSIS 2011). Luxembourg, 23-25 May 2011.
54.
United Nations Economic Commission for Europe, UNECE (2011). Strategic Vision of the High-level Group for Strategic Developments in Business Architecture in Statistics. ECE/CES/2011/1. Economic Commission for Europe, Conference for European Statisticians, Geneva, 14-16 June 2011. United Nations Economic Commission for Europe, UNECE (2012a). Strategy to implement the vision of the High-level Group for Strategic Developments in Business Architecture in Statistics. ECE/CES/2012/10. Economic Commission for Europe, Conference for European Statisticians, Paris 6-8 June 2012.
55.
United Nations Economic Commission for Europe, UNECE (2012b). The Generic Statistical Information Model (GSIM): Version 0.8. Available at: http://www1.unece.org/stat/platform/display/metis/GSIMþv0.8. (accessed February 12, 2013).
56.
Vaccari, C. (2009). Common Reference Architecture (CORA) ESSnet: history and next activities, paper presented at the Meeting on the Management of Statistical Information Systems (MSIS 2009). Oslo, Norway, 18-20 May 2009. Available at: http://www.unece.org/stats/documents/2009.05.msis.htm. (accessed February 12, 2013).
57.
Vale, S. (2009). The Generic Statistical Business Process Model, Version 4.0. Available at: http://www1.unece.org/stat/platform/display/metis/TheþGenericþStatisticalþBusinessþProcessþModel. (accessed February 12, 2013).
58.
Wagner, J. (2008). Adaptive Survey Design to Reduce Nonresponse Bias. Unpublished Ph.D. dissertation, University of Michigan, Ann Arbor, MI.