Determination of the Threshold in Cutoff Sampling Using Response Burden with an Application to Intrastat

Abstract

Statistical offices frequently use cutoff sampling to determine which businesses in a population should be surveyed. Examples include business surveys about international trade, production, innovation, ICT usage and so on. Cutoff thresholds are typically set in terms of key variables of interest and aim to satisfy a minimum coverage ratio–the share of aggregate values of reporting units. In this article we propose a simple cost-benefit approach to determination of the sampling cutoff by taking into account the response burden. In line with existing practice, we use the coverage ratio as our measure of accuracy and provide either analytical or numerical solutions to cutoff determination. Using a business survey on response burden of reporting trade flows within the EU (Intrastat), we present an application that illustrates our approach to cutoff determination. An important practical implication is the possibility to set industry-contingent cutoffs.

Keywords

Accuracy cost-benefit analysis reporting costs social benefits social welfare function

References

Abowd, J.M., and I.M. Schmutte. 2019. “An Economic Analysis of Privacy Protection and Statistical Accuracy as Social Choices.” American Economic Review 109(1): 171–202. DOI: https://doi.org/10.1257/aer.20170627.

Adler, M.D. 2019. Measuring Social Welfare: An Introduction. Oxford: Oxford University Press.

Adler, M.D., and E.A. Posner. 1999. “Rethinking Cost-Benefit Analysis.” The Yale Law Journal 109(2): 165–247. DOI: https://doi.org/10.2307/797489.

AJPES. 2014. “Annual Balance Sheets and Income Statements Data, 2013.” [data collection]. Available at: https://www.ajpes.si/Letna_porocila/Splosno. (accessed January 2021).

Axtell, R. 2001. “Zipf Distribution of U.S. Firm Sizes.” Science 293: 1818–1820. DOI: https://www.ajpes.si/Letna_porocila/Splosno.

Baillargeon, S., and L.-P. Rivest. 2009. “A general algorithm for univariate stratification.” International Statistical Review 77(3): 331–344. DOI: https://doi.org/10.1111/j.1751-5823.2009.00093.x.

Bavdaž, M., D. Giesen, S. KorenjakČerne, T. Löfgren, and V. Raymond-Blaess. 2015. “Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes.” Journal of Official Statistics 31(4): 559–588. DOI: https://doi.org/10.1515/jos-2015-0035.

Bee M., R. Benedetti, E. Espa, and F. Piersimoni. 2011. “Cut-Off Approach To The Design Of Longitudinal Business Surveys,” In Proceedings of the Section on Government Statistics, Joint Statistical Meetings, July 30–August 4, 2011: 2488–2500, Alexandria, VA: American Statistical Association.

Bee, M., and S. Schiavo. 2018. “Powerless: gains from trade when firm productivity is not Pareto distributed.” Review of World Economics 154(1): 15–45. DOI: https://doi.org/10.1007/s10290-017-0295-z.

10.

Benedetti, R., M. Bee, and G. Espa. 2010. “A framework for cut-off sampling in business survey design.” Journal of Official Statistics 26(4): 651–671. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/20200206/jos_cutoff_benedetti-mfl.pdf. (accessed January 2021).

11.

Bikhchandani, S., and J.W. Mamer. 2013. “Decreasing marginal value of information under symmetric loss.” Decision Analysis 10(3): 245–256. DOI: https://doi.org/10.1287/deca.2013.0274.

12.

Bradburn, N. 1978. “Respondent burden.” In Proceedings of the Survey Research Methods Section, Joint Statistical Meetings. August 14–17, 1978: 35–40, Alexandria, VA: American Statistical Association. Available at: http://www.asasrms.org/Proceedings/papers/1978_007.pdf. (accessed July 2021).

13.

Cabral, L.M.B., and J. Mata. 2003. “On the Evolution of the Firm Size Distribution: Facts and Theory.” American Economic Review 93(4): 1075–1090. DOI: https://doi.org/10.1257/000282803769206205.

14.

Chaudron, R., and K. Carlier. 2015. “The advantages of random sampling versus cutting-of-the-tail: the application of a stratified sample design for the collection of data on special financial institutions in the Netherlands.” IFC Bulletin 39: 37. Available at: https://EconPapers.repec.Org/RePEc:bis:bisifc:39-37. (accessed July 2021).

15.

Christensen, L.R., and W. Green. 1976. “Economies of Scale in U.S. Electric Power.” Journal of Political Economy 84(4): 655–676. Available at: https://econpapers.repec.org/RePEc:ucp:jpolec:v:84:y:1976:i:4:p:655-76. (accessed January 2021).

16.

Commission Implementing Regulation (EU) 2020/1197 of 30 July 2020 laying down technical specifications and arrangements pursuant to Regulation (EU) 2019/2152 of the European Parliament and of the Council on European business statistics repealing 10 legal acts in the field of business statistics (Text with EEA relevance) (2020). Official Journal L271, 18.8.2020: 1–170. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32020R1197. (accessed July 2021).

17.

Dalén, J. 2005. “Sampling Issues in Business Surveys.” PHARE 2002 Multi Beneficiary Statistics Programme (Lot 1). Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/20-Quality-in-statistics-sampling-issues-in-business-statistics-2005.pdf. (accessed July 2021).

18.

Eurostat. 2007a. Impact of a change in threshold on the number of PSIs. Doc QA/G3/05-07/10 (SIM A/10).

19.

Eurostat. 2007b. Impact of a change in threshold on data quality at detailed level. Doc QA/G3/05-07/11 (SIM A/11).

20.

Eurostat. 2018. European Statistics Code of Practice. Luxembourg: Publications Office of the European Union. Available at: https://ec.europa.eu/eurostat/documents/4031688/8971242/KS-02-18-142-EN-N.pdf. (accessed January 2021).

21.

Eurostat. 2020a. ICT usage in enterprises (isoc_e). Available at: https://ec.europa.eu/eurostat/cache/metadata/en/isoc_e_esms.htm. (accessed January 2021).

22.

Eurostat. 2020b. Labour costs survey 2008, 2012 and 2016–NACE Rev. 2 activity (lcs_r2). Available at: https://ec.europa.eu/eurostat/cache/metadata/en/lcs_r2_esms.htm. (accessed January 2021).

23.

Eurostat. 2020c. Results of the community innovation survey (CIS2-CIS7). Available at: https://ec.europa.eu/eurostat/cache/metadata/en/inn_cis2_esms.htm. (accessed January 2021).

24.

Eurostat. 2020d. Statistics on the production of manufactured goods (prom). Available at: https://ec.europa.eu/eurostat/cache/metadata/en/prom_esms.htm. (accessed January 2021).

25.

Eurostat. 2021a. European business statistics compilers’ manual for international trade in goods statistics. Luxembourg: Publications Office of the European Union. Available at: https://ec.europa.eu/eurostat/documents/3859598/12399711/KS-GQ-21-004-ENN.pdf. (accessed June 2021).

26.

Eurostat. 2021b. National requirements for the Intrastat system. Luxembourg: Publications Office of the European Union. Available at: https://ec.europa.eu/eurostat/documents/3859598/12889594/KS-GQ-21-011-EN-N.pdf. (accessed June 2021).

27.

Fernandes, A.M., P.J. Klenow, S. Meleshchuk, M.D. Pierola, and A. Rodriguez-Clare. 2018. The Intensive Margin In Trade. World Bank Policy Research Working Paper. DOI: https://doi.org/10.1596/1813-9450-8625.

28.

Gabaix, X. 2009. “Power Laws in Economics and Finance.” Annual Review of Economics 1: 255–294. DOI: https://doi.org/10.1146/annurev.economics.050708.142940.

29.

Giesen, D., M. Vella, C.F. Brady, P. Brown, D. Ravindra, and A. Vaasen-Otten. 2018. “Response Burden Management for Establishment Surveys at Four National Statistical Institutes.” Journal of Official Statistics 34(2): 397–418. DOI: https://doi.org/10.2478/jos-2018-0018.

30.

Guadarrama, M., I. Molina, and Y. Tillé. 2020. “Small area estimation methods under cutoff sampling.” Survey Methodology 46(1): 51–75. Available at: https://www150.statcan.gc.ca/n1/en/catalogue/12-001-X202000100004. (accessed July 2021).

31.

Haziza, D., G. Chauvet, and J.C. Deville. 2010. “Sampling and estimation in the presence of cut-off sampling.” Australian & New Zealand Journal of Statistics 52(3): 303–319. DOI: https://doi.org/10.1111/j.1467-842X.2010.00584.x.

32.

Head, K., T. Mayer, and M. Thoenig. 2014. “Welfare and Trade Without Pareto.” American Economic Review: Papers & Proceedings 104(5): 310–316. DOI: https://doi.org/10.1257/aer.104.5.310.

33.

Knaub, J.R. 2007. Cutoff Sampling and Inference. InterStat. DOI: https://doi.org/10.1111/j.1467-842X.2010.00584.x. (accessed January 2021).

34.

Lavallée, P., and M.A. Hidiroglou. 1988. “On the stratification of skewed populations.” Survey Methodology 14: 33–43. Available at: https://www150.statcan.gc.ca/n1/en/catalogue/12-001-X198800114602. (accessed July 2021).

35.

Layard, R., and S. Gleister. 1994. “Introduction.” In Cost-Benefit Analysis, edited by R. Layard, and S. Gleister: 1–56, Cambridge (UK): Cambridge University Press. DOI: http://dx.doi.org/10.1017/CBO9780511521942.001.

36.

Lee, S.E., and K.-I. Shin. 2016. “The cut-off point based on underlying distribution and cost function.” Journal of Applied Statistics 43(6): 1061–1073. DOI: http://dx.doi.org/10.1080/02664763.2015.1089222.

37.

Melitz, M.J. 2003. “The Impact of Trade on Intra-Industry Reallocations and Aggregate Industry Productivity.” Econometrica 71(6): 1695–1725. DOI: https://doi.org/10.1111/1468-0262.00467.

38.

Nerlove, M. 1963. “Returns to scale in electricity supply”. In Measurement in Economics-Studies in Mathematical Economics and Econometrics in Memory of Yehuda Grunfeld, by C.F. Christ, M. Friedman, L.A. Goodman, Z. Griliches, A.C. Harberger, Y. Mundlak, M. Nerlove, D. Patinkin, L.G. Telser, and H. Theil: 167–198, Stanford (Calif.): Stanford University Press.

39.

Office for Statistics Regulation. 2020. The Public Good of Statistics: What we know so far. London: Office for Statistics Regulation. Available at: https://osr.statisticsauthority.gov.uk/publication/the-public-good-of-statistics-what-we-know-so-far/. (accessed January 2021).

40.

Perše, S. 2016. Analiza stroskov in koristi moznih razvojnih scenarijev sistema Intrastat v Sloveniji (Cost-Benefit Analysis of Intrastat Development Options in Slovenia). MSc dissertation. Ljubljana: University of Ljubljana, School of Economics and Business. Available at: http://www.cek.ef.uni-lj.si/magister/perse5068.pdf. (accessed January 2021)

41.

Prest, A.R., and R. Turvey. 1965. “Cost-Benefit Analysis: A Survey.” The Economic Journal 75(300): 683–735. DOI: http://dx.doi.org/10.2307/2229670.

42.

Regulation (EC) No 638/2004 of the European Parliament and of the Council of 31 March 2004 on Community statistics relating to the trading of goods between Member States and repealing Council Regulation (EEC) No 3330/91 (2004). Official Journal L102, 7.4.2004: 1–8. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32004R0638 (accessed July 2021).

43.

Regulation (EC) No 222/2009 of the European Parliament and of the Council of 11 March 2009 amending Regulation (EC) No 638/2004 on Community statistics relating to the trading of goods between Member States (2004). Official Journal L87, 31.3.2009: 160–163. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32009R0222 (accessed July 2021).

44.

Samuelson, P.A. 1954. “The Pure Theory of Public Expenditure.” Review of Economics and Statistics 36: 386–389. DOI: https://doi.org/10.2307/1925895.

45.

Smith, P.A., and W. Yung. 2023. “Introduction to sampling and estimation for business surveys.” In Advances in Business Statistics, Methods and Data Collection edited by G. Snijkers, M. Bavdaž, S. Bender, J. Jones, S. MacFeely, J.W. Sakshaug, K.J. Thompson, and A. van Delden: chap. 27. Hoboken: Wiley.

46.

Spencer, B.D. 1985. “Optimal Data Quality.” Journal of the American Statistical Association 80(391): 564–573. DOI: https://doi.org/10.1080/01621459.1985.10478155.

47.

Spencer, B.D., and L.E. Moses. 1990. “Needed Data Expenditure for an Ambiguous Decision Problem.” Journal of the American Statistical Association 85(412): 1099–1104. DOI: https://doi.org/10.1080/01621459.1990.10474981.

48.

Spencer, B.D., and Z.H. Seeskin. 2015. “Effects of Census Accuracy on Apportionment of Congress and Allocations of Federal Funds.” In JSM Proceedings, Government Statistics Section, August 8–13: 3061–3075. Alexandria, VA: American Statistical Association. Available at: https://www.norc.org/PDFs/Publications/SeeskinZ_Effects%20of%20Census%20Accuracy_2015.pdf. (accessed January 2021).

49.

Stern, N.H. 1976. “On the specification of models of optimum income taxation.” Journal of Public Economics 6(1–2): 123–162. DOI: https://doi.org/10.1016/0047-2727(76)90044-X.

50.

SURS. 2014. Survey on current response burden in Intrastat, 2013. [data collection].

51.

SURS and FURS. 2014. Intra-EU and Extra-EU Trade Flows, 2013. [data collection]. Available at: https://www.stat.si/StatWeb/en/DataCollection/Intrastat (accessed January 2021).

52.

Trottini, M., and S.E. Fienberg. 2002. “Modelling User Uncertainty for Disclosure Risk and Data Utility.” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 10(5): 511–528. DOI: https://doi.org/10.1142/S0218488502001612.

53.

Yorgason, D., B. Bridgman, Y. Cheng, A.H. Dorfman, J. Lent, Y.K. Liu, J. Miranda, and S. Rumburg. 2011. “Cutoff Sampling in Federal Surveys: an Inter-Agency Review.” In Proceedings of the American Statistical Association, Section on Government Statistics: 76–90. Available at: https://www.bls.gov/osmr/research-papers/2011/pdf/st110050.pdf. (accessed June 2021).