Setting M-Estimation Parameters for Detection and Treatment of Influential Values

Abstract

Recent research on the use of M-estimation methodology for detecting and treating verified influential values in economic surveys found that initial parameter settings affect effectiveness. In this article, we explore the basic question of how to develop initial settings for the M-estimation parameters. The economic populations that we studied are highly skewed and are consequently highly stratified. While we investigated settings for several parameters, the most challenging problem was to develop an “automatic” data-driven method for setting the initial value of the tuning constant φ, the parameter with the greatest influence on performance of the algorithm. Of all the methods that we considered, we found that methods defined in terms of the accuracy of published estimates can be implemented on a large scale and yielded the best performance. We illustrate the methodology with an empirical analysis of 36 consecutive months of data from 19 industries in the Monthly Wholesale Trade Survey.

Keywords

Outlier economic surveys Monthly Wholesale Trade Survey

References

Beaumont, J.-F. 2004. “Robust Estimation of a Finite Population Total in the Presence of Influential Units.” Report for the Office for National Statistics. Newport, U.K.: Office for National Statistics.

Beaumont, J.-F. and A. Alavi. 2004. “Robust Generalized Regression Estimation.” Survey Methodology 30(2): 195–208. Available at: http://www.statcan.gc.ca/pub/12-001-x/2004002/article/7752-eng.pdf (accessed August 2017).

Beaumont, J.-F., D. Haziza, and A. Ruiz-Gazen. 2013. “A Unified Approach to Robust Estimation in Finite Population Sampling.” Biometrika 100: 555–569. Doi: https://doi.org/10.1093/biomet/ast010.

Binder, D.A., C. Babyak, M. Brodeur, M. Hidiroglou, and W. Jocelyn. 2000. “Variance Estimation for Two-Phase Stratified Sampling.” The Canadian Journal of Statistics 28(4): 751–764. Doi: https://doi.org/10.2307/3315914.

Clark, R.G. 1995. “Winsorization Methods in Sample Surveys.” Masters Thesis. Department of Statistics. Canberra: Australia National University. Available at: http://hdl.handle.net/10440/1031 (accessed August 2017).

Clark, R.G., P. Kokic, and P.A. Smith. 2017 “A Comparison of two Robust Estimation Methods for Business Surveys.” International Statistical Review 85: 270–289. Doi: https://doi.org/10.1111/insr.12177.

Duchesne, P. 1999. “Robust Calibration Estimators.” Survey Methodology 25(1): 43–56. Available at: http://hbanaszak.mjr.uw.edu.pl/TempTxt/Duchesne_1999_RobustCalibrationEstimators.PDF (accessed August 2017).

Gwet, J.-P. and L.-P. Rivest. 1992. “Outlier Resistant Alternatives to Ratio Estimator.” Journal of the American Statistical Association 87: 1174–1182. Available at: https://www.mat.ulaval.ca/fileadmin/mat/documents/lrivest/Publications/26-GwetRivest1992.pdf (accessed August 2017).

Hampel, F.R., E.M. Ronchetti, P.J. Rousseeuw, and S.A. Werner. 1986. Robust Statistics. An Approach Based on Influence Functions. New York, NY: John Wiley & Sons.

10.

Huang, E.T. 1984. “An Imputation Study for the Monthly Retail Trade Survey.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Philadelphia, Pennsylvania, August 13–16, 1984. Alexandria, VA: American Statistical Association. 610–615. Available at: http://ww2.amstat.org/sections/srms/Proceedings/papers/1984_117.pdf (accessed August 2017).

11.

Huang, E.T. 1986. “Report on the Imputation Research for the Monthly Retail Trade Survey.” Statistical Research Report Series No. CENSUS/SRD/RR-86-09. U.S. Census Bureau. Washington, DC. Available at: https://www.census.gov/srd/papers/pdf/rr86-09.pdf (accessed August 2017).

12.

Huber, P.J. 1964. “Robust Estimation of a Location Parameter.” Annals of Mathematical Statistics 35: 73–101. Doi: https://doi.org/10.1214/aoms/1177703732.

13.

Hulliger, B. 1995. “Outlier Robust Horvitz-Thompson Estimators.” Survey Methodology 21(1): 79–87. Available at: http://www.statcan.gc.ca/pub/12-001-x/1995001/article/14407-eng.pdf (accessed August 2017).

14.

Hulliger, B. 1999. “Simple and Robust Estimators for Sampling.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Baltimore, MD, August 8–12. Alexandria, VA: American Statistical Association. 54–63. Available at: http://ww2.amstat.org/sections/srms/Proceedings/papers/1999_009.pdf (accessed August 2017).

15.

Martinoz, C.F., D. Haziza, and J.-F. Beaumont. 2015. “A Method of Determining the Winsorization Threshold, with an Application to Domain Estimation.” Survey Methodology 41(1): 57–77.

16.

Mulry, M.H. and R. Feldpausch. 2007. “Investigation of Treatment of Influential Values.” Proceedings of the Third International Conference on Establishment Surveys, Montreal, Quebec, Canada, June 18–21, 2007. Alexandria, VA: American Statistical Association. 1173–1179. Available at: http://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000229.PDF (accessed April 2018).

17.

Mulry, M.H., S. Kaputa, and K.J. Thompson. 2016. “A Cautionary Note on Clark Winsorization.” Survey Methodology 42(2): 297–305. Available at: http://www.statcan.gc.ca/pub/12-001-x/2016002/article/14676-eng.pdf (accessed August 2017).

18.

Mulry, M.H., B. Oliver, and S. Kaputa. 2014. “Detecting and Treating Verified Influential Values in a Monthly Retail Trade Survey.” Journal of Official Statistics 30(4): 721–747. Doi: http://dx.doi.org/10.2478/JOS-2014-0045.

19.

Mulry, M.H., B. Oliver, S. Kaputa, and K.J. Thompson. 2013. “Setting M-Estimation Parameters for Detection and Treatment of Influential Values.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Montréal, Quebéc, Canada, August 3–8, 2013. Alexandria, VA: American Statistical Association. 1424–1438. Available at: http://ww2.amstat.org/sections/srms/Proceedings/y2013/Files/308309_80627.pdf (accessed August 2017).

20.

Mulry, M.H., B. Oliver, and S. Kaputa. 2012. “Several Scenarios for Influential Values in Business Surveys and Methods for Their Treatment.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, San Diego, CA, July 28–August 2, 2012. Alexandria, VA: American Statistical Association. 4015–4029. Available at: http://ww2.amstat.org/sections/srms/Proceedings/y2012/Files/304652_73493.pdf (accessed August 2017).

21.

Särndal, C.-E., B. Swensson, and J. Wretman. 1992. Model Assisted Survey Sampling. Springer-Verlag. New York, NY.

22.

SAS/STAT(R) 9.22 User’s Guide. Web. 06 Apr. 2016.

23.

Thompson, K.J. and R.S. Sigman. 1996. Evaluation of Statistical Methods for Developing Ratio Edit Module Parameters. Technical report #ESM-9610. Washington, DC: U.S. Bureau of the Census. Available upon request.

24.

Wang, Y.-G., X. Lin, M. Zhu, and Z. Bai. 2007. “Robust Estimation Using the Huber Function with a Data-Dependent Tuning Constant.” Journal of Computational and Graphical Statistics 16(2): 468–481.Doi: http://dx.doi.org/10.1198/106186007X180156.

25.

White, H. 1980. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica 48(4): 817–838. Doi: https://doi.org/10.2307/1912934.