Speaking Stata: Trimming to Taste

Abstract

Trimmed means are means calculated after setting aside zero or more values in each tail of a sample distribution. Here we focus on trimming equal numbers in each tail. Such trimmed means define a family or function with mean and median as extreme members and are attractive as simple and easily understood summaries of the general level (location, central tendency) of a variable. This article provides a tutorial review of trimmed means, emphasizing the scope for trimming to varying degrees in describing and exploring data. Detailed remarks are included on the idea's history, plotting of results, and confidence interval procedures. Examples are given using astronomical and medical data. The new Stata commands trimmean and trimplot are also included.

Keywords

st0313 trimmean trimplot trimming means medians midmeans Winsorizing robust resistant graphics

References

Aldrich

2007. “But you have to remember P. J. Daniell of Sheffield”. Electronic Journal for History of Probability and Statistics 3(2): 1–58.

Andrews

D. F.

, Bickel

P. J.

, Hampel

F. R.

, Huber

P. J.

, Rogers

W. H.

, and Tukey

J. W.

1972. Robust Estimates of Location: Survey and Advances. Princeton, NJ: Princeton University Press.

Anonymous. 1821. Probabilités. Dissertation sur la recherche du milieu le plus probable, entre les résultats de plusieurs observations ou expériences. Annales de Mathématiques pures et appliquées 12: 181–204. http://www.numdam.org/item?id=AMPA_1821-1822__12__181_0.

Barnett

, and Lewis

1994. Outliers in Statistical Data. 3rd ed. Chichester, UK: Wiley.

Bickel

P. J.

1965. On some robust estimates of location. Annals of Mathematical Statistics 36: 847–858.

Bickel

P. J.

, and Lehmann

E. L.

1975. Descriptive statistics for nonparametric models II. Location. Annals of Statistics 3: 1045–1069.

Breiman

1973. Statistics: With a View Toward Applications. Boston: Houghton Mifflin.

Cox

N. J.

1999. gr42: Quantile plots, generalized. Stata Technical Bulletin 51: 16–18. Reprinted in Stata Technical Bulletin Reprints, vol. 9, pp. 113–116. College Station, TX: Stata Press.

Cox

N. J.

2003. Stata tip 2: Building with floors and ceilings. Stata Journal 3: 446–447.

10.

Cox

N. J.

2005. Speaking Stata: The protean quantile plot. Stata Journal 5: 442–460.

11.

Cox

N. J.

2012a. Software update: gr42_6: Quantile plots, generalized. Stata Journal 12: 167.

12.

Cox

N. J.

2012b. Speaking Stata: Axis practice, or what goes where on a graph. Stata Journal 12: 549–561.

13.

Daniell

P. J.

1920. Observations weighted according to order. American Journal of Mathematics 42: 222–236.

14.

David

H. A.

, and Nagaraja

H. N.

2003. Order Statistics. 3rd ed. Hoboken, NJ: Wiley.

15.

Davison

A. C.

, and Hinkley

D. V.

1997. Bootstrap Methods and Their Application. Cambridge: Cambridge University Press.

16.

Dixon

W. J.

1960. Simplified estimation from censored normal samples. Annals of Mathematical Statistics 31: 385–391.

17.

Dixon

W. J.

, and Massey

F. J.

Jr. 1983. Introduction to Statistical Analysis. 4th ed. New York: McGraw–Hill.

18.

Dixon

W. J.

, and Tukey

J. W.

1968. Approximate behavior of the distribution of Winsorized t (trimming/Winsorization 2). Technometrics 10: 83–98.

19.

Dixon

W. J.

, and Yuen

K. K.

1974. Trimming and Winsorization: A review. Statistische Hefte 15: 157–170.

20.

Donoho

D. L.

, and Huber

P. J.

1983. The notion of breakdown point. In A Festschrift For Erich L. Lehmann, ed. Bickel

P. J.

, Doksum

, and Hodges

J. L.

Jr. , 157–184. Belmont, CA: Wadsworth.

21.

Efron

, and Tibshirani

R. J.

1993. An Introduction to the Bootstrap. New York: Chapman & Hall/CRC.

22.

Ender

P. B.

2009. robmean: Trimmed, Winsorized means & Huber 1-step estimator. UCLA: Statistical Consulting Group. http://www.ats.ucla.edu/stat/stata/ado/analysis.

23.

Erickson

B. H.

, and Nosanchuk

T. A.

1977. Understanding Data. Toronto, Canada: McGraw–Hill.

24.

Erickson

B. H.

, and Nosanchuk

T. A.

1992. Understanding Data. 2nd ed. Toronto, Canada: University of Toronto Press.

25.

Feigelson

E. D.

, and Babu

G. J.

2012. Modern Statistical Methods for Astronomy: With R Applications. Cambridge: Cambridge University Press.

26.

Flournoy

1993. A conversation with Wilfrid J. Dixon. Statistical Science 8: 458–477.

27.

Flournoy

2010. Wilfrid Joseph Dixon, 1915–2008. Journal of the Royal Statistical Society, Series A 173: 455–457.

28.

Gauss

C. F.

1816. Bestimmung der Genauigkeit der Beobachtungen. Zeitschrift fü r Astronomie und verwandte Wissenschaften 1: 187–197.

29.

Gordin

M. D.

2004. A Well-Ordered Thing: Dmitrii Mendeleev and the Shadow of the Periodic Table. New York: Basic Books.

30.

Hamilton

L. C.

1991. sed4: Resistant normality check and outlier identification. Stata Technical Bulletin 3: 15–18. Reprinted in Stata Technical Bulletin Reprints, vol. 1, pp. 86–90. College Station, TX: Stata Press.

31.

Hampel

F. R.

1985. The breakdown points of the mean combined with some rejection rules. Technometrics 27: 95–107.

32.

Hampel

F. R.

1997. Some additional notes on the “Princeton robustness year”. In The Practice of Data Analysis: Essays in Honor of John W. Tukey, ed. Brillinger

D. R.

, Fernholz

L. T.

, and Morgenthaler

, 133–153. Princeton, NJ: Princeton University Press.

33.

Hampel

F. R.

, Ronchetti

E. M.

, Rousseeuw

P. J.

, and Stahel

W. A.

1986. Robust Statistics: The Approach Based on Influence Functions. New York: Wiley.

34.

Harter

H. L.

1974a. The method of least squares and some alternatives—Part I. International Statistical Review 42: 147–174.

35.

Harter

H. L.

1974b. The method of least squares and some alternatives—Part II. International Statistical Review 42: 235–264 and 282.

36.

Helsel

D. R.

, and Hirsch

R. M.

1992. Statistical Methods in Water Resources. Amsterdam: Elsevier. http://pubs.usgs.gov/twri/twri4a3/.

37.

Hill

, and Dixon

W. J.

1982. Robustness in real life: A study of clinical laboratory data. Biometrics 38: 377–396.

38.

Huber

P. J.

1964. Robust estimation of a location parameter. Annals of Mathematical Statistics 35: 73–101.

39.

Huber

P. J.

1972. The 1972 Wald lecture: Robust statistics: A review. Annals of Mathematical Statistics 43: 1041–1067.

40.

Huber

P. J.

1981. Robust Statistics. New York: Wiley.

41.

Huber

P. J.

, and Ronchetti

E. M.

2009. Robust Statistics. 2nd ed. Hoboken, NJ: Wiley.

42.

Jennrich

R. I.

2007. BMDP and some statistical computing history. Statistical Computing and Graphics 18: 17–23.

43.

Jurečková

, and Picek

2006. Robust Statistical Methods with R. Boca Raton, FL: Chapman & Hall/CRC.

44.

Kafadar

2003. John Tukey and robustness. Statistical Science 18: 319–331.

45.

Kim

S.-J.

1992. The metrically trimmed mean as a robust estimator of location. Annals of Statistics 20: 1534–1547.

46.

Kleiner

, and Graedel

T. E.

1980. Exploratory data analysis in the geophysical sciences. Reviews of Geophysics and Space Physics 18: 699–717.

47.

Lehmann

E. L.

1983. Theory of Point Estimation. New York: Wiley.

48.

Mazliak

2012. Poincaré's odds. Séminaire Poincaré 16: 173–210.

49.

Miller

R. G.

1986. Beyond ANOVA: Basics of Applied Statistics. New York: Wiley. Reprinted, London: Chapman & Hall, 1997.

50.

Mosteller

, and Tukey

J. W.

1977. Data Analysis and Regression: A Second Course in Statistics. Reading, MA: Addison–Wesley.

51.

Olive

D. J.

2001. High breakdown analogs of the trimmed mean. Statistics & Probability Letters 51: 87–92.

52.

Pearson

R. K.

2011. Exploring Data in Engineering, the Sciences, and Medicine. New York: Oxford University Press.

53.

Poincaré

1896. Calcul des probabilités. Paris: Gauthier-Villars.

54.

Poincaré

1912. Calcul des probabilité s. 2nd ed. Paris: Gauthier-Villars. http://archive.org/details/calculdeprobabil00poinrich.

55.

Reimann

, Filzmoser

, Garrett

, and Dutter

2008. Statistical Data Analysis Explained: Applied Environmental Statistics with R. Chichester, UK: Wiley.

56.

Rice

J. A.

2007. Mathematical Statistics and Data Analysis. 3rd ed. Belmont, CA: Duxbury.

57.

Rosenberger

J. L.

, and Gasko

1983. Comparing location estimators: Trimmed means, medians, and trimean. In Understanding Robust and Exploratory Data Analysis, ed. Hoaglin

D. C.

, Mosteller

, and Tukey

J. W.

, 297–338. New York: Wiley.

58.

Short

1763. Second paper concerning the parallax of the sun determined from the observations of the late transit of Venus, in which this subject is treated of more at length, and the quantity of the parallax more fully ascertained. Philosophical Transactions of the Royal Society of London 53: 300–345.

59.

Siegel

A. F.

1988. Statistics and Data Analysis: An Introduction. New York: Wiley.

60.

Sprent

, and Smeeton

N. C.

2007. Applied Nonparametric Statistical Methods. 4th ed. Boca Raton, FL: Chapman & Hall/CRC.

61.

Staudte

R. G.

, and Sheather

S. J.

1990. Robust Estimation and Testing. New York: Wiley.

62.

Stewart

C. A.

1947. P. J. Daniell. Journal of the London Mathematical Society 22: 75–80.

63.

Stigler

S. M.

1973. Simon Newcomb, Percy Daniell, and the history of robust estimation 1885–1920. Journal of the American Statistical Association 68: 872–879.

64.

Stigler

S. M.

1976. The anonymous Professor Gergonne. Historia Mathematica 3: 71–74.

65.

Stigler

S. M.

1977. Do robust estimators work with real data? Annals of Statistics 5: 1055–1098.

66.

Stigler

S. M.

2010. The changing history of robustness. American Statistician 64: 277–281.

67.

Tilanus

C. B.

, and Rey

1964. Input-output volume and value predictions for the Netherlands, 1948–1958. International Economic Review 5: 34–45.

68.

Tukey

J. W.

1960. A survey of sampling from contaminated distributions. In Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, ed. Olkin

, Ghurye

, Hoeffding

, Madow

, and Mann

, 448–485. Stanford, CA: Stanford University Press.

69.

Tukey

J. W.

1962. The future of data analysis. Annals of Mathematical Statistics 33: 1–67.

70.

Tukey

J. W.

1970a. Exploratory Data Analysis. Limited preliminary ed. Reading, MA: Addison–Wesley.

71.

Tukey

J. W.

1970b. Some further inputs. In Geostatistics: A Colloquium, ed. Merriam

D. F.

, 163–174. New York: Plenum.

72.

Tukey

J. W.

1977. Exploratory Data Analysis. Reading, MA: Addison–Wesley.

73.

Tukey

J. W.

1986. Choosing techniques for the analysis of data. In The Collected Works of John W. Tukey: Philosophy and Principles of Data Analysis: 1965–1986, Volume IV, ed. Jones

L. V.

, 869–874. Monterey, CA: Wadsworth and Brooks/Cole.

74.

Tukey

J. W.

, and McLaughlin

D. H.

1963. Less vulnerable confidence and significance procedures for location based on a single sample: Trimming/Winsorization 1. Sankhyā 25: 331–352.

75.

Turner

G. L'E.

1969. James Short, F.R.S., and his contribution to the construction of reflecting telescopes. Notes and Records of the Royal Society of London 24: 91–108.

76.

van Belle

, Fisher

L. D.

, Heagerty

P. J.

, and Lumley

2004. Biostatistics: A Methodology For the Health Sciences. 2nd ed. Hoboken, NJ: Wiley.

77.

Venables

W. N.

, and Ripley

B. D.

2002. Modern Applied Statistics with S. 4th ed. New York: Springer.

78.

Wilcox

R. R.

2003. Applying Contemporary Statistical Techniques. San Diego: Academic Press.

79.

Wilcox

R. R.

2009. Basic Statistics: Understanding Conventional Methods and Modern In-sights. New York: Oxford University Press.

80.

Wilcox

R. R.

2010. Fundamentals of Modern Statistical Methods: Substantially Improving Power and Accuracy. 2nd ed. New York: Springer.

81.

Wilcox

R. R.

2012a. Introduction to Robust Estimation and Hypothesis Testing. 3rd ed. Waltham, MA: Elsevier.

82.

Wilcox

R. R.

2012b. Modern Statistics for the Social and Behavioral Sciences: A Practical Introduction. Boca Raton, FL: CRC Press.

83.

Woolf

1959. The Transits of Venus: A Study of Eighteenth-century Science. Princeton, NJ: Princeton University Press.

84.

Wulf

2012. Chasing Venus: The Race to Measure the Heavens. New York: Alfred A. Knopf.