Simultaneous tests and confidence bands for Stata estimation commands

Abstract

Stata estimation commands that implement frequentist methods produce an output table that contains multiple tests and multiple confidence intervals. Presumably, the multiple tests and multiple confidence are designed to help determine which parameters are responsible for a possible rejection of the overall null hypothesis of no effect. When taken by itself, each test and each confidence interval provides valid inference about the null hypothesis of no effect for each parameter at the specified error rate. However, simultaneously using two or more of these tests or confidence intervals provides inference at an error rate that exceeds the one specified.

In this article, I discuss the sotable command, which provides p-values that are adjusted for the multiple tests and a confidence band that can be used to simultaneously test multiple parameters for no effect after almost all frequentist estimation commands. I also provide an introduction to the literature on simultaneous inference.

Keywords

st0718 sotable multiple comparisons confidence bands simultaneous inference multiple testing simultaneous tests

Get full access to this article

View all access options for this article.

References

Cameron

A. C.

Trivedi

P. K.

2005. Microeconometrics: Methods and Applications. New York: Cambridge University Press.

Casella

Berger

R. L.

2002. Statistical Inference. 2nd ed. Pacific Grove, CA: Duxbury.

Clarke

Romano

J. P.

Wolf

2020. The Romano–Wolf multiple-hypothesis correction in Stata. Stata Journal 20: 812–843. https://doi.org/10.1177/1536867X20976314.

Dunnett

C. W.

1955. A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association 50: 1096–1121. https://doi.org/10.2307/2281208.

Elfenbein

D. W.

McManus

2010. A greater price for a greater good? Evidence that consumers pay more for charity-linked products. American Economic Journal: Economic Policy 2: 28–60. https://doi.org/10.1257/pol.2.2.28.

Genz

Bretz

2002. Comparison of methods for the computation of multivariate t probabilities. Journal of Computational and Graphical Statistics 11: 950–971. https://doi.org/10.1198/106186002394.

Genz

Bretz

2009. Computation of Multivariate Normal and t Probabilities. Berlin: Springer. https://doi.org/10.1007/978-3-642-01689-9.

Hochberg

Tamhane

A. C.

1987. Multiple Comparison Procedures. New York: Wiley. https://doi.org/10.1002/9780470316672.

Hothorn

Bretz

Westfall

2008. Simultaneous inference in general parametric models. Biometrical Journal 50: 346–363. https://doi.org/10.1002/bimj.200810425.

10.

Hsu

J. C.

1996. Multiple Comparisons: Theory and Methods. London: Chapman and Hall/CRC. https://doi.org/10.1201/b15074.

11.

Miller

R. G.

Jr.

1966. Simultaneous Statistical Inference. New York: McGraw–Hill.

12.

Montiel Olea

J. L.

Plagborg-Møller

2019. Simultaneous confidence bands: Theory, implementation, and an application to SVARs. Journal of Applied Econometrics 34: 1–17. https://doi.org/10.1002/jae.2656.

13.

Romano

J. P.

Shaikh

A. M.

Wolf

2010. Hypothesis testing in econometrics. Annual Review of Economics 2: 75–104. https://doi.org/10.1146/annurev.economics.102308.124342.

14.

Romano

J. P.

Wolf

2005. Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association 100: 94–108. https://doi.org/10.1198/016214504000000539.

15.

Roy

S. N.

1953. On a heuristic method of test construction and its use in multivariate analysis. Annals of Mathematical Statistics 24: 220–238. https://doi.org/10.1214/aoms/1177729029.

16.

Roy

S. N.

Bose

R. C.

1953. Simultaneous confidence interval estimation. Annals of Mathematical Statistics 24: 513–536. https://doi.org/10.1214/aoms/1177728912.

17.

Savin

N. E.

1984. Multiple hypothesis testing. In Vol. 2 of Handbook of Econometrics, ed. Griliches

Intriligator

M. D.

, 827–879. Amsterdam: Elsevier. https://doi.org/10.1016/S1573-4412(84)02006-7.

18.

Tamhane

A. C.

1996. Multiple comparisons. In Design and Analysis of Experiments, ed. Ghosh

Rao

C. R.

Vol. 13 of Handbook of Statistics, 587–630. New York: Elsevier. https://doi.org/10.1016/S0169-7161(96)13020-0.

19.

Wasserman

2004. All of Statistics: A Concise Course in Statistical Inference. New York: Springer. https://doi.org/10.1007/978-0-387-21736-9.

20.

Westfall

P. H.

Young

S. S.

1993. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. New York: Wiley.

21.

Wooldridge

J. M.

1999. Distribution-free estimation of some nonlinear panel data models. Journal of Econometrics 90: 77–97. https://doi.org/10.1016/S0304-4076(98)00033-5.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.22 MB

0.00 MB