In this article, I discuss the main approaches to resampling variance estimation in complex survey data: balanced repeated replication, the jackknife, and the bootstrap. Balanced repeated replication and the jackknife are implemented in the Stata svy suite. The bootstrap for complex survey data is implemented by the bsweights command. I describe this command and provide working examples.
BinderD. A.1983. On the variances of asymptotically normal estimators from complex surveys. International Statistical Review51: 279–292.
2.
BinderD. A., and RobertsG. R.2003. Design-based and model-based methods for estimating model parameters. In Analysis of Survey Data, ed. ChambersR. L., and SkinnerC. J., 29–48. New York: Wiley.
3.
CantyA. J., DavisonA. C., HinkleyD. V., and VenturaV.2006. Bootstrap diagnostics and remedies. Canadian Journal of Statistics34: 5–27.
4.
ChambersR. L., and SkinnerC. J. ed. 2003. Analysis of Survey Data.New York: Wiley.
5.
CochranW. G.1977. Sampling Techniques. 3rd ed. New York: Wiley.
EfronB., and TibshiraniR. J.1993. An Introduction to the Bootstrap.New York: Chapman & Hall/CRC.
8.
EltingeJ.1996. Discussion of “Resampling methods in sample surveys” by J. Shao. Statistics27: 241–244.
9.
GleasonJ. R.1988. Algorithms for balanced bootstrap simulations. American Statistician42: 263–266.
10.
GrahamR. L., HinkleyD. V., JohnP. W. M., and ShiS.1990. Balanced design of bootstrap simulations. Journal of the Royal Statistical Society, Series B52: 185–202.
11.
GuptaV. K., and NigamA. K.1987. Mixed orthogonal arrays for variance estimation with unequal numbers of primary selections per stratum. Biometrika74: 735–742.
12.
GurneyM., and JewettR. S.1975. Constructing orthogonal replications for variance estimation. Journal of the American Statistical Association70: 819–821.
13.
HedayatA. S., SloaneN. J. A., and StufkenJ.1999. Orthogonal Arrays: Theory and Applications.New York: Springer.
14.
HeeringaS. G., WestB. T., and BerglundP. A.2010. Applied Survey Data Analysis.Boca Raton, FL: Chapman & Hall/CRC.
15.
HuberP. J.1967. The behavior of maximum likelihood estimates under nonstandard conditions. In Vol. 1 of Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 221–233. Berkeley: University of California Press.
16.
JudkinsD. R.1990. Fay's method for variance estimation. Journal of Official Statistics6: 223–239.
17.
KishL.1995. Survey Sampling. 3rd ed. New York: Wiley.
18.
KornE. L., and GraubardB. I.1995. Analysis of large health surveys: Accounting for the sampling design. Journal of the Royal Statistical Society, Series A158: 263–295.
19.
KottP. S.2001. The delete-a-group jackknife. Journal of Official Statistics17: 521–526.
20.
KovarJ. G., RaoJ. N. K., and WuC. F. J.1988. Bootstrap and other methods to measure errors in survey estimates. Canadian Journal of Statistics16(Suppl.): 25–45.
21.
KrewskiD., and RaoJ. N. K.1981. Inference from stratified samples: Properties of the linearization, jackknife, and balanced repeated replication methods. Annals of Statistics9: 1010–1019.
22.
LehtonenR., and PahkinenE.2004. Practical Methods for Design and Analysis of Complex Surveys. 2nd ed. New York: Wiley.
23.
LohrS. L.2010. Sampling: Design and Analysis. 2nd ed. Pacific Grove, CA: Duxbury.
24.
LumleyT. S.2010. Complex Surveys: A Guide to Analysis Using R.Hoboken, NJ: Wiley.
25.
McCarthyP. J.1969. Pseudo-replication: Half samples. Review of the International Statistical Institute37: 239–264.
26.
McCarthyP. J., and SnowdenC. B.1985. The bootstrap and finite population sampling. In Vital and Health Statistics, 1–23. Washington, DC: U.S. Government Printing Office.
27.
NigamA. K., and RaoJ. N. K.1996. On balanced bootstrap for stratified multistage samples. Statistica Sinica6: 199–214.
28.
PhillipsO.2004. Using bootstrap weights with WesVar and SUDAAN. Research Data Centres Information and Technical Bulletin1: 6–15.
29.
RaoJ. N. K., and WuC. F. J.1985. Inference from stratified samples: Second-order analysis of three methods for nonlinear statistics. Journal of the American Statistical Association80: 620–630.
30.
RaoJ. N. K.1988. Resampling inference with complex survey data. Journal of the American Statistical Association83: 231–241.
31.
RaoJ. N. K., WuC. F. J., and YueK.1992. Some recent work on resampling methods for complex surveys. Survey Methodology18: 209–217.
32.
RustK. F., and RaoJ. N. K.1996. Variance estimation for complex surveys using replication techniques. Statistical Methods in Medical Research5: 283–310.
33.
SärndalC.-E., SwenssonB., and WretmanJ.1992. Model Assisted Survey Sampling.New York: Springer.
34.
ShaoJ.1996. Resampling methods in sample surveys (with discussion). Statistics27: 203–254.
35.
ShaoJ.2003. Impact of the bootstrap on sample surveys. Statistical Science18: 191–198.
36.
ShaoJ., and TuD.1995. The Jackknife and Bootstrap.New York: Springer.
37.
SitterR. R.1993. Balanced repeated replications based on orthogonal multi-arrays. Biometrika80: 211–221.
38.
SkinnerC. J.1989. Domain means, regression and multivariate analysis. In Analysis of Complex Surveys, ed. SkinnerC. J., HoltD., and SmithT. M. F., 59–88. New York: Wiley.
39.
SkinnerC. J., HoltD., and SmithT. M. F.1989. Analysis of Complex Surveys. NewYork: Wiley.
ThompsonM. E.1997. Theory of Sample Surveys.London: Chapman & Hall.
42.
ValliantR.1996. Discussion of “Resampling methods in sample surveys” by J. Shao. Statistics27: 247–251.
43.
ValliantR., BrickJ. M., and DeverJ. A.2008. Weight adjustments for the grouped jackknife variance estimator. Journal of Official Statistics24: 469–488.
44.
WestB. T., BerglundP., and HeeringaS. G.2008. A closer examination of subpopulation analysis of complex-sample survey data. Stata Journal8: 520–531.
45.
WhiteH.1982. Maximum likelihood estimation of misspecified models. Econometrica50: 1–26.
46.
WolterK. M.2007. Introduction to Variance Estimation. 2nd ed. New York: Springer.
47.
WuC. F. J.1991. Balanced repeated replications based on mixed orthogonal arrays. Biometrika78: 181–188.
48.
YeoD., MantelH., and LiuT.-P.1999. Bootstrap variance estimation for the National Population Health Survey. In Proceedings of the Survey Research Methods Section, 778–785. American Statistical Association.
49.
YungW.1997. Variance estimation for public use files under confidentiality constraints. In Proceedings of the Survey Research Methods Section, 434–439. American Statistical Association.
