Abstract
Statistical agencies depend on responses to inquiries made to the public, and occasionally conduct experiments to improve contact procedures. Agencies may wish to assess whether there is significant change in response rates due to an operational refinement. This work considers the assessment of response rates when up to L attempts are made to contact each subject, and subjects receive one of J possible variations of the operation under experimentation. In particular, the continuation-ratio logit (CRL) model facilitates inference on the probability of success at each step of the sequence, given that failures occurred at previous attempts. The CRL model is investigated as a basis for sample size determination– one of the major decisions faced by an experimenter–to attain a desired power under a Wald test of a general linear hypothesis. An experiment that was conducted for nonresponse followup in the United States 2020 decennial census provides a motivating illustration.
Keywords
References
1.
Agresti, A. 2013. Categorical Data Analysis . Wiley, 3rd edition.
2.
Albert, J.H., and S. Chib. 2001. “Sequential ordinal modeling with applications to survival data.” Biometrics 57(3): 829–836,. DOI: https://doi.org/10.1111/j.0006-341X.2001.00829.x.
3.
Alho., J.M. 1990. “Adjusting for nonresponse bias using logistic regression.” Biometrika 77(3): 617–624. DOI: https://doi.org/10.1093/biomet/77.3.617.
4.
Ashmead, R., E. Slud, and T. Hughes. 2017. “Adaptive intervention methodology for reduction of respondent contact burden in the American Community Survey.” Journal of Official Statistics 33(4): 901–919. DOI: https://doi.org/10.1515/jos-2017-0043.
5.
Barboza, G.E., and S. Dominguez. 2016. “A sequential logit model of caretakers’ decision to vaccinate children for the human papillomavirus virus in the general population.” Preventive Medicine 85: 84–89. DOI: https://doi.org/10.1016/j.ypmed.2016.01.010.
6.
Blumenson, L.E. 1960. “A derivation of n-dimensional spherical coordinates.” The American Mathematical Monthly , 67(1): 63–66. DOI: https://doi.org/10.2307/2308932.
7.
Boes, S., and R. Winkelmann. 2006. “Ordered response models.” Allgemeines Statistisches Archiv 90: 167–181. DOI: https://doi.org/10.1007/s10182-006-0228-y.
8.
Brown, J.D., M.L. Heggeness, S.M. Dorinski, L. Warren, and M. Yi. 2018. Understanding the quality of alternative citizenship data sources for the 2020 Census. CES Working Paper Series: CES 18–38, Center for Economic Studies, U.S. Census Bureau. Available at: https://www2.census.gov/ces/wp/2018/CES-WP-18-38.pdf (accessed October 2022).
9.
Bush, S. 2015. “Sample size determination for logistic regression: A simulation study.” Communications in Statistics – Simulation and Computation 44(2): 360–373. DOI: 10.1080/03610918.2013.777458.
10.
Chow, S.-C., J. Shao, H. Wang, and Y. Lokhnygina. 2017. Sample Size Calculations in Clinical Research . Chapman and Hall/CRC, 3rd edition.
11.
Compton, E., and M. Bentley. 2012. 2010 Census nonresponse followup (NRFU) contact strategy experiment report. 2010 Census Planning Memoranda Series: CPEX–174, Decennial Statistical Studies Division, U.S. Census Bureau. Available at: https://www.census.gov/programs-surveys/decennial-census/decade/2010/program-management/cpex/2010-cpex-174.html.
12.
Cordeiro, G.M., D.A. Botter, and S.L.D.P. Ferrari. 1994. “Nonnull asymptotic distributions of three classic criteria in generalised linear models.” Biometrika 81(4): 709–720. DOI: https://doi.org/10.1093/biomet/81.4.709.
13.
Cox, D.R. 1972. “Regression models and life-tables.” Journal of the Royal Statistical Society. Series B 34(2): 187–220. DOI: https://doi.org/10.1111/j.2517-6161.1972.tb00899.x.
14.
Daas, P.J.H., M.J. Puts, B. Buelens, and P.A.M. van den Hurk. 2015. “Big Data as a Source for Official Statistics.” Journal of Official Statistics 31(2): 249–262. DOI: https://doi.org/10.1515/jos-2015-0016.
15.
Davern, M., M. Roemer, and W. Thomas. 2009. “Investing in a data quality research program for administrative data linked to survey data for policy research purposes is essential.” In Federal Committee on Statistical Methodology Research Conference, November 2–4, Washington, D.C. U.S.A, Available at: https://nces.ed.gov/FCSM/pdf/2009FCSM_Davern_IX-A.pdf (accessed October 2022).
16.
Demidenko, E. 2007. “Sample size determination for logistic regression revisited.” Statistics in Medicine 26(18): 3385–3397. DOI: https://doi.org/10.1002/sim.2771.
17.
Demidenko, E. 2008. “Sample size and optimal design for logistic regression with binary interaction.” Statistics in Medicine 27(1): 36–46. DOI: https://doi.org/10.1002/sim.2980.
18.
Deming, W.E. 1953. “On a probability mechanism to attain an economic balance between the resultant error of non-response and the bias of non-response.” Journal of the American Statistical Association 48: 743 – 772. DOI: https://doi.org/10.1080/01621459.1953.10501197.
19.
Ellis, R., P. Goerman, K. Kephart, A.C. Fobia, A.S. Giron, M. Meyers, R. Terry, L. Fernandez, F. Lineback, M. Berger, A. Bruce, and E. Jensen. 2018. Research on coverage of underrepresented populations in anticipation of a records-based census . 2020 Census: Evaluation, Experiment, and Research and Testing Study. Internal report.
20.
Erdman, C., and N. Bates. 2016. “The low response score (LRS): A metric to locate, predict, and manage hard-to-survey populations.” Public Opinion Quarterly 81(1): 144–156. DOI: https://doi.org/10.1093/poq/nfw040.
21.
Erdman. C., and N. Bates. 2017. Available at: https://www.census.gov/topics/research/-guidance/planning-databases.html (accessed October 2022).
22.
Fienberg, S.E. 2007. The analysis of cross-classified categorical data . Springer Science & Business Media, 2nd edition.
23.
Fullerton, A.S. 2009. “A conceptual framework for ordered logistic regression models.” Sociological Methods & Research 38(2): 306–347. DOI: https://doi.org/10.1177/0049124109346162.
24.
Ghosal, S., and A. van der Vaart. 2017. Fundamentals of Nonparametric Bayesian Inference . Cambridge University Press.
25.
Groves, R.M., and G.J. Schoeffel. “Use of administrative records in evidence-based policymaking.” The ANNALS of the American Academy of Political and Social Science 678(1): 71–80. DOI: https://doi.org/10.1177/0002716218766508.
26.
Hansen, M.H., and W.N. Hurwitz. 1946. “The problem of non-response in sample surveys.” Journal of the American Statistical Association 41(236): 517–529. DOI: https://doi.org/10.1080/01621459.1946.10501894.
27.
Lohr, S.L. 2010. Sampling: Design and Analysis . Brooks/Cole, Boston, MA, 2nd edition, 2010.
28.
Lyles, R.H., H.-M. Lin, and J.M. Williamson. 2007. “A practical approach to computing power for generalized linear models with nominal, count, or ordinal responses.” Statistics in Medicine 26(7): 1632–1648. DOI: https://doi.org/10.1002/sim.2617.
29.
Molfino, E., G. Korkmaz, S.A. Keller, A. Schroeder, S. Shipp, and D.H. Weinberg. 2017. “Can administrative housing data replace survey data?” Cityscape 19(1): 265–292.
30.
Morris, D.S., A. Keller, and B. Clark. 2016. “An approach for using administrative records to reduce contacts in the 2020 Decennial Census.” Statistical Journal of the IAOS 32(2): 177–188. DOI: https://doi.org/10.3233/SJI-161002.
31.
Myers, R.H. 2000. Classical and Modern Regression with Applications . Duxbury Press, 2nd edition.
32.
National Research Council. 2010. In Envisioning the 2020 Census, edited by L.D. Brown, M.L. Cohen, D.L. Cork, and C.F. Citro,. The National Academies Press, Washington, D.C. DOI: https://doi.org/10.17226/12865.
33.
Oehlert, G.W. 2000. A first course in design and analysis of experiments . W.H. Freeman. Available at: http://users.stat.umn.edu/~gary/Book.html.
34.
Pan, Y., and S. Lubkemann. 2013. Observing census enumeration of non-English speaking households in the 2010 Census: Evaluation report. Research Report Series: Survey Methodology 2013(2). Center for Survey Measurement, U.S. Census Bureau. Available at: https://www.census.gov/library/working-papers/2013/adrm/ssm2013-02.html (accessed October 2023).
35.
Peyhardi, J., C. Trottier, and Y. Guédon. 2015. “A new specification of generalized linear models for categorical responses.” Biometrika 102(4): 889–906. DOI: https://doi.org/10.1093/biomet/asv042.
36.
Politz, A., and W. Simmons. 1949. “An attempt to get the “not at homes” into the sample without callbacks.” Journal of the American Statistical Association 44(245): 9–16. DOI: https://doi.org/10.2307/2280347.
37.
Praestgaard, J. 2012. “A note on the power superiority of the restricted likelihood ratio test.” Journal of Multivariate Analysis 104(1): 1–15. DOI: https://doi.org/10.1016/j.jmva.2011.04.004.
38.
R Core Team. 2021. R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria. Available at: https://www.R-project.org/ (accessed October 2023).
39.
Rao, p. 1983 “Callbacks, follow-ups, and repeated telephone calls.” In Incomplete Data in Sample Surveys , edited by W.G. Madow, I. Olkin, and D.B. Rubin 33–44. Academic Press, New York.
40.
Rigon, T., and D. Durante. 2021. “Tractable Bayesian density regression via logit stick-breaking priors.” Journal of Statistical Planning and Inference 211: 131–142. DOI: https://doi.org/10.1016/j.jspi.2020.05.009.
41.
Särndal, C.-E., B. Swensson, and J. Wretman. 1992. Model Assisted Survey Sampling . Springer-Verlag New York, Inc., New York.
42.
SAS Institute Inc. 2018. The GENMOD Procedure 3407–3607. SAS Publishing. Available at: http://support.sas.com/documentation/onlinedoc/stat/151/genmod.pdf (accessed October 2023).
43.
Scheuren, F. 1999. “Administrative records and census taking.” Survey Methodology 25(2): 151–160. Available at: https://www150.statcan.gc.ca/n1/pub/12-001-x/1999002/article/4878-eng.pdf (accessed October 2022).
44.
Self, S.G., and R.H. Mauritsen. 1988. “Power/sample size calculations for generalized linear models.” Biometrics 44(1): 79–86. DOI: https://doi.org/10.2307/2531897.
45.
Self, S.G., R.H. Mauritsen, and J. Ohara. 1992. “Power calculations for likelihood ratio tests in generalized linear models.” Biometrics 48(1): 31–39. DOI: https://doi.org/10.2307/2532736.
46.
Shieh, G. 2000. “On power and sample size calculations for likelihood ratio tests in generalized linear models.” Biometrics 56(4): 1192–1196. DOI: https://doi.org/10.1111/j.0006-341X.2000.01192.x.
47.
Shieh, G. 2005. “On power and sample size calculations for Wald tests in generalized linear models.” Journal of Statistical Planning and Inference 128(1): 43–59. DOI: https://doi.org/10.1016/j.jspi.2003.09.017.
48.
Silvapulle, M.J., and P.K. Sen. 2004. Constrained Statistical Inference: Order, Inequality, and Shape Constraints . Wiley-Interscience.
49.
Singer, E. 2006. “Introduction: Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70(5): 637–645. DOI: https://doi.org/10.1093/poq/nfl034.
50.
Slud, E., and B. Kedem. 1994. “Partial likelihood analysis of logistic regression and autoregression.” Statistica Sinica 4(1): 89–106.
51.
Tutz, G. 1991. “Sequential models in categorical regression.” Computational Statistics & Data Analysis 11(3): 275–295. DOI: https://doi.org/10.1016/0167-9473(91)90086-H.
52.
Tutz, G. 2022. “Ordinal regression: A review and a taxonomy of models.” WIREs Computational Statistics 14(2): e1545. DOI: https://doi.org/10.1002/wics.1545.
53.
U.S. Census Bureau. 2013. U.S. Census Bureau statistical quality standards. Available at: https://www.census.gov/content/dam/Census/about/about-the-bureau/policies_and_notices/quality/statistical-quality-standards/Quality_Standards.pdf (accessed October 2022).
54.
U.S. Census Bureau. 2019. 2020 Census detailed operational plan for: 18. nonresponse followup operation (NRFU). Available at: https://www.census.gov/programs-surveys/-decennial-census/2020-census/planning-management/planning-docs/NRFU-detailed-op-plan.html.
55.
Walker, S., S. Winder, G. Jackson, and S. Heimel. 2012. 2010 census nonresponse followup operations assessment. Technical Report 190, 2010 Census Planning Memoranda Series. Available at: https://www.census.gov/2010census/pdf/2010_Census_NRFU_Operations_Assessment.pdf (accessed October 2022).
56.
Wood, A.M., I.R. White, and M. Hotopf. 2006. “Using number of failed contact attempts to adjust for non-ignorable non-response.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 169(3): 525–542. DOI: https://doi.org/10.1111/j.1467-985X.2006.00405.x.