Least Squares and Robust Rank-Based Double Bootstrap Analyses for Time-Series Intervention Designs

Abstract

Time-series intervention designs that include two or more phases have been widely discussed in the healthcare literature for many years. A convenient model for the analysis of these designs has a linear model part (to measure changes in level and trend) plus a second part that measures the random error structure; the error structure is assumed to follow an autoregressive time-series process. Traditional generalized linear model approaches widely used to estimate this model are less than satisfactory because they tend to provide substantially biased intervention tests and confidence intervals. We describe an updated version of the original double bootstrap approach that was developed by McKnight et al. (2000) to correct for this problem. This updated analysis and a new robust version were recently implemented in an R package (McKean & Zhang, 2018). The robust method is insensitive to outliers and problems associated with common departures from normality in the error distribution. Monte Carlo studies as well as published data are used to demonstrate the properties of both versions. The R code required to perform the analyses is provided and illustrated.

Keywords

autoregressive interrupted time-series linear model Monte Carlo quasi-experiment robust methods intervention analysis

Get full access to this article

View all access options for this article.

References

Biglan

Ary

Wagenaar

A. C

(2000). The value of interrupted time-series experiments for community intervention research. Prevention Science, 1(1), 31–49. https://doi.org/10.1023/a:1010024016308

Boel

Andreasen

Jarlov

J. O.

Ostergaard

Gjorup

Boggild

Arpi

(2016). Impact of antibiotic restriction on resistance levels of Escherichia coli: A controlled interrupted time series study of a hospital-wide antibiotic stewardship programme. Journal of Antimicrobial Chemotherapy, 71(7), 2047–2051. https://doi.org/10.1023/a:1010024016308

Campbell

D. T.

Stanley

(1966). Experimental and quasi-experimental designs for research. Rand McNally.

Chang

McKean

J. W.

Naranjo

Sheather

(1999). High-breakdown rank regression. Journal of the American Statistical Association, 94(445), 205–219. https://doi.org/10.1080/01621459.1999.10473836

Cheng

Benassi

De Oliveira

Zaheer

Collins

Kurdyak

(2016). Impact of a mass media mental health campaign on psychiatric emergency department visits. Canadian Journal of Public Health, 107(3), 303–311. https://doi.org/10.17269/cjph.107.5265

Chib

Greenberg

(1994). Bayes inference in regression models wirh ARMA (p, q) errors. Journal of Econometrics, 64(1–2), 183–206.

Cochrane

Orcutt

G. H

(1949). Application of least squares regression to relationships containing autocorrelated error terms. Journal of the American Statistical Association, 44(245), 32–61. https://doi.org/10.2307/2280349

Collini

S. A

(2019). An experimental comparison of visual and statistical analyses. Unpublished Doctoral Dissertation. Western Michigan University.

Durbin

(1960). Estimation of parameters in time-series regression models. Journal of the Royal Statistical Society. Series B (Methodological), 22(1), 139–153. https://doi.org/10.1111/j.2517-6161.1960.tb00361.x

10.

Ewusie

J. E.

Blondal

Soobiah

Beyene

Thabane

Straus

S. E.

Hamid

J. S

(2017). Methods, applications, interpretations, and challenges of interrupted time series (ITS) data: Protocol for a scoping review. British Medical Journal Open, 7(6), e016018. https://doi.org/10.1136/bmjopen-2017-016018

11.

Gebski

Ellingson

Edwards

Jernigan

Kleinbaum

(2012). Modelling interrupted time series to evaluate prevention and control of infection in healthcare. Epidemiology and Infection, 140(12), 2131–2141. https://doi.org/10.1017/S0950268812000179

12.

Hall

(1988). Theoretical comparison of bootstrap confidence intervals. Annals of Statistics, 16(3), 927–953. https://doi.org/10.1214/aos/1176350933

13.

Harrop

J. W.

Velicer

W. F

(1985). A comparison of alternative approaches to the analysis of interrupted time-series. Multivariate Behavioral Research, 20(1), 27–44. https://doi.org/10.1207/s15327906mbr2001._2

14.

Hettmansperger

T. P.

McKean

J. W

(2011). Robust nonparametric statistical methods (2nd ed.). Chapman-Hall.

15.

Hudson

Fielding

Ramsay

C. R

(2019). Methodology and reporting characteristics of studies using interrupted time series design in healthcare. BioMed Central Medical Research Methods, 19(1), 137. https://doi.org/10.1186/s12874-019-0777-x

16.

Huitema

B. E

(2011). The analysis of covariance and alternatives: Statistical methods for experiments, quasi-experiments, and single-case studies (2nd ed.). John Wiley and Sons.

17.

Huitema

B. E.

McKean

J. W

(1998). Irrelevant autocorrelation in least-squares intervention models. Psychological Methods, 3(1), 104–116. https://doi.org/10.1037/1082-989x.3.1.104

18.

Huitema

B. E.

McKean

J. W

(2000). Design specification issues in time-series intervention models. Educational and Psychological Measurement, 60(1), 38–58. https://doi.org/10.1177/00131640021970358

19.

Huitema

B. E.

McKean

J. W.

McKnight

(1994). Small-sample time-series intervention analysis: Problems and solutions. Paper presented at the meeting of the American Psychological Association, Los Angeles, August 25, 1994.

20.

Huitema

B. E.

McKean

J. W

Mcknight

(1999). Autocorrelation effects on least-squares intervention analysis of short time series. Educational and Psychological Measurement, 59(5), 767–786. https://doi.org/10.1177/00131649921970134

21.

Jandoc

Burden

A. M.

Mamdani

Levesque

L. E.

Cadarette

S. M

(2015). Interrupted time series analysis in drug utilization research is increasing: Systematic review and recommendations. Journal of Clinical Epidemiology, 68(8), 950–956. https://doi.org/10.1016/j.jclinepi.2014.12.018

22.

Johnston

(2013). Useful theories should apply to individuals. British Journal of Health Psychology, 18(3), 469–473. https://doi.org/10.1111/bjhp.12049

23.

Kloke

J. D.

McKean

J. W

(2010). Rfit: Rank estimation for linear models. R package version 0.09. http://cran.r-project.org/web/packages/Rfit/

24.

Kloke

J. D.

McKean

J. W

(2012). Rfit: Rank-based estimation for linear models. The R Journal, 4(2), 57–64. https://doi.org/10.32614/rj-2012-014

25.

Kloke

J. D.

McKean

J. W

(2015). Nonparametric statistical methods using R. Chapman-Hall.

26.

Martinez

J. P.

Huitema

B. E

(2022). A historical account of single-subject time-series methodology. Manuscript submitted for publication.

27.

Matowe

Ramsay

C. R.

Grimshaw

J. M.

Gilbert

F. J.

Macleod

M. J.

Needham

(2002). Effects of mailed dissemination of the Royal College of Radiologists’ guidelines on general practitioner referrals for radiography: A time series analysis. Clinical Radiology, 57(7), 575–578. https://doi.org/10.1053/crad.2001.0894

28.

McKean

J. W.

Schrader

(1980). The geometry of robust procedures in linear models. Journal of the Royal Statistical Society, Series B, 42(3), 366–371. https://doi.org/10.1111/j.2517-6161.1980.tb01137.x

29.

McKean

J. W.

Zhang

(2018). Package ‘DBfit’. R package version 1.0. https://cran.r-project.org/web/packages/DBfit/

30.

McKnight

McKean

J. W.

Huitema

B. E

(2000). A double bootstrap method to analyze linear models with autoregressive error terms. Psychological Methods, 5(1), 87–101. https://doi.org/10.1037/1082-989x.5.1.87

31.

Penfold

Zhang

(2013). Use of interrupted time series analysis in evaluating health care quality improvements. Academic Pediatric Association, 13(6), 38–44. https://doi.org/10.1016/j.acap.2013.08.002

32.

Prais

S. J.

Winsten

C. B

(1954). Trend estimators and serial correlation. Cowles Commission Discussion Paper No. 383.

33.

Ramsey

C. R.

Matowe

Grilli

Grimshaw

J. M.

Thomas

R. E

(2004). Interrupted time series designs in health technology assessment: Lessons from two systematic reviews of behavior change strategies. International Journal of Technology Assessment in Health Care, 19(4), 613–623. https://doi.org/10.1017/s0266462303000576

34.

Runkle

J.D.

Zhang

Karmaus

Martin

A. B.

Svendsen

E. R

(2012). Prediction of unmet primary care needs for the medically vulnerable post-disaster: An interrupted time- series analysis of health system responses. International Journal of Environmental Research and Public Health, 9(10), 3384–3397. https://doi.org/10.3390/ijerph9103384

35.

Schaffer

A. L.

Dobbins

T. A.

Pearson

S. A

(2021). Interrupted time series analysis using autoregressive integrated moving average (ARIMA) models: A guide for evaluating large- scale health interventions. BMC Medical Research Methodology, 21(1), 58. https://doi.org/10.1186/s12874-021-01235-8

36.

Taljaard

McKenzie

Ramsay

Grimshaw

(2014). The use of segmented regression in analyzing interrupted time series studies: An example in pre-hospital ambulance care. Implementation Science, 9(1), 77. https://doi.org/10.1186/1748-5908-9-77

37.

Turner

S. L.

Karahalios

Forbes

A. B.

Taljaard

Grimshaw

J. M.

Cheng

A. C.

Bero

McKenzie

J. E

(2020). Design characteristics and statistical methods used in interrupted time series studies evaluating public health interventions: A review. Journal of Clinical Epidemiology, 122, 1–11. https://doi.org/10.1016/j.jclinepi.2020.02.006. PMID: 32109503.

38.

Van Houten

Malenfant

J. E.

Reagan

Sifrit

Compton

Tenenbaum

(2010). Increasing seat belt use in service vehicle drivers with a gearshift delay. Journal of Applied Behavior Analysis, 43(3), 369–380. https://doi.org/10.1901/jaba.2010.43-369

39.

Wagner

A. K.

Soumerai

S. B.

Zhang

Ross-Degnan

(2002). Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics, 27(4), 299–309. https://doi.org/10.1046/j.1365-2710.2002.00430.x

40.

Zhang

(2017). Development of traditional and rank-based algorithms for linear models with autoregressive errors and multivariate logistic regression with spatial random effects. Unpublished Doctoral Dissertation. Western Michigan University.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.66 MB