A New Approach to Composite Estimation for Repeated Surveys with Rotating Panels

Abstract

Composite estimation in repeated surveys with rotating panels refers to methods of estimation that exploit correlations in the data in the sample overlap between survey times to improve the precision of estimates. In this article a novel approach to composite estimation is proposed, in which composite regression estimators of current totals for a number of key variables are generated from a simultaneous calibration of the sampling weights of the overlapping samples of the current and previous survey time. In this procedure, in addition to the usual calibration to known population totals, differences of estimates for the key variables based on the full sample and the common sample from the two consecutive times are calibrated to each other. The resulting multivariate composite regression estimator, which is constructed as an approximate best linear unbiased estimator, incorporates effectively information from the samples of both survey times for enhanced estimation efficiency. Unlike other composite regression estimators, the proposed estimator does not require micro-matching of data in the overlap sample, and, therefore, is free of potential issues associated with it. It is also considerably more practical than existing composite regression estimators and the traditional AK-composite estimator.

1. Introduction

Some repeated surveys, typically Labor Force Surveys, use a sampling design with rotating panels for operational and statistical efficiency. In such a design, a large overlap in the samples between successive survey times allows the improvement of estimates of population parameters, especially for those variables for which there is a strong correlation between the values reported by the same units in successive times. Composite estimation refers to estimation methods that use information from previous times to improve the precision of both the point-in-time (“level”) estimates and estimates of change between consecutive times, by exploiting correlations in the data of the overlap sample. This improvement in precision can in turn reduce the volatility in the time series of estimates, especially of estimates with high sampling variability associated with subpopulations of interest.

The earliest composite estimation method, known as “K-composite estimation,” was introduced for the US Current Population Survey (CPS) by Hansen et al. (1955), and extended later to the “AK-composite estimation” by Gurney and Daly (1965), and to the “AK-composite weighting”; see Fuller (1990), Cantwell and Ernst (1992), and Lent et al. (1994, 1999).

Later, a type of regression method of composite estimation introduced for the Canadian Labor Force Survey, called modified regression (MR) composite estimation, was developed to overcome certain shortcomings of the AK method; see Singh and Merkouris (1995), Singh et al. (1997, 2001), Gambino et al. (2001), Fuller and Rao (2001), Bell (2001), Beaumont and Bocci (2005). A recent evaluation study by Bonnéry et al. (2020), devised to assess the design-based properties of different composite estimators using CPS data and CPS sample design, found that the currently used Fuller-Rao version of the MR composite estimator performed the best. A more recent regression estimator has been proposed by Konrad and Berger (2023).

In a time series approach, other authors (Bell and Hillmer 1990; Binder and Dick 1989; Jones 1980; Pfeffermann 1991; Scott et al. 1977; Tiller 1989) developed estimators for repeated surveys allowing for stochastic variation in the parameters being estimated.

In this article, a novel approach to composite estimation is proposed, in which composite regression estimators of current totals for a number of key variables are generated from a simultaneous calibration of the sampling weights of the overlapping samples of the current and previous survey time. In this procedure, in addition to the usual calibration to known population totals of auxiliary variables, differences of estimates for the key variables based on the full sample and the common sample of the two consecutive surveys are calibrated to each other.

The proposed approach is motivated by considerations of most effective use of data collected in successive survey times. Earlier work that adopted this approach (e.g., Gurney and Daly 1965; Jones 1980) centered on the formulation of best linear unbiased estimators (BLUE) of a population parameter, involving linear combinations of estimators from a number of survey times. These estimators are rather unrealistic in practice to use when the series of available survey data is long, and depend on the intractable covariance structure of the combined estimators. In contrast, the proposed composite regression estimator of a vector of population totals is constructed as a practical approximation of a BLUE that involves a particular set of estimators from the latest two survey times.

The proposed composite regression estimators of current totals and changes can be particularly efficient because the regression coefficients incorporate information from the samples of both current and previous survey time, as do the coefficients of the BLUE. Furthermore, the simultaneous calibration of the samples of consecutive survey times facilitates greatly variance estimation by resampling methods. Unlike the MR-composite method, the proposed method of composite regression estimation does not require micro-matching of data in the common sample, and therefore is free of potential quality issues associated with it. It is also considerably more practical than the MR-composite estimation and the traditional AK-composite estimation. The comparative merits of the proposed estimator are discussed in detail in Section 4. Section 2 provides the notation and estimation preliminaries. Section 3 describes the construction of the composite regression estimators of level and change for any survey variable, as approximate BLUEs, through a suitable calibration procedure. A concluding discussion is provided in Section 5.

2. Notation and Preliminaries

The sample of a repeated survey with a rotating panel design is typically made up of a number (say $r$ ) of subsamples (“panels,” or “rotation groups”) of approximately equal size, each one staying in the sample for $k\le r$ consecutive survey times and then rotated out of the survey and replaced by a newly selected panel. The panel sizes may vary in time in accordance with necessary adjustments in the survey design. Each panel is designed to be a sample of units (e.g., dwellings) representative of the survey population, and so can provide estimates of population parameters by a proper scaling-up of its sampling weights. For any two consecutive survey times there is a partial panel overlap of $100(r-r/k)/r$ %, defining the “matched sample.” The samples at current time $t$ and previous time $t-1$ are denoted by $s_t$ and $s_{t-1}$ , respectively, and the vector of sampling weights in $s_t$ is denoted by ${\mathbf{w}}_t$ . Adjustment of these weights for nonresponse is necessary in practice but not considered in the theory expounded in this article.

Let $\mathbf{y}$ be a vector of $q$ key variables to be used in composite estimation, with vector of current-time totals ${\mathbf{\tau }}_{\mathbf{y}\mathbf{,}\mathbf{t}}$ , and let $\mathbf{x}$ be a vector of $p$ auxiliary variables used in calibration, with vector of current-time known totals $\mathbf{\tau }\mathbf{x}\mathbf{,}\mathbf{t}$ . Denote then the sample matrix of $\mathbf{y}$ , of dimension $n_t\times q$ , where $n_t$ is the sample size at time $t$ , by ${\mathbf{Y}}_t$ , partitioned by the unmatched $\left(u\right)$ and matched $\left(m\right)$ part of the sample into ${\mathbf{Y}}_{u,t}$ and ${\mathbf{Y}}_{m,t}$ , respectively. Similar is the notation for the previous survey time $t-1$ , but ${\mathbf{Y}}_{u,t-1}$ and ${\mathbf{Y}}_{m,t-1}$ refer to unmatched and matched samples with respect to time $t$ . The sample matrix of $\mathbf{x}$ , of dimension $n_t\times p$ , is denoted by ${\mathbf{X}}_t$ .

The current-time Horvitz-Thompson (HT) estimators of the totals ${\mathbf{\tau }}_{y,t}$ and ${\mathbf{\tau }}_{x,t}$ , based on the full sample, are the weighted sums ${\hat{\tau }}_{y,t}=\mathbf{Y}{\prime }_t{\mathbf{w}}_t$ , and ${\hat{\tau }}_{x,t}=\mathbf{X}{\prime }_t{\mathbf{w}}_t$ , respectively. The estimator of ${\mathbf{\tau }}_{y,t}$ based on the matched sample is ${\hat{\tau }}_{y,m,t}=\mathcal{R}{\mathbf{Y}}_{m,t}\prime {\mathbf{w}}_{m,t}$ , where ${\mathbf{w}}_{m,t}$ is the subvector of sampling weights of units in the matched sample, and $\mathcal{R}=1_{s_t}\prime {\mathbf{w}}_t/1_{s_{m,t}}\prime {\mathbf{w}}_{m,t}$ , where $1$ denotes the unit vector and $s_t$ and $s_{m,t}$ are the full and matched samples. $\mathcal{R}$ is the ratio (weighted version of $r/(r-r/k)$ ) that adjusts for the fact that $r-r/k$ of the $r$ panels between times $t-1$ and $t$ are common. When the panel sizes are approximately equal and procedures to balance the weights by panel are used (e.g., nonresponse adjustment is done separately by panel), then $\mathcal{R}$ is approximately equal to $r/(r-r/k)$ .

The standard regression estimator ${\hat{\tau }}_{y,t}^R$ of ${\mathbf{\tau }}_{y,t}$ involving the vector $\mathbf{x}$ of auxiliary variables, and ${\mathbf{X}}_t$ as the associated regression matrix, is given by

$${\hat{\tau }}_{y,t}^R={\hat{\tau }}_{y,t}+\hat{\mathbf{B}}({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t}),$$

(1)

where $\hat{\mathbf{B}}={\mathbf{Y}}_t\prime {\mathbf{W}}_t{\mathbf{X}}_t{({\mathbf{X}}_t\prime {\mathbf{W}}_t{\mathbf{X}}_t)}^{-1}$ is the $q\times p$ matrix of regression coefficients, with ${\mathbf{W}}_t$ being the diagonal “weighting” matrix with diagonal elements the elements of the vector ${\mathbf{w}}_t$ . The estimator ${\hat{\tau }}_{y,t}^R$ can be also written in the form of a calibration estimator, that is, as the weighted sum ${\mathbf{Y}}_t\prime {\mathbf{c}}_{x,t}$ , where ${\mathbf{c}}_{x,t}$ is the vector of the calibrated sampling weights

$${\mathbf{c}}_{x,t}={\mathbf{w}}_t+{\mathbf{W}}_t{\mathbf{X}}_t{({\mathbf{X}}_t\prime {\mathbf{W}}_t{\mathbf{X}}_t)}^{-1}({\mathbf{\tau }}_{\mathbf{x}\mathbf{,}\mathbf{t}}-{\mathbf{X}}_t\prime {\mathbf{w}}_t),$$

(2)

derived by minimizing the generalized least-squares distance $({\mathbf{c}}_{x,t}-{\mathbf{w}}_t)\prime {\mathbf{W}}_t^{-1}({\mathbf{c}}_{x,t}-{\mathbf{w}}_t)$ subject to the constraints ${\mathbf{X}}_t\prime {\mathbf{c}}_{x,t}={\mathbf{\tau }}_{x,t}$ . Of course, ${\hat{\tau }}_{x,t}^R={\mathbf{\tau }}_{x,t}$ , and for any other single variable $z$ with sample vector ${\mathbf{Z}}_t$ at time $t$ , the regression estimator ${\hat{\tau }}_{z,t}^R$ of its total ${\tau }_{z,t}$ is of the form of Equation (1) and is obtained as ${\mathbf{Z}}_t\prime {\mathbf{c}}_{x,t}$ .

The regression estimator of ${\mathbf{\tau }}_{y,t}$ based on the matched sample, to be used in composite estimation, is

$${\hat{\tau }}_{y,m,t}^R={\hat{\tau }}_{y,m,t}+{\hat{\mathbf{B}}}_m({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t}),$$

where ${\hat{\mathbf{B}}}_m=\mathcal{R}{\mathbf{Y}}_{m,t}\prime {\mathbf{W}}_{m,t}{\mathbf{X}}_{m,t}{({\mathbf{X}}_t\prime {\mathbf{W}}_t{\mathbf{X}}_t)}^{-1}$ , with the obvious notation for ${\mathbf{W}}_{m,t}$ and ${\mathbf{X}}_{m,t}$ .

3. A New Method of Composite Estimation

3.1. Constructing a Composite Regression Estimator

The proposed method of composite estimation arises from the search for a most effective use of information on the vector $\mathbf{y}$ from successive survey times, in addition to the use of the current-time data on the auxiliary vector $\mathbf{x}$ that gives the regression estimator in Equation (1). Thus, in addition to the estimates ${\hat{\tau }}_{y,t}$ and ${\hat{\tau }}_{x,t}$ used in Equation (1) for current survey time $t$ , we consider the matched-sample estimate ${\hat{\tau }}_{y,m,t}$ defined above, plus the full-sample and matched-sample estimates for time $t-1$ . In the time series of composite estimates, these two estimates for time $t-1$ are given by $\widehat{{\tau }_{y,t-1}^c}={\mathbf{Y}}_{t-1}\prime {\mathbf{c}}_{t-1}$ and $\widehat{{\tau }_{y,m,t-1}^c}={\mathbf{Y}}_{m,t-1}\prime {\mathbf{c}}_{m,t-1}$ , where ${\mathbf{c}}_{t-1}$ is the vector of composite weights for time $t-1$ . It is important to note that in this time series the vector ${\mathbf{c}}_{t-1}$ is constructed to incorporate information on $\mathbf{y}$ from time $t-1$ and past times. The first time of employing composite estimation, the estimates ${\hat{\tau }}_{y,t-1}^c$ and ${\hat{\tau }}_{y,m,t-1}^c$ are just the regression estimates ${\hat{\tau }}_{y,t-1}^R$ and ${\hat{\tau }}_{y,m,t-1}^R$ for previous time $t-1$ , and ${\mathbf{c}}_{t-1}$ is the vector of calibrated weights, of the form of Equation (2), that generates these estimates. We thus have a second estimate of ${\mathbf{\tau }}_{y,t}$ formed as ${\hat{\tau }}_{y,t-1}^c+({\hat{\tau }}_{y,m,t}-{\hat{\tau }}_{y,m,t-1}^c)$ , that is, the previous-time composite estimate updated with the change estimate (estimate of ${\mathbf{\tau }}_{y,t}-{\mathbf{\tau }}_{y,t-1}$ ) based on the matched sample. We then consider the best linear unbiased estimator (BLUE) ${\hat{\tau }}_{y,t}^B$ of ${\mathbf{\tau }}_{y,t}$ , which is the minimum-variance linear unbiased combination of the three estimates ${\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t}$ , ${\hat{\tau }}_{y,t}$ , and ${\hat{\tau }}_{y,t-1}^c+({\hat{\tau }}_{y,m,t}-{\hat{\tau }}_{y,m,t-1}^c)$ , that is,

$${\hat{\tau }}_{y,t}^B={\mathbf{B}}_1({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t})+{\mathbf{B}}_2{\hat{\tau }}_{y,t}+{\mathbf{B}}_3({\hat{\tau }}_{y,t-1}^c+({\hat{\tau }}_{y,m,t}-{\hat{\tau }}_{y,m,t-1}^c)).$$

Using the condition of unbiasedness $E\left({\hat{\tau }}_{y,t}^B\right)={\mathbf{\tau }}_{y,t}$ (satisfied approximately for the terms ${\hat{\tau }}_{y,t-1}^c$ and ${\hat{\tau }}_{y,m,t-1}^c$ ), and the fact that $E\left({\hat{\tau }}_{x,t}\right)={\mathbf{\tau }}_{x,t}$ , it follows immediately that ${\mathbf{B}}_2+{\mathbf{B}}_3=1$ , and thus ${\hat{\tau }}_{y,t}^B$ takes the extended regression form

$${\hat{\tau }}_{y,t}^B={\hat{\tau }}_{y,t}+{\mathbf{B}}_1({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t})+{\mathbf{B}}_3[{\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c-({\hat{\tau }}_{y,t}-{\hat{\tau }}_{y,m,t})],$$

which can be written as

$$\widehat{{\tau }_{y,t}^B}=\widehat{{\tau }_{y,t}}+\mathbf{B}\left(\begin{array}{c}{\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t}\hfill \\ \multicolumn{1}{c}{{\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c-({\hat{\tau }}_{y,t}-{\hat{\tau }}_{y,m,t})}\end{array}\right),$$

(3)

where $\mathbf{B}=({\mathbf{B}}_1,{\mathbf{B}}_3)$ has the easily derived variance-minimizing value

$$B=-\mathrm{Cov}\left[{\widehat{\tau }}_{y,t},\left(\begin{array}{c}{\tau }_{x,t}-{\widehat{\tau }}_{x,t}\\ \left(\begin{array}{c}{\widehat{\tau }}_{y,t-1}^c-{\widehat{\tau }}_{y,m,t-1}^c-\left({\widehat{\tau }}_{y,t}-{\widehat{\tau }}_{y,m,t}\right)\end{array}\right)\end{array}\right)\right]{\left[\mathrm{Var}\left(\begin{array}{c}{\tau }_{x,t}-{\widehat{\tau }}_{x,t}\\ {\widehat{\tau }}_{y,t-1}^c-{\widehat{\tau }}_{y,m,t-1}^c-\left({\widehat{\tau }}_{y,t}-{\widehat{\tau }}_{y,m,t}\right)\end{array}\right)\right]}^{-1}.$$

(4)

It is pointed out here that a more typical formulation of the BLUE ${\hat{\tau }}_{y,t}^B$ is given by

$${\hat{\tau }}_{y,t}^B={(\mathbf{U}\mathbf{\text{'}}{\mathbf{K}}^{-1}\mathbf{U})}^{-1}\mathbf{U}\mathbf{\text{'}}\mathbf{K}\hat{\mathbf{\Theta }},$$

where $\hat{\mathbf{\Theta }}=({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t},{\hat{\tau }}_{y,t},{\hat{\tau }}_{y,t-1}^c+({\hat{\tau }}_{y,m,t}-{\hat{\tau }}_{y,m,t-1}^c))\prime$ , $\mathbf{K}$ is the covariance matrix of $\hat{\mathbf{\Theta }}$ , and $\mathbf{U}$ is a design matrix of 0’s and 1’s that satisfies $E(\hat{\mathbf{\Theta }})=\mathbf{U}{\mathbf{\tau }}_{y,t}$ . The variance of ${\hat{\tau }}_{y,t}^B$ is $\mathrm{Var}\left({\hat{\tau }}_{y,t}^B\right)={(\mathbf{U}\prime {\mathbf{K}}^{-1}\mathbf{U})}^{-1}$ . Such formulation of a best linear unbiased estimator, for a scalar parameter, was used by Gurney and Daly (1965) and Jones (1980) in the setting of a repeated survey, with $\hat{\mathbf{\Theta }}$ being a vector of unbiased estimators of the parameter for a number of survey times.

The more transparent formulation of the BLUE of ${\mathbf{\tau }}_{y,t}$ in Equation (3), with the specified $\hat{\mathbf{\Theta }}=({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t},{\hat{\tau }}_{y,t},{\hat{\tau }}_{y,t-1}^c+({\hat{\tau }}_{y,m,t}-{\hat{\tau }}_{y,m,t-1}^c))\prime$ , involving two estimates of ${\mathbf{\tau }}_{y,t}$ and the HT estimate of ${\mathbf{\tau }}_{x,t}$ , is used here to motivate a more practicable estimation procedure for the current-time total of any survey variable. Thus we set

$$\mathbf{w}=\left(\begin{array}{c}{\mathbf{w}}_t\hfill \\ \multicolumn{1}{c}{{\mathbf{c}}_{t-1}}\end{array}\right),{\mathbf{Y}}_{(t)}=\left(\begin{array}{c}{\mathbf{Y}}_t\hfill \\ \multicolumn{1}{c}{\begin{array}{c}0\hfill \end{array}}\end{array}\right),\mathcal{X}=\left(\begin{array}{c}\begin{array}{c}{\mathbf{X}}_t{\mathbf{\Psi }}_t\hfill \\ \multicolumn{1}{c}{0-{\mathbf{\Psi }}_{t-1}}\end{array}\hfill \end{array}\right),$$

(5)

where ${\mathbf{\Psi }}_t=({\mathbf{Y}}_{u,t}\prime ,(1-\mathcal{R}){\mathbf{Y}}_{m,t}\prime )\prime$ and ${\mathbf{\Psi }}_{t-1}=({\mathbf{Y}}_{u,t-1}\prime ,(1-\mathcal{R}){\mathbf{Y}}_{m,t-1}\prime )\prime$ . Then ${\mathbf{Y}}_{(t)}\prime \mathbf{w}={\hat{\tau }}_{y,t}$ and

$$\mathcal{X}\prime \mathbf{w}=\left(\begin{array}{c}{\mathbf{X}}_t\prime {\mathbf{w}}_t\hfill \\ \multicolumn{1}{c}{\mathbf{\Psi }\prime {\mathbf{w}}_t-{\mathbf{\Psi }}_{t-1}\prime {\mathbf{c}}_{t-1}}\end{array}\right)=\left(\begin{array}{c}{\hat{\tau }}_{x,t}\hfill \\ \multicolumn{1}{c}{{\hat{\tau }}_{y,t}-{\hat{\tau }}_{y,m,t}-({\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c)}\end{array}\right),$$

and thus the matrix coefficient $\mathbf{B}$ in Equation (4) can be expressed as

$$\mathbf{B}=\mathrm{Cov}({\mathbf{Y}}_{(t)}\prime \mathbf{w},\mathcal{X}\prime \mathbf{w}){\left[\mathrm{Var}(\mathcal{X}\prime \mathbf{w})\right]}^{-1}=\mathrm{Cov}({\hat{\tau }}_{y,t},{\hat{\tau }}_{\mathcal{X}}){\left[\mathrm{Var}\left({\hat{\tau }}_{\mathcal{X}}\right)\right]}^{-1},$$

(6)

where $\widehat{{\tau }_{\mathcal{X}}}=\mathcal{X}\prime \mathbf{w}$ . An estimate of $\mathbf{B}$ through the estimated variance $\widehat{\mathrm{Var}}(\mathbf{w})$ , having the form $\hat{\mathbf{B}}={\mathbf{Y}}_{(t)}\prime \widehat{\mathrm{Var}}(\mathbf{w})\mathcal{X}{[\mathcal{X}\prime \widehat{\mathrm{Var}}(\mathbf{w})\mathcal{X}]}^{-1}$ , would give the “optimal” estimator of ${\mathbf{\tau }}_{y,t}$ , of minimum asymptotic variance, in analogy to the single-sample optimal estimator (see, e.g., Rao 1994). However, computation of the matrix $\widehat{\mathrm{Var}}(\mathbf{w})$ is intractable in the considered sampling setting. Besides, such an estimate $\hat{\mathbf{B}}$ might be quite suboptimal.

A practical substitute can be obtained by replacing $\widehat{\mathrm{Var}}(\mathbf{w})$ in $\hat{\mathbf{B}}$ with the weighting matrix $\mathcal{W}=\mathrm{diag}({\mathbf{W}}_t,{\mathbf{C}}_{t-1})$ associated with $\mathbf{w}$ . This defines the $q\times (p+q)$ matrix of (generalized) regression coefficients $\hat{\mathcal{B}}={\mathbf{Y}}_{(t)}\prime \mathcal{W}\mathcal{X}{(\mathcal{X}\prime \mathcal{W}\mathcal{X})}^{-1}$ , and the composite regression estimator, as an approximation of the BLUE in Equation (3),

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}={\hat{\tau }}_{y,t}+\hat{\mathcal{B}}({\mathbf{\tau }}_{\mathcal{X}}-{\hat{\tau }}_{\mathcal{X}}),$$

(7)

where ${\mathbf{\tau }}_{\mathcal{X}}=({\mathbf{\tau }}_{x,t}\prime ,0\mathbf{\text{'}})\prime$ , and $0$ denotes a $q$ -dimensional vector of zeros.

The composite regression estimator in Equation (7) is generated by an extended calibration procedure which involves both samples $s_t$ and $s_{t-1}$ . This procedure is specified by the augmented regression matrix $\mathcal{X}$ defined in Equation (5), the associated vector of calibration totals ${\mathbf{\tau }}_{\mathcal{X}}=({\mathbf{\tau }}_{x,t}\prime ,0\mathbf{\text{'}})\prime$ and the weight vector $\mathbf{w}=({\mathbf{w}}_t\prime ,{\mathbf{c}}_{t-1}\prime )\prime$ . The vector of calibrated weights for the combined sample $s_t\cup s_{t-1}$ is given then by

$$\mathbf{c}=\mathbf{w}+\mathcal{W}\mathcal{X}{(\mathcal{X}\prime \mathcal{W}\mathcal{X})}^{-1}({\mathbf{\tau }}_{\mathcal{X}}-\mathcal{X}\prime \mathbf{w}).$$

(8)

The vector $c={\left({c^{\prime }}_t,{{c^{\prime }}^{\star }}_{t-1}\right)}^{\prime }$ , where ${\mathbf{c}}_{t-1}^{*}$ is the vector ${\mathbf{c}}_{t-1}$ recalibrated at time $t$ , satisfies the calibration constraints $\mathcal{X}\prime \mathbf{c}={\mathbf{\tau }}_{\mathcal{X}}$ , that is, ${\mathbf{X}}_t\prime {\mathbf{c}}_t={\mathbf{\tau }}_{x,t}$ and $\mathbf{\Psi }{\prime }_t{\mathbf{c}}_t=\mathbf{\Psi }{\prime }_{t-1}{\mathbf{c}}_{t-1}^{*}$ . The second constraint means that the differences in full-sample and matched-sample estimates from previous and current time are equated. Clearly then, with ${\mathbf{Y}}_{(t)}$ defined in Equation (5), the composite regression estimator in Equation (7) can be obtained as a composite calibration estimator, that is, ${\hat{\tau }}_{y,t}^{\mathrm{CR}}={\mathbf{Y}}_{(t)}\prime \mathbf{c}={\mathbf{Y}}_t\prime {\mathbf{c}}_t$ . Evidently, it suffices to use only the current-time component ${\mathbf{c}}_t$ , which incorporates micro-level information on $\mathbf{y}$ from the previous time, together with the current-time data matrix ${\mathbf{Y}}_t$ to obtain the composite regression estimator of ${\mathbf{\tau }}_{y,t}$ .

3.2. Analytical Expressions of CR Estimates of Levels and Changes

3.2.1. Estimates of Levels

The composite regression estimator of ${\mathbf{\tau }}_{y,t}$ given by Equation (7) can be decomposed into the standard and additional (composite) regression terms as

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}={\hat{\tau }}_{y,t}+{\hat{\mathbf{B}}}_x({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t})+{\hat{\mathbf{B}}}_y^c[{\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c-({\hat{\tau }}_{y,t}-{\hat{\tau }}_{y,m,t})],$$

(9)

where ${\hat{\mathbf{B}}}_x$ and ${\hat{\mathbf{B}}}_y^c$ are the partial regression coefficients, components of $\hat{\mathcal{B}}$ . Clearly, the estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ is recursive, through ${\hat{\tau }}_{y,t-1}^c$ and ${\hat{\tau }}_{y,m,t-1}^c$ , which are composite regression estimators carrying information from previous surveys to the current survey—the notation CR is reserved for estimators produced by the current-time $t$ composite calibration.

Partitioning the matrix $\mathcal{X}$ by the column submatrices in Equation (5) as $\mathcal{X}=(\mathbf{X},\mathbf{\Psi })$ , the vector $\mathbf{c}$ in Equation (8) can be decomposed (Merkouris 2004, 1132) as

$$\mathbf{c}={\mathbf{c}}_x+{\mathbf{L}}_x\mathbf{\Psi }{(\mathbf{\Psi }\prime {\mathbf{L}}_x\mathbf{\Psi })}^{-1}(0-\mathbf{\Psi }\prime {\mathbf{c}}_x),$$

(10)

where ${\mathbf{c}}_x=\mathbf{w}+\mathcal{W}\mathbf{X}{(\mathbf{X}\mathbf{\text{'}}\mathcal{W}\mathbf{X})}^{-1}({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t})$ is the vector of calibrated weights based on the regression matrix $\mathbf{X}$ , so that $\mathbf{X}\mathbf{\text{'}}{\mathbf{c}}_x={\mathbf{\tau }}_{x,t}$ , and ${\mathbf{L}}_x=\mathcal{W}(\mathbf{I}-{\mathbf{P}}_x)$ , with ${\mathbf{P}}_x=\mathbf{X}(\mathbf{X}\mathbf{\text{'}}\mathcal{W}\mathbf{X}{)}^{-1}\mathbf{X}\mathbf{\text{'}}\mathcal{W}$ . Note that $\mathbf{\Psi }\prime \mathbf{c}=0$ , this being the partial calibration constraint associated with the differences in the last term in Equation (9).

The vector ${\mathbf{c}}_x$ can be written analytically as

$${\mathbf{c}}_x=\left(\begin{array}{c}{\mathbf{c}}_{x,t}\hfill \\ \multicolumn{1}{c}{{\mathbf{c}}_{x,t-1}}\end{array}\right)=\left(\begin{array}{c}{\mathbf{w}}_t+{\mathbf{W}}_t{\mathbf{X}}_t{\left(\mathbf{X}{\prime }_t{\mathbf{W}}_t{\mathbf{X}}_t\right)}^{-1}({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t})\hfill \\ \multicolumn{1}{c}{{\mathbf{c}}_{t-1}}\end{array}\right),$$

(11)

with ${\mathbf{c}}_{x,t}$ as in Equation (2). Then, straightforward calculations give

$${\mathbf{Y}}_{(t)}\prime {\mathbf{c}}_x={\hat{\tau }}_{y,t}^R$$

and

$$\mathbf{\Psi }\mathbf{\text{'}}{\mathbf{c}}_x={\hat{\tau }}_{y,t}^R-{\hat{\tau }}_{y,m,t}^R-({\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c),$$

where ${\hat{\tau }}_{y,t}^R$ and ${\hat{\tau }}_{y,m,t}^R$ are respectively the full-sample and matched-sample regression estimators defined in Section 2.

Now, using Equation (10) and Equation (11), the composite calibration estimator ${\mathbf{Y}}_{(t)}\prime \mathbf{c}$ can be expressed in the (alternative to Equation (7)) composite regression form

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}={\hat{\tau }}_{y,t}^R+{\hat{\mathbf{B}}}_y^c({\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c-({\hat{\tau }}_{y,t}^R-{\hat{\tau }}_{y,m,t}^R)),$$

(12)

where ${\hat{\mathbf{B}}}_y^c={\mathbf{Y}}_{(t)}\prime {\mathbf{L}}_x\mathbf{\Psi }{(\mathbf{\Psi }\prime {\mathbf{L}}_x\mathbf{\Psi })}^{-1}$ is the partial regression coefficient in Equation (9). A more explicit expression of ${\hat{\mathbf{B}}}_y^c$ is derived upon noting that

$${\mathbf{L}}_x=\left(\begin{array}{c}\begin{array}{c}{\mathbf{L}}_{x,t}0\hfill \\ \multicolumn{1}{c}{0{\mathbf{C}}_{t-1}}\end{array}\hfill \end{array}\right),$$

where ${\mathbf{L}}_{x,t}={\mathbf{W}}_t(\mathbf{I}-{\mathbf{P}}_{x,t})$ , and ${\mathbf{P}}_{x,t}={\mathbf{X}}_t{({\mathbf{X}}_t\prime {\mathbf{W}}_t{\mathbf{X}}_t)}^{-1}{\mathbf{X}\mathbf{\text{'}}}_t{\mathbf{W}}_t$ . It follows then that ${\hat{\mathbf{B}}}_y^c={\mathbf{Y}}_t\prime {\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t{({\mathbf{\Psi }}_t\prime {\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t+{\mathbf{\Psi }}_{t-1}\prime {\mathbf{C}}_{t-1}{\mathbf{\Psi }}_{t-1})}^{-1}$ , which shows how previous-time information on $\mathbf{y}$ is incorporated in ${\hat{\mathbf{B}}}_y^c$ .

Expression (12) of the composite regression estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ allows a direct comparison with the current-time regression estimator ${\hat{\tau }}_{y,t}^R$ , separating the effect of incorporating previous-time information. We can write Equation (12) alternatively in the more interpretative form

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}=(\mathbf{I}-{\hat{\mathbf{B}}}_y^c){\hat{\tau }}_{y,t}^R+{\hat{\mathbf{B}}}_y^c({\hat{\tau }}_{y,t-1}^c+{\hat{\tau }}_{y,m,t}^R-{\hat{\tau }}_{y,m,t-1}^c).$$

(13)

Expression (13) shows that the composite regression estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ is a weighted average of the current-time regression estimator and the previous-time composite regression estimator updated with the change estimator based on the matched sample.

Expression (13) gives the composite regression estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ in multivariate form for the vector $\mathbf{y}$ . It follows from Equation (13) that for any of the $q$ components of $\mathbf{y}$ , say $y_g$ , the composite regression estimator of its total is (using for simplicity only the subscript $g$ )

$$\begin{array}{c}\multicolumn{1}{c}{{\hat{\tau }}_{g,t}^{\mathrm{CR}}=(1-{\hat{\beta }}_g^c){\hat{\tau }}_{g,t}^R+{\hat{\beta }}_g^c({\hat{\tau }}_{g,t-1}^c+{\hat{\tau }}_{g,m,t}^R-{\hat{\tau }}_{g,m,t-1}^c)}\\ \multicolumn{1}{c}{+{\hat{\mathbf{\beta }}}_{\overline{g}}^c({\hat{\tau }}_{\overline{g},t-1}^c-{\hat{\tau }}_{\overline{g},m,t-1}^c-({\hat{\tau }}_{\overline{g},t}^R-{\hat{\tau }}_{\overline{g},m,t}^R))}\end{array}$$

(14)

where ${\hat{\beta }}_g^c$ is the $g$ -th diagonal element of ${\hat{\mathbf{B}}}_y^c$ , ${\hat{\mathbf{\beta }}}_{\overline{g}}^c$ is the $g$ -th row vector of ${\hat{\mathbf{B}}}_y^c$ without the $g$ -th element, and the quantities in the last bracket of Equation (14) are the indicated vector estimators for the other $q-1$ components of $\mathbf{y}$ . Thus, although the composite estimator ${\hat{\tau }}_{g,t}^{\mathrm{CR}}$ incorporates all information on $y_g$ available in the two overlapping samples, in the manner of the weighted average in the first two terms of Equation (14), the additional third term suggests that ${\hat{\tau }}_{g,t}^{\mathrm{CR}}$ may realize additional efficiency due to correlation of $y_g$ with the rest of the components of $\mathbf{y}$ . Of course, ${\hat{\tau }}_{g,t}^{\mathrm{CR}}$ can be conveniently obtained as calibration estimator ${\mathbf{Y}}_{g,t}\prime {\mathbf{c}}_t$ , where ${\mathbf{Y}}_{g,t}$ is the $g$ -th column of ${\mathbf{Y}}_t$ . Generally, for any linear combination $\mathbf{\lambda }\mathbf{\text{'}}\mathbf{y}$ of the components of $\mathbf{y}$ , where $\mathbf{\lambda }$ is a $q\times 1$ vector of constants, the composite regression estimator of $\mathbf{\lambda }\mathbf{\text{'}}{\mathbf{\tau }}_{y,t}$ is given by $\mathbf{\lambda }\mathbf{\text{'}}{\hat{\mathbf{\tau }}}_{\mathbf{y}\mathbf{,}\mathbf{t}}^{\mathbf{CR}}\mathbf{=}\mathbf{\lambda }\mathbf{\text{'}}{\mathbf{Y}}_{\mathbf{t}}\mathbf{\text{'}}{\mathbf{c}}_{\mathbf{t}}$ .

Let now $z$ be any other single variable, with current-time total ${\tau }_{z,t}$ and current-sample matrix ${\mathbf{Z}}_t$ of dimension $n_t\times 1$ . Setting ${\mathbf{Z}}_{(t)}=({\mathbf{Z}}_t\prime ,0\mathbf{\text{'}})\prime$ and using Equation (10) we obtain the composite calibration estimator ${\mathbf{Z}}_t\prime {\mathbf{c}}_t$ of ${\tau }_{z,t}$ in composite regression form, analogous to Equation (12), as

$${\hat{\tau }}_{z,t}^{\mathrm{CR}}={\hat{\tau }}_{z,t}^R+{\hat{\mathbf{B}}}_z^c({\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c-({\hat{\tau }}_{y,t}^R-{\hat{\tau }}_{y,m,t}^R)),$$

(15)

where ${\hat{\mathbf{B}}}_z^c={\mathbf{Z}}_{(t)}\prime {\mathbf{L}}_x\mathbf{\Psi }{(\mathbf{\Psi }\prime {\mathbf{L}}_x\mathbf{\Psi })}^{-1}=\mathbf{Z}{\prime }_t{\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t({\mathbf{\Psi }}_t\prime {\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t+{\mathbf{\Psi }}_{t-1}\prime {\mathbf{C}}_{t-1}{\mathbf{\Psi }}_{t-1})$ , and ${\hat{\tau }}_{z,t}^R$ is the regression estimator of the form (1). It is seen from Equation (15) that the efficiency of the composite regression estimator ${\hat{\tau }}_{z,t}^{\mathrm{CR}}$ relative to the standard regression estimator ${\hat{\tau }}_{z,t}^R$ depends on the strength of correlation of $z$ with $\mathbf{y}$ , in the presence of the auxiliary variable $\mathbf{x}$ .

It is noteworthy that the simultaneous calibration of the two samples results also in an updated estimator for the previous time, incorporating information from current time. Setting ${\mathbf{Y}}_{(t-1)}=(0\mathbf{\text{'}},{\mathbf{Y}}_{t-1}\prime )\prime$ , we obtain the updated calibration estimator ${\mathbf{Y}}_{(t-1)}\prime \mathbf{c}={\mathbf{Y}}_{t-1}\prime {\mathbf{c}}_{t-1}^{*}$ (in place of ${\mathbf{Y}}_{t-1}\prime {\mathbf{c}}_{t-1}={\hat{\tau }}_{y,t-1}^c$ ) in the composite regression form, similar to Equation (13),

$${\hat{\tau }}_{y,t-1}^{\mathrm{CR}}=(\mathbf{I}-{\hat{\mathbf{B}}}_y^{c^{*}}){\hat{\tau }}_{y,t-1}^c+{\hat{\mathbf{B}}}_y^{c^{*}}({\hat{\tau }}_{y,t}^R-({\hat{\tau }}_{y,m,t}^R-{\hat{\tau }}_{y,m,t-1}^c)),$$

where ${\hat{\mathbf{B}}}_y^{c^{*}}={\mathbf{Y}}_{t-1}\prime {\mathbf{C}}_{t-1}{\mathbf{\Psi }}_{t-1}{({\mathbf{\Psi }}_t\prime {\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t+{\mathbf{\Psi }}_{t-1}\prime {\mathbf{C}}_{t-1}{\mathbf{\Psi }}_{t-1})}^{-1}$ . This shows that the updated composite regression estimator ${\hat{\tau }}_{y,t-1}^{\mathrm{CR}}$ is a weighted average of the initial previous-time composite regression estimator and the current-time regression estimator reduced by the change estimator based on the matched sample. The possible use of ${\hat{\tau }}_{y,t-1}^{\mathrm{CR}}$ will be considered in the following section.

3.2.2. Estimates of Change

In the simultaneous calibration of the previous-time and current-time samples that generates the composite estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ , the differences in estimates based on full and matched samples from previous and current time are calibrated to each other. This is due to the partial calibration constraint $\mathbf{\Psi }\prime \mathbf{c}=0$ noted above. Writing $\mathbf{\Psi }\prime \mathbf{c}=\mathbf{\Psi }{\prime }_t{\mathbf{c}}_t-\mathbf{\Psi }{\prime }_{t-1}{\mathbf{c}}_{t-1}^{*}$ , we easily verify that

$$\mathbf{\Psi }{\prime }_t{\mathbf{c}}_t=\mathbf{\Psi }{\prime }_{t-1}{\mathbf{c}}_{t-1}^{*}=(\mathbf{I}-{\hat{\mathbf{B}}}_d^c)({\hat{\tau }}_{y,t}^R-{\hat{\tau }}_{y,m,t}^R)+{\hat{\mathbf{B}}}_d^c({\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c),$$

(16)

where ${\hat{\mathbf{B}}}_d^c=\mathbf{\Psi }{\prime }_t{\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t{({\mathbf{\Psi }}_t\prime {\mathbf{L}}_{x,t}{\mathbf{\Psi }}_t+{\mathbf{\Psi }}_{t-1}\prime {\mathbf{C}}_{t-1}{\mathbf{\Psi }}_{t-1})}^{-1}$ . This shows that the calibration equates both differences ${\hat{\tau }}_{y,t}^R-{\hat{\tau }}_{y,m,t}^R$ and ${\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c$ to their combination in Equation (16). Since $\mathbf{\Psi }{\prime }_t{\mathbf{c}}_t={\hat{\tau }}_{y,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,m,t}^{\mathrm{CR}}$ and $\mathbf{\Psi }{\prime }_{t-1}{\mathbf{c}}_{t-1}^{*}={\hat{\tau }}_{y,t-1}^{\mathrm{CR}}-{\hat{\tau }}_{y,m,t-1}^{\mathrm{CR}}$ , the composite calibration results in the equality

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,m,t}^{\mathrm{CR}}={\hat{\tau }}_{y,t-1}^{\mathrm{CR}}-{\hat{\tau }}_{y,m,t-1}^{\mathrm{CR}}.$$

(17)

Interestingly,

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}={\hat{\tau }}_{y,t-1}^{\mathrm{CR}}+{\hat{\tau }}_{y,m,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,m,t-1}^{\mathrm{CR}},$$

which shows that the composite regression estimate at time $t$ is simply the updated composite regression estimate at time $t-1$ plus the change estimate based on the matched sample at times $t-1$ and $t$ .

It follows from Equation (17) that

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,t-1}^{\mathrm{CR}}={\hat{\tau }}_{y,m,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,m,t-1}^{\mathrm{CR}},$$

(18)

which means that the estimate of the change ${\mathbf{\tau }}_{y,t}-{\mathbf{\tau }}_{y,t-1}$ based on the full samples $s_{t-1}$ , $s_t$ is equal to the estimate of change based on the matched sample. The estimate ${\hat{\tau }}_{y,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,t-1}^{\mathrm{CR}}$ in Equation (18) involves the updated previous-time estimates ${\hat{\tau }}_{y,t-1}^{\mathrm{CR}}$ and ${\hat{\tau }}_{y,m,t-1}^{\mathrm{CR}}$ , and is obtained through composite calibration as ${\mathbf{Y}}_t\prime {\mathbf{c}}_t-{\mathbf{Y}}_{t-1}\prime {\mathbf{c}}_{t-1}^{*}$ . On the other hand, if the initial previous-time estimates ${\hat{\tau }}_{y,t-1}^c$ and ${\hat{\tau }}_{y,m,t-1}^c$ are used ( ${\hat{\tau }}_{y,t-1}^c$ being the already published estimate), then it follows easily from Equation (13) that the estimate of change ${\hat{\tau }}_{y,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,t-1}^c$ can be expressed as the weighted average of full-sample and matched-sample estimates of change

$${\hat{\tau }}_{y,t}^{\mathrm{CR}}-{\hat{\tau }}_{y,t-1}^c=(\mathbf{I}-{\hat{\mathbf{B}}}_y^c)({\hat{\tau }}_{y,t}^R-{\hat{\tau }}_{y,t-1}^c)+{\hat{\mathbf{B}}}_y^c({\hat{\tau }}_{y,m,t}^R-{\hat{\tau }}_{y,m,t-1}^c),$$

and conveniently obtained as ${\mathbf{Y}}_t\prime {\mathbf{c}}_t-{\mathbf{Y}}_{t-1}\prime {\mathbf{c}}_{t-1}$ . Like the level estimate ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ , this estimate of change arises as an approximate BLUE of change.

Using notation defined in Section 3.2.1, the estimate of change for the component $y_g$ of $\mathbf{y}$ is clearly obtained as ${\mathbf{Y}}_{g,t}\prime {\mathbf{c}}_t-{\mathbf{Y}}_{g,t-1}\prime {\mathbf{c}}_{t-1}$ , and for any other single variable $z$ the estimate of change is obtained as ${\mathbf{Z}}_t\prime {\mathbf{c}}_t-{\mathbf{Z}}_{t-1}\prime {\mathbf{c}}_{t-1}$ .

4. Comparisons with Other Methods

To facilitate the comparison of the proposed composite regression estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ of ${\mathbf{\tau }}_{y,t}$ with the current Fuller and Rao (2001) version of the MR composite estimator, a formulation of the MR estimator is presented here using present notation. This version is an extension of the regression estimator in Equation (1), of the form

$${\hat{\tau }}_{y,t}^{\mathrm{MR}}={\hat{\tau }}_{y,t}+{\hat{\mathbf{B}}}_x({\mathbf{\tau }}_{x,t}-{\hat{\tau }}_{x,t})+{\hat{\mathbf{B}}}^{\mathrm{MR}}({\hat{\tau }}_{y,t-1}^{\mathrm{MR}}-{\hat{\tau }}_{\psi ,t}^{\mathrm{MR}}),$$

(19)

where ${\hat{\tau }}_{y,t-1}^{\mathrm{MR}}$ is the MR composite estimate of ${\mathbf{\tau }}_{y,t-1}$ , and ${\hat{\tau }}_{\psi ,t}^{\mathrm{MR}}=(1-\alpha ){\hat{\tau }}_{\psi ,t}^{\mathrm{MR}1}+\alpha {\hat{\tau }}_{\psi ,t}^{\mathrm{MR}2}$ , with $\alpha \in (0,1)$ , is a weighted average of two HT estimates that use data from both times $t-1$ and $t$ but current-time $t$ sampling weights. The values $\alpha =0$ and $\alpha =1$ give the composite estimators MR1 (Singh and Merkouris 1995) and MR2 (Singh et al. 1997, 2001), respectively. Specifically, ${\hat{\tau }}_{\psi ,t}^{\mathrm{MR}1}=\left({\mathbf{\Psi }}_t^{\mathrm{MR}1}\right)\prime {\mathbf{w}}_t$ , where the $n_t\times q$ sample matrix ${\mathbf{\Psi }}_t^{\mathrm{MR}1}$ is defined as ${\mathbf{\Psi }}_t^{\mathrm{MR}1}=(\left({\hat{\tau }}_{y,t-1}^{\mathrm{MR}}\right)\prime 1_u\prime ,{\mathbf{Y}}_{m,t-1}\prime )\prime$ , where $1_u$ is a column of ones for the units in the unmatched sample, and ${\hat{\tau }}_{\psi ,t}^{\mathrm{MR}2}=\left({\mathbf{\Psi }}_t^{\mathrm{MR}2}\right)\prime {\mathbf{w}}_t$ , where ${\mathrm{Y}}_t^{MR2}={\left(Y_{u,t}^{\prime },Y_{m,t}^{\prime }+R\left(Y_{m,t-1}^{\prime }-Y_{m,t}^{\prime }\right)\right)}^{\prime }$ .

Writing the extended design matrix for the MR estimator in Equation (19) as ${\mathcal{X}}_t^{\mathrm{MR}}=({\mathbf{X}}_t,{\mathbf{\Psi }}_t^{\mathrm{MR}})$ , where ${\mathbf{\Psi }}_t^{\mathrm{MR}}=(1-\alpha ){\mathbf{\Psi }}_t^{\mathrm{MR}1}+\alpha {\mathbf{\Psi }}_t^{\mathrm{MR}2}$ is the sample matrix of the composite auxiliary variables, and the associated vector of totals as ${\mathbf{\tau }}_{\mathcal{X}}^{\mathrm{MR}}=({\mathbf{\tau }}_{x,t}\prime ,\left({\hat{\mathbf{\tau }}}_{y,t-1}^{\mathrm{MR}}\right)\prime )\prime$ , this estimator can be written as

$${\hat{\tau }}_{y,t}^{\mathrm{MR}}={\hat{\tau }}_{y,t}+{\hat{\mathcal{B}}}^{\mathrm{MR}}({\mathbf{\tau }}_{\mathcal{X}}^{\mathrm{MR}}-{\hat{\tau }}_{\mathcal{X}}^{\mathrm{MR}}),$$

where ${\hat{\tau }}_{\mathcal{X}}^{\mathrm{MR}}=\left({\mathcal{X}}_t^{\mathrm{MR}}\right)\prime {\mathbf{w}}_t$ and ${\hat{\mathcal{B}}}^{\mathrm{MR}}={\mathbf{Y}}_t\prime {\mathbf{W}}_t{\mathcal{X}}_t^{\mathrm{MR}}{[\left({\mathcal{X}}_t^{\mathrm{MR}}\right)\prime {\mathbf{W}}_t{\mathcal{X}}_t^{\mathrm{MR}}]}^{-1}$ . Note that the coefficients ${\hat{\mathbf{B}}}_x$ and ${\hat{\mathbf{B}}}^{\mathrm{MR}}$ in Equation (19) are the two components of ${\hat{\mathcal{B}}}^{\mathrm{MR}}$ . It is clear then that ${\hat{\tau }}_{y,t}^{\mathrm{MR}}$ can be expressed as a calibration estimator, that is, ${\hat{\tau }}_{y,t}^{\mathrm{MR}}=\mathbf{Y}{\prime }_t{\mathbf{c}}_t^{\mathrm{MR}}$ , where the vector of calibrated weights

$${\mathbf{c}}_t^{\mathrm{MR}}={\mathbf{w}}_t+{\mathbf{W}}_t{\mathcal{X}}_t^{\mathrm{MR}}{[\left({\mathcal{X}}_t^{\mathrm{MR}}\right)\prime {\mathbf{W}}_t{\mathcal{X}}_t^{\mathrm{MR}}]}^{-1}({\mathbf{\tau }}_{\mathcal{X}}^{\mathrm{MR}}-{\hat{\mathcal{X}}}_t^{\mathrm{MR}})$$

satisfies the initial constraints ${\mathbf{X}}_t\prime {\mathbf{c}}_t^{\mathrm{MR}}={\mathbf{\tau }}_{x,t}$ and the additional constraints $\left({\mathbf{\Psi }}_t^{\mathrm{MR}}\right)\prime {\mathbf{c}}_t^{\mathrm{MR}}={\hat{\tau }}_{y,t-1}^{\mathrm{MR}}$ .

The composite auxiliary variables used in the MR composite estimation, with the previous-time MR estimator used as corresponding calibration total, involve data from the current and previous survey time, and sample matching between the two times is done at the individual record level. In this matching procedure, missing values are imputed using mean imputation and carry-backward imputation for the MR1 and MR2 components, respectively. The choice of the value of the tuning constant $\alpha$ depends on the relative importance of the level and change estimates, and leads also to a compromise between the two imputation methods. Satisfying the two sets of calibration constraints requires that the previous-time survey data be calibrated to the current-time demographic totals. Detailed discussion of implementation issues related to the choice of the tuning constant and the imputation required in specifying the composite auxiliary variables is found in Gambino et al. (2001) and Statistics Canada (2017).

The proposed composite regression estimators for levels and changes derive their efficiency from the fact that they are approximate BLUEs, with the partial regression coefficient ${\hat{\mathbf{B}}}_y^c$ incorporating information on $\mathbf{y}$ from both previous-time and current-time samples. Note that the estimate $\hat{\mathbf{B}}={\mathbf{Y}}_{(t)}\prime \widehat{\mathrm{Var}}(\mathbf{w})\mathcal{X}{[\mathcal{X}\prime \widehat{\mathrm{Var}}(\mathbf{w})\mathcal{X}]}^{-1}$ of the BLUE coefficient in (6) is a function of data from both survey times. The regression coefficient $\hat{\mathcal{B}}={\mathbf{Y}}_{(t)}\prime \mathcal{W}\mathcal{X}{(\mathcal{X}\prime \mathcal{W}\mathcal{X})}^{-1}$ in Equation (7), a practicable substitute of $\hat{\mathbf{B}}$ , and hence the partial regression coefficient ${\hat{\mathbf{B}}}_y^c$ in Equation (12), are also functions of weighted data from both survey times. It should be also noted that the submatrix ${\mathbf{C}}_{t-1}$ of the weighting matrix $\mathcal{W}$ incorporates past information through the composite calibrated weights ${\mathbf{c}}_{t-1}$ of time $t-1$ .

In contrast, the MR-composite estimator is generated by a calibration of current-time weights, whereby current-time estimates are calibrated to previous-time estimates, the latter being treated as constants in calibration, and thus the regression coefficient incorporates weighted data from current time only. This becomes evident in the MR2 composite estimator (the Fuller-Rao version with $\alpha =1$ ), which has the form in Equation (19) with the second regression term written explicitly as ${\hat{\tau }}_{y,t-1}^{\mathrm{MR}2}-{\hat{\tau }}_{y,m,t-1}^{*}-({\hat{\tau }}_{y,t}-{\hat{\tau }}_{y,m,t})$ . In this expression, ${\hat{\tau }}_{y,m,t-1}^{*}$ estimates current population totals for the previous-time variables $\mathbf{y}$ , and the previous-time MR2 estimator ${\hat{\tau }}_{y,t-1}^{\mathrm{MR}2}$ is the calibration total, also transformed to reflect possible changes in demographic totals from $t-1$ to $t$ . On the other hand, in the comparable form in Equation (9) of the proposed composite estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ (which is generated by the simultaneous calibration of $s_{t-1}$ and $s_t$ ) the calibration total associated with the second regression term ${\hat{\tau }}_{y,t-1}^c-{\hat{\tau }}_{y,m,t-1}^c-({\hat{\tau }}_{y,t}-{\hat{\tau }}_{y,m,t})$ is zero.

In view of the construction of the proposed composite estimators of level and change, there is no need for searching a compromise between estimation of level and change using a tuning constant, as with the Fuller-Rao estimator. A similar comment is made in Bonnéry et al. (2020) regarding a BLUE of level and a BLUE of change, in a comparison with the Fuller-Rao estimator.

The proposed composite estimation is free of problems with sample matching between two consecutive times at the individual record level, as required in the MR-composite estimation. These problems arise when, for a given matched sample, data is available only for one survey time. This may occur due to nonresponse in either survey time or when a move or change in scope has taken place between the two consecutive survey times; see Gambino et al. (2001). Micro-matching involves imputation, that is, artificial creation of data, which may not reflect the actual correlation of data collected from the same units in successive survey times. This may create a false effect on the efficiency of the MR estimator, and may also introduce bias. Such bias, which may be accumulated over time due to the recursive nature of the composite estimator, is avoided in the proposed estimation procedure. The proposed method is also free of operational complexities of the MR-composite estimation, which include the extra calibration of past-month data to the current-month population totals, and the cumbersome variance estimation by resampling methods; see Statistics Canada (2017). In current MR methodology, bootstrap replicates of the composite calibration totals of previous month are computed, adjusted to current month population totals, and used in current month calculations using bootstrap sample coordination between survey times. In the proposed method, estimation and variance estimation can be done conveniently in one step, with replication of the composite calibration of the combined sample to generate replicate estimates for the variance calculations.

The form of the composite regression estimator in Equation (12) is similar to that of the K-composite estimator, with the regression coefficient ${\hat{\mathbf{B}}}_y^c$ in place of the coefficient K. In K-composite and AK-composite estimation, values of A and K that are optimal over time, in the sense of minimum variance of the estimator, are empirically chosen for each variable of interest, which may cause inconsistencies among estimates. In contrast, the proposed composite estimator, with the time-dependent matrix coefficient ${\hat{\mathbf{B}}}_y^c$ , is multivariate and thus the efficiency of estimation for each of the components of $\mathbf{y}$ (or a linear combination of any subset of them) may be enhanced by the correlation with other components, as indicated by Equation (14). Also unlike AK-composite estimation, in which only estimates for selected key variables are true composite estimates, the proposed composite calibration generates composite regression estimates for any variable, as shown in Equation (15). Operationally, unlike the AK-estimation, where calibration to satisfy known population totals and composite estimation are separate steps, calibration weighting in the proposed composite regression estimation is done in one step, that is, simultaneously with weighting to satisfy the standard calibration constraints.

Like the proposed multivariate composite regression estimator, the regression estimator of Konrad and Berger (2023) also involves the combined sample $s_{t-1}\cup s_t$ and does not require imputation, though it has a different structure: It is multivariate with respect to the totals of a single variable $y$ in two consecutive survey times. It involves the vector $\mathbf{x}$ in both $s_{t-1}$ and $s_t$ and the vector of the additional auxiliary variables represents sampling design information in terms of stratification and sample rotation, instead of information on the key variables of interest from both survey times (as in Equation (9)). Furthermore, this estimator is not recursive and is not formulated as a calibration estimator.

5. Discussion

We have developed a new method of composite regression estimation, based on the principle of best linear unbiased estimation and with a transparent structure of the multivariate composite estimators of levels and changes for any survey variable. These estimators are generated by a suitable simultaneous calibration of the weights of the combined sample of current and previous survey times. This theoretically well-founded calibration procedure allows a most effective incorporation of past information in the current time weights, and incorporation of current information in the previous time weights, resulting in composite estimators whose efficiency is expected to compare favorably with that of existing estimators.

The proposed composite estimation method is considerably more practical than the existing composite estimation methods. This is a significant advantage of the new method, considering the operational complexity of a composite estimation process.

The proposed method can be extended to rotation schemes that are more general than the typical scheme outlined in Section 2.

An important issue with repeated surveys with rotating panels is the possible rotation bias due to the differential nonresponse and measurement error for the different panels. It is known that the birth panel usually differs most from the others, so that the matched and unmatched samples differ. This is a intrinsic problem with repeated surveys with such design, and affects not only composite estimation but also the basic Horvitz-Thompson estimation and the standard regression estimation. The reduction of the effect of rotation group bias on composite estimation by addressing the cause of this bias (e.g., differences in nonresponse rates and in the mode of data collection) has been discussed in the literature; see Gambino et al. (2001). In this connection, an extension of the composite regression estimator ${\hat{\tau }}_{y,t}^{\mathrm{CR}}$ , analogous to the extension of the K-composite to the AK-composite estimator, could involve adding the regression term ${\hat{\tau }}_{y,u,t}-{\hat{\tau }}_{y,m,t}$ in Equation (9), where ${\hat{\tau }}_{y,u,t}=r{\mathbf{Y}}_{u,t}\prime {\mathbf{w}}_{u,t}$ is the estimate of ${\mathbf{\tau }}_{y,t}$ based on the unmatched (“birth”) panel at time $t$ . This is done by augmenting the matrix $\mathcal{X}$ in Equation (5) by the column $({\overline{\mathbf{\Psi }}}_t\prime ,0\mathbf{\text{'}})\prime$ , where ${\overline{\mathbf{\Psi }}}_t=-r{\mathbf{\Psi }}_t$ , and using the vector of calibration totals $({\mathbf{\tau }}_{x,t}\prime ,0\mathbf{\text{'}},0\mathbf{\text{'}})\prime$ . The extended calibration corresponding to this extended regression estimation will result in ${\hat{\tau }}_{y,u,t}^{\mathrm{CR}}=\widehat{{\tau }_{y,m,t}^{\mathrm{CR}}}$ , which may help to reduce the birth rotation bias due to the usual difference of the birth panel from the other panels.

The performance of the proposed composite regression estimators for levels and changes needs to be assessed through an extensive empirical study using actual data from a repeated survey with rotating panels (e.g., data from a Labour Force Survey). These estimators should be evaluated for multiple survey characteristics using data over a sufficient period of time, to generate the effect of the recursive incorporation of past information, and their advantages should be judged not only on their statistical efficiency but also on their impact on various time series with respect to stability and seasonal adjustment. Such is the study for the MR estimator in Gambino et al. (2001). Considering that the Bonnéry et al. (2020) evaluation study declared the current MR method preferable to the AK method, the comparison could be limited to one between the proposed method and the MR method and the method of Konrad and Berger (2023). Such an empirical study is beyond the scope of the present article.