Despite the increasing attention devoted to the study and analysis of longitudinal
data, relatively little consideration has been directed toward understanding the
issues of reliability and measurement error. Perhaps one reason for this neglect has
been that traditional methods of estimation (e.g. generalisability theory) require
assumptions that are often not tenable in longitudinal designs. This paper first
examines applications of generalisability theory to the estimation of m easurement
error and reliability in longitudinal research, and notes how factors such as
missing data, correlated errors, and true score instability prohibit traditional
variance com ponent estimation. Next, we discuss how estimation methods using
restricted maximum likelihood can account for these factors, thereby providing m any
advantages over traditional estimation methods. Finally, we provide a substantive
exam ple illustrating these advantages, and include brief discussions of programming
and software considerations.
Get full access to this article
View all access options for this article.
References
1.
Anderson, R.D.
(1979). Estimating variance components from balanced data:
Optimum properties of REML solutions and MIVQUE estimators. In
L.D. V and Vleck
and
S.R. Searle
(Eds.), Variance components and animal breeding.
Animal Science Department, Cornell University,
Ithaca: New York
.
2.
Arbuckle, J.L.
(1996). Full information estimation in the presence of
incomplete data. In
G.A. Marcoulides
&
R.E. Schumacker
(Eds.), Advanced structural equation modeling: Issues and
techniques.
Mahwah, N J: LE A
.
3.
Bollen, K.A.
(1989). Structural equations with latent variables.
New York: Wiley
.
4.
Bost, J.E.
(1995).
The effects of correlated errors on generalizability and depend ability coefficients
. Applied Psychological Measurement, 19,
191-203
.
5.
Brennan, R.L.
(1983). Elements of generalizability theory.
Iowa City, IA: A C T Publications
.
6.
Brennan, R.L.
(1992). Elements o f generalizability theory (2nd
ed.).
Iowa City, IA: A C T Publication s
.
7.
Brown, R.L.
(1994).
Efficacy of the indirect app roach for estimating structural equation
models with missing data: A comparison o f five methods
. Structural Equation Modeling, 1,
287-316
.
8.
Cronbach, L.J.
(1951).
C o efficient alpha and the internal structure o f tests
. Psychometrika, 16,
297-334
.
9.
Cronbach, L.J.
,
G lese r, G.C.
,
Nanda, H.
, &
Rajaratnam, N.
(1972). T h e dependability of behavioral m easurements:
Theory of generalizability for scores an d profiles.
New York: Wiley
.
10.
Duncan, S.C.
, &
D un can, T.E.
(1994).
Modeling incomplete longitudinal sub stance abuse data using latent
variable growth curve methodology
. Multivariate Behavioral Research, 29,
313-338
.
11.
G ood man, J.S.
, &
B lum, T.C.
(1996).
Assessing the no n-random sampling effects of subject attrition in
longitudinal research
. Journal of Management, 22,
627-652
.
12.
Graham, J.W.
,
Hofer, S.M.
, &
M cK inn on, D.P.
(1996).
Maximizing the usefulness o f data obtained with planned missing value
pattern s: A n application o f maximum likelihood procedures
. Multivariate Behavioral Research, 31,
197-218
.
13.
Hartley, H.O.
, &
R ao, J.N.K.
(1967).
Maximum likelihood estimation forth e mixed analysis of variance mo del
. Biometrika, 54,
93-108
.
14.
Heise, D.R.
(1969).
Separating reliability an d stability in test-retest correlation
. American Sociological Review, 34,
93-101
.
15.
Hocking, R.R.
(1993). Variance component estimation in mixed linear
models. In
L.K. Edward s
(Ed.), A p plied analysis of variance in behavioral science.
New York: Marcel Dekker
.
16.
Hocking, R.R.
, &
Kutner, M.H.
(1975).
Some analytic an d numerical comparisons of estimators for the mixed A.O.V. model
. Biometrics, 31,
19-28
.
17.
Jagodzinski, W.
, &
Kuhnel, S.M.
(1987).
Estimation of reliability and stability in single-indicator multiple-wave models
. Sociological Method s & Research, 15,
219-258
.
Little, R.J.A.
(1982).
Models for non response in sample surveys
. Journal of the American Statistical Association,
77,
237-250
.
20.
Little, R.J.A.
(1988).
A test of missing completely at random for multivariate data with missing values
. Journal o f the American Statistical Association,
83,
1198-1202
.
21.
Little, R.J.A.
, &
Rubin, D.B.
(1987). Statistical analysis with missing data.
New York: Wiley
.
22.
Little, R.J.A.
, &
Rubin, D.B.
(1989).
T he analysis of social science data with missing values
. Sociological Method s and Research, 18,
292-326
.
23.
Little, R.J.A.
, &
Schenker, N.
(1995). Missing data. In
G. Arminger
,
C.C. Clogg
, &
M.E. Sobel
(Eds.), Handbook of statistical modeling for the social an d behavioral
sciences (pp. 39-75).
New York: Plenum
.
24.
Marcoulides, G. (1987). A n alternative method for variance component
estimation: Applications to generalizability theory. Unpublished doctoral
dissertation, University o f California, Los Angeles.
25.
McArdle, J.J.
, &
E p stein, D.B.
(1987).
Latent growth curves within developmental structural equation models
. Child Development, 58,
110-133
.
26.
McArdle, J.J.
, &
Hamagami, F.
(1991). Modeling incomplete longitudinal data u sing
latent growth structural equation models. In
L. Collins
&
J. Horn
(Eds.), Best methods for the analysis of change.
Washington, D C: American Psychological Association
.
27.
Meredith, W.
, &
Tisak, J.
(1990).
Latent curve an analysis
. Psychometrik a, 55,
107-122
.
28.
M u then, B.O.
,
Kaplan, D.
, &
Hollis, M.
(1987).
O n structural equation modeling with data that are not missing completely
at random
. Psychometrika, 52,
431-467
.
29.
Patterson, H.D.
, &
Thompson, R.
(1971).
Recovery of inter-block information when block sizes are unequal
. Biometrik a, 58,
545-554
.
30.
Rao, C.R.
(1971).
Estimation o f variance and covariance components—M IN Q U E theory
. Journal of Multivariate Analysis, 1,
257-275
.
31.
Rubin, D.B.
(1976).
Inference an d missing data
. Biometrika, 63,
581-592
.
32.
Searle, S.R.
(1971). Linear models.
New York: Wiley
.
33.
Searle, S.R.
,
Casella, G.
, &
McCulloch, C.E.
(1992). Variance components.
New York: Wiley
.
34.
Shavelson, R.J.
, &
W ebb, N.M.
(1991). Generalizability theory: A primer.
Newbury Park, C A: Sage
.
35.
Smith, P.
(1978).
A confidence interval approach for variance component estimates in the
context o f generalizability theory
. Educational and Psychological Measurement, 42,
459-466
.
36.
Smith, P.
(1982).
Sampling errors o f variance components in small sample multifacet
generalizability studies
. Journal of Educational Statistics, 3,
319-346
.
Tisak, J.
, &
Meredith, W.
(1990). Descriptive and associated developmental
models. In
A. V on E ye
(Ed.), Statistical methods in longitudinal research (V o l. 2, pp.
387-406).
Sa n Diego, C A: Academic Press
.
39.
Werts, C.E.
,
Breland, H.M.
,
G randy, J.
, &
R o ck, D.R.
(1980).
U sing longitudinal data to estimate reliability in the presence of
correlated measurement errors
. Educational an d Psychological Measurement, 40,
19-29
.
40.
Wiley, D.E.
, &
Wiley, J.A.
(1970).
The estimation of measurement error in p an el data
. American Sociological Review, 35,
112-117
.
41.
Willett, J.B.
, &
Sayer, A.G.
(1994).
U sin g covariance structure analysis to detect correlates an d predictors
of individual change over time
. Psychological Bulletin, 116,
363-381
.
42.
Wolfinger, R.
(1993).
Covariance structure selection in general mixed models
. Communications in statistics—Simulation and
computation, 22,
1079-1106
.
