Quantitative industrial relations research frequently relies on data collected from large surveys of establishments that use complex sampling designs, such as stratified and unequal probability sampling. The authors analyze two complex surveys of establishments, the National Organizations Survey and the National Survey of Establishments. They discuss design-based (survey-weighted) and model-based (unweighted) strategies for analyzing these data. They show that the choice of strategy can affect inferences about parameters, and hence conclusions drawn from analyses. They discuss the advantages of model-based approaches that include independent variables corresponding to design features, such as functions of size measures or indicator variables for strata or clusters, relative to purely design-based approaches.
Get full access to this article
View all access options for this article.
References
1.
AlthauserRobert P.1989. “Internal Labor Markets.”Annual Review of Sociology, Vol. 15, pp. 143–61.
2.
AlthauserRobert P.KallebergArne L., 1981. “Firms, Occupations, and the Structure of Labor Markets: A Conceptual Analysis.” In BergIvar, ed., Sociological Perspectives on Labor Markets. New York: Academic Press, pp. 119–49.
3.
AngristJoshua D.ImbensGuido W.RubinDonald B., 1996. “Identification of Causal Effects Using Instrumental Variables.”Journal of the American Statistical Association, Vol. 91, No. 433, pp. 444–55.
4.
BelsleyDavid A.KuhEdwinWelschRoy E., 1980. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: John Wiley & Sons.
5.
BinderDavid A.1983. “On the Variances of Asymptotically Normal Estimators from Complex Surveys.”International Statistical Review, Vol. 51, No. 2, pp. 279–92.
6.
BinderDavid A.1992. “Fitting Cox's Proportional Hazards Model from Survey Data.”Biometrika, Vol. 79, No. 1 (March), pp. 139–47.
7.
BrewerKenneth R. W.MellorR. W., 1973. “The Effect of Sample Structure on Analytical Surveys.”Australian Journal of Statistics, Vol. 15, No. 3, pp. 145–52.
8.
BrykAnthony S.RaudenbushStephen W., 1992. Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, Calif.: Sage.
9.
CappelliPeter. 2001. “The National Employer Survey: Employer Data on Employment Practices.”Industrial Relations, Vol. 40, No. 3 (July), pp. 635–47.
10.
CappelliPeterNeumarkDavid. 2001. “Do ‘High-Performance’ Work Practices Improve Establishment-Level Outcomes?”Industrial and Labor Relations Review, Vol. 54, No. 4 (July), pp. 737–75.
11.
ChamblessLloyd E.BoyleKerrie E., 1985. “Maximum Likelihood Methods for Complex Sample Data: Logistic Regression and Discrete Proportional Hazards Models.”Communications in Statistics, Part A—Theory and Methods, Vol. 14, No. 6, pp. 1377–92.
CohenSteven B.1997. “An Evaluation of Alternative PC-Based Packages for the Analysis of Complex Survey Data.”American Statistician, Vol. 51, No. 3, pp. 285–92.
DuMouchelWilliam H.DuncanGreg J., 1983. “Using Sample Survey Weights in Multiple Regression Analyses of Stratified Samples.”Journal of the American Statistical Association, Vol. 78, No. 383, pp. 535–43.
16.
GelmanAndrewCarlinJohn B.SternHal S.RubinDonald B., 1995. Bayesian Data Analysis. New York: Chapman & Hall.
17.
GerhartBarryWrightPatrick M.McMahanGary C.SnellScott A., 2000. “Measurement Error in Research on Human Resources and Firm Performance: How Much Error Is There and How Does It Influence Effect Size Estimates?”Personnel Psychology, Vol. 53, No. 4 (Winter), pp. 803–34.
18.
GrammCynthia L.SchnellJohn F., 2001. “The Use of Flexible Staffing Arrangements in Core Production Jobs.”Industrial and Labor Relations Review, Vol. 54, No. 2 (January), pp. 245–58.
19.
HansenMorris H.MadowWilliam G.TeppingBenjamin J., 1983. “An Evaluation of Model-Dependent and Probability-Sampling Inferences in Sample Surveys.”Journal of the American Statistical Association, Vol. 78, No. 388, pp. 776–93.
20.
HeckmanJames J.1979. “Sample Selection Bias as a Specification Error.”Econometrica, Vol. 47, No. 1, pp. 153–61.
21.
HeckmanJames J.1990. “Varieties of Selection Bias.”American Economic Review, Vol. 80, No. 3 (June), pp. 313–18.
22.
HeckmanJames J.RobbRichard. 1985. “Alternative Methods for Evaluating the Impact of Interventions: An Overview.”Journal of Econometrics, Vol. 30 (October-November), pp. 239–67.
23.
HorvitzDaniel G.ThompsonD. J., 1952. “A Generalization of Sampling without Replacement from a Finite Universe.”Journal of the American Statistical Association, Vol. 47, No. 260, pp. 663–85.
HunterLarry W.PilFrits. 1995. “How Do You Survey Firms?” Proceedings of the Forty-Seventh Annual Meetings. Madison, Wisconsin: Industrial Relations Research Association, pp. 152–62.
26.
HuselidMark A.1995. “The Impact of Human Resource Management Practices on Turnover, Productivity, and Corporate Financial Performance.”Academy of Management Journal, Vol. 38, No. 3 (June), pp. 635–72.
27.
KallebergArne L.KnokeDavid L.MarsdenPeter V.SpaethJoe L., 1996a. “Formalizing the Employment Relation.” In KallebergArne L.KnokeDavid L.MarsdenPeter V.SpaethJoe L., eds., Organizations in America: Analyzing Their Structures and Human Resource Practices. Thousand Oaks, Calif.: Sage, pp. 87–112.
28.
KallebergArne L.MarsdenPeter V.KnokeDavid L.SpaethJoe L., 1996b. Organizations in America. Analyzing Their Structures and Human Resource Practices. Thousand Oaks, Calif.: Sage.
29.
KaltonGraham. 1983. “Models in the Practice of Survey Sampling.”International Statistical Review, Vol. 51, No. 1, pp. 175–88.
30.
KaltonGrahamKasprzykDaniel. 1986. “The Treatment of Missing Survey Data.”Survey Methodology, Vol. 12, No. 1, pp. 1–16.
31.
KornEdward L.GraubardBarry I., 1995. “Examples of Differing Weighted and Unweighted Estimates from a Sample Survey.”American Statistician, Vol. 49, No. 3, pp. 291–95.
32.
KottPhillip S.1991. “A Model-Based Look at Linear Regression with Survey Data.”American Statistician, Vol. 45, No. 2, pp. 107–12.
33.
LittleRoderick J. A.RubinDonald B., 2002. Statistical Analysis with Missing Data, 2nd ed.New York: John Wiley & Sons.
34.
LohrSharon L.LiuJoanna. 1994. “A Comparison of Weighted and Unweighted Analyses in the National Crime Victimization Survey.”Journal of Quantitative Criminology, Vol. 10, No. 4, pp. 343–60.
35.
LynchLisa M.BlackSandra E., 1998. “Beyond the Incidence of Employer-Provided Training.”Industrial and Labor Relations Review, Vol. 52, No. 1 (October), pp. 64–81.
36.
MacDuffieJohn P.1995. “Human Resource Bundles and Manufacturing Performance: Organizational Logic and Flexible Production Systems in the World Auto Industry.”Industrial and Labor Relations Review, Vol. 48, No. 2 (January), pp. 199–221.
37.
NordbergLennart. 1989. “Generalized Linear Modeling of Sample Survey Data.”Journal of Official Statistics, Vol. 5, No. 3, pp. 223–39.
38.
OstermanPaul. 1984. “Introduction: The Nature and Importance of Internal Labor Markets.” In OstermanPaul, ed., Internal Labor Markets. Cambridge, Mass.: MIT Press.
39.
OstermanPaul. 1994. “How Common Is Workplace Transformation and Who Adopts It?”Industrial and Labor Relations Review, Vol. 47, No. 2 (January), pp. 175–88.
40.
OstermanPaul. 2000. “Work Reorganization in an Era of Restructuring: Trends in Diffusion and Effects on Employee Welfare.”Industrial and Labor Relations Review, Vol. 53, No. 2 (January), pp. 179–96.
41.
PfefferJeffreyCohenYinon. 1984. “Determinants of Internal Labor Markets in Organizations.”Administrative Science Quarterly, Vol. 29, No. 4 (December), pp. 550–72.
42.
PfeffermannDanny. 1993. “The Role of Sampling Weights When Modeling Survey Data.”International Statistical Review, Vol. 61, No. 2, pp. 317–37.
43.
PilFrits K.MacDuffieJohn Paul. 1996. “The Adoption of High-Involvement Work Practices.”Industrial Relations, Vol. 35, No. 3 (Fall), pp. 423–55.
44.
ReiterJerome P.RaghunathanThivellore E., 2003. “Multiple Imputation for Missing Data in Surveys with Complex Designs.” Unpublished paper, Duke University.
45.
RosenbaumPaul R.2002. Observational Studies. New York: Springer-Verlag.
46.
RosenbaumPaul R.RubinDonald B., 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.”Biometrika, Vol. 70, No. 1 (March), pp. 41–55.
47.
RosenbaumPaul R.RubinDonald B., 1984. “Reducing Bias in Observational Studies Using Subclassification on the Propensity Score.”Journal of the American Statistical Association, Vol. 79, No. 387, pp. 516–24.
48.
RoyallRichard M.1983. “Comments on An Evaluation of Model-Dependent and Probability-Sampling Inferences in Sample Surveys.”Journal of the American Statistical Association, Vol. 78, No. 384, pp. 794–96.
49.
RubinDonald B.1976. “Inference and Missing Data.”Biometrika, Vol. 63, No. 3 (September), pp. 581–90.
50.
RubinDonald B.1984. “Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.”Annals of Statistics, Vol. 12, pp. 1151–72.
51.
RubinDonald B.1987. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons.
52.
SarndalCarl-ErikSwenssonBengtWretmanJan. 1992. Model Assisted Survey Sampling. New York: Springer-Verlag.
53.
SchaferJoseph L.1997. Analysis of Incomplete Multivariate Data. New York: Chapman & Hall.
54.
SchaferJoseph L.GrahamJohn W., 2002. “Missing Data: Our View of the State of the Art.”Psychological Methods, Vol. 7, No. 2, pp. 147–77.
55.
SpaethJoe L.O'RourkeDiane P., 1996. “Design of the National Organizations Study.” In KallebergArne L.KnokeDavid L.MarsdenPeter V.SpaethJoe L., eds., Organizations in America: Analyzing Their Structures and Human Resource Practices. Thousand Oaks, Calif.: Sage, pp. 23–39.
56.
SugdenR. A.SmithT. M. F., 1984. “Ignorable and Informative Designs in Survey Sampling Inference.”Biometrika, Vol. 71, No. 3 (September), pp. 495–506.
57.
WinshipChristopherRadbillLarry. 1994. “Sampling Weights and Regression Analysis.”Sociological Methods & Research, Vol. 23, No. 2, pp. 230–57.
58.
WolterKirk M.1985. Introduction to Variance Estimation. New York: Springer-Verlag.
59.
ZaslavskyAlan. 2004. “Summary of Survey Analysis Software.” American Statistical Association Survey Research Methods Section Webpage. Retrieved January 27, 2004 from the World Wide Web http://www.hcp.med.harvard.edu/statis-tics/survey-soft/.
