Range restriction is a common problem in organizational research and is an important statistical artifact to correct for in meta-analysis. Historically, researchers have had to rely on range-restriction corrections that only make use of range-restriction information for one variable, but it is not uncommon for researchers to have such information for both variables in a correlation (e.g., when studying the correlation between two predictor variables). Existing meta-analytic methods incorporating bivariate range-restriction corrections overlook their unique implications for estimating the sampling variance of corrected correlations and for accurately assigning weights to studies in individual-correction meta-analyses. We introduce new methods for computing individual-correction and artifact-distribution meta-analyses using the bivariate indirect range restriction (BVIRR; “Case V”) correction and describe improved methods for applying BVIRR corrections that substantially reduce bias in parameter estimation. We illustrate the effectiveness of these methods in a large-scale simulation and in meta-analyses of expatriate data. We provide R code to implement the methods described in this article; more comprehensive and robust functions for applying these methods are available in the psychmeta package for R.
AitkenA. C. (1934). Note on selection from a multivariate normal population. Proceedings of the Edinburgh Mathematical Society (Series 2), 4(2), 106–110. https://doi.org/10/bmqrbz
2.
AlbrechtA.-G.OnesD. S.SinangilH. K. (2018). Expatriate management. In OnesD. S.AndersonN.ViswesvaranC.SinangilH. K. (Eds.), The SAGE handbook of industrial, work and organizational psychology (2nd ed., Vol. 3, pp. 429–465). London, UK: Sage. https://doi.org/10/c4xf
3.
AlexanderR. A. (1990). Correction formulas for correlations restricted by selection on an unmeasured variable. Journal of Educational Measurement, 27(2), 187–189. https://doi.org/10/cjrr9c
4.
AlexanderR. A.CarsonK. P.AlligerG. M.CarrL. (1987). Correcting doubly truncated correlations: An improved approximation for correcting the bivariate normal correlation when truncation has occurred on both variables. Educational and Psychological Measurement, 47(2), 309–315. https://doi.org/10/bc6dzj
5.
BeattyA. S.BarrattC. L.BerryC. M.SackettP. R. (2014). Testing the generalizability of indirect range restriction corrections. Journal of Applied Psychology, 99(4), 587–598. https://doi.org/10.1037/a0036361
6.
BlackJ. S.GregersenH. B. (1991). The other half of the picture: Antecedents of spouse cross-cultural adjustment. Journal of International Business Studies, 22(3), 461–477. https://doi.org/10/b2nbxx
7.
BryantN. D.GokhaleS. (1972). Correcting correlations for restrictions in range due to selection on an unmeasured variable. Educational and Psychological Measurement, 32(2), 305–310. https://doi.org/10/fc4533
8.
ColquittJ. A.ScottB. A.RodellJ. B.LongD. M.ZapataC. P.ConlonD. E.WessonM. J. (2013). Justice at the millennium, a decade later: A meta-analytic test of social exchange and affect-based perspectives. Journal of Applied Psychology, 98(2), 199–236. https://doi.org/10/q6c
9.
DahlkeJ. A.WiernikB. M. (2018). psychmeta: An R package for psychometric meta-analysis. Applied Psychological Measurement, 43(5), 415–416. https://doi.org/10/gfgt9t
10.
DahlkeJ. A.WiernikB. M. (2019). psychmeta: Psychometric meta-analysis toolkit [R Package]. Retrieved fromhttps://cran.r-project.org/package=psychmeta(Original work published 2017)
11.
DeRueD. S.NahrgangJ. D.WellmanN. E. D.HumphreyS. E. (2011). Trait and behavioral theories of leadership: An integration and meta-analytic test of their relative validity. Personnel Psychology, 64(1), 7–52. https://doi.org/10/fwzt2t
12.
ElwertF.WinshipC. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40(1), 31–53. https://doi.org/10/gdnb59
13.
HarariM. B.ReavesA. C.BeaneD. A.LaginessA. J.ViswesvaranC. (2018). Personality and expatriate adjustment: A meta-analysis. Journal of Occupational and Organizational Psychology, 91(3), 486–517. https://doi.org/10/gdnb6c
14.
HeckmanJ. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161. https://doi.org/10/c62z76
15.
HumphreyS. E.NahrgangJ. D.MorgesonF. P. (2007). Integrating motivational, social, and contextual work design features: A meta-analytic summary and theoretical extension of the work design literature. Journal of Applied Psychology, 92(5), 1332–1356. https://doi.org/10/dch52t
16.
HunterJ. E.SchmidtF. L.LeH. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91(3), 594–612. https://doi.org/10/bt4t68
17.
JonesJ. A.WallerN. G. (2013). Computing confidence intervals for standardized regression coefficients. Psychological Methods, 18(4), 435–453. https://doi.org/10/gckfx4
18.
KostalJ. W.WiernikB. M.AlbrechtA.-G.OnesD. S. (2018). Expatriate personality: Facet-level comparisons with domestic counterparts. In WiernikB. M.RügerH.OnesD. S. (Eds.), Managing expatriates: Success factors in private and public domains (pp. 33–52). Opladen, Germany: Budrich. https://doi.org/10/c4xg
19.
LawleyD. N. (1943). A note on Karl Pearson’s selection formulae. Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences, 62(1), 28–30. https://doi.org/10/ckc2
20.
LeH.OhI.-S.SchmidtF. L.WooldridgeC. D. (2016). Correction for range restriction in meta-analysis revisited: Improvements and implications for organizational research. Personnel Psychology, 69(4), 975–1008. https://doi.org/10/gckf2p
21.
LeH.SchmidtF. L. (2006). Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic procedure. Psychological Methods, 11(4), 416–438. https://doi.org/10/fv7g5s
22.
MeyerS. L. (1975). Data analysis for scientists and engineers. New York, NY: John Wiley.
23.
MurrayA. L.JohnsonW.McGueM.IaconoW. G. (2014). How are conscientiousness and cognitive ability related to one another? A re-examination of the intelligence compensation hypothesis. Personality and Individual Differences, 70, 17–22. https://doi.org/10/f6gvt6
24.
OehlertG. W. (1992). A note on the delta method. American Statistician, 46(1), 27–29. https://doi.org/10/d27qpx
25.
OnesD. S.SinangilH. K.WiernikB. M. (2018). Validity of Big Five personality traits for expatriate success: Results from Turkey. In WiernikB. M.RügerH.OnesD. S. (Eds.), Managing expatriates: Success factors in private and public domains (pp. 83–102). Opladen, Germany: Budrich. https://doi.org/10/c4xh
26.
OnesD. S.ViswesvaranC. (2003). Job-specific applicant pools and national norms for personality scales: Implications for range-restriction corrections in validation research. Journal of Applied Psychology, 88(3), 570–577. https://doi.org/10/c2rvxw
27.
OswaldF. L.McCloyR. A. (2003). Meta-analysis and the art of the average. In MurphyK. R. (Ed.), Validity generalization: A critical review (pp. 311–338). Mahwah, NJ: Lawrence Erlbaum. https://doi.org/10/ckc3
28.
RajuN. S.BurkeM. J. (1983). Two new procedures for studying validity generalization. Journal of Applied Psychology, 68(3), 382–395. https://doi.org/10/fq5thj
29.
RajuN. S.BurkeM. J.NormandJ.LangloisG. M. (1991). A new meta-analytic approach. Journal of Applied Psychology, 76(3), 432–446. https://doi.org/10/dcrgkf
30.
RohrerJ. M. (2018). Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in Methods and Practices in Psychological Science, 1(1), 27–42. https://doi.org/10/gcvj3r
SackettP. R.LievensF.BerryC. M.LandersR. N. (2007). A cautionary note on the effects of range restriction on predictor intercorrelations. Journal of Applied Psychology, 92(2), 538–544. https://doi.org/10/ch6qk2
33.
SackettP. R.OstgaardD. J. (1994). Job-specific applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research. Journal of Applied Psychology, 79(5), 680–684. https://doi.org/10/fkpnv2
34.
SackettP. R.YangH. (2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85(1), 112–118. https://doi.org/10/c6npmd
35.
SchmidtF. L.HunterJ. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage. https://doi.org/10/b6mg
36.
SteinS. K.BarcellosA. (1992). Calculus and analytic geometry (5th ed.). New York, NY: McGraw-Hill.
37.
WiernikB. M.DahlkeJ. A. (in press). Obtaining unbiased results in meta-analysis: The importance of correcting for statistical artefacts. Advances in Methods and Practices in Psychological Science.
38.
WiernikB. M.KostalJ. W. (2019). Protean and boundaryless career orientations: A critical review and meta-analysis. Journal of Counseling Psychology, 66(3), 280–307. https://doi.org/10/gfxrgp
39.
WiernikB. M.KostalJ. W.WilmotM. P.DilchertS.OnesD. S. (2017). Empirical benchmarks for interpreting effect size variability in meta-analysis. Industrial and Organizational Psychology, 10(3), 472–479. https://doi.org/10/ccnv
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
