Invariant relationships in the internal mechanisms of estimating achievement scores on educational tests serve as the basis for concluding that a particular test is fair with respect to statistical bias concerns. Equating invariance and differential item functioning are both concerned with invariant relationships yet are treated separately in the psychometric literature. Connecting these two facets of statistical invariance is critical for developing a holistic definition of fairness in educational measurement, for fostering a deeper understanding of the nature and causes of equating invariance and a lack thereof, and for providing practitioners with guidelines for addressing reported score-level equity concerns. This study hypothesizes that differential item functioning manifested in anchor items of an assessment will have an effect on equating dependence. Findings show that when anchor item differential item functioning varies across forms in a differential manner across subpopulations, population invariance of equating can be compromised.
AckermanT. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.
2.
ACT. (2010). QualityCore®: Technical manual. Iowa City, IA: Author.
3.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
4.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2011). Draft standards for educational and psychological testing. Washington, DC: American Psychological Association.
5.
AngoffW. H. (1971). Scales, norms, and equivalent scores. In ThorndikeR. L. (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.
6.
AngoffW. H.CowellW. R. (1985). An examination of the assumption that the equating of parallel forms is population independent (Report No. 85-22). Princeton, NJ: Educational Testing Service.
7.
BabcockB.AlbanoA. D. (2012). Rasch scale stability in the presence of item parameter and trait drift. Applied Psychological Measurement, 36, 565-580.
8.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397-460). Reading, MA: Addison-Wesley.
9.
BockR.MurakiE.PfeiffenbergerW. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275-285.
10.
BroerM.LeeY.RizaviS.PowersD. (2005). Ensuring the fairness of GRE writing prompts: Assessing differential difficulty (ETS RR-05-11). Princeton, NJ: Educational Testing Service.
11.
CamilliG.ShepardL. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
12.
CookL. L.PetersenN. S. (1987). Problems related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225-244.
13.
DeMarsC. E. (2004). Detection of item parameter drift over multiple test administrations. Applied Measurement in Education, 17, 265-300.
14.
DonoghueJ. R.IshamS. P. (1998). A comparison of procedures to detect item parameter drift. Applied Psychological Measurement, 22, 33-51.
15.
DoransN. J. (2004). Using subpopulation invariance to assess test score equity. Journal of Educational Measurement, 41, 43-68.
16.
DoransN. J. (2008, March). Three facets of fairness. Paper presented at the annual meeting of the National Council on Measurement in Education, New York, NY.
17.
DoransN. J.FeigenbaumM. D. (1994). Equating issues engendered by changes to the SAT and PSAT/NMSQT. In LawrenceI. M.DoransN. J.FeigenbaumM. D.FeryokN. J.SchmittA. P.WrightN. K. (Eds.), Technical issues related to the introduction of the new SAT and PSAT/NMSQT (pp. 91-122) (ETS Research Memorandum No. RM-94-10). Princeton, NJ: Educational Testing Service.
18.
DoransN. J.HollandP. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
19.
DoransN. J.HollandP. W. (2000). Population invariance and the equatability of tests: Basic theory and the linear case. Journal of Educational Measurement, 37, 281-306.
20.
DrasgowF. (1987). Study of the measurement bias of two standardized psychological tests. Journal of Applied Psychology, 72, 19-29.
21.
FlanaganJ. C. (1951). Units, scores, and norms. In LindquistE. F. (Ed.), Educational measurement (pp. 695-793). Washington, DC: American Council on Education.
22.
HaebaraT. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149.
23.
HanK. T. (2008). Impact of item parameter drift on test equating and proficiency estimates (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (UMI No. 3325324)
24.
HollandP. W. (2007). A framework and history for score linking. In DoransN. J.PommerichM.HollandP. W. (Eds.), Linking and aligning scores and scales (pp. 5-30). New York, NY: Springer.
25.
HollandP. W.DoransN. J. (2006). Linking and equating. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 187-220). Westport, CT: American Council on Education & Praeger.
26.
HugginsA. C.PenfieldR. D. (2012). An NCME instructional module on population invariance in linking and equating. Educational Measurement: Issues and Practices, 31, 27-40.
27.
Joint Committee on Testing Practices. (2004). Code of fair testing practices in education. Washington, DC: Author.
28.
KleinA. (2011, November2). Clock ticks on senate bill to overhaul NCLB. Education Week, pp. 18-19.
29.
KleinA. (2012, January11). House ESEA draft would reduce federal school role. Education Week, pp. 16-17.
KolenM. J. (2004). Population invariance in equating and linking: Concept and history. Journal of Educational Measurement, 41, 3-14.
32.
KolenM. J.BrennanR. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd ed.). New York, NY: Springer.
33.
LiX. (2008). An investigation of the item parameter drift in the examination for the certificate of proficiency in English (ECPE). Spaan Fellow Working Papers in Second or Foreign Language Assessment, 6, 1-28.
34.
LoydB. H.HooverH. D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179-193.
35.
MarcoG. L. (1977). Item characteristic curve solutions to three intractable test problems. Journal of Educational Measurement, 14, 139-160.
36.
McNeilM. (2011, December7). Waiver plans would scrap parts of NCLB. Education Week, pp. 1, 28-29.
37.
NandakumarR. (1993). Amplification and cancellation of DIF. Journal of Educational Measurement, 30, 293-311.
38.
No Child Left Behind: State Plans, 20 U.S.C. § 1111 (2001).
39.
OshimaT. C.MorrisS. B. (2008). Raju’s differential functioning of items and tests (DFIT). Educational Measurement: Issues and Practices, 27, 43-50.
PenfieldR. D.CamilliG. (2007). Differential item functioning and item bias. In SinharayS.RaoC. R. (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 125-167). New York, NY: Elsevier.
42.
R Development Core Team. (2005). R: A language and environment for statistical computing, reference index version 2.2.1. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org
43.
RajuN. S.van der LindenW. J.FleerP. F. (1995). IRT-based internal measures of differential functioning of items and tests. Applied Psychological Measurement, 19, 353-368.
44.
StockingM. L.LordF.M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
45.
van der LindenW. J. (2000). A test-theoretic approach to observed-score equating. Psychometrika, 65, 437-456.
46.
von DavierA. A.WilsonC. (2007). IRT true-score test equating: A guide through assumptions and applications. Educational and Psychological Measurement, 67, 940-957.
47.
von DavierA. A.WilsonC. (2008). Investigating the population sensitivity assumption of item response theory true-score equating across two subgroups of examinees and two test formats. Applied Psychological Measurement, 32, 11-26.
48.
WellsC. S.SubkoviakM. J.SerlinR. C. (2002). The effect of item parameter drift on examinee ability estimates. Applied Psychological Measurement, 26, 77-87.
49.
YangW. L. (2004). Sensitivity of linkings between AP multiple-choice scores and composite scores to geographical region: An illustration of checking for population invariance. Journal of Educational Measurement, 41, 33-41.
50.
YangW. L.GaoR. (2008). Invariance of score linkings across gender groups for forms of a testlet-based college-level examination program examination. Applied Psychological Measurement, 32, 45-61.
51.
YiQ.HarrisD. J.GaoX. (2008). Invariance of equating functions across different subgroups of examinees taking a science achievement test. Applied Psychological Measurement, 32, 62-80.
52.
ZimowskiM. F.MurakiE.MislevyR. J.BockR.D. (2003). BILOG-MG 3 for Windows: Multiple-group IRT analysis and test maintenance for binary items [Computer software]. Lincolnwood, IL: Scientific Software International.
