Equating is a family of statistical models and methods used to adjust scores on different test forms so that they can be comparable and used interchangeably. Equated scores are obtained estimating the equating transformation function, which maps the scores on the scale of one test form into their equivalents on the scale of other one. All the statistical models that have been proposed for estimating this function are based on continuous approximations of the score distributions, leading to equated scores lying on a continuous scale even though score scales are usually subsets of the integer numbers (e.g., the total number of correct answers). In this article, we develop a new equating method from which equated scores defined on the original discrete scale are obtained. Considering scores as ordinal random variables, we propose a continuous latent variable formulation to perform an equipercentile-like equating based on a Bayesian nonparametric model for score distributions. The proposed model is applied to simulated and real data collected under an equivalent group design. Some methods to assess the performance of our model are also discussed. Compared with discrete versions of equated scored obtained from traditional equating methods, the results show that the proposed method has better performance.
AlbertJ.ChibS. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88(422), 669–679.
2.
AngoffW. H. (1984). Scales, norms and equivalent scores. In ThorndikeR. L. (Ed.), Educational measurement (2nd ed., pp. 508–600). American Council on Education. (Original work published 1971)
3.
BlackwellD.MacQueenJ. B. (1973). Ferguson distributions via Pólya urn schemes. The Annals of Statistics, 1(2), 353–355.
4.
BraunH.HollandP. (1982). Observed-score test equating: A mathematical analysis of some ETS equating procedures. In HollandP.RubinD. (Eds.), Test equating (Vol. 1, pp. 9–49). Academic Press.
5.
BrooksS. P.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455.
6.
ChenM.-H.DeyD. K. (2000). Bayesian analysis for correlated ordinal data models. In DeyD. K.GhoshS.MallickB. K. (Eds.), Generalized linear models: A Bayesian perspective (pp. 135–162). Marcel Dekker.
7.
CullenB.O’NeillB.EvansJ.CoenR.LawlorB. (2007). A review of screening tests for cognitive impairment. Journal of Neurology, Neurosurgery, and Psychiatry, 78(8), 790–799.
8.
De IorioM.MüllerP.RosnerG.MacEachernS. (2004). An ANOVA model for dependent random measures. Journal of the American Statistical Association, 99(465), 205–215.
9.
DeyD.MuellerP.SinhaD. (1998). Practical nonparametric and semiparametric Bayesian statistics. Springer.
10.
FergusonT. S. (1973). A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1, 209–230.
11.
FergusonT. S. (1983). Bayesian density estimation by mixtures of normal distributions. In SiegmundD.RustageJ.RizviG. G. (Eds.), Recent advances in statistics: Papers in Honor of Herman Chernoff on his sixtieth birthday (pp. 287–302). Bibliohound.
12.
FriedE. I. (2016). Are more responsive depression scales really superior depression scales?Journal of Clinical Epidemiology, 77, 4–6.
13.
GelmanA.RubinD. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457–511.
14.
GhosalS. (2010). The Dirichlet process, related priors and posterior asymptotics. In HjortN.HolmesC.MüllerP.WalkerS. G. (Eds.), Bayesian nonparametrics (pp. 22–34). Cambridge University Press.
15.
GhoshS. K.BurnsC. B.PragerD. L.ZhangL.HuiG. (2018). On nonparametric estimation of the latent distribution for ordinal data. Computational Statistics and Data Analysis, 119, 86–98.
16.
GonzálezJ. (2014). SNSequate: Standard and nonstandard statistical models and methods for test equating. Journal of Statistical Software, 59(7), 1–30.
17.
GonzálezJ.BarrientosA. F.QuintanaF. A. (2015a). A dependent Bayesian nonparametric model for test equating. In MillsapR.BoltD.van der ArkL.WangW.-C. (Eds.), Quantitative psychology research (pp. 213–226). Springer.
18.
GonzálezJ.BarrientosA. F.QuintanaF. A. (2015b). Bayesian nonparametric estimation of test equating functions with covariates. Computational Statistics & Data Analysis, 89, 222–244.
19.
GonzálezJ.von DavierM. (2013). Statistical models and inference for the true equating transformation in the context of local equating. Journal of Educational Measurement, 50(3), 315–320.
20.
GonzálezJ.WibergM. (2017). Applying test equating methods using R. Springer.
21.
HaberM. (1985). Maximum likelihood methods for linear and log-linear models in categorical data. Computational Statistics and Data Analysis, 3, 1–10.
22.
HansonT.JohnsonW. (2002). Modeling regression error with a mixture of Polya trees. Journal of the American Statistical Association, 97(460), 1020–1033.
23.
HjortN. L.HolmesC.MüllerP.WalkerS. (2010). Bayesian nonparametrics. Cambridge University Press.
24.
HollandP.ThayerD. (1989). The kernel method of equating score distributions(Technical report). Educational Testing Service.
25.
IshwaranH.JamesL. F. (2001). Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96(453), 161–173.
26.
JohnsonV.AlbertJ. (1999). Ordinal data modeling (1st ed.). Springer.
27.
KarabatsosG.WalkerS. (2009a). A Bayesian nonparametric approach to test equating. Psychometrika, 74(2), 211–232.
28.
KarabatsosG.WalkerS. (2009b). Coherent psychometric modelling with Bayesian nonparametrics. British Journal of Mathematical and Statistical Psychology, 62(1), 1–20.
29.
KarabatsosG.WalkerS. (2011). A Bayesian nonparametric model for test equating. In vonDavierA. (Ed.), Statistical models for test equating, scaling, and linking (Vol. 1, pp. 175–184). Springer.
30.
KassR. E.CarlinB. P.GelmanA.NealR. M. (1998). Markov chain Monte Carlo in practice: A roundtable discussion. The American Statistician, 52(2), 93–100.
31.
KolenM.BrennanR. (2014). Test equating, scaling, and linking: Methods and practices (3rd ed.). Springer.
32.
KottasA.MullerP.QuintanaF. (2005). Nonparametric Bayesian modeling for multivariate ordinal data. Journal of Computational and Graphics Statistics, 14(3), 610–625.
33.
LavineM. (1992). Some aspects of Polya tree distributions for statistical modelling. The Annals of Statistics, 20(3), 1222–1235.
34.
LijoiA.PrünsterI. (2010). Models beyond the Dirichlet process. In HjortN.HolmesC.MüllerP.WalkerS. G. (Eds.), Bayesian nonparametrics (pp. 80–136). Cambridge University Press.
35.
LoA. Y. (1984). On a class of Bayesian nonparametric estimates I: Density estimates. The Annals of Statistics, 12, 351–357.
36.
LordF. (1982). The standard error of equipercentile equating. Journal of Educational and Behavioral Statistics, 7(3), 165.
37.
MacEachernS. N. (1999). Dependent nonparametric processes. In ASA proceedings of the section on Bayesian statistical science (pp. 50–55).
38.
MacEachernS. N. (2000). Dependent Dirichlet processes(Technical report). Department of Statistics, The Ohio State University.
39.
MauldinR.SudderthW.WilliamsS. (1992). Polya trees and random distributions. The Annals of Statistics, 20(3), 1203–1221.
40.
McCullaghP. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society, Series B (Methodology), 42(2), 109–142.
41.
McCullaghP.NelderJ. (1989). Generalized linear models (2nd ed.). Chapman & Hall.
42.
MitraR.MüllerP. (2015). Nonparametric Bayesian inference in biostatistics. Springer.
43.
PitmanJ. (1996). Some developments of the Blackwell-MacQueen urn scheme, Volume 30 of Lecture Notes–Monograph Series (pp. 245–267). Institute of Mathematical Statistics.
44.
PitmanJ.YorM. (1997). The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. The Annals of Probability, 25(2), 855–900.
45.
R Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing.
46.
SantorD.GregusM.WelchA. (2009). Eight decades of measurement in depression. Measurement, 4, 135–155.
47.
SethuramanJ. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4, 639–650.
48.
ShahD. A.MaddenL. V. (2004). Nonparametric analysis of ordinal data in designed factorial experiments. Phytopathology, 91(1), 33–43.
49.
von DavierA. A.HollandP.ThayerD. (2004). The kernel method of test equating. Springer.
50.
WinshipC.MareR. D. (1984). Regression models with ordinal variables. American Sociological Review, 49(4), 512–525.
51.
ZegerS. L.KarimM. R. (1991). Generalized linear models with random effects: A Gibbs sampling approach. Journal of the American Statistical Association, 86(413), 79–86.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
