The item-position effect describes how an item’s position within a test, that is, the number of previous completed items, affects the response to this item. Previously, this effect was represented by constraints reflecting simple courses, for example, a linear increase. Due to the inflexibility of these representations our aim was to examine whether adapted representations are more appropriate than the existing ones. Models of confirmatory factor analysis were used for testing the different representations. Analyses were conducted by means of simulated data that followed the covariance pattern of Raven’s Advanced Progressive Matrices (APM) items. Since the item-position effect has been demonstrated repeatedly for the APM, it is a very suitable measure for our investigations. Results revealed no remarkable improvement by using an adapted representation. Possible reasons causing these results are discussed.
BandalosD. L.GagnéP. (2012). Simulation methods in structural equation modeling. In HoyleR. H. (Ed.), Handbook of structural equation modeling (pp. 92-108). New York, NY: Guilford.
CarpenterP. A.JustM. A.ShellP. (1990). What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97, 404-431. doi:10.1037/0033-295X.97.3.404
4.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233-255. doi:10.1207/S15328007SEM0902_5
5.
DiStefanoC.MorganG. B. (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 21, 425-438. doi:10.1080/10705511.2014.915373
6.
EmbretsonS. E. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-515. doi:10.1007/BF02294487
7.
FinneyS. J.DiStefanoC. (2013). Nonnormal and categorical data in structural equation modeling. In HancockG. R.MuellerR. O. (Eds.), Structural equation modeling: A second course (2nd ed., pp. 439-492). Charlotte, NC: Information Age.
8.
GrahamJ. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930-944. doi:10.1177/0013164406288165
9.
HamiltonJ. C.ShuminskyT. R. (1990). Self-awareness mediates the relationship between serial position and item reliability. Journal of Personality and Social Psychology, 59, 1301-1307. doi:10.1037/0022-3514.59.6.1301
10.
HartigJ.BuchholzJ. (2012). A multilevel item response model for item position effects and individual persistence. Psychological Test and Assessment Modelling, 54, 418-431.
11.
HartigJ.HölzelB.MoosbruggerH. (2007). A confirmatory analysis of item reliability trends (CAIRT): Differentiating true score and error variance in the analysis of item context effects. Multivariate Behavioral Research, 42, 157-183.
12.
HohensinnC.KubingerK. D.ReifM.Holocher-ErtlS.KhorramdelL.FrebortM. (2008). Examining item-position effects in large-scale assessment using the linear logistic test model. Psychology Science Quarterly, 50, 391-402.
13.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. doi:10.1080/10705519909540118
KlineR. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, England: Guilford.
17.
KnowlesE. S. (1988). Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 55, 312-320. doi:10.1037/0022-3514.55.2.312
18.
KnowlesE. S.ByersB. (1996). Reliability shifts in measurement reactivity: Driven by content engagement or self-engagement?Journal of Personality and Social Psychology, 70, 1080-1090. doi:10.1037//0022-3514.70.5.1080
19.
KubingerK. D. (2008). On the revival of the Rasch model-based LLTM: From constructing tests using item generating rules to measuring item administration effects. Psychology Science Quarterly, 50, 311-327.
20.
LiC.-H. (2015). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods. Advance online publication. doi:10.3758/s13428-015-0619-7
21.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
22.
LozanoJ. H. (2015). Are impulsivity and intelligence truly related constructs? Evidence based on the fixed-links model. Personality and Individual Differences, 85, 192-198. doi:10.1016/j.paid.2015.04.049
23.
McCullaghP. J.NelderJ. A. (1984). Generalized linear Models (Monographs on Statistics and Applied Probability; Vol. 37). London, England: Chapman & Hall.
24.
McDonaldR. P.AhlawatK. S. (1974). Difficulty factors in binary data. British Journal of Mathematical and Statistical Psychology, 27, 82-99. doi:10.1111/j.2044-8317.1974. tb00530.x
25.
MollenkopfW. G. (1950). An experimental study of the effects on item-analysis data of changing item placement and test time limit. Psychometrika, 15, 291-315. doi:10.1007/BF02289044
26.
MuthénB. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132. doi:10.1007/BF02294210
27.
RaschG. (1980). Probabilistic models for some intelligence and attainment tests (Expended ed.). Chicago, IL: University of Chicago Press.
28.
RavenJ. C.RavenJ.CourtJ. H. (1997). Raven’s Progressive Matrices and Vocabulary Scales. Edinburgh, England: J. C. Raven.
29.
RenX.GoldhammerF.MoosbruggerH.SchweizerK. (2012). How does attention relate to the ability-specific and position-specific components of reasoning measured by APM?Learning and Individual Differences, 22(1), 1-7. doi:10.1016/j.lindif.2011.09.009
30.
RenX.WangT.AltmeyerM.SchweizerK. (2014). A learning-based account of fluid intelligence from the perspective of the position effect. Learning and Individual Differences, 31, 30-35. doi:10.1016/j.lindif.2014.01.002
31.
SchweizerK. (2011). Scaling variances of latent variables by standardizing loadings: Applications to working memory and the position effect. Multivariate Behavioral Research, 46, 938-955. doi:10.1080/00273171.2011.625312
32.
SchweizerK. (2012). The position effect in reasoning items considered from the CFA perspective. International Journal of Educational and Psychological Assessment, 11(2), 44-58.
33.
SchweizerK. (2013). A threshold-free approach to the study of the structure of binary data. International Journal of Statistics and Probability, 2(2). doi:10.5539/ijsp.v2n2p67
34.
SchweizerK.RenX.WangT. (2015). A comparison of confirmatory factor analysis of binary data on the basis of tetrachoric correlations and of probability-based covariances: A simulation study. In MillsapR. E.BoltD. M.van der ArkL. A.WangW.-C. (Eds.), Springer proceedings in mathematics & statistics. Quantitative psychology research (Vol. 89, pp. 273-292). Berlin, Germany: Springer.
35.
SchweizerK.SchreinerM.GoldA. (2009). The confirmatory investigation of APM items with loadings as a function of the position and easiness of items: A two-dimensional model of APM. Psychology Science Quarterly, 51(1), 47-64.
36.
SchweizerK.TrocheS. J.RammsayerT. H. (2011). On the special relationship between fluid and general intelligence: New evidence obtained by considering the position effect. Personality and Individual Differences, 50, 1249-1254. doi:10.1016/j.paid.2011.02.019
37.
VergutsT.De BoeckP. (2000). A Rasch model for detecting learning while solving an intelligence test. Applied Psychological Measurement, 24, 151-162. doi:10.1177/014662 10022031589
