Reliability and validity are integral concepts in assessment design. Test speededness, the influence of time constraints on test taker performance, is often an overlooked threat to reliability and validity, especially in classroom-based testing. The purpose of this study is to evaluate the degree of test speededness of classroom-based assessments that have been developed and tested for a curricular reading intervention study with third-grade gifted students. The degree of speededness for the assessments was calculated using a mixture Rasch model. The results indicate that a large proportion of students were influenced by test speededness for both posttests in both treatment groups. The implications of these results are discussed.
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
2.
BaxterB. (1941). An experimental analysis of speed and level in an intelligence test. Journal of Educational Psychology, 32, 285-296.
3.
BejarI. I. (1985). Test speededness under number-right scoring: An analysis of the Test of English As a Foreign Language (Report No. ETS-RR-85-11). Princeton, NJ: Educational Testing Service.
4.
BoltD.CohenA.WollackJ. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints. Journal of Educational Measurement, 39, 331-348.
5.
CamilliG.ShepardL. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
6.
CrockerL.AlginaJ. (1986). Factors that affect reliability coefficients. In CrockerL.AlginaJ. (Eds.), Introduction to classical and modern test theory (pp. 143-146). Fort Worth, TX: Harcourt Brace Jovanovich.
7.
DavidsonW. M.CarrollJ. B. (1945). Speed and level components in time limit scores: A factor analysis. Educational and Psychological Measurement, 5, 411-427.
8.
DouglasJ.KimH. R.HabingB.GaoF. (1998). Investigating local dependence with conditional covariance functions. Journal of Educational and Behavioral Statistics, 23, 129-151.
9.
EmbretsonS.ReiseS. (2009). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
10.
EvansF. R.ReillyR. R. (1972). A study of speededness as a source of test bias. Journal of Educational Measurement, 9, 123-131.
11.
FurrR. M.BacharachV. R. (2008). Psychometrics: An introduction. Thousand Oaks, CA: Sage.
12.
GreenS. B.HershbergerS. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7, 251-270.
13.
GulliksenH. (1950). Theory of mental tests. Hillsdale, NJ: Lawrence Erlbaum.
HendersonW. (2004). The LSAT, law school exams, and meritocracy: The surprising and undertheorized role of test-taking speed. Texas Law Review, 82, 975-1052.
16.
KaplanS. A. (2001). Layering differentiated curriculum for the gifted and talented. In KarnesF.BeanS. (Eds.), Methods and materials for teaching the gifted. Waco, TX: Prurock Press.
17.
KurpiusS.StaffordM. (2006). Testing and measurement: A user-friendly guide. Thousand Oaks, CA: Sage.
18.
LawrenceI. M. (1993). The effect of test speededness on subgroup performance (Report no. ETS-RR-93–49). Princeton, NJ: Educational Testing Service.
19.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
20.
LuY.SireciS. (2007). Validity issues in test speededness. Educational Measurement: Issues and Practice, 26(4), 29-37.
21.
MeyerJ. P. (2010). Understanding measurement: Reliability. New York, NY: Oxford University Press.
MrochA. A.BoltD. M.WollackJ. A. (2005, April). A new multi-class mixture Rasch model for test speededness. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec.
24.
No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107–110. § 115, Stat. 1425. (2002).
25.
OshimaT. C. (1994). The effect of speededness on parameter estimation in item response theory. Journal of Educational Measurement, 21, 200-219.
26.
RenzulliJ. S.ReisS. M. (1985). The schoolwide enrichment model: A comprehensive plan for educational excellence. Mansfield Center, CT: Creative Learning Press.
27.
SchnipkeD. L. (1996). How contaminated by guessing are item-parameter estimates and what can be done about it? Paper presented at the annual meeting of the National Council on Measurement in Education, New York, NY.
ThorndikeR.Thorndike-ChristT. (2009). Measurement and evaluation in psychology and education. Englewood Cliffs, NJ: Prentice Hall.
30.
TomlinsonC. A. (2001). How to differentiate instruction in mixed-ability classrooms (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
31.
van der LindenW. J. (2011). Setting time limits on tests. Applied Psychological Measurement, 35, 183-199.
32.
van der LindenW. J.ScramsD. J.SchnipkeD. L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195-210.
33.
WollackJ. A.CohenA. S.WellsC. S. (2003). A method for maintaining scale stability in the presence of test speededness. Journal of Educational Measurement, 40, 307-330.
34.
ZeniskyA.HambletonR.SireciS. (2002). Identification and evaluation of local item dependencies in the medical college admissions test. Journal of Educational Measurement, 39, 291-309.
