Alternatives to the standard statewide assessment often are necessary for valid measurement of students with significant disabilities. These alternate assessments must be carefully developed and evaluated with respect to generally accepted psychometric standards. Ideally, these measures should be sensitive to growth and scaled to state tests that are aligned with grade level content standards. In this study, we investigated the technical adequacy of early reading performance tasks for a Grade 3 statewide alternate reading test. Our research hypotheses pertain to (a) equitable measurement function across populations of students, (b) measurement dimensionality, and (c) item difficulty. Using test results obtained from annual testing in one state, our findings suggest that performance tasks are adequate for use in the alternate assessment, and they can be scaled in terms of the statewide testing metric.
Get full access to this article
View all access options for this article.
References
1.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
BennettR. E. (1993). On the meanings of constructed response. In BennettR. E.WardW. C. (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 1–28). Hillsdale, NJ: Lawrence Erlbaum.
3.
BennettR. E.WardW. C. (1993). Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment. Hillsdale, NJ: Lawrence Erlbaum.
4.
BerkR. A. (1986). A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56, 137–172.
5.
BoomsmaA.van DuijnM. A. J.SnijdersT. A. B. Eds. (2001). Essays on item response theory. New York: Springer.
6.
BrowderD.FlowersC.Ahlgrim-DelzellL.KarvonenM.SpoonerF.AlgozzineR. (2004). The alignment of alternate assessment content with academic and functional curricula. The Journal of Special Education, 37, 211–223.
7.
CizekG. J. (Ed.). (2001). Setting performance standards: Concepts, methods, and perspectives. Mahwah, NJ: Lawrence Erlbaum.
8.
EmbretsonS. E.ReiseS. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
9.
FordA.DavernL.SchnorrR. (2001). Learners with significant disabilities. Remedial and Special Education, 22, 214–222.
GierlM. J. (2005). Using dimensionality-based DIF analyses to identify and interpret constructs that elicit group differences. Educational Measurement Issues and Practice, 24, 3–14.
12.
HasbrouckJ.TindalG. (1992). Curriculum-based oral reading fluency norms for students in grades 2–5. TEACHING Exceptional Children, 24, 41–44.
13.
HintzJ. M.OwenS. V.ShapiroE. S.DalyE. J. (2000). Generalizability of oral reading fluency measures: Application of G theory to curriculum-based measurement. School Psychology Quarterly, 15 (1), 52–68.
14.
HintzJ. M.PetitteH. A. P. (2001). The generalizability of CBM oral reading fluency measures across general and special education. Journal of Psychoeducational Assessment, 19, 158–170.
15.
HollandP.WainerH. (Eds.). (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum.
16.
HollandP. W.ThayerD. T. (1988). Differential item performance and the Mantel-Hanzel procedure. In WainerH.BrauH. (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Lawrence Erlbaum.
17.
KampferS.HorvathL.KleinhertH.KearnsJ. (2001). Teachers' perceptions of one state's alternate assessment: Implications for practice and preparation. Exceptional Children, 67, 361–374.
18.
KaneM.CrooksT.CohenA. (1999). Validating measures of performance. Educational Measurement: Issues and Practice, 18, 5–17.
19.
KimS.CohenA. S. (1998). A comparison of linking and concurrent calibration under item response theory. Applied Psychological Measurement, 22, 131–143.
20.
KleinhertH.KearnsJ.KennedyS. (1997). Accountability for all students: Kentucky's alternate portfolio assessment for students with moderate and severe cognitive disabilities. Journal of the Association for Persons With Severe Handicaps, 24, 88–101.
21.
KleinhertH.KearnsJ. F. (1999). A validation study of the performance indicators and learner outcomes of Kentucky's alternate assessment for students with significant disabilities. Journal of the Association for Persons With Severe Handicaps, 24, 100–110.
22.
KleinhertH.KennedyS.KearnsJ. (1999). The impact of alternate assessments: A statewide teacher survey. Journal of Special Education, 33, 93–102.
23.
KolenM. J. (1981). Comparison of traditional and item response methods for equating tests. Journal of Educational Measurement, 19, 1–11.
24.
KolenM. J.BrennanR. L. (1995). Test equating: Methods and practices. New York: Springer-Verlag.
25.
KoretzD. M.BertenthalM. W.GreenB. F. (Eds.). (1999). Embedding questions. The pursuit of common measure in uncommon tests. Washington, DC: National Academy Press.
26.
LevineM. V.DrasgowF. (1983). Appropriateness measurement: Validating studies and variable ability models. In WeissD. J. (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 109–131). New York: Academic Press.
27.
LiM. F.OlejnikS. (1977). The power of the Rasch person-fit statistics in detecting unusual response patterns. Applied Psychological Measurement, 21, 215–231.
LinnR. L.BakerE. L.DunbarS. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20 (8), 15–21.
30.
LittleR. J. A.SchenkerN. (1995). Missing data. In ArmingerG.CloggC. C.SobelM. E. (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 339–369). New York: Plenum.
31.
LivingstonA.ZiekyM. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
32.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
33.
MacCallumR. C.ZhangS.PreacherK. J.RuckerD. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19–40.
34.
MastersG.WrightB. D. (1984). The essential process in a family of measurement models. Psychometrika, 49, 529–544.
35.
MeijerR. R.SijtsmaK. (2001). Methodological review: Evaluating person fit. Applied Psychological Measurement, 25, 107–135.
36.
MessickS. (1996). Validity of performance assessment. In PhillipsG. W. (Ed.), Technical issues in large-scale performance assessment (pp. 1–18). Washington, DC: National Center for Education Statistics.
37.
MurakiE.HomboC. M.LeeYong-Won. (2000). Equating and linking of performance assessments. Applied Psychological Measurement, 24, 325–337.
38.
NandakumarR. (1993). Assessing essential unidimensionality for real data. Applied Psychological Measurement, 17, 29–38.
39.
RaschG. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danmarks Paedagogiske Institut.
40.
RobinsonS. P. (1993). The politics of multiple-choice versus free-response assessment. In BennettR. E.WardW. C. (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 313–324). Hillsdale, NJ: Lawrence Erlbaum.
41.
RosaK.SwygertK. A.NelsonL.ThissenD. (2001). Item response theory applied to combinations of multiple-choice and constructed-response items—scale scores for patterns of summed scores. In ThissenD.WainerH. (Eds.), Test scoring (pp. 253–292). Hillsdale, NJ: Lawrence Erlbaum.
42.
SmithR. M. (2000). Fit analysis in latent trait measurement models. Journal of Applied Measurement, 1, 199–218.
43.
StockingM. L.LordF. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201–210.
44.
ThissenD.NelsonL.RosaK.McLeodL. D. (2001). Item response theory for items scored in more than two categories. In ThissenD.WainerH. (Eds.), Test scoring (pp. 141–186). Hillsdale, NJ: Lawrence Erlbaum.
45.
ThissenD.WainerH. (Eds.). (2001). Test scoring. Hillsdale, NJ: Lawrence Erlbaum.
46.
ThissenD.WainerH.WangX. (1994). Are tests comprising both multiple-choice and free-response items necessarily less unidimensional than multiple-choice tests? An analysis of two tests. Journal of Educational Measurement, 31, 113–123.
47.
TindalG. (2005). Alignment of alternate assessments using the Webb system. Washington, DC: Council of Chief State School Officers.
48.
TindalG.GlasgowA.GallJ.VanLooD.ChowE. (2002). Oregon Extended Assessment: Technical adequacy analysis summary. Salem: Oregon Department of Education.
49.
TindalG.McDonaldM.TedescoM.GlasgowA.AlmondP.CrawfordL. (2003). Alternate assessments in reading and math: Development and validation for students with significant disabilities. Exceptional Children, 69, 481–494.
50.
TindalJ.HaladynaT. M. (Eds.). (2002). Large-scale assessment programs for all students. Mahwah, NJ: Lawrence Erlbaum.
51.
TownsendR.ZhangL.VestermanB.WiseL.TindalG.WinterP. (2004, June). Meeting alignment challenges: A computer-based alignment analysis tool with models for analyzing vertical alignment and alternate assessments. Symposium conducted at the meeting of the National Conference on Large-Scale Assessment, Boston, MA.
52.
TraubR. E.FisherC. W. (1977). On the equivalence of constructed-response and multiple choice tests. Applied Psychological Measurement, 1 (3), 355–369.
53.
TurnerM.BaldwinL.KleinhertH.KearnsJ. (2000). The relation of a statewide alternate assessment for students with severe disabilities to other measures of instructional effectiveness. Journal of Special Education, 34, 69–76.
54.
U.S. Department of Education. Title I—Improving the Academic Achievement of the Disadvantaged; Final Rule. Fed. Reg. 68,236 (Dec. 9, 2003).
55.
U.S. Department of Education. Title I—Improving the Academic Achievement of the Disadvantaged: Final Rule. 68 Fed. Reg. 68,699 (Dec. 9, 2003).
56.
U.S. National Reading Panel. (2000). Teaching children to read. Washington, DC: National Institute of Child Health and Human Development, National Institutes of Health.
57.
ValeD. C. (1986). Linking item parameters onto a common scale. Applied Psychological Measurement, 10, 333–344.
58.
van der LindenW. J.HambletonR. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.
59.
WebbN. L. (1999). Alignment of science and mathematics standards and assessments in four states. Washington, DC: Council of Chief State School Officers.
60.
WilsonD. T.WoodR.GibbonsR. (2003). TESTFACT: Test scoring, item statistics, and item factor analysis (Version 4.0.2) [Computer software]. Chicago: Scientific Software International.
61.
YsseldykeJ. E.OlsenK. R. (1997). Putting alternate assessments into practice: What to measure and possible sources of data. Exceptional Children, 65, 175–185.
