State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily available for the research community. The purpose of this article is to demonstrate a validity argument for a large-scale alternate assessment, using one state program as a model. Specifically, the authors review the documented purposes of the assessment program and the assumptions related to teaching and learning subsumed within those purpose statements. Potential sources of evidence to support or refute those assumptions are also presented.
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1974). Standards for educational and psychological tests. Washington, DC: American Psychological Association.
3.
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
4.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
5.
BehuniakP.AmentaJ. (2009). Assessing all students in Connecticut. In SchaferW. M.LissitzR. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 157–170). Baltimore, MD: Brookes.
6.
BrowderD. M.FallinK.DavisS.KarvonenM. (2003). A consideration of what may influence student outcomes on alternate assessment. Education and Training in Developmental Disabilities, 38, 255–270.
7.
BrowderD. M.FlowersC.Ahlgrim-DelzellL.KarvonenM.SpoonerF.AlgozzineR. (2004). The alignment of alternate assessment content to academic and functional curricula. Journal of Special Education, 37, 211–223.
8.
BrowderD. M.SpoonerF.AlgozzineB.Ahlgrim-DelzellL.FlowersC.KarvonenM. (2003). What we know and need to know about alternate assessment. Exceptional Children, 70, 45–62.
9.
BrowderD.WakemanS.FlowersC. (2009). Which came first: The curriculum or the assessment? In SchaferW. M.LissitzR. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 329–334). Baltimore, MD: Brookes.
10.
ChapelleC. A.EnrightM. K.JamiesonJ. M. (2010). Does an argument-based approach to validity make a difference?Educational Measurement: Issues and Practice, 29(1), 3–13.
11.
CizekG. J.RosenbergS.KoonsH. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–412.
12.
Connecticut State Department of Education. (2006). CMT/CAPT Skills Checklist Second Generation: Technical manual. Hartford: Connecticut State Department of Education.
13.
CronbachL. J. (1988). Five perspectives on validity argument. In WainerH. (Ed.), Test validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.
14.
ElliottS. N.ComptonE.RoachA. T. (2007). Concurrent and predictive validity evidence for Idaho Alternate Assessment scores of students with and without significant disabilities. Educational Measurement: Issues and Practice, 26(2), 30–42.
15.
ElliottS. N.RoachA. T. (2007). Alternate assessments of students with significant disabilities: Alternative approaches, common technical challenges. Applied Measurement in Education, 20, 301–333.
16.
EricssonK. A.SimonH. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press.
17.
FlowersC.BrowderD.Ahlgrim-DelzellL. (2006). An analysis of three states’ alignment between language arts and mathematics standards and alternate assessments. Exceptional Children, 72, 201–215.
18.
GoldsteinJ.BehuniakP. (in press). Assessing students with significant cognitive disabilities on academic content: The teachers’ perspective. Journal of Special Education.
19.
HaertelE. (1999). Validity arguments for high-stakes testing: In search of the evidence. Educational Measurement: Issues and Practice, 18(4), 5–9.
20.
HaladynaT. M. (2006). Roles and importance of validity studies in test development. In DowningS. M.HaladynaT. M. (Eds.), Handbook of test development (pp. 739–758). Mahwah,NJ: Lawrence Erlbaum.
21.
HoganT. P.AgnelloJ. (2004). An empirical study of reporting practices concerning measurement validity. Educational and Psychological Measurement, 64, 802–812.
22.
Individuals with Disabilities Education Improvement Act, Pub. L. No. 108-446, 118 Stat. 2647 (2004).
23.
JohnsonE.ArnoldN. (2004). Validating an alternate assessment. Remedial and Special Education, 25, 266–275.
24.
KampferS.HorvathL.KleinertH.KearnsJ. (2001). Teachers’ perceptions of one state’s alternate assessment portfolio program: Implications for practice and teacher preparation. Exceptional Children, 67, 361–374.
25.
KaneM. T. (2006). Validation. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 17–64). Washington, DC: National Council on Measurement in Education and American Council on Education.
26.
KaneM. (2008). Terminology, emphasis, and utility in validation. Educational Researcher, 37, 76–82.
27.
KarvonenM.FlowersC. P.BrowderD. M.WakemanS.AlgozzineB. (2006). Case study of the influences on alternate assessment outcomes for students with disabilities. Education and Training in Developmental Disabilities, 41, 95–110.
28.
KearnsJ. F.Towles-ReevesE.KleinertH. L.KleinertJ.ThomasM. (in press). Characteristics of and implications for students participating in alternate assessments based on alternate academic achievement standards. Journal of Special Education.
29.
KleinertH. L.KearnsJ. F.KennedyS. (1997). Accountability for all students: Kentucky’s alternate portfolio assessment for students with moderate and severe cognitive disabilities. Journal of the Association for Persons with Severe Handicaps, 22, 88–101.
30.
LinnR. (2002). Validation of the uses and interpretations of results of state assessment and accountability systems. In TindalG.HaladynaT. (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 27–48). Mahwah, NJ: Lawrence Erlbaum.
31.
LinnR. (2009). The concept of validity in the context of NCLB. In LissitzR. (Ed.), The concept of validity: Revisions, new directions and applications (pp. 195–212). Charlotte, NC: Information Age.
32.
LissitzR. W.SamuelsenK. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437–448.
33.
MarionS.PerieM. (2009). An introduction to validity arguments for alternative assessments. In SchaferW. M.LissitzR. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 329–334). Baltimore, MD: Brookes.
34.
MehrensW. (2002). Consequences of assessment: What is the evidence? In TindalG.HaladynaT. (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation. Mahwah, NJ: Lawrence Earlbaum.
35.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education and National Council on Measurement in Education.
36.
MossP. A. (2007). Reconstructing validity. Educational Researcher, 36, 470–476.
37.
No Child Left Behind Act, Pub. L. No. 107-110, 115 Stat. 1425 (2002).
38.
QuenemoenR. (2008). A brief history of alternate assessments based on alternate achievement standards (Synthesis Report 68). Minneapolis: University of Minnesota, National Center on Educational Outcomes.
39.
QuenemoenR. (2009). Toward a straighter road. In SchaferW. M.LissitzR. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 359–362). Baltimore, MD: Brookes.
40.
ReckaseM. (1998). Consequential validity from the test developer’s perspective. Educational Measurement: Issues and Practice, 17(2), 13-16.
41.
RigneyS. (2009). Public policy and the development of alternate assessments for students with cognitive disabilities. In SchaferW. M.LissitzR. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 41–60). Baltimore, MD: Brookes.
42.
RyanK. (2002). Assessment validation in the context of high stakes assessments. Educational Measurement: Issues and Practice, 2(1), 7–15.
43.
SchaferW. D.WangJ.WangV. (2009). Validity in action: State assessment validity evidence for compliance with NCLB.In LissitzR. (Ed.), The concept of validity: Revisions, new directions and applications (pp. 173–193). Charlotte, NC: Information Age.
44.
SireciS. G. (2007). On validity theory and test validation. Educational Researcher, 36, 477–481.
45.
SireciS. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In LissitzR. (Ed.), The concept of validity: Revisions, new directions and applications (pp. 19–37). Charlotte, NC: Information Age.
TindalG.McDonaldM.TedescoM.GlasgowA.AlmondP.CrawfordL.HollenbeckK. (2003). Alternate assessments in reading and math: Development and validation for students with significant disabilities. Exceptional Children, 49, 481–494.
48.
Towles-ReevesE.KearnsJ.KleinertH.KleinertJ. (2009). An analysis of the learning characteristics of students taking alternate assessments based on alternate achievement standards. Journal of Special Education, 42, 241–254.
49.
U.S. Department of Education. (2003). Title I–Improving the academic achievement of the disadvantaged: Final rule, 68 Fed. Reg. 236 (December 9, 2003). Washington, DC: U.S. Department of Education, Office of Elementary and SecondaryEducation.
50.
U.S. Department of Education. (2005, August). Alternate achievement standards for students with the most significant cognitive disabilities: Non-regulatory guidance. Washington, DC: Author.
51.
U.S. Department of Education. (2009). State and local implementation of the No Child Left Behind Act, volume V: Implementation of the 1 percent rule and 2 percent interim policy options. Washington, DC: Office of Planning, Evaluation, and Policy Development. Retrieved from http://www.ed.gov/rschstat/eval/disadv/nclb-disab/nclb-disab.pdf
