The dependability of academic achievement measures for English language learners (ELLs) is influenced by three facts: (a) Each ELL has unique strengths and weaknesses in each language mode (listening, speaking, reading, and writing) both in English and in his or her first language, (b) each test item poses a different set of linguistic demands that do not replicate across languages, and (c) raters may vary in their ability to properly interpret students' responses. Generalizability theory allows examination of the magnitude of score variation due to the language (or dialect) in which tests are administered. The interaction of student, item, and language (and dialect) is the most important source of score variation. The language in which students should be tested and the minimum number of items needed to produce dependable measures of academic achievement for ELLs may be different across communities within a broad group of native speakers of a language.
Abedi, J. (2002). Standardized achievement tests and English language learners: Psychometric issues. Educational Assessment, 8, 231-257.
2.
Aguirre-Muñoz, Z., & Baker, E.L. (1997). Improving the equity and validity of assessment-based information systems (CSE Technical Rep. 462). Los Angeles : Center for the Study of Evaluation; National Center for Research on Evaluation, Standards, and Student Testing; University of California, Los Angeles, Graduate School of Education & Information Studies .
3.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing . Washington, DC: American Educational Research Association.
4.
Ball, A.F. (1992). Cultural preference and the expository writing of African-American adolescents. Written Communication, 9, 501-532.
5.
Ball, A.F. (1997). Expanding the dialogue on culture as a critical component when assessing writing. Assessing Writing, 4, 169-202.
6.
Baxter, G.P., & Shavelson, R.J. (1994). Science performance assessments: Benchmarks and surrogates. International Journal of Educational Research, 21, 279-298.
7.
Beaumont, C., de Valenzuela, J.S., & Trumbull, E. (2002). Alternative assessment for transitional readers . Bilingual Research Journal, 26, 213-240.
8.
Bialystok, E. (1997). Effects of bilingualism and biliteracy on children's emerging concepts of print. Developmental Psychology, 3(3), 429-440.
9.
Bransford, J.D., Brown, A.L., & Cocking, R.R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
10.
Brennan, R. (2001). Generalizability theory. New York: Springer Verlag.
11.
Casanova, U., & Arias, B. (1993). Contextualizing bilingual education. In M. B. Arias & U. Casanova (Eds.), Bilingual education: Politics, practice, and research: Ninety-second yearbook of the National Society for the Study of Education, Part II. Chicago: University of Chicago Press.
12.
Cocking, R.R., & Mestre, J.P. (1988). Linguistic and cultural influences on learning mathematics. Hillsdale, NJ: Lawrence Erlbaum.
13.
Cronbach, L.J., Gleser, G.C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements. New York: John Wiley .
14.
Cummins, J. (2000). Language, power, and pedagogy: Bilingual children in the crossfire. Clevedon, UK: Multilingual Matters.
15.
Dunbar, S.B., Koretz, D.M., & Hoover, H.D. (1991). Quality control in the development and use of performance assessments. Applied Measurement in Education , 4, 298-303.
16.
Fashola, O.S., Drum, P.A., Mayer, R.E., & Kang, S. (1996). A cognitive theory of orthographic transitions: Predictable errors in how Spanish-speaking children spell English words. American Educational Research Journal, 33, 825-844.
17.
Figueroa, R.A. (1980). Intersection of special education and bilingual education. In J. E. Alatis (Ed.), Georgetown University roundtable on languages and linguistics . Washington, DC: Georgetown University Press.
18.
Freedle, R.O. (2003). Correcting the SAT's ethnic and social-class bias: A method for reestimating SAT scores. Harvard Educational Review, 73, 1-43.
19.
Genesee, F. (Ed.). (1994). Introduction . In F. Genesee (Ed.), Educating second language children: The whole child, the whole curriculum, the whole community (pp. 1-11). Cambridge, UK: Cambridge University Press.
20.
Grosjean, F. (1999). Bilingualism, individual. In B. Spolsky (Ed.), Concise encyclopedia of educational linguistics. Amsterdam: Elsevier.
21.
Hambleton, R.K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 147-200). New York: American Council on Education and Macmillan .
22.
Heath, S.B. (1986). Sociocultural contexts of language development . In Beyond language: Social and cultural factors in schooling language minority students (pp. 143-186). Los Angeles: Bilingual Education Office, California State Department of Education and the Evaluation, Dissemination and Assessment Center, California State University , Los Angeles.
23.
Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC : National Academy Press, Committee on Appropriate Test Use, Board on Testing and Assessment, Commission on Behavioral and Social Sciences in Education, National Research Council.
24.
Hofstetter, C.J. (2003). Contextual and mathematics accommodation test effects for English-language learners. Applied Measurement in Education, 16, 159-188.
25.
Kochman, T. (1989). Black and White cultural styles in pluralistic perspective. In B. Gifford (Ed.), Test policy and test performance: Education, language, and culture (pp. 259-296). Boston: Kluwer.
26.
Kopriva, R. (1999). A conceptual framework for the valid and comparable measurement of all students. Washington, DC: Council of Chief State School Officers.
27.
LaCelle-Peterson, M.W., & Rivera, C. (1994). "Is it real for all kids?" A framework for equitable assessment policies for English language learners. Harvard Educational Review, 64, 55-75.
28.
Lee, O. (1999). Equity implications based on the conceptions of science achievement in major reform documents. Review of Educational Research, 69, 83-115.
29.
Lee, O. (2002). Promoting scientific inquiry with elementary students from diverse cultures and languages. Review of Research in Education, 26, 23-69.
30.
Lee, O., & Fradd, S.H. (1998). Science for all, including students from non-English language backgrounds. Educational Researcher, 27(4), 12-21.
31.
Linn, R.L., Baker, E.L., & Betebenner, D.W. (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31(6), 3-16.
32.
MacSwan, J. (2000). The threshold hypothesis, semilingualism, and other contributions to a deficit view of linguistic minorities. Hispanic Journal of Behavioral Sciences, 22, 3-45.
33.
Marzzatto, R.L. (2002). Idioms under the microscope. ATA Chronicle, 31(1), 30-35.
34.
Moschkovich, J.N. (2000). Learning mathematics in two languages: Moving from obstacles to resources. In W. G. Secada (Ed.), Changing the faces of mathematics: Perspectives on multiculturalism and gender equity (pp. 85-93). Reston, VA: National Council of Teachers of Mathematics.
35.
National Assessment of Educational Progress. ( 1996a). Mathematics items public release. Washington, DC: Author.
36.
National Assessment of Educational Progress. ( 1996b). Science items public release. Washington, DC: Author.
37.
Pellegrino, J.W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
38.
Romaine, S. (1995). Bilingualism (2nd ed). Malden, MA: Blackwell.
39.
Ruiz-Primo, M.A., Baxter, G.P., & Shavelson, R.J. (1993). On the stability of performance assessments. Journal of Educational Measurement, 30, 41-53.
40.
Sandoval, J., & Durán, R.P. (1998). Language. In J. Sandoval, C. Frisby, K. F. Geisinger, J. D. Scheuneman , & J. R. Grenier (Eds.), Test interpretation and diversity: Achieving equity in assessment (pp. 181-212). Washington, DC: American Psychological Association .
41.
Shavelson, R.J., Baxter, G.P., & Gao, X. (1993). Sampling variability of performance assessment . Journal of Educational Measurement, 30, 215-232.
42.
Shavelson, R.J., Baxter, G.P., & Pine, J. (1992). Performance assessments: Political rhetoric and measurement reality. Educational Researcher, 21(4), 22-27.
43.
Shavelson, R.J., & Webb, N.M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
44.
Solano-Flores, G. (2006). Language, dialect, and register: Sociolinguistics and the estimation of measurement error in the testing of English-language learners. Teachers College Record, 108, 2354-2379.
45.
Solano-Flores, G., Jovanovic, J., Shavelson, R.J., & Bachman, M. (1999). On the development and evaluation of a shell for generating science performance assessments. International Journal of Science Education, 21, 293-315.
46.
Solano-Flores, G., Lara, J., & Li, M. (2003, April). Assessment comparability across languages: Testing linguistic minorities. Symposium presented at the annual meeting of the National Council on Measurement in Education , Chicago.
47.
Solano-Flores, G., Lara, J., Sexton, U., & Navarrete, C. (2001). Testing English language learners: A sampler of student responses to science and mathematics test items. Washington, DC: Council of Chief State School Officers .
48.
Solano-Flores, G., & Li, M. (2006). The use of generalizability (G) theory in the testing of linguistic minorities. Educational Measurement: Issues and Practice, 25, 13-22.
49.
Solano-Flores, G., & Nelson-Barber, S. (2001). On the cultural validity of science assessments. Journal of Research in Science Teaching , 38, 553-573.
50.
Solano-Flores, G., & Trumbull, E. (2003). Examining language in context: The need for new research and practice paradigms in the testing of English-language learners . Educational Researcher, 32(2), 3-13.
51.
Solano-Flores, G., & Trumbull, E. (2008). In what language should English language learners be tested? In R. J. Kopriva (Ed.), Improving testing for English language learners. New York : Routledge.
52.
Stevens, R.A., Butler, F.A., & Castellon-Wellington, M. (2000). Academic language and content assessment: Measuring the progress of English language learners (ELLs) (CSE Technical Rep. 552). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing; University of California, Los Angeles, Graduate School of Education & Information Studies.
53.
Valdés, G., & Figueroa, R.A. (1994). Bilingualism and testing: A special case of bias. Norwood, NJ: Ablex.
54.
Wei, L. (2000). Dimensions of bilingualism. In L. Wei (Ed.), The bilingualism reader (pp. 3-25). London: Routledge.
55.
Wolfram, W., Adger, C.T., & Christian, D. (1999). Dialects in schools and communities. Mahwah, NJ: Lawrence Erlbaum.
