This study piloted test items that will be used in a computer-delivered and scored test of productive grammatical ability in English as a second language (ESL). Findings from research on learners’ development of morphosyntactic, syntactic, and functional knowledge were synthesized to create a framework of grammatical features. We outline the interpretive argument and present results from four pilot test administrations in terms of (a) reliability, (b) relationships between item difficulties and developmental stages, (c) correlations with other English tests, and (d) predictability of test scores in relation to proficiency levels. The results support the potential of assessing productive ESL grammatical ability by targeting areas identified in SLA research, and the plausibility of moving forward with computer delivery and scoring.
American Council for the Teaching of Foreign Languages ( 1983). ACTFL Proficiency Guidelines. Revised 1985. Hastings-on-Hudson, NY: ACTFL Materials Center.
2.
Alderson JC ( 2007). The challenge of (diagnostic) testing: Do we know what we are measuring? In J Fox, M Wesche, D Bayliss, L Cheng, CE Turner and C Doe (eds), What are we measuring? Language testing reconsidered (pp. 21-39). Ottawa : University of Ottawa Press.
3.
Andersen R. ( 1978). An implicational model for second language research. Language Learning, 28, 221-282.
4.
Bachman LF ( 2005). Building and supporting a case for test use. Language Assessment Quarterly, 2(1), 1-34.
5.
Bailey N., Madden C., and Krashen S. ( 1974). Is there a ‘natural sequence’ in adult second language learning? Language Learning, 24(2), 235-243.
6.
Bardovi-Harlig K. ( 2000). Tense and aspect in second language acquisition: Form, meaning and use. Malden, MA: Blackwell .
7.
Bayley RJ ( 1999). The primacy of aspect hypothesis revisited: Evidence from language shift. Southwest Journal of Linguistics, 18(2), 1-22.
8.
Council of Europe. (2001). Common European Framework of Reference for languages: Learning, teaching, assessment . Cambridge: Cambridge University Press.
9.
Chapelle AC, Enright MK, and Jamieson JM (eds) (2008). Building a validity argument for the Test of English as a Foreign LanguageTM . New York/London: Routledge.
10.
Cotos E., Pendar N. ( 2007). Automated diagnostic writing tests: Why? How? In CA Chapelle, Y-R Chung and J Xu (eds) Towards adaptive CALL: Natural language processing for diagnostic language assessment (pp. 65-81). Ames, IA: Iowa State University. Retrieved 8 September 2008 from http://apling.public.iastate.edu//TSLL/5th_2007/proceedings2007/papers/TSLL07_07_Cotos.pdf.
11.
Davies A., Brown A., Elder C., Hill K., Lumley T., and McNamara T. ( 1999). Dictionary of language testing. Cambridge : Cambridge University Press and UCLES.
12.
Dulay H., Burt M. ( 1973). Should we teach children syntax? Language Learning, 23, 245-258.
13.
Dulay H., Burt M. ( 1974). Natural sequences in child second language acquisition . Language Learning, 24, 37-53.
14.
Grabe W., Kaplan RB ( 1996). Theory and practice of writing. London : Longman.
15.
Hawkins R. ( 2001). Second language syntax: A generative introduction. Cornwall, UK: Blackwell.
16.
Haegeman L. ( 1994). Introduction to government and binding theory (2nd ed.). Oxford: Blackwell .
17.
Kane MT ( 2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342.
18.
Kane M. ( 2006). Validation. In R Brennen (ed.), Educational measurement, (4th ed., pp. 17-64). Westport, CT: Greenwood Publishing.
19.
Krashen S., Madden C., and Bailey N. ( 1975). Theoretical aspects of grammatical sequencing. In MK Burt, H Dulay (eds), On TESOL ‘75 (pp. 44-54). Washington, DC: TESOL.
20.
Krashen S., Sferlazza V., Feldman L., and Fathman AK ( 1976). Adult performance on the slope test: More evidence for a natural sequence in adult second language acquisition. Language Learning, 26(1), 146-151.
21.
Larsen-Freeman D. ( 1975a). The acquisition of grammatical morphemes by adult learners of English as a second language. Unpublished PhD dissertation, University of Michigan.
22.
Larsen-Freeman D. ( 1975b). The acquisition of grammatical morphemes by adult ESL students. TESOL Quarterly, 9, 409-419.
23.
Lardiere D. ( 2007). Ultimate attainment in second language acquisition: A case study. Mahwah, NJ: Lawrence Erlbaum.
24.
Lardiere D. ( 2008). Feature assembly in second language acquisition. In JM Liceras, H Zobl and H Goodluck (eds), The role of formal features in second language acquisition (pp. 107-140). New York: Lawrence Erlbaum.
25.
Lightbown PM, Spada N. ( 1993). How languages are learned. Oxford : Oxford University Press.
26.
Meisel JM, Clahsen H., and Pienemann M. ( 1981). On determining development stages in natural second language acquisition. Studies in Second Language Acquisition, 3(2), 109-135.
27.
Mislevy RJ, Steinberg LS, and Almond RG ( 2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-62.
28.
Milon JP ( 1974). The development of negation in English by a second language learner. TESOL Quarterly, 8(2), 137-143.
29.
Norris J. ( 2005). Using developmental sequences to estimate ability with English grammar: Preliminary design and investigation of a web-based test . Second Language Studies, 24(1), 24-128. Retrieved 4 March 2008 from www.hawaii.edu/sls/uhwpesl/on-line_cat.html.
30.
Pienemann M. ( 1999). Language processing and second language development: Processability Theory. Amsterdam/ Philadelphia: John Benjamins.
31.
Pienemann M., Johnston M. ( 1987). Factors influencing the development of language proficiency . In D Nunan (ed.) Applying second language acquisition research (pp. 45-141). Adelaide : National Curriculum Resource Center.
32.
Pienemann M., Johnston M., and Brindley G. ( 1988). Constructing an acquisition-based procedure for second language assessment. Studies in Second Language Acquisition , 10(2), 217-243.
33.
Purpura J. ( 2004). Assessing grammar. Cambridge: Cambridge University Press.
34.
Stauble A-M ( 1984). A comparison of a Spanish-English and a Japanese-English second language continuum: Negation and verb morphology. In R Andersen (Ed.), Second language: A cross-linguistic perspective. Rowley, MA: Newbury House.
