Diagnosing the English Speaking Ability of College Students in China – Validation of the Diagnostic College English Speaking Test

Abstract

This study investigates the validity of the Diagnostic College English Speaking Test (DCEST) in the context of EFL teaching and learning in China. The experiment was conducted in three stages over the course of eight weeks at a national key university in China. By means of test administration and questionnaire survey, the researcher gathered various types of data from the control and the experiment group. The results demonstrated that the DCEST had achieved satisfactory reliability and validity. The DCEST displayed a high intra-rater reliability (r = .966) and inter-rater reliability (r = .884). The construct validity of the DCEST was evidenced by the means of factor analysis, which showed that three factors loaded heavily on each of the three test tasks and accounted for a total of 78.69% of the score variance. The analysis of the questionnaire data provided supportive evidence to the face validity of the DCEST. The paper concludes with suggestions for further research into the impact of the DCEST on oral English teaching.

Keywords

Diagnostic testing oral English assessment test validation construct validity diagnostic speaking test

Get full access to this article

View all access options for this article.

References

Alderson

(2005) Diagnosing Foreign Language Proficiency: the Interface between Learning and Assessment. London: Continuum.

Alderson

Huhta

(2005) The development of a suite of computer-based diagnostic tests based on the common European framework. Language Testing 22(3): 301–20.

Alderson

Wall

(1993) Does washback exist? Applied Linguistics 14(2): 115–29.

Bachman

(1990) Fundamental Considerations in Language Testing. Oxford: Oxford University Press.

Black

Wiliam

(1998) Assessment and classroom learning. Assessment in Education 5(1): 7–74.

Chapelle

(1999) Validity in language assessment. Annual Review of Applied Linguistics 19: 254–72.

Cheng

(1998) Impact of a public English examination change on students’ perceptions and attitudes toward their English learning. Studies in Educational Evaluation 24: 279–301.

Cheng

Watanabe

Curtis

(eds) (2004) Washback in Language Testing: Research Contexts and Methods. Mahwah, NJ: Laurence Erlbaum & Associates.

Jang

(2005) A validity narrative: effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Illinois, USA.

10.

Jang

(2009) Cognitive diagnostic assessment of L2 reading comprehension ability: validity arguments for fusion model application to LanguEdge assessment. Language Testing 26: 31–73.

11.

Jin

(2000) The washback of CET-SET on teaching. Foreign Languages World (4): 57–62.

12.

Jin

(2004) The reform of CET. Foreign Languages in China (1): 27–29.

13.

Kane

(2013) Validating the interpretations and use of test scores. Journal of Educational Measurement 50(1): 1–73.

14.

Messick

(1989) Meaning and values in test validation: the science and ethics of assessment. Educational Researcher 18(2): 5–11.

15.

Ministry of Education in China (2007) College English Curriculum Requirements. Shanghai: Shanghai Foreign Language Education Press.

16.

Richards

(2008) Formative assessment in teacher education: the development of a diagnostic language test for trainee teachers of German. British Journal of Educational Studies 56(2): 184–204.

17.

Sewell

(2004) Diagnostic assessment within the skills for life strategy. Paper presented at the Conference of the International Association for Educational Assessment, Philadelphia, PA.

18.

Weir

(2005) Language Testing and Validation: An Evidence-based Approach. Basingstoke: Palgrave Macmillan.

19.

Yang

Gui

(2007) The sociological view of language testing. Modern Foreign Languages (4): 368–74.

20.

Zhao

(2011) Development and validation of the diagnostic college English speaking test. Unpublished doctoral dissertation, Shanghai Jiao Tong University, Shanghai, China.