The effects of group members' personalities on a test taker's L2 group oral discussion test scores

Abstract

The second language group oral is a test of second language speaking proficiency, in which a group of three or more English language learners discuss an assigned topic without interaction with interlocutors. Concerns expressed about the extent to which test takers' personal characteristics affect the scores of others in the group have limited its attractiveness. This study investigates the degree to which assertive and non-assertive test takers' scores are affected by the levels of assertiveness of their group members. The sample of test takers was Japanese first year university students who were studying English in Japan. The students took the revised NEO-PI-R (Costa & McCrae, 1992; Shimanoka et al., 2002), a group oral test, and PhonePass SET-10 (Ordinate, 2004). Two separate MANCOVA analyses were conducted, one designed to determine the extent to which assertive test takers' scores are affected by the levels of assertiveness of group members (N = 112), and one designed to determine the extent to which non-assertive test takers' scores are affected by the levels of assertiveness of group members (N = 113). The analyses indicated that assertive test takers were assigned higher scores than expected when grouped with only non-assertive test takers and lower scores than expected when grouped with only assertive test takers, while the study failed to find an effect for grouping based on assertiveness for non-assertive test takers' scores. The findings of the study suggest that when the group oral is used, rater-training sessions should include guidance on how to evaluate a test taker in the context of the group in which the test taker is assessed and assign scores that are not based on a comparison of proficiencies of group members.

Keywords

group oral oral assessment personality speaking test speaking proficiency

Get full access to this article

View all access options for this article.

References

Bachman, L.F. (2001). February: Speaking as a realization of communicative competence. Paper presented at the meeting of the American Association of Applied Linguistics - International Language Testing Association (AAAL-ILTA) Symposium. St. Louis, Missouri.

Bachman, L.F. & Palmer, A.S. (1996). Language testing in practice. Hong Kong Oxford University Press.

Berry, V. (2004). A study of the interaction between individual personality differences and oral performance test facets. Unpublished doctoral dissertation. King's College, University of London.

Bonk, W.J. & Ockey, G.J. (2003). A many-facet Rasch analysis of the second language group oral discussion task. Language Testing, 20, 89-110.

Bonk, W.J. & Van Moere, A. (2004, March). L2 group oral testing: The influence of shyness/outgoingness, match of interlocutors' proficiency level, and gender on individual scores. Paper presented at the annual meeting of the Language Testing Research Colloquium, Temecula , California.

Brown, A. (2003). Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20, 1-25.

Costa, P. & McCrae, R. (1992). Revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.

Eysenck, H. & Eysenck, S. (1991). Eysenck personality questionnaire - Revised. London: Hodder.

Folland, D. & Robertson, D. (1976). Towards objectivity in group oral testing. English Language Teaching Journal, 30, 156-167.

10.

Fulcher, G. (1996). Testing tasks: Issues in task design and the group oral. Language Testing, 13, 23-51.

11.

He, L. & Dai, Y. (2006). A corpus-based investigation into the validity of the CET-SET group discussion. Language Testing, 23, 370-402.

12.

Hilsdon, J. (1995). The group oral exam: Advantages and limitations . In Alderson, J. & North, B. , editors, Language testing in the 1990s: The communicative legacy (pp. 189-197). Hertfordshire : Prentice Hall International.

13.

Hymes, D. (1972). On communicative competence. In Pride, J. & Holmes, J. , editors, Sociolinguistics: Selected readings (pp. 269-293). Harmondsworth, Middlesex: Penguin.

14.

Johnson, M. & Tyler, A. (1998). Re-analyzing the OPI: How much does it look like natural conversation? In Young, R & He, W. , editors, Talking and testing: Discourse approaches to the assessment of oral proficiency (pp. 27-51). Philadelphia, PA: John Benjamins.

15.

Kenyon, D. (1992, February). Development and use of rating scales in language testing. Paper presented at the annual meeting of the Language Testing Research Colloquium, Vancouver , Canada.

16.

Keppel, G. & Wickens, T. (2004). Design and analysis: A researcher's handbook, fourth edition. Upper Saddle River, NJ: Pearson-Prentice Hall.

17.

Kormos, J. (1999). Simulating conversations in oral-proficiency assessments: A conversation analysis of role plays and non-scripted interviews in language exams. Language Testing, 16, 163-188.

18.

Lazaraton, A. (1996). Interlocutor support in oral proficiency interviews: The case of CASE. Language Testing, 13, 151-172.

19.

Liski, E. & Puntanen, S. (1983). A study of the statistical foundations of group conversation tests in spoken English. Language Learning, 33, 225-246.

20.

McNamara, T.F. (1996). Measuring second language performance. London: Longman.

21.

Monte, C. (1991). Beneath the mask: An introduction to theories of personality, fourth edition. Fort Worth, TX: Harcourt Brace College.

22.

Nevo, D. & Shohamy, E. (1984). Applying the joint committee's evaluation standards for the assessment of alternative testing methods. Paper presented at the Annual Meeting of the American Educational Research Association , New Orleans, Louisiana. (ED. 243 934).

23.

Ockey, G.J. (2001). Is the oral interview superior to the group oral? Working Papers on Language Acquisition and Education, International University of Japan, 11, 22-41.

24.

Ordinate. (2004). SET-10 test description validation summary, Retrieved February 25, 2005, from www.ordinate.com.

25.

Raudenbush, S.W. & Bryk, S.A. (2002). Hierarchical linear models: Applications and data analysis methods, second edition. London: Sage Publications.

26.

Ross, S. & Berwick, R. (1992). The discourse of accommodation in oral proficiency interviews. Studies in Second Language Acquisition, 14, 159-176.

27.

Shimanoka, Y. , Nakazato, K. , Gondo, Y. & Takayama, M. (1997). Construction and factorial validity of the Japanese NEO-PI-R. The Japanese Journal of Personality, 6, 138-147.

28.

Shimanoka, Y. , Nakazato, K. , Gondo, Y. & Takayama, M. (2002). NEO-PI-R NEO-FFI manual. Tokyo : Tokyo Shinri.

29.

Shohamy, E. , Reves, E. & Bejarano, Y. (1986). Introducing a new comprehensive test of oral proficiency . ELT Journal, 40, 212-220.

30.

Skehan, P. (1998). Processing perspectives to second language development, instruction, performance, and assessment. Thames Valley Working Papers in Applied Linguistics, 4, 70-88.

31.

SPSS 11.5 for Windows. (1989-2002). Computer software. SPSS.

32.

Stata 8.2 for Windows. (1985- 2005). Computer software. College Station, TX: StataCorp.

33.

Tabachnick, B. & Fidell, L. (2001). Using multivariate statistics. Boston, MA: Allyn & Bacon.

34.

van Lier, L. (1989). Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 23, 489-508.

35.

Van Moere, A. (2006). Validity evidence in a group oral test. Language Testing, 23, 411-440.

36.

Van Moere, A. & Kobayashi, M. (2003, July). Who speaks most in this group? Does that matter? Paper presented at the annual meeting of the Language Testing and Research Colloquium, Reading, UK.

37.

Webb, N.M. , Nemer, K.M. , Chizhik, A.W. & Sugrue, B. (1998). Equity issues in collaborative group assessment: Group composition and performance. American Educational Research Journal, 35, 607-651.

38.

Young, R. & He, W. (1998). Language proficiency interviews: A discourse approach . In Young, R & He, W. , editors, Talking and testing: Discourse approaches to the assessment of oral proficiency (pp. 1-24). Philadelphia, PA: John Benjamins.