This study investigates the impact of performance conditions on perceptions of task
difficulty in a test of spoken language, in light of the cognitive complexity
framework proposed by Skehan (1998). Candidates performed a series of narrative
tasks whose characteristics, and the conditions under which they were performed,
were manipulated, and the impact of these on task performance was analysed.
Test-takers recorded their perceptions of the relative difficulty of each task and
their attitudes to them. Results offered little support for Skehan’s
framework in the context of oral proficiency assessment, and also raise doubts about
post hoc estimates of task difficulty by test-takers.
Get full access to this article
View all access options for this article.
References
1.
Alderson, J. C.1988:
New procedures for validating proficiency tests of ESP? Theory and practice
. Language Testing5,
220-232
.
2.
Bachman, L. F.1990: Fundamental considerations in language testing.
Oxford and New York: Oxford University Press
.
3.
Bachman, L. F.
and
Palmer, A.1996: Language testing in practice.
Oxford and New York: Oxford University Press
.
4.
Bradshaw, J.1990:
Test-takers’ reactions to a placement test
. Language Testing7,
13-30
.
5.
Brindley, G.1987: Factors affecting task difficulty. In
Nunan, D.
, editor, Guidelines for the development of curriculum resources.
Adelaide: Adelaide National Curriculum Resource Centre
, 45-56.
6.
Brooks, L. 1999: Adult ESL student attitudes towards performance-based
assessment. Unpublished MA thesis, University of Toronto.
7.
Brown, A.1993:
The role of test-taker feedback in the test development process:
test-takers’ reactions to a tape-mediated test of proficiency in
spoken Japanese
. Language Testing10,
277-303
.
8.
Brown, J. D.
,
Hudson, T.
and
Norris, J. M.
1999:
Validation of test-dependent and task-independent ratings of performance assessment
. Paper presented at the 21st Language Testing Research
Colloquium, Tsukuba, Japan, July
9.
Clarke, J. L. D.1985: Development of tape-mediated, ACTFL/ILR scale-based test of
Chinese speaking proficiency. In
Stansfield, C. W.
, editor, Technology and language testing.
Princeton, NJ: Education Testing Service
, 129-146.
10.
Davies, A.
,
Brown, A.
,
Elder, C.
,
Hill, K.
,
Lumley, T.
and
McNamara, T.1999: Dictionary of language testing.
Cambridge: UCLES, Cambridge University Press
.
11.
Douglas, D.2000: Language testing for specific purpose.
Cambridge: Cambridge University Press
.
12.
Elder, C.
and
Lynch, B.
1996:
Public perceptions of basic skills tests and their ethical implications
. Paper presented at the 20th Language Testing Research
Colloquium, Monterey, March.
13.
Foster, P.
and
Skehan, P.1996:
The influence of planning and task type on second language performance
. Studies in Second Language Acquisition18,
299-323
.
14.
Hill, K.1998: The effect of test-taker characteristics on reactions to
and performance on an oral English proficiency test. In
Kunnan, A. J.
, editor, Validation in language assessment.
Mahwah, NJ: Lawrence Erlbaum
, 209-229.
15.
Iwashita, N.
and
Elder, C.1997:
Expert feedback? Assessing the role of test-taker reactions to a
proficiency test for teachers of Japanese
. Melbourne Papers in Language Testing6,
53-67
.
16.
Iwashita, N.
,
McNamara, T.
and
Elder, C.2001:
Can we predict task difficulty in an oral proficiency test? Exploring the
potential of an information processing approach to task design
. Language Learning21,
401-436
.
17.
Kenyon, D.
and
Stansfield, C.
1991:
A method for improving tasks on the performance assessments through field testing
. Paper presented in the annual meeting of the National Council on
Measurement in Education, Chicago, IL,
April 1991.
Linacre, J. M.1992: FACETS Computer program for many faceted Rasch
Measurement.
Chicago, IL: Mesa Press
.
20.
McNamara, T.F. 1990: Assessing the second language proficiency of health
professionals. Unpublished PhD Dissertation, The University of Melbourne.
21.
Linacre, J. M.1996: Measuring second language performance.
London and New York: Addison Wesley Longman
.
22.
McNamara, T.
,
Elder, C.
and
Iwashita, N.
in preparation: Investigating predictors of task difficulty in the
measurement of speaking proficiency. Final Report, TOEFL 2000 Research
Project.
Princeton; NJ: Educational Testing Center
.
23.
Messick, S.1989: Validity. In
Linn, R. J.
, editor, Educational measurement. 3rd edition.
New York: American Council on Education/Macmillan
.
24.
Nevo, B.1985:
Face validity revisited
. Journal of Educational Measurement22,
287-293
.
25.
Norris, J. M.
,
Brown, J. D.
,
Hudson, T. D.
and
Bonk, W.
2000:
Assessing performance on complex L2 tasks: investigating raters, examinees
and tasks
. Paper presented at the 22nd Language Testing Research
Colloquium, Vancouver, March.
26.
Norris, J. M.
,
Brown, J. D.
,
Hudson, T.
and
Yoshioka, J.1998: Designing second language performance assessments.
Technical Report 18, Second Language Teaching and Curriculum Center, University of
Hawaii at Manoa.
Honolulu: University of Hawaii Press
.
27.
Nunan, D.1989: Designing tasks for the communicative classroom.
Cambridge: Cambridge University Press
.
28.
Robinson, P.1995:
Task complexity and second language narrative discourse
. Language Learning45,
141-175
.
29.
Robinson, P.1996: Connecting tasks, cognition and syllabus design.
In
Robinson, P.
, editor, Task complexity and second language syllabus design: databased
studies and speculations. University of Queensland Working Papers in
Applied Linguistics (Special Issue).
Brisbane: University of Queensland
, 1-16.
30.
Robinson, P.2001:
Task complexity, task difficulty and task production: Exploring
interactions in a componential framework
. Applied Linguistics21,
27-57
.
31.
Robinson, P.
in press: Attention and memory during SLA. In
Doughty, C.
and
Long, M.
, editors, Handbook of research in second language acquisition.
Oxford: Blackwell
.
32.
Scott, M. L.1986:
Student affective reactions to oral language tests
. Language Testing, 3,
99-118
.
33.
Scott, M. L.
and
Madsen, H. S.1983: The influence of retesting on test affect. In
Oller, J. W.
, editor, Issues in language testing research.
Rowley, MA: Newbury House
, 270-279.
34.
Shohamy, E.1982.
Affective considerations in language testing
. The Modern Language Journal66,
13-17
.
35.
Skehan, P.1996:
A framework for the implementation of task-based instruction
. Applied Linguistics17,
38-62
.
36.
Skehan, P.1998: A cognitive approach to language learning.
Oxford: Oxford University Press
.
37.
Skehan, P.
and
Foster, P.1997:
Task type and task processing conditions as influences on foreign language performance
. Language Teaching Research1,
185-211
.
38.
Skehan, P.
and
Foster, P.1999:
The influence of task structure and processing conditions on narrative retellings
. Language Learning49,
93-120
.
39.
Slatyer, H.
,
Brindley, G.
and
Wigglesworth
. 2000:
Task difficulty in ESL listening assessment
. Paper presented at the 22nd Language Testing Research
Colloquium, Vancouver, March.
40.
Spolsky, B.1995: Measured words: the development of objective language
testing.
Oxford: Oxford University Press
.
41.
Stansfield, C.1991: A comparative analysis of simulated and direct oral
proficiency interviews. In
Anivan, S.
, editor, Current developments in language testing.
Singapore: SEAMEO RELC
, 199-209.
42.
Stansfield, C. W.
,
Kenyon, D. M.
,
Paiva, R.
,
Doyle, F.
,
Ulsh, I.
and
Cowles, M. A.1990:
The development and validation of the Portuguese speaking test
. Hispania73,
641-651
.
43.
Widdowson, H.2001: Communicative language testing. In
Elder, C.
,
Brown, A.
,
Grove, E.
,
Hill, K.
,
Iwashita, N.
,
Lumley, T.
,
McNamara, T.
and
O’Loughlin, K.
, editors, Experimenting with uncertainty: essays in honour of Alan
Davies.
Cambridge: UCLES
, 12-21.
44.
Wigglesworth, G.1997:
An investigation of planning time and proficiency level on oral test discourse
. Language Testing14,
85-106
.
45.
Zeidner, M.1988:
Sociocultural differences in examinees’ attitudes toward
scholastic ability exams
. Journal of Educational Research80,
352-258
.
46.
Zeidner, M.1990:
College students’ reactions towards key facets of classroom testing
. Assessment and Evaluation in Higher Education15,
151-169
.
47.
Zeidner, M.
and
Bensoussan, M.1988:
College students’ attitudes towards written versus oral of EFL
. Language Testing5,
100-114
.
