This article reports on an exploratory study that investigated the comparability of
listening assessment tasks used to assess and report learning outcomes of adult ESL
learners in Australia. The study focused on the effects of task characteristics and
task conditions on learners’ performance in competency-based listening
assessment tasks that require learners to demonstrate specific listening behaviours.
Key variables investigated included the nature of the input and the response mode.
Quantitative and qualitative analyses of test scores suggest that speech rate and
item format influence task and item difficulty. However, the complexity of the
interaction between text, item and response makes it difficult to isolate the
effects of specific variables. Implications of these findings for assessment task
validity and reliability are considered and practical consequences for assessment
task design in outcomes-based systems are discussed.
Get full access to this article
View all access options for this article.
References
1.
Adams, R. J.
and
Khoo, S. T.1993: QUEST.
Hawthorn, Victoria: Australian Council for
Educational Research
.
2.
Alderson, J. C.1991: Bands and scores. In
Alderson, J. C.
and
North, B.
, editors, Language testing in the 1990s.
London: Macmillan
, 71-86.
3.
Association of Language Testers in Europe (ALTE)2001: Glossary. Available at
http://www.alte.org/glossary/index.cfm (May 2002).
4.
Bachman, L. F.
and
Palmer, A. S.1996: Language testing in practice.
Oxford: Oxford University Press
.
5.
Bejar, I.
,
Douglas, D.
,
Jamieson, J.
,
Nissan, S.
and
Turner, J.2000: TOEFL 2000 listening framework: a working paper.
Princeton, NJ: Educational Testing Service
.
6.
Berne, J. E.
1993:
The role of text type, assessment task and target language experience in L2
listening comprehension assessment
. Paper presented at the Annual Meetings of the American
Association of Applied Linguistics and the American Association of Teachers of
Spanish and Portuguese, Cancun, Mexico,
August 9-13, 1992.
ED 358737-ED 358737
.
7.
Breiner-Sanders, K.
,
Lowe, P.
,
Miles, J.
and
Swender, E.2000:
ACTFL Proficiency Guidelines-Speaking
. Revised 1999. Foreign Language Annals33,
13-18
.
8.
Brindley, G.1997: Investigating second language listening ability: listening
skills and item difficulty. In
Brindley, G.
and
Wigglesworth, G.
, editors, Access: issues in language test design and delivery.
Sydney: National Centre for English Language
Teaching and Research, Macquarie University
, 65-86.
9.
Brindley, G.1998a:
Outcomes-based assessment and reporting in language learning programs
. Language Testing15,
45-85
.
Brindley, G.1998c. Describing language development? Rating scales and second
language acquisition. In
Bachman, L. F.
and
Cohen, A. D.
, editors, Interfaces between second language acquisition and language
testing research.
Cambridge: Cambridge University Press
, 112-140.
12.
Brindley, G.2000: Task difficulty and task generalisability in
competency-based writing assessment. In
Brindley, G.
, editor, Studies in immigrant English language assessment. Volume
1.
Sydney: National Centre for English Language
Teaching and Research
, Macquarie University, 125-157.
13.
Brindley, G.2001: Investigating rater consistency in competency-based
language assessment. In
Brindley, G.
and
Burrows, C.
, editors, Studies in immigrant English language assessment. Volume
2.
Sydney: National Centre for English Language
Teaching and Research, Macquarie University
, 59-80.
14.
Buck, G. 1990: The testing of second language listening comprehension.
Unpublished doctoral dissertation, University of Lancaster, UK.
15.
Brindley, G.1991a:
The testing of listening comprehension: an introspective study
. Language Testing8,
67-91
.
16.
Brindley, G.
1991b:
Expert estimates of test item characteristics
. Paper presented at Language Testing Research
Colloquium, Princeton, NJ.
17.
Brindley, G.1994:
The appropriacy of psychometric measurement models for testing second
language listening comprehension
. Language Testing11,
145-170
.
18.
Brindley, G.2001: Assessing listening.
Cambridge: Cambridge University Press
.
19.
Buck, G.
and
Tatsuoka, K.1998:
Application of the rule-space procedure to language testing: examining
attributes of a free response listening test
. Language Testing15,
119-157
.
20.
Chaudron, C.
and
Richards, J.1986:
The effect of discourse markers on the comprehension of lectures
. Applied Linguistics7,
113-127
.
21.
Christie, J.
and
Delaruelle, S.1997: Assessment and moderation.
Sydney: National Centre for English Language
Teaching and Research, Macquarie University
.
22.
Davies, A.
,
Brown, A.
,
Elder, C.
,
Hill, K.
,
Lumley, T.
and
McNamara, T.1999: Dictionary of language testing.
Cambridge: Cambridge University Press
.
23.
Dunkel, P.
,
Henning, G.
and
Chaudron, C.1993:
The assessment of a listening comprehension construct: a tentative model
for test specification and development
. Modern Language Journal77,
180-191
.
24.
Flowerdew, J.1994: Research of relevance to second language lecture
comprehension-an overview. In
Flowerdew, J.
, editor, Academic listening: research perspectives.
Cambridge: Cambridge University Press
, 7-29.
25.
Freedle, R.
and
Kostin, I.1996: The prediction of TOEFL listening comprehension item
difficulty for minitalk passages: implications for construct validity.
TOEFL Research Report 56.
Princeton, NJ: Educational Testing Service
.
26.
Griffiths, R.1990:
Speech rate and nonnative speaker comprehension: a preliminary study in
time benefit analysis
. Language Learning40,
311-336
27.
Griffiths, R.1992:
Speech rate and listening comprehension: further evidence of the relationship
. TESOL Quarterly26,
385-389
.
28.
Hansen, C.
and
Jensen, C.1994: Evaluating lecture comprehension. In
Flowerdew, J.
, editor, Academic listening.
Cambridge: Cambridge University Press
, 241-268.
29.
Henning, G.1991: A study of the effects of variation of short-term memory load,
reading response length, and processing hierarchy on TOEFL listening
comprehension item performance.
Princeton, NJ: Educational Testing Service
.
30.
Ingram, D.
and
Wylie, E.1984: Australian second language proficiency ratings.
Canberra: Australian Government Publishing Service
.
31.
Ingram, D.
and
Wylie, E.1997: International second language proficiency ratings.
Nathan, Queensland: Griffith University
.
32.
Jensen, C.
,
Hansen, C.
,
Green, S.
and
Akey, T.1997: An investigation of item difficulty incorporating the
structure of listening tests: a hierarchical linear modeling analysis.
In
Huhta, A.
,
Kohonen, V.
,
Kurki-Suonio, L.
and
Luoma, S.
, editors, Current developments and alternatives in language
assessment.
Jyväskylä: University of Jyväskylä
, 151-164.
33.
McKay, P.2000:
On ESL standards for school-age learners
. Language Testing17,
185-214
.
34.
McKay, P.2001.
Why standards
? Babel36,
16-20
.
35.
NSW AMES (New South Wales Adult Migrant English Service)1998: Certificates in spoken and written English I, II, III and
IV.
Sydney: Author
.
36.
Nissan, S.
,
DeVincenzi, F.
and
K. L. Tang1996: An analysis of factors affecting the difficulty of dialogue
items in TOEFL listening comprehension.
Princeton, NJ: Educational Testing Service
.
37.
Norris, J.
,
Brown, J. D.
,
Hudson, T.
and
Yoshioka, J.1998: Designing second language performance tasks.
Honolulu: University of Hawaii Press
.
38.
Robinson, P.2001:
Task complexity, task difficulty, and task production: exploring
interactions in a componential framework
. Applied Linguistics22,
27-57
.
39.
Ross, S.2000: Individual differences and learning outcomes in the
Certificates in Spoken and Written English. In
Brindley, G.
, editor, Studies in immigrant English language assessment. Volume
1.
Sydney: National Centre for English Language
Teaching and research
, Macquarie University, 191-214.
40.
Ross, S.
and
Langille, J.1997: Negotiated discourse and interlanguage accent effects on a
second language listening test. In
Brindley, G.
and
Wigglesworth, G.
, editors, Access: issues in language test design and delivery.
Sydney: National Centre for English Language
Teaching and Research, Macquarie University
, 87-116.
41.
Rost, M.1990: Listening in language learning.
London: Longman
.
42.
Rost, M.
and
Ross, S.1991:
Learner use of strategies in interaction: typology and teachability
. Language Learning41,
235-273
.
43.
Rubin, J.1994:
A review of second language listening comprehension research
. Modern Language Journal78,
199-221
.
44.
Shohamy, E.
and
Inbar, O.1991:
Validation of listening comprehension tests: the effect of text and
question type
. Language Testing8,
23-40
.
45.
Skehan, P.1998: A cognitive approach to language learning.
Oxford: Oxford University Press
.
46.
Smith, D.2000: Rater judgements in the direct assessment of
competency-based second language writing ability. In
Brindley, G.
, editor, Studies in immigrant English language assessment. Volume
1.
Sydney: National Centre for English Language
Teaching and research, Macquarie University
, 159-189.
47.
Tauroza, S.
and
Allison, D
: 1990.
Speech rates in British English
. Applied Linguistics11,
90-105
.
48.
Wigglesworth, G.2000a: Issues in the development of oral tasks for
competency-based assessments of second language performance. In
Brindley, G.
, editor, Studies in immigrant English language assessment. Volume
1.
Sydney: National Centre for English Language
Teaching and Research
, Macquarie University, 81-124.
49.
Wigglesworth, G.2000b:
Relating research and professional development: assessment task design
. Prospect15, (3),
52-64
.
50.
Wright, B.
and
Linacre, J. M.1991: A user’s guide to Bigsteps Rasch Model computer
program. Version 2.2.
Chicago: Mesa Press
.
51.
Wu, Y.1998:
What do tests of listening comprehension test? A retrospective study of EFL
test-takers performing a multiple-choice task
. Language Testing15,
21-44
.
52.
Zhao, Y.1997:
The effects of listeners’ control of speech rate on second
language comprehension
. Applied Linguistics18,
49-68
.
