This study, conducted by two researchers who were also multiple-choice question (MCQ) test item writers at a private English-medium university in an English as a foreign language (EFL) context, was designed to shed light on the factors that influence test-takers’ perceptions of difficulty in English for academic purposes (EAP) vocabulary, with the aim of improving test writers’ judgments on difficulty. The research consisted of a survey of 588 test-takers, followed by a focus group interview, aimed at investigating the relative influences of test-taker factors and word factors on difficulty perceptions. Results reveal a complex interaction of factors influencing perceived difficulty dominated by the educational, and particularly, the social context. Factors traditionally associated with vocabulary difficulty, such as abstractness and word length, appeared to have little influence. The researchers concluded that rather than basing their intuitions regarding vocabulary difficulty on language-lesson input or surface features of words, EAP vocabulary test writers need a clear understanding of test-takers’ difficulty perceptions, and how these emerge from interactions between academic, social and linguistic factors. As a basis for EAP vocabulary item writer training, four main implications are drawn, related to test-takers’ social and educational background, field of study, the features of academic words, and the test itself.
AndersonN. J.BachmanL. FPerkinsK.CohenA. (1991). An exploratory study of construct validity of a reading comprehension test: Triangulation of data sources. Language Testing, 8(1), 41–66.
2.
BachmanL. F. (2003). Fundamental considerations in language testing (6th ed.). Oxford: Oxford University Press.
3.
BaddeleyA.GathercoleS.PapagnoC. (1998).The phonological loop as a learning device. Psychological Review, 105(1), 158–173.
4.
BakerT. L. (1994). Doing social research (2nd ed.). New York: McGraw-Hill.
5.
BonaccioS.ReeveC. L. (2010).The nature and relative importance of students’ perceptions of the sources of test anxiety. Learning and individual differences, 20(6), 617–625.
6.
BradshawJ. (1990). Test-takers’ reactions to a placement test. Language Testing, 7(1), 13–30.
7.
BrownA. (1993). The role of test-taker feedback in the test development process: Test-takers’ reactions to a tape-mediated test of proficiency in spoken Japanese. Language Testing, 10(3), 277–301.
ClassonD. L.DormodyT. J. (1999). Analyzing data measured by individual Likert-Type items. Journal of Agricultural Education, 35(4), 31–35.
10.
CohenA. D. (2006). The coming of age of research on test-taking strategies. Language Assessment Quarterly, 3(4), 307–331.
11.
ConnellL.LynottD. (2012). Strength of perceptual experience predicts word processing performance better than concreteness or imageability. Cognition, 125(3), 452–465.
12.
CorsonD. (1997). The learning and use of academic English words. Language Learning, 47(4), 671–718.
13.
CoxheadA. (2000). A new academic word list. TESOL Quarterly, 34(2), 213–238.
14.
De GrootA. M. B.KeijzerR. (2000). What is hard to learn is easy to forget: The roles of word concreteness, cognate status, and word frequency in foreign language learning and forgetting. Language Learning, 50(1),1–56.
ElliottM. (2013).Test-taker characteristics. In GeranpayehA.TaylorL. (Eds.), Examining listening: Research and practice in assessing second language listening. Studies in Language Testing, 35 (pp. 36–76). Cambridge: Cambridge University Press.
17.
EllisN.BeatonA. (1993). Psycholinguistic determinants of Foreign Language vocabulary learning. Language Learning, 43(4), 559–617.
18.
EllisR.ShintaniN. (2014). Exploring language pedagogy through second language acquisition research. London: Routledge.
19.
FalchikovN. (2005). Improving assessment through student involvement: Practical solutions for aiding learning in higher and further education. London: Routledge Falmer.
20.
GeorgeD.MalleryP. (2003). SPSS for Windows step by step: A simple guide and reference, 11.0 update (4th ed.). Boston, MA: Allyn and Bacon.
21.
GernsbacherM. A. (1984). Resolving 20 years of inconsistent interactions between lexical familiarity, and orthography, concreteness, and polysemy. Journal of Experimental Psychology: General, 113(2), 256–281.
22.
Hansen EdwardsJ. G. (2008). Social factors and variation in production in L2 phonology. In Hansen EdwardsJ. G.ZampiniM. L. (Eds.). Phonology and second language acquisition (pp. 251–282). Amsterdam: John Benjamins.
23.
HarrellM. C.BradleyM. A. (2009). Data collection methods: Semi-structured interviews and focus groups. Santa Monica, CA: National Defense Research Institute.
24.
HuhtaA.KalajaP.Pitkanen-HuhtaA. (2006). Discursive construction of a high-stakes test: The many faces of a test-taker. Language Testing, 23(3), 326–350.
25.
HulstijnJ. H. (2003). Incidental and intentional learning. In DoughtyC. J.LongM. H. (Eds.), The handbook of second language acquisition (pp. 349–381). Oxford: Blackwell.
26.
HylandK.TseP. (2007). Is there an “Academic Vocabulary?”TESOL Quarterly, 41(2), 235–253.
27.
IwashitaN.McNamaraT.ElderC. (2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information processing approach to task design. Language Learning, 21(3), 401–436.
28.
KimH.ElderC. (2014). Interrogating the construct of aviation English: Feedback from test-takers in Korea. Language Testing, 32(2), 129–149.
29.
LauferB.GoldsteinZ. (2004). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399–436.
30.
LiuJ. Y.ChangY. J.YangF. Y.SunY. C. (2011). Is what I need what I want? Reconceptualising college students’ needs in English courses for general and specific/academic purposes. Journal of English for Academic Purposes, 10(4), 271–228.
31.
MartinK. I.EllisN. C. (2012). The roles of phonological short-term memory and working memory in L2 grammar and vocabulary learning. Studies in Second Language Acquisition, 34, 379–413.
32.
MiltonJ. (2009). Measuring second language vocabulary acquisition. Bristol, UK: Multilingual matters.
33.
MurphyJ. M. (2004). Attending to word-stress while learning new vocabulary. English for Specific Purposes Journal, 23(1), 67–83.
34.
O’SullivanB. (2000).Towards a model of performance in oral language testing. Unpublished PhD thesis. University of Reading, UK.
35.
PaquotM. (2010). Academic vocabulary in learner writing: From extraction to analysis. London: Continuum International.
36.
ReadJ. (2000). Assessing vocabulary. Cambridge: Cambridge University Press.
ScarcellaR. C.ZimmermanC. B. (2005). Cognates, cognition, and writing: An investigation of the use of cognates by university second language learners. In TylerA. E.TakadaM.KimY.MarinovaD. (Eds.), Language in use: Cognitive and discourse perspectives on language and language learning (pp.123–136). Washington, DC: Georgetown University Press.
39.
SchmittN.ZimmermanC. B. (2002). Derivative word forms: What do learners know?TESOL Quarterly, 36(2), 145–173.
40.
ShohamyE. (2001). Democratic assessment as an alternative. Language Testing, 18(4), 373–391.
41.
Van de WateringG.Van der RijtJ. (2006). Teachers’ and students’ perceptions of assessments: A review and a study into the ability and accuracy of estimating the difficulty levels of assessment items. Educational Research Review, 1(2), 133–147.
42.
Wang TzuK.NationP. (2004). Word meaning in academic English: Homography in the academic word list. Applied Linguistics, 25(3), 291–314.
43.
WaringR.NationP. (2004). Reading and incidental vocabulary learning. In AlbrechtsenD.HaastrupK.HenriksenB. (Eds.), Writing and vocabulary in foreign language acquisition (pp. 97–110). Copenhagen: Museum Tusculanum Press.
44.
WebbR. (2007). Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Applied Linguistics, 28(1), 46–65.
45.
WescheM. B.ParibakhtT. S. (2009). Lexical inferencing in a first and second language: Cross linguistic dimensions. Bristol, UK: Multilingual Matters.
46.
XioaY.CarlessD. R. (2013). Illustrating students’ perceptions of English language assessment: Voices from China. RELC Journal, 44(3), 319–340.
