Abstract
The current study investigated 29 first-year Japanese university students’ usage of an online vocabulary notebook that automatically searched eight different word and phrase lists to provide students with reference information about their self-selected vocabulary. Over the course of a 14-week period, participants read English books and articles and added self-selected vocabulary that they wanted to learn to individual online vocabulary notebooks. The notebooks immediately and automatically showed whether the vocabulary appeared on any of eight different reference vocabulary lists. The distribution of participant vocabulary across lists was examined and participant surveys and interviews were conducted to understand usage of the system. Analysis of participant vocabulary, learner surveys, and interviews indicated that participants selected relatively high percentages of standardized-test related vocabulary (i.e. TOEIC and TOEFL), chose individual vocabulary items over multi-word expressions, and studied vocabulary items even when they did not appear on any reference lists. Learner surveys and interview results suggested that use of the system directly or indirectly influenced half of the participants’ decisions about which vocabulary to include in their notebooks, though participants reported that it did not affect their choice of reading material.
Keywords
I Introduction
Foreign language learners face a challenge trying to develop their knowledge of the L2 lexicon: They know that learning more vocabulary is valuable, but with the tens of thousands of words and phrases in any language, language learners must prioritize learning some vocabulary items over others. One response to this need to prioritize vocabulary for study has been the development of English word lists (e.g. Brezina & Gablasova, 2013; Coxhead, 2000; Gardner & Davies, 2013) and multi-word expression lists (Garnier & Schmitt, 2015; Martinez & Schmitt, 2012; Simpson-Vlach, & Ellis, 2010) based on large computerized corpora. In creating such lists, researchers have cited ‘pedagogic . . . purposes’ (Brezina & Gablasova, 2013, p. 18), ‘teaching’ (Coxhead, 2000, p. 227), the importance of academic vocabulary in ‘ “gate-keeping” tests’ (Gardner & Davies, 2013, p. 305), ‘pedagogic utility’ (Martinez & Schmitt, 2012, p. 302), or the usefulness of lists for ‘a wide range of English language teaching professionals and students’ (Garnier & Schmitt, 2015, p. 16). Sorell (2013) has asserted that ‘probably the foremost application of core vocabulary lists is the prioritizing of vocabulary teaching and learning in foreign languages’ (p. 61). Suggestions to use such lists pedagogically have appeared elsewhere as well (e.g. Coxhead, 2011; Folse, 2011; Schmitt & Schmitt, 2014).
There is evidence that some core word lists have been integrated with teaching materials (Coxhead, 2011; Lessard-Clouston, 2010), but it is not clear that they are being used pedagogically. Almost no accounts of pedagogical applications of word or phrase lists have been published. Youngblood and Folse (2017) have commented that ‘now more than ever, English language teachers’ classroom-based research on the application of corpus-based vocabulary lists is needed to fill a gap in our understanding of real vocabulary development in the L2 classroom’ (p. 12). The purpose of the current study was to begin filling the gap and enable English language learners to gain a practical learning benefit from word and phrase lists.
II Background
1 The development and pedagogical use of word and phrase lists
Especially in the past 20 years, a multitude of new word and phrase lists have appeared in publications (Lessard-Clouston, 2013; Nation, 2016). Some lists have been developed for academic fields, such as science word lists (e.g. Coxhead & Hirsh, 2007), engineering word lists (e.g. Ward, 2009), business word lists (e.g. Konstantakis, 2007), medical word lists (e.g. Wang, Liang, & Ge, 2008), and theology word lists (e.g. Lessard-Clouston, 2010). In addition to genre-specific lists, most commercial standardized English proficiency test preparation materials offer word or multi-word expression lists to study, and some such lists have been corpus-based and published for public use (e.g. Browne & Culligan, 2016). Finally, less genre-specific lists have been created from corpora of diverse selections of academic texts (e.g. Coxhead, 2000; Gardner & Davies, 2013) or general texts (e.g. Brezina & Gablasova, 2013).
Vocabulary lists have been used pedagogically mainly in the development of reference and teaching materials. The lists that have been used most widely are the Academic Word List (AWL) (Coxhead, 2000) and the General Service List (GSL) (West, 1953). The AWL has been used in many textbooks, dictionaries, and on websites (Coxhead, 2011), including being the primary study focus in certain texts (e.g. Schmitt & Schmitt, 2005; Wells & Valcourt, 2010; Zwier, 2013). It seems likely the AWL has been the most widely applied reference vocabulary list in modern pedagogical materials. The GSL has also been used or adapted in numerous English language learning textbooks (e.g. Butler, 2010; Dingle, 2008; Jeffries & Mikulecky, 2009), and Coxhead (2000) has reported that previous word lists such as the American University Word List (Praninskas, 1972) and the University Word List (Xue & Nation, 1984) have also appeared in pedagogical materials.
Despite their use in pedagogical materials, there are few reports of direct pedagogical application of published word and phrase lists in classrooms. Brown and Perry (1991) conducted a study in which the American University Word List (AUWL) (Praninskas, 1972) had a peripheral role in pedagogy. The study’s primary purpose was to compare word list study with other vocabulary learning strategies (e.g. keyword techniques). Thirty-one of the 40 words used in the study were chosen from the AUWL. Thus, students were learning words from a word list even though teaching of the AUWL was not the main focus of the study. In another study, Shillaw (1995) reported using the GSL (West, 1953) with a group of Japanese university students over the course of a semester. Students studied GSL words while concurrently reading Cry freedom in class and later watching the film. While the paper was anecdotal rather than empirical, Shillaw suggested that students learned more than 300 words over the course of a semester. In a third study, Van Benthuysen (2003) used the University Word List (UWL) (Xue & Nation, 1984) to determine whether explicit word list study would lead to gains on the Vocabulary Levels Test (VLT) (Nation, 1990), a test which evaluates learners’ vocabulary knowledge at different frequency levels: 2,000 words, 3,000 words, 5,000 words, and a UWL section. Learners were given the VLT at the beginning and the end of an eight-month period. During the eight-month period, the 14 study participants were given one of the 11 sections of the UWL to study every two weeks. At the end of every two-week period, they were given a multiple-choice test, mainly to motivate them to study the words (i.e. not as a research instrument). The second administration of the VLT showed the strongest gains in the UWL section, as compared with the 2,000-, 3,000-, and 5,000-frequency level words. With a maximum score of 18 on any of the sections, scores in the UWL section improved 3.9 points while improvements on other sections were all fewer than two points.
More recently, McDonough, Neumann, and Hubert-Smith (2018) reported on a study in which AWL word families were taught through textbooks, and learners’ accuracy using AWL words in argumentative essays was assessed. Participants were students in an English for Academic Purposes (EAP) course at a Canadian university. Study materials used in the course included excerpts from an academic English textbook (Williams, 2012) and a book specifically designed to teach the AWL (Schmitt & Schmitt, 2005). Students’ final argumentative essays for the course were collected as a written corpus of data to analyse for the accuracy of AWL word use. The corpus contained a total of 202 AWL word families, comprising 5,390 tokens. Sixty of those word families (4,434 tokens), each with at least 20 tokens in the corpus, were analysed for accuracy. Results showed that 67% of the 4,434 tokens were used accurately. The authors identified collocational (18%) or morphosyntactic errors (11%) as the most common types of errors. Among implications of the study, the authors suggested that EAP instruction should include a specific focus on collocation information to help learners use EAP lexis more accurately.
2 Vocabulary learning and tools
Notwithstanding the four studies reviewed above, a large gap remains between the number of reported pedagogical uses of word and phrase lists and the amount of research carried out creating such lists. The reason word and phrase lists have not often been utilized pedagogically might be that they are not easily accessible as pedagogical tools. First, many teachers are likely not aware of available lists in the first place (Nation, 2016). Even if teachers are aware of lists, choosing which list to teach is an additional hurdle. There are not clear criteria for choosing one list over another. A popular list like the AWL (Coxhead, 2000) might seem like a safe choice, but there is no easy way to determine if that list is most appropriate for a given group of learners. In fact, multiple lists could be relevant for any group of learners, but directly teaching most vocabulary lists (i.e. thousands of words and phrases) would be impractical in most instructed language settings (Nation, 2016). Teachers could create their own lists, but many lack the time and expertise to do so. Thus, it is not surprising that current vocabulary list pedagogy has primarily been confined to using textbooks that already include the lists.
An additional concern with direct instruction of vocabulary lists is that learners have less opportunity to regulate their own strategic vocabulary study. L2 vocabulary research has consistently shown that self-regulated learners who use metacognitive strategies learn vocabulary more effectively than those who do not (e.g. Kojic-Sabo & Lightbown, 1999; Mizumoto & Takeuchi, 2009; Winke & Abbuhl, 2007). Tseng and Schmitt (2008) have posited the idea of ‘self-motivated experts in vocabulary learning’ (p. 388, emphasis original), which ‘stresses the need for learners to develop self-regulating capacity by proactively generating personal control of vocabulary learning’ (p. 389). When learners make individual choices about vocabulary study, they exercise control over their learning and can focus on their individual needs. However, the common problem with learners selecting their own words has been that they will do so without regard to the frequency or usefulness of vocabulary (e.g. McCrostie, 2007), which is precisely the problem word and phrase lists are intended to address.
Vocabulary learning tools offer one method for letting learners select their own vocabulary while still allowing them to benefit from published word and phrase lists. Tools have long been used to mediate vocabulary learning. The design and use of vocabulary learning tools such as dictionaries (e.g. Chan, 2011; Kirkness, 2004; Laufer & Kimmel, 1997; Schofield, 1982), flashcards (e.g. Nakata, 2011; Seibert Hanson & Brown, 2020), language corpora and concordancers (e.g. Cobb, 1997; Li, 2017; Philip, 2010; Poole, 2012; ) and vocabulary notebooks (e.g. Dubiner, 2017; Fowle, 2002; McCrostie, 2007; Schmitt & Schmitt, 1995) have received plentiful attention, and technological advances have afforded the development of new multi-functional digital learning tools which integrate multiple traditional tools (e.g. a dictionary with a flash card function). In the same way, technology can be used to make word and phrase lists accessible while still allowing learners to control their own vocabulary study through a technologically enhanced form of the traditional vocabulary notebook.
3 Vocabulary notebooks
Vocabulary notebooks have been widely regarded as an effective tool for learning new words in a language (e.g. Folse, 2004; McCarthy, 1990; Nation, 2013; Schmitt & Schmitt, 1995). Two vocabulary notebook characteristics that have been discussed in the literature are the notebook format and the choice of vocabulary items.
Some researchers have provided learners with guidance about notebook design while still allowing learners to choose the final format (e.g. Baierschmidt, 2011; Dubiner, 2017; McCrostie, 2007). In contrast, Fowle (2002), regarded vocabulary notebooks as personal dictionaries and had learners structure them in that way. Other reports of vocabulary notebook use do not detail the exact format of the learners’ notebooks (e.g. Bozkurt, 2007; Hirschel & Fritz, 2013; Walters & Bozkurt, 2009). Left to their own individually devised methods, learners seem to make diverse format choices (Leeke & Shaw, 2000).
In terms of which vocabulary items to study, some vocabulary notebooks have included items pre-selected by instructors (e.g. Bozkurt, 2007; Hirschel & Fritz, 2013; Walters & Bozkurt, 2009), but others have involved learners exclusively or primarily self-selecting their vocabulary (e.g. Choi & Ma, 2015; Dubiner, 2017; Fowle, 2002; McCrostie, 2007). Recent research has suggested that learners prefer self-selection of vocabulary over teacher-mandated study (Reynolds & Shih, 2019). Vocabulary learning autonomy can provide learners a motivational benefit (Barker, 2007; Hunt & Beglar, 2005) while allowing them to self-regulate. Nevertheless, even self-regulated vocabulary learners likely benefit from assistance deciding which vocabulary items are most valuable to learn, because ‘in students’ minds, all words are created equal’ (McCrostie, 2007, p. 254). Many researchers have advocated for teacher guidance and support to help learners make more informed choices about what vocabulary should be studied (Barker, 2007; Folse, 2004, 2011; Reynolds & Shih, 2019; Schmitt & Schmitt, 1995). McCrostie identified five areas in which learners’ self-selected vocabulary for notebooks were less than optimal. These areas included learners choosing words primarily from textbooks and written sources, preferences for vocabulary items of certain parts of speech (i.e. nouns), difficulties writing quality example sentences, and – most relevant to word and phrase lists – difficulty identifying collocations and ‘difficulty determining the frequency or usefulness of words’ (p. 254).
4 The research questions for the study
The literature reviewed above shows that despite considerable effort expended to create pedagogically useful English word and multi-word expression lists, there has been little direct pedagogical use of such lists in classrooms. Also, though researchers have suggested how teachers can guide learners toward making informed choices about self-selected vocabulary, there are no investigations of how vocabulary word and phrase lists can be integrated into such guidance.
The current study describes use of an online vocabulary notebook system through which learners received instant, automated reference information about their self-selected vocabulary choices. The purpose of the study was twofold. A first purpose was to understand what percentage of participant vocabulary items appeared on eight different reference word and phrase lists and what portions of the reference and phrase lists appeared in participants’ self-selected vocabulary list. A second purpose was to understand whether the practice of using the vocabulary notebook affected participants’ vocabulary selection and sources of vocabulary (i.e. reading material). The three research questions for the study were as follows:
To what extent do participants’ self-selected words and phrases appear on reference vocabulary lists?
To what extent are reference vocabulary list contents covered by participants’ self-selected vocabulary?
Does the introduction of a vocabulary reference system affect participants’ choice of reading material and/or choice of vocabulary to study?
III Method
1 Participants
There were 29 participants in the study, all of whom were either 18 or 19 years old (M = 18.72). All participants were first-year female Japanese university EFL learners majoring in English. Participants were a convenience sample, invited to participate in the study by virtue of being enrolled in two different sections of a course focused on reading and writing. They were intermediate-proficiency learners with TOEIC (Test of English for International Communication) scores between 500–700, roughly equivalent to CEFR (Common European Framework of Reference for Languages) level B1 (Educational Testing Service, 2016).
The study was explained to participants orally and in writing in their L1, and all participants gave written consent to use their anonymous vocabulary data and learner survey comments in the study. Six participants who participated in interviews signed additional L1 consent forms for being audio recorded. At the beginning of the study, there were 30 participants. However, one learner did not fully complete the vocabulary notebook activity researched in the study, leaving 29 participants with complete data sets.
2 Study design
The study took place during a 15-week semester. Actual data collection occurred over the course of 14 weeks (i.e. starting from the second week through the final week). Data included participant vocabulary notebooks, written open-ended surveys, and transcripts from semi-structured interviews.
a The online vocabulary notebook system
In brief, the online vocabulary notebook system functioned as follows:
Learners read self-selected reading and kept track of new English vocabulary words and phrases from their reading.
Learners input the new English vocabulary words and phrases into the online vocabulary notebook along with L1 Japanese definitions and any other notes about the vocabulary item.
The vocabulary notebook automatically and instantly searched eight different reference word and phrase lists and showed learners on which lists their vocabulary term appeared.
Each of the above three steps in the process are further explained below.
b Self-selected reading: The vocabulary source
Participants’ weekly activities in and outside class time included completing self-selected reading. The self-selected reading was the source of vocabulary for the online vocabulary notebook. Self-selected reading consisted of any English text chosen by participants. Participants were encouraged to choose texts with no more than five unknown words per page (excluding proper nouns), a heuristic derived from Hu and Nation’s (2000) suggestion that readers should understand 98% of vocabulary for satisfactory reading comprehension. Nevertheless, participants were given leeway to make individual choices on reading material that was more difficult than the guideline, as motivation to read was considered to be an important factor that could lead students toward more difficult, yet satisfying, reading material. Most students chose short novels (e.g. Little Women, Peter Rabbit), online news articles (e.g. The Japan Times) or non-fiction texts (e.g. Who was Mother Teresa, How Starbucks Saved My Life).
Participants brought their self-selected reading materials to class each week to read for 10 minutes of in-class silent reading time. They also read twice outside of class each week. The two periods of reading outside class had to equal at least one hour (e.g. 30 minutes twice a week, or 20 minutes + 40 minutes, etc.). After each reading session, students completed a reading log questionnaire online in which they recorded what they read, how many words they read, how long they read, and a short comment about the reading. They were also directed to write down new vocabulary, which they added to their online vocabulary notebooks at least once a week.
c Online vocabulary notebook
Participant online vocabulary notebooks were individual sheets on a Google Sheets workbook that learners could access anytime with an Internet connection. Participants recorded new words and phrases that they wanted to learn from their self-selected reading on their individual online vocabulary notebook sheets.
A screenshot of an example vocabulary notebook is shown in Figure 1. As seen in Figure 1, participants wrote their new words/phrases (column A), L1 Japanese meanings (column B), reading sources where they found the words/phrases (column C), and optional notes about the words/phrases (column D). When participants entered a new word or phrase on their individual sheet, they immediately saw whether it appeared in any of the eight reference lists (column F). Technically speaking, formulas were written in the Google Sheets vocabulary notebook so the eight reference lists were searched when participants entered a new word or phrase. This was accomplished by having hidden helper columns in the vocabulary notebook for each of the eight lists. Each of the helper columns contained an vlookup function which made the list name appear if the word or phrase was in the list. Then, in column F, a concatenate statement was used to join any of the list names that appeared in the eight hidden helper columns.
Screenshot of the online vocabulary notebook.
Eight different reference vocabulary lists were used in the study. The foci of the lists (i.e. academic vocabulary, general vocabulary) were explained orally and in writing when learners initially started using the vocabulary notebooks. Table 1 gives an overview of the reference lists used, a brief description, the reference list pseudonyms that learners saw appearing in the vocabulary notebook, and the number of vocabulary items on each list.
The eight reference vocabulary lists used in the study.
The eight word and phrase reference lists were chosen for having filled at least one of two criteria. The first criterion was that a word or phrase list be relatively well known in vocabulary research. The requirement for the first criterion was that the list appeared in a peer-reviewed article in a highly ranked journal related to English language or pedagogy. Lists which fulfilled the first criterion were the AWL (Coxhead, 2000), AVL (Academic Vocabulary List; Gardner & Davies, 2013), the New-GSL (Brezina & Gablasova, 2013) the PHRASE List (Martinez & Schmitt, 2012) and the PHaVE List (Garnier & Schmitt, 2015). For lists that did not fulfill the first criterion, the second criterion was that a list be relevant to the participants’ English learning needs. All participants were required to take the TOEFL (Test of English as a foreign language) and TOEIC tests multiple times during university study, and test outcomes could affect their opportunities for study abroad and future employment prospects. Thus, the TOEIC Service List (Browne & Culligan, 2016), the TOEIC-2, and the TOEFL lists were included.
The Phrase list used for the study was a mixture of different lists. The Phrase list was originally going to be composed of the PHRASE List (Martinez & Schmitt, 2012) and the PHaVE List (Garnier & Schmitt, 2015). However, despite the research integrity involved in creating both lists, the PHRASE List comprises only 505 phrasal expressions and the PHaVE list comprises only 150 phrasal verbs. Thus, the two lists were combined with other phrasal verb lists freely available on the internet. Therefore, the Phrase list used in the current study is by no means a reliable frequency list. However, I vetted phrases subjectively for inclusion in the list, and subjective judgment can be valuable in constructing vocabulary lists (Sorell, 2013).
Learners were required to add 10 words or phrases per week to their individual vocabulary notebooks over the course of 14 weeks, though they were encouraged to add more. The vocabulary notebook was graded for completeness (i.e. 10 words per week and having definitions and the vocabulary source), and learners also took unannounced, 15-item, graded vocabulary quizzes on their individual words four times during the semester to encourage them to learn the vocabulary. Individual vocabulary quizzes were created through a system of randomizing individual lists and sending the top fifteen terms to a preformatted quiz template in a Microsoft Word mail merge document. The vocabulary notebook itself was a relatively small percentage (i.e. 10%) of the overall work and grade in the class, as were the vocabulary quizzes (i.e. 5%).
d Learner survey and semi-structured interviews
Two sources of data used to understand the effects the online vocabulary notebooks had on participants’ reading and vocabulary choice were an open-ended learner survey for all participants and semi-structured interviews held with six participants toward the end of the semester.
During the twelfth week of the study, participants completed a short, anonymous, open-ended survey in Japanese concerning their experience using the vocabulary notebook system. The open-ended questions were designed to assess how the vocabulary notebook’s automated reference information affected learners’ choice of reading material and their selection of vocabulary for their vocabulary notebooks. The questions asked were as follows:
Did the vocabulary notebook system affect the kinds of books you chose to read? (e.g. fiction, non-fiction, academic, novels, online, etc.)
Did the vocabulary notebook system affect the words and phrases you decided to put in your notebook?
What did you think of the vocabulary notebook system?
In addition to the survey, volunteers for semi-structured interviews were solicited. Six of the 29 participants volunteered to participate in semi-structured interviews. The questions asked during the interviews were identical to those asked in the survey. The purpose of the semi-structured interviews was to gain more detail about individual experiences. Interviews lasted an average of nine minutes each. Interviews were initiated in participants’ L1, though most participants chose to use L2 English as well. Interviews were recorded and transcribed (using pseudonyms). Transcription conventions, adapted from Mackey and Gass (2005), were as follows: ? rising intonation , pause of less than 1 second with nonfinal intonation . . . pause of 1–2 seconds { } translation
IV Results
A total of 7,627 words or phrases were self-selected by the 29 participants over the course of 14 weeks. There were 4,809 unique words or phrases. Of these, there were 3,246 words or phrases that occurred only one time, and there were 1,563 that had duplicates, meaning they appeared in more than one participant’s vocabulary notebook. The number of vocabulary items included in individual participants’ notebooks ranged from 127 to 483 (M = 263, SD = 85.27).
1 Participant vocabulary on reference lists
The first research question concerning the extent to which participant vocabulary appeared on reference lists was answered by calculating the percentage of total learner vocabulary items appearing on each of the reference lists. The results appear in Table 2; this table shows that the COCA-Academic list contained the most participant vocabulary items at 72.49%. The other two lists containing relatively high percentages of participant vocabulary were the TOEIC-2 list (43.73%) and the TOEFL list (34.08%). The New-GSL and the AVL had roughly equal percentages of participant vocabulary at a little more than 12 percent each. Finally, the AWL, the TOEIC Service List, and the Phrase list each contained fewer than 10% of participant vocabulary items. Meanwhile, more than one-fifth of the vocabulary did not appear on any of the reference lists.
The percentages of self-selected vocabulary items on reference lists.
The longest lists (i.e. the COCA-Academic, the TOEIC-2, and the TOEFL) contained the most participant vocabulary, and the shortest list (i.e. TOEIC Service List) contained relatively few participant items. However, there was some differentiation among lists of relatively equal length (i.e. the AWL, the AVL, and the Phrase list). Especially, very few participant items appeared on the Phrase list. Though learners were encouraged to include multi-word expressions as an essential component of vocabulary study (Dóczi & Kormos, 2016; Schmitt, 2010; Siyanova-Chanturia, 2017; Wood, 2002), examination of participants’ vocabulary notebooks revealed that more than 90% of vocabulary items were individual words. Even for the phrases that participants did include, only 38% appeared on reference lists as compared with individual words, 81% of which appeared on reference lists.
The percentage of offlist words – more than one-fifth of participant items – was higher than I expected given the inclusion of the long COCA-Academic list. Further examination of participants’ vocabulary showed that 60% of offlist items were uncommon or technical terms (e.g. slacker, purr, apothecary), 25% were multi-word expressions, and 15% were inflected or derivative word forms that did not appear on reference lists even though their roots words did (e.g. merrily, antibiotics, untie).
2 List coverage by participant vocabulary
The second research question concerning how much of each reference list was covered by participant vocabulary items was answered by reporting the percentage of each reference list’s items that were contained in the total list of participant vocabulary items. Table 3 shows the percentages. The TOEFL list had the highest percentage of coverage by participant-selected vocabulary. That is, more than half of the TOEFL list (56.02%) was contained in participants’ self-selected vocabulary. The TOEIC-2 list was next highest at 34.35%, followed by the TOEIC Service List (30.98%), the New-GSL (26.65%), the AVL (20.54%), the COCA-Academic list (18.79%), the AWL (14.63%), and the Phrase list (5.64%).
The results in Table 3 show that the extent to which reference list vocabulary was covered by participant vocabulary had a different hierarchy than the percentage of participants’ vocabulary appearing on each list (i.e. Table 2). For example, the COCA-Academic list was shown to contain the highest percentage of participant vocabulary items in Table 2, yet given the size of the list, the ratio of participant vocabulary on the COCA-Academic list was, as expected, lower than most other lists. In contrast, the TOEIC Service List, the shortest of all lists, was the third most covered reference list.
The percentage of reference lists covered by participant vocabulary.
The first two research questions were posed to analyse participant vocabulary from two perspectives because the two questions in combination give a fuller understanding of the relevance of the reference lists for the participants of the study. Although the first research question analysis results were unsurprising – the long lists generally contained the most participant vocabulary items – the results for the second research question showed that participants had relatively high quantities of TOEFL and TOEIC vocabulary items, as the TOEFL, TOEIC-2, and TOEIC Service List contained the highest percentages of participant vocabulary items. Thus, the combined results suggest that the test-related lists might have been more relevant to the study participants.
3 Effects on reading and vocabulary choice
The third research question concerned how the online vocabulary notebook system affected learners’ selection of reading material and vocabulary. The question was answered according to the results of the open-ended survey and six participant interviews.
The first open-ended survey question asked whether participants’ self-selected reading material was influenced by the vocabulary notebook system. None of the participants reported that their reading selection was influenced through use of the online vocabulary notebook system. Interviews also suggested that participants did not choose books based on the vocabulary notebook system. For example, in the following exchange, Kumiko commented that she read whatever types of books she liked because she could learn vocabulary for the TOEIC in other classes.
1 Kumiko: Um, I read a lots of fantasy books, so, it’s not useful for talking, but, it’s 2 fun to know 3 R: Uhhuh, cause that’s what I was going to ask you, like, the one question 4 here is, does it, kinds of books, you choose, you’re just choosing what you 5 like? 6 Kumiko: Yeah 7 R: You’re not trying to get, vocabulary, like academic? or TOEIC? no 8 Kumiko: No 9 R: No, 10 Kumiko: No, like I can, learn, like TOEIC words by, like TOEIC class 11 R: Uhhuh 12 Kumiko: In the TOEIC class so, I decided to read what I like, and, what I interested 13 in
Other interview participants similarly expressed that they chose books that they wanted to read, not because of the type of vocabulary they thought they would encounter.
The second survey question asked whether participants’ decisions about which vocabulary to include were influenced by the vocabulary notebook system. Nine out of 29 participants directly reported that the vocabulary system had affected their choice of what vocabulary items to include in their vocabulary notebooks. For example, three wrote that their vocabulary choices were affected by the number of lists on which their words appeared. One participant wrote: ‘I erased words that were only on 1 or 0 lists.’ Another participant wrote: ‘If it showed 0 and I thought I would not use the word, then I deleted it.’ Two other participants cited specific words that they deleted because they did not appear on lists, such as ‘vicar’, ‘parrish’, and ‘wheelbarrow’. The other four participants expressed less specific responses such as, ‘I sometimes didn’t include words if they were not on the lists.’ Of the remaining 20 participants, seven explicitly wrote that the vocabulary notebook system did not influence their choice. However, of the other 13 participants, nine commented on some form of personal screening of vocabulary without mentioning influence of the lists. For example, one participant wrote that ‘I didn’t put many specialized vocabulary in the notebook’, while another wrote ‘I only deleted phrases that I really thought were not necessary.’ Such comments could not be interpreted as direct influence of the notebook system, but they were evidence that participants were screening vocabulary. It is possible that the use of the vocabulary system might have inspired some participants to think more carefully about which vocabulary terms they were selecting for inclusion, even if the reference lists in the vocabulary notebook were not a direct influence.
During interviews, I elicited more specific details about how the vocabulary notebook system influenced which words participants included in the notebook. Among the six interviewees, three (Anika, Satomi, and Fufumi) said the vocabulary notebook system influenced which vocabulary they included, two (Kumiko and Takako) said it did not, and one (Madoka) implied that she was not influenced by the system, but did state that she screened vocabulary. For example, Madoka said she screened words as follows: 1 Madoka: I write, write on the paper all, isshuukan no tango, and, I, look 2 up for dictionary, and, I didn’t write, ah, ah, nani yarou . . . nanka, 3 hougen, toka, nanka, hougen toka, nanka, muzukashii 1 I write, write on the paper all, {a week’s words}, and, I, look 2 up for dictionary, and, I didn’t write, ah, ah, {what is it . . . like, 3 dialect, or, like, dialect or, like, difficult} 4 R: like a dialect, hougen 4 like a dialect, {dialect} 5 Madoka: dialect, or, specific words, 6 R: uh huh, 7 Madoka ha, kaiteinai 7 {those, I didn’t write} 8 R: uh huh, 9 Madoka nihongo demo tsukawahen yatsu ha 9 {things I wouldn’t use in Japanese}
In the transcript, Madoka suggested that she used Japanese as her main guide about whether to include new vocabulary terms in her vocabulary notebook. Thus, she was screening vocabulary to some extent, but not by using the reference information generated in the vocabulary notebook. In contrast, another participant, Anika, described using a combination of the vocabulary system reference information, Japanese, and her own feelings about the vocabulary item.
1 Anika: at first, I keep, nanka ‘Looking for Alaska’, saigo no hen ha, 1 at first, I keep, {like} ‘Looking for Alaska’, {toward the end} 2 R: tokidoki you skipped, and not keep, 2 {sometimes} you skipped, and not keep, 3 Anika: uh huh, 4 R: How do you decide, to skip, or not skip 5 Anika: I, write, in my paper, every word, zenbu no kotoba wo kaite, nihongo wo 6 shirabete kara, kore, shite, sukunakattara, keshita 5 I, write, in my paper, every word, {I write all of the terms, and after I find 6 out the Japanese, this, I do, if there are few, I delete it} 7 R: a, kore sukunakattara, 7 {ah, if this was few} 8 Anika: ato nanka, sukina tango ka dou ka, nanka, jibun no kimochi 8 {then like, if it was a word I liked or not, like, my own feeling} 9 R: uh huh, 10 Anika: nanka kono tango chotto, nanka, amari awanai shi, I don’t know 11 how to explain, kibun de erandeita 10 {a little like this word, like, it doesn’t fit and}, I don’t know 11 how to explain, {I chose by my mood}
In the transcript, Anika described a process in which she wrote all vocabulary items down on paper (i.e., line 5), found Japanese meanings for items, and finally deleted words if they appeared on few reference lists (i.e., line 6). However, she added that her personal feelings about vocabulary items also influenced whether she included them or not on her personal vocabulary list.
In summary, according to surveys and interviews, the vocabulary system did not influence participants’ reading choices, and it only directly influenced about one-third of the participants’ decisions about which vocabulary they studied for the class. However, the interview data showed that other factors, such as L1 definitions and personal feelings, were also important for participants when they screened vocabulary for inclusion in their personal vocabulary notebooks.
V Discussion
Data from learner surveys and interviews indicated that the vocabulary notebook system did not affect participants’ reading choices, and fewer than half of the participants mentioned screening their vocabulary based on the reference data available in the vocabulary notebook system. Meanwhile, the analysis of participants’ vocabulary showed that TOEFL and TOEIC reference lists contained relatively high percentages of learner vocabulary. To be clear, there is no comparison data with which to define what would be high or low representation on the reference lists. Nevertheless, I had some expectation that lists that had been developed through rigorous corpus analysis procedures for pedagogical use (e.g. the AWL, the New-GSL, the AVL) would contain higher percentages of vocabulary items chosen by participants. However, it is possible that learners already knew high percentages of vocabulary items that appeared on those lists. In other words, the proficiency of the participants might be at a level in which the TOEFL and TOEIC lists presented vocabulary that was more relevant or new to learners than the vocabulary on the other lists. Further investigation into participants’ knowledge of individual lists could be useful to determine if proficiency level and vocabulary knowledge were significant factors.
Another observation that was salient from the results for the first two research questions was that participants recorded relatively few multi-word expressions in their vocabulary notebooks, and the multi-word expressions that were recorded generally did not appear in the reference Phrase list. The relative lack of participant-selected multi-word expressions, the fact that they did not appear in reference lists, and the difficulty of learning multi-word expressions versus individual words (Peters, 2014) suggests that multi-word expressions might require special attention in the vocabulary notebooks. Spending class time discussing the value of learning multi-word expressions and discussing those chosen by learners could be worthwhile, especially when multi-word expression usage contrasts with the L1 translation (Laufer & Girsai, 2008). Also, separate logs for individual words and multi-word expressions could be kept so that the two could be analysed separately (He & Godfroid, 2018). Finally, more multi-word expression lists could be included for reference (e.g. Ackermann & Chen, 2013).
One subjective impression from the participant vocabulary data was that there was a high percentage of offlist words. As stated at the beginning of this article, the prioritization of vocabulary for study is the main purpose behind many published word and phrase lists, and the vocabulary notebook in this study was designed to alert participants to the relative frequency and usefulness of the vocabulary they were self-selecting. If ‘frequency of occurrence should be the guiding force . . . regarding what should be taught to L2 learners and when’ (Siyanova-Chanturia & Webb, 2016, p. 230), then more effort to reduce offlist words might be warranted. For example, a strong directive to avoid adverbial forms ending in -ly might result in fewer offlist words. Expanding the number and breadth of reference lists or encouraging learners to read non-fiction instead of fiction might also reduce the number of offlist words. However, there are contrary arguments. Limiting learners’ choice of inflected and derivative forms can make the self-selection process more cumbersome, and there are certainly inflected and derivative forms that are frequent and worth learning (Gardner & Davies, 2013). Expanding the number or breadth of reference lists would be possible, but should be done thoughtfully, not just to reduce offlist words. Finally, having learners reading non-fiction over fiction reduces learner autonomy and does not necessarily solve the offlist concern. Perhaps the best approach is to discuss the value of offlist words directly with learners. As other researchers have pointed out, low-frequency vocabulary items might seem unnecessary from a teacher’s standpoint but still be valued by learners (Barker, 2007).
VI Limitations and conclusions
The purpose of this study was to understand how an online vocabulary notebook with automated referencing of word and phrase lists would function and support the participants’ choice of English language vocabulary for study. However, the study did have limitations. First, reading materials and vocabulary were selected by participants, and the reading material was only monitored in class. Because participants chose their own vocabulary, it was difficult to assess to what degree participants already knew the vocabulary items they chose. There were four quizzes given to participants over the course of the semester, but they were not used as a source of data for this study because they were short quizzes that would not necessarily reveal participants’ vocabulary knowledge. More frequent or comprehensive vocabulary assessments during the term could provide a useful perspective on participants’ knowledge of their individual vocabulary notebook items. Finally, the study does not lend itself to generalization for other learner groups. The sample size was small, and the proficiency and age range of participants was narrow. The vocabulary notebook system could lead to significantly different results with a different group of learners.
Despite the lack of generalizability of participants’ performance, the current study was innovative. Research has shown that L2 learners are not always able to determine by themselves the relative frequency and usefulness of vocabulary for study (McCrostie, 2007). The current study was the first investigation of an system that offered automated feedback to learners about the potential value of their self-selected vocabulary based on well-researched word and phrase lists developed for pedagogical purposes. Furthermore, the flexibility and scalability of the notebook system should be apparent. In different contexts, the online vocabulary notebook can be set up with different reference lists according to the learning context (e.g. ESP contexts, low-proficiency learners, etc.). The system can also be used with large numbers of students without the need for significant modification. In short, the online vocabulary notebook is an adaptable system which any teacher or learner can manipulate to support vocabulary learning goals.
The vocabulary notebook system had only a moderate influence on which vocabulary items participants chose to study. However, unless a researcher or teacher had a specific aim to alter learner choice, how learners change or do not change their behavior need not be considered either positive or negative. In the current study, participants were given autonomy choosing vocabulary. The main goal was to offer participants valuable information gained from English language research that they would be unlikely to find on their own and allow participants to utilize the enhanced information in order to make self-regulated decisions about which words and phrases they would study.
The vocabulary notebook system in this study was an effort to bridge the gap between the substantial work corpus analysts have done producing English language word and phrase lists and their pedagogical application in classroom settings. It is hoped that this research inspires further efforts to build useful pedagogical tools on the strong foundation of current English language vocabulary research.
