Vocabulary-Size VIBE Test
Hananto
PelitaHarapanUniversity
Abstract
The original study was the development of a computer-based lexical tool named Vocabulary Item Bank of English (VIBE). The software was designed for both learning and testing purposes. The focus, however, was on the use of the tool for measurement purposes, i.e. as a vocabulary-size test. This article focuses on the development and validation of the paper-and-pencil test forms generated by the computer-based VIBE test. This article is divided into two parts. The first part, in this volume, focuses on the development of the paper-and-pencil test forms, and the second part (next volume) will discuss the validation of the test forms.The aim of this paper is to introduce an innovative vocabulary-size test as an alternative to the existing vocabulary-size tests and to invite Indonesian EFL teachers to try it out.
Key words: vocabulary-size test, vocabulary-depth test, validation
Introduction
After a lengthy period of being preoccupied with the development of grammatical competence, there has been a renewed recognition of the importance vocabulary studies for second/foreign language (L2/FL) learners for the last fifteen years or more. Vocabulary was neglected and was seen just one of the many components that contribute to the learner’s overall competence. However, there seems to be a growing recognition among L2/FL teachers and applied linguistic researchers of the central role of vocabulary in language study. Words are regarded as the basic building blocks of language, the units of meaning from which larger structures such as sentences, paragraphs and whole texts are created.
Vocabulary Knowledge and Vocabulary Tests
Vocabulary knowledge has been approached and defined differently by different researchers. Vocabulary knowledge can be approached as the knowledge of discrete word items independent of context in which they appear. This approachto vocabulary knowledge is known as the ‘trait’ view (Read and Chapelle 2001). In this perspective, Nation’s (1990, 2001) lexical knowledge definition has been widely used. In Nation’s definition, vocabulary knowledge is divided into ‘subknowledges’ such as knowledge of the spoken and written form, morphological knowledge, knowledge of word meaning, collocational and grammatical knowledge, etc.
This knowledge component model has been commonly used in vocabulary tests. However, no vocabulary test currently available is able to tap into all subknowledges of lexical knowledge. Some vocabulary testsmeasure just one of the subknowledges (e.g. Laufer & Nation 1999 and Nation 1983) and referred to as vocabulary-size or breadth tests. Other tests attempt to measure several subknowledges at the same time (e.g. Wesche and Paribakht 1996) and known as vocabulary-depth tests.
There is clearly a trade-off between how much can be elicited about each word and how many different words can be included. Vocabulary-depth tests better represent the vocabulary knowledge of the test-takers than the vocabulary-size tests. However, vocabulary-depth tests usually have one limitation: the number of lexical items is relatively small because of the time needed to complete the test.On the other hand, vocabulary-size tests try to cover as many words as possible within the allotted time. There is a deliberate attempt to test only the decontextualized word form in a written form, a single meaning of word, and the connection of the two. The test is deliberately limited to the most important aspect of vocabulary knowledge – the form/meaning connection. Although vocabulary-size tests may seem superficial, “they can give a more representative picture of the overall state of the learner’s vocabulary than in-depth probe of a limited number of words” (Read 2000:115).
This study adheres to the trait view of vocabulary and focuses on vocabulary-size tests because of the following advantages. They can predict success in reading, writing, general language proficiency, and academic achievement (Laufer 1997). Additionally, Vocabulary-size tests offer more efficient placement in language teaching program (Meara 1987). Finally, vocabulary-size tests also offer relatively simple and efficient research instruments to measure vocabulary growth after certain treatments.
Vocabulary-Size Studies in Indonesia
Several researchers have conducted vocabulary-size studies in Indonesia by administering vocabulary-size tests to certain groups of Indonesian learners.
The first and widely cited vocabulary-size study in Indonesiais Quinn’s (1968) study at Satya Wacana Christian University (UKSW) in Salatiga, Central Java. He administered a translation test to his first semester students and found that on average the students had mastered less than 1000 of the most frequent English words after six years of study in high school.
Kweldju has conducted several nation-wide vocabulary studies in Indonesia. For example, Kweldju (1997) conducted a nation-wide vocabulary-size study at 15 teachers' colleges in Sumatra, Java, Kalimantan, Sulawesi, Bali and Nusa Tenggara. She discovered that English department students or prospective English teachers only had 4664 base-words or 2800 family words, a size right at the threshold level of reading shorter academic texts.
Nurweni has also conducted several vocabulary-size studies in Indonesia. For example,Nurweni and Read (1999)investigated the vocabulary-size (and depth) of first-year Indonesian university students to find out whether they have enough English words for their academic reading. They got a mean vocabulary size of 1226 words and concluded that the students’ total vocabulary size was very fairly small although they had spent six years studying English in junior and senior high school.
The most recent vocabulary study is perhaps Suteja (2007). She conducted a vocabulary-size (and depth) study with Indonesian national plus high school (year 10) students. She found that the subjects had an average vocabulary size of 5.000 words which were enough to be able to read authentic texts required for their study. It should be noted that the result is an exception rather than a norm because the subjects in this study did not represent typical Indonesian students.
Underlying Assumptions
Vocabulary-size test developers have some theoretical assumptions on which the test is based. The underlying assumptions of vocabulary-size tests are best represented by Laufer and Goldstein (2004:409), as follows:
- The most important component of word knowledge is the ability to establish the link between word form and word meaning.
- Knowledge of meaning is not an all-or-nothing phenomenon. The form-meaning link in the mental lexicon can have four degrees of strength: active recall (which is the strongest and presupposes the other three degrees of strength), passive recall, active recognition, and passive recognition, which is the weakest.
- Knowing many words (units of meaning) is more important than knowing few words in depth. Hence, a good vocabulary test should test how many words are known. In other words, it should try to provide a picture of the learner’s overall vocabulary.
This study follows the assumptions that underlie the principle for the VIBE test developed in this study.The test, however, does not cover the four degrees of strength of word knowledge but only two: active recall and passive recognition. Additionally, the test format is innovative, as discussed below.
Test Specifications
The paper-and-pencil vocabulary-size test forms developed in this study aregenerated by a computer-based Vocabulary Item Bank of English (VIBE). The target words in VIBE were taken from West’s (1953) General Service List (GSL) and Coxhead’s (2000) Academic Word List (AWL). Except some short words such as the article a and the personal pronoun I, almost all of the 2000 target words from theGLS and 570 target words from the AWLareincluded in the VIBE. The whole target words are divided into three word levels: (1) the 1000-word level, (2) the 2000-word level, and (3) the academic-word level. The first two word levels are taken from the GSL and the third word level is takenfrom the AWL.
The target items were taken from the base lists (BASEWRD1.txt, BASEWRD2.txt, and BASEWRD3.txt) of the Range Program( three base word lists contain the word families of the target words, i.e. the base form and derived forms. The following a sample entry taken from the BASEWRD1.txt:
ACCOUNT 0
ACCOUNTED 0
ACCOUNTING 0
ACCOUNTS 0
ACCOUNTANT 0
ACCOUNTANTS 0
ACCOUNTANCY 0
ACCOUNTABLE 0
ACCOUNTABILITY 0
The head words are selected as target items to represent the word family. In the example above, ACCOUNT and ACCOUNTABLE are taken as the target words representing the two different word families for the reason that the two words have different meanings despite their similar word forms.In the VIBE, the whole target items are kept in several small files. Each file consisted of approximately 50 target items, as illustrated in Table 1 below.
Table 1: VIBE Files
Files / Items1 / 1 – 50
2 / 51 – 100
3 / 101 – 150
4 / 151 – 200
… / …
The test items can be selected either randomly or sequentially by the computer-based VIBE test.In random selection, the first question can be any item from 1 to 50 from file 1. Similarly, the second question can be any item 51 to 100 from file 2, and so on. In sequential selection, however, the first item is item 1 from file 1, the second item is item 52 of file 2, the third item is item 1003 from file 3, etc. When VIBE reaches the last file of the selected word-level, it comes back to the first file again and repeats the cycle.
The VIBE test forms have double tasks: active recall and passive recognition.In active recall, test-takers have to supply the target word form (i.e. English). The clues are the first and the last few letters. Only the middle letter is deleted. For example, if the target word form is teacher, the prompt is tea _her. In passive recognition, the test-takers have to select a number to indicate the meaning of the target words.
Because of the double tasks, the VIBE test tasks are designed to be as easy as possible so that more test items could be administered within a manageable test time. They include the presentation of target items in isolation without any sentence-context and the use of objective test formats (i.e. gap-filling and matching formats).
The current development of VIBE focuses on the bilingual (English – Indonesian) version. Below are the instruction andten sample-items of the paper-and-pencil test form from three different word-frequency levels: 1000-word, 2000-word, and academic-word level.
Figure 1: VIBE Test Instruction
Isi SATU HURUF yang hilang dan ANGKA yang menunjukkan artinya.Contoh Soal: / Contoh Jawaban:
1 / buku / = ho _ se / (…) / = ho u se (3)
2 / kucing / = b _ ok / (…) / = b o ok (1)
3 / rumah / = c _ t / (…) / = c a t (2)
Figure 2: Sample items from the 1000-word level
1000-Word Level1 / (satu) yang lain / = mo _ her / (…)
2 / dia (perempuan) / = po _ nt / (…)
3 / harga / = s _ e / (…)
4 / ibu, induk / = t _ y / (…)
5 / kawan, teman, sahabat / = ano _ her / (…)
6 / mencoba; percobaan, usaha / = con _ inue / (…)
7 / meneruskan, melanjutkan / = fr _ end / (…)
8 / terlambat, terbaru; almarhum / = l _ te / (…)
9 / tidak pernah / = ne _ er / (…)
10 / ujung, titik, angka; menunjuk / = pr _ ce / (…)
Figure 3: Sample items from the 2000-word level
2000-Word Level1 / bangku, meja tulis / = ab _ oad / (…)
2 / kapur / = au _ umn / (…)
3 / luar negeri / = bo _ row / (…)
4 / membeku / = ch _ lk / (…)
5 / meminjam, pinjam / = c _ py / (…)
6 / menyalin; salinan / = d _ sk / (…)
7 / musim gugur / = esse _ tial / (…)
8 / mutlak, penting, mendasar / = fr _ eze / (…)
9 / rahang / = h _ ap / (…)
10 / tumpukan; menimbun / = j _ w / (…)
Figure 4: Sample items from the academic-word level
Academic-Word Level1 / bersama, saling / = aba _ don / (…)
2 / bersamaan, berbarengan / = auth _ rity / (…)
3 / jeda waktu, jarak waktu / = conc _ rrent / (…)
4 / karenanya, sebab itu / = de _ ote / (…)
5 / memutuskan, memecahkan / = esta _ lish / (…)
6 / mendirikan / = he _ ce / (…)
7 / meninggalkan / = int _ rval / (…)
8 / merupakan, menunjukkan / = mu _ ual / (…)
9 / tepat, seksama / = pre _ ise / (…)
10 / wewenang / = res _ lve / (…)
Figure 5 illustrates the complete paper-and-pencil VIBE test form. It has three parts and each part represents one word-frequency level. Each part has four blocks (40 items). Therefore, the complete test forms consist of 120 items which can fit nicely on a legal-size paper. The complete test form cannot be written here due to the paper-size limitation of this journal but can be sent to those who want it by sending an e-mail to .
Figure 5: The VIBE Paper-and-Pencil Test Form Layout
Instruction and Sample Items1000-Word Level / 2000-Word Level
10 items / 10 items
10 items / 10 items
10 items / 10 items
10 items / 10 items
Academic-Word Level
10 items / 10 items
10 items / 10 items
It should be noted that the three word-frequency levels do not have to be administered together. A teacher may select a certain word level(s) to estimate the vocabulary-size of certain groups of Indonesian EFL learners. Table 4 provides some guidelines to use certain word level to certain proficiency level students:
Table 2: The Target Users of the VIBE Test
Word-Frequency Levels / Proficiency Levels1000-word / Elementary (junior high school)
2000-word / Intermediate (senior high school)
Academic-word / Advanced (university)
Scoring
Because the test has a double task, it can be scored in several different ways. The following three scoring systems are recommended:
(a) by looking at the word form only (word-from score): One point is awarded if the word form is correct, even though the word meaning is wrong.
(b) by looking at the word meaning only (word-meaning score): One point is given if the word meaning is right, even though the word form is incorrect
(c) by looking at both the word form and the word meaning: One point is awarded only if both the word form and the word meaning are right.
Table 6 illustrates the two parts of the paper-and-pencil VIBE test. Depending on one’s interest, the scoring can be based on the word-form only (by ignoring the word-meaning part), or based on the word-meaning only (by disregarding the word-form part), or based on both word-form and word-meaning. Each scoring system reflects the different part of word knowledge component.
Table 3: Word-Form and Word Meaning Part
Word-Form / Word-Meaning= mo _ her / (…)
= po _ nt / (…)
= s _ e / (…)
= t _ y / (…)
= ano _ her / (…)
The criterion mastery of 90%is recommended. Test-takers who do not reach the criterion mastery are likely to have problems in their English studies. They should pay a serious attention to the high-frequency words because of their importance. One approach is direct vocabulary learning, especially the high-frequency words from the GLS.
Advantages and Limitations of the VIBE Test
The VIBE test offers several advantages. First and the most obvious advantage of the VIBE test is its large number of test items (120 questions) that can be put in a legal-size paper. Second, it measures several aspects of word knowledge at the same time. Third, the test can be objectively and quickly scored. Fourth, it is very economical to administer the VIBE test in terms of cost and time. Finally, the VIBE test forms offer relatively simple and efficient research instruments.
The limitation of the VIBE test is that it deals with word-form and word-meaning link only. That is why researchers who are interested in other aspects of lexical knowledge will have to administer another vocabulary-depth test. As discussed before, some lexical researchers combine both vocabulary-size and vocabulary-depth tests to get more comprehensive picture of learners’ vocabulary knowledge. Those who want to do research related to vocabulary-size and/or vocabulary-depth studies are advised to read Suteja’s article (2007).
REFERENCES
Coxhead, A. (2000). "A New Academic Word List." TESOL Quarterly 34: 213-38.
Kweldju, S. (1997). English Department Students' Vocabulary Size and the Development of a Model of Extensive Reading with Individualized Vocabulary Learning. Singapore, SEAMO-Regional Language Centre
Laufer, B. and Z. Goldstein (2004). "Testing Vocabulary Knowledge: Size, Strength, and Computer Adaptiveness." Language Learning Vol. 54, No. 3: 399-436.
Laufer, B. and K. Shmueli (1997). "Memorizing new words: Does teaching have anything to do with it?" RELC Journal 28(1): 89-108.
Laufer, B. and P. Nation (1999). "A Vocabulary-Size Test of Controlled Productive Ability." Language Testing 16/1: 33-51.
Meara, P. and B. Buxton (1987). "An Alternative to Multiple Choice Vocabulary Tests." Language Testing 4.2(142-151).
Nation, I. S. P. (1983). "Testing and teaching vocabulary." Guidelines (RELC supplement) 5: 12-25.
Nation, I. S. P. (1990). Teaching and Learning Vocabulary. New York, Heinle and Heinle.
Nation, I. S. P. (2001). Learning Vocabulary in Another Language. New York, CambridgeUniversity Press.
Quinn, G. (1968). The English Vocabulary of some IndonesianUniversity Entrants: A Report on a Survey Conducted at the ChristianUniversity and TeacherCollege of Satya Wacana. Salatiga, Indonesia., IKIP Kristen Satya Wacana.
Nurweni, A. and J. Read (1999). "The English Vocabulary Knowledge of IndonesianUniversity Students." English for Specific Purposes 18: 161-175.