Partial dictation is a measure of EFL listening proficiency that can be easily constructed, administered, and scored by EFL teachers. However, it is controversial whether this form of test measures lower-order abilities exclusively or involves both lower- and higher-order abilities. In order to answer this question, a study was designed to examine the difference between partial dictation and test forms believed to measure more higher-order abilities. In a series of confirmatory factor analyses, the simplex, second-order, and bi-factor models were fitted to the scores of 367 college-level EFL learners in China in a listening test composed of partial dictation, gap-filling and constructed response tasks. The bi-factor model was identified as the best-fitting and this supports the view that partial dictation measures the same construct as test forms believed to measure more higher-order abilities. Concomitant statistical analyses also showed that the partial dictation tasks were suited to the ability level of the test takers and had high internal consistency.
AkaikeH. (1987). Factor analysis and AIC. Psychometrika, 52(3), 317–332.
2.
AldersonJ. C. (1978). The use of cloze procedure with native and non-native speakers of English. Ph.D. dissertation, University of Edinburgh.
3.
AldersonJ. C. (1984). Reading in a foreign language: A reading problem or a language problem? In AldersonJ. C.UrquhartA. H. (Eds.), Reading in a foreign language (pp. 1–27). London: Longman.
4.
AldersonJ. C. (2000). Assessing reading. Cambridge: Cambridge University Press.
5.
AldersonJ. C. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.
6.
AldersonJ. C.BachmanL. F. (2001). Series editors’ preface. In BuckG., Assessing listening (pp. x–xi). Cambridge: Cambridge University Press.
7.
AldersonJ. C.ClaphamC.WallD. (1995). Language test construction and evaluation. New York: Cambridge University Press.
8.
AllowayT. P.GathercoleS. E.WillisC.AdamsA. M. (2004). A structural analysis of working memory and related cognitive skills in young children. Journal of Experimental Child Psychology, 87, 85–106.
9.
AndersonA.LynchT. (1988). Listening. Oxford: Oxford University Press.
10.
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
11.
BachmanL. F. (2004). Statistical analyses for language assessment. Cambridge: Cambridge University Press.
12.
BachmanL. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2(1), 1–34.
13.
BachmanL. F.PalmerA. S. (1996). Language testing in practice. Oxford: Oxford University Press.
14.
BachmanL. F.PalmerA. S. (2010). Language assessment in practice. Oxford: Oxford University Press.
15.
BernhardtE. B.KamilM. L. (1995). Interpreting relationships between L1 and L2 reading: Consolidating the linguistic interdependent hypotheses. Applied Linguistics, 16, 15–34.
16.
BighamC. C. (1932). A study of error. New York: College Entrance Examination Board.
17.
BloomB. S.EngelhartM. B.FurstE. J.HillW. H.KrathwohlD. R. (1956). Taxonomy of educational objectives, Handbook I: Cognitive domain. New York: McKay.
18.
BossersB. (1991). On thresholds, ceilings, and short-circuits: The relation between L1 reading, L2 reading and L2 knowledge. AILA Review, 8, 45–60.
19.
BrownH. D. (2004). Language assessment: Principles and classroom practices. White Plains, NY: Pearson Education.
20.
BrownR. W. (1915). How the French boy learns to write. Cambridge, MA: Harvard University Press.
21.
BuckG. (1991). The testing of listening comprehension: An introspective study. Language Testing, 8(1), 67–91.
22.
BuckG. (1992). Listening comprehension: Construct validity and trait characteristics. Language Learning, 42(3), 313–357.
23.
BuckG. (1994). The appropriacy of psychometric measurement models for testing second language listening comprehension. Language Testing, 11(2), 145–170.
24.
BuckG. (2001). Assessing listening. Cambridge: Cambridge University Press.
25.
CarrellP. L. (1991). Second language reading: Reading ability or language proficiency?Applied Linguistics, 12, 223–242.
26.
CohenA. D. (1980). Testing language ability in the classroom. Rowley, MA: Newbury House.
27.
CohenA. D. (1994). Assessing language ability in the classroom (2nd ed.). Boston, MA: Heinle & Heinle.
28.
ConiamD. (1998). Interactive evaluation of listening comprehension: How the context may help. Computer Assisted Language Learning, 11(1), 35–53.
29.
ConiamD. (2009). Investigating the quality of teacher-produced tests for EFL students and the impact of training in test development principles on improving test quality. System, 37(2), 226–242.
30.
DavisP.RinvolucriM. (1988). Dictation: New methods, new possibilities. Cambridge: Cambridge University Press.
31.
ElmanJ. L.McClellandJ. L. (1983). Speech perception as a cognitive process: The interactive activation model. ICS Report No. 8302, Institute for Cognitive Science, University of California, San Diego, La Jolla, CA 92093.
32.
ForeroC. G.Maydeu-OlivaresA.Gallardo-PujolD. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling, 16, 625–641.
GuttmanL. (1954). A new approach to factor analysis: The radex. In LazarsfeldP. F. (Ed.), Mathematical thinking in the social sciences. Glencoe, IL: The Free Press.
35.
HarmerJ. (2007). The practice of English language teaching (4th ed.). Harlow: Longman.
36.
HenningG.GaryN.GaryJ. (1983). Listening recall: A listening comprehension test for low proficiency learners. System, 11, 287–293.
37.
HillP. W.McGawB. (1981). Testing the simplex assumption underlying Bloom’s taxonomy. American Educational Research Journal, 18, 93–101.
38.
HolzingerK. J.SwinefordF. (1937). The bi-factor method. Psychometrika, 2, 41–54.
39.
HolzingerK. J.SwinefordF. (1939). A study in factor analysis: The stability of a bi-factor solution. Supplementary Educational Monographs, 48, 81–91.
40.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
41.
HughesA. (2003). Testing for language teachers (2nd ed.). Cambridge and New York: Cambridge University Press.
42.
IrvineP.AtaiP.OllerJ. W.Jr. (1974). Cloze, dictation, and the Test of English as a Foreign Language. Language Learning, 24, 245–252.
43.
JohanssonS. (1973). The partial dictation as a test of foreign language proficiency. Swedish-English Contrastive Studies, Report No. 3.
44.
JonesR. L. (1977). Testing: A vital connection. In PhillipsJ. K. (Ed.) The language connection: From the classroom to the world (pp. 237–265). Skokie, IL: National Textbook Company.
45.
JöreskogK. G. (1970). Estimation and testing of simplex models. The British Journal of Mathematical and Statistical Psychology, 23, 121–145.
46.
JöreskogK. G.SörbomD. (1996). LISREL 8: Structural equation modeling with the SIMPLIS command language. Chicago, IL: Scientific Software International.
47.
JöreskogK. G.SörbomD. (2006). LISREL 8.8 for Windows [Computer software]. Lincolnwood, IL: Scientific Software International.
48.
KaneM. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.
49.
LadoR. (1961). Language testing: The construction and use of language tests. New York: McGraw-Hill.
50.
LewkowiczJ. A. (1991). Testing listening comprehension: A new approach?Hong Kong Papers in Linguistics and Language Teaching, 14, 25–31.
51.
MarkhamP. L. (1988). Gender and the perceived expertness of the speaker as factors in ESL listening recall. TESOL Quarterly, 22(3), 397–406.
52.
McClellandJ. L.St. JohnM.TarabanR. (1989). Sentence comprehension: A parallel distributed processing approach. Language and Cognitive Processes, 4(3), SI287–SI335.
53.
McDonaldR. P. (1965). Difficulty factors and nonlinear factor analysis. British Journal of Mathematical and Statistical Psychology, 18, 11–23.
54.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan.
55.
NeisserU. (1967). Cognitive psychology. New York: Appleton-Century-Crofts.
56.
OllerJ. W.Jr. (1971). Dictation as a device for testing foreign language proficiency. English Language Teaching, 15, 254–259.
57.
OllerJ. W.Jr. (1979). Language tests at school: A pragmatic approach. London: Longman.
58.
OllerJ. W.Jr.ChenL. (2007). Episodic organization in discourse and valid measurement in the sciences, 14, 127–144.
59.
OllerJ. W.Jr.StreiffV. (1975). Dictation: A test of grammar-based expectancies. English Language Teaching, 30, 25–36.
60.
PASW Statistics. (2009). PASW Statistics Version 17.0. Chicago, IL: SPSS.
61.
ReiseS. P.MorizotJ.HaysR. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19–31.
62.
RichardsJ. C. (1990). The language teaching matrix. Cambridge: Cambridge University Press.
63.
SatorraA.BentlerP. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514.
64.
StansfieldC. W. (1985). A history of dictation in foreign language teaching and testing. The Modern Language Journal, 69(2), 121–128.
65.
SteigerJ. H.ShapiroA.BrowneM. W. (1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50, 253–264.
66.
SternH. H. (1983). Fundamental concepts of language teaching. Oxford: Oxford University Press.
67.
TempletonH. (1977). A new technique for measuring listening comprehension. ELT Journal, 31(4), 292–299.
68.
TreimanR. (2001). Linguistics and reading. In AronoffM.Rees-MillerJ. (Eds.), Handbook of linguistics (pp. 664–672). Oxford: Blackwell.
69.
TylerL. K.FrauenfelderU. H. (1987). The process of spoken word recognition: An introduction. In TylerL. K.FrauenfelderU. H. (Eds), Spoken word recognition (pp. 1–20). Cambridge, MA: The MIT Press.
70.
WeirC. J. (1993). Understanding and developing language tests. New York: Prentice Hall.
71.
WeirC. J. (2005). Language testing and validation: An evidence-based approach. Basingstoke: Palgrave Macmillan.
72.
WertsC. E.LinnR. L.JöreskogK. G. (1977). A simplex model for analyzing academic growth. Educational and Psychological Measurement, 37, 745–756.
73.
WirthR. J.EdwardsM. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12(1), 58–79.
