Motivated by cognitive theories of graph comprehension, this study systematically manipulated characteristics of a line graph description task in a speaking test in ways to mitigate the influence of graph familiarity, a potential source of construct-irrelevant variance. It extends Xi (2005), which found that the differences in holistic scores on graph tasks with varying characteristics, although significant, were small. Using an analytic scoring approach, it re-examined how visual chunks in line graphs and planning time influenced some specific components of examinees’ performance on line graph description tasks. The analytic dimensions examined were determined based on results of previous studies and hypotheses about the relationships among visual chunks, planning, graph familiarity, and features of elicited discourse. It was found that participants less familiar with graphs described the line graphs in a less organized fashion and that their descriptions were weaker in content. Graph familiarity thus introduced construct-irrelevant variance. However, providing planning time and using less complex graphical displays improved the fluency, organization and content of the elicited oral discourse and helped lessen the influence of graph familiarity, thus enhancing the validity of this task. The theoretical and practical implications of the findings are discussed.
Bagozzi RP and Edwards JR ( 1998). A general approach for representing constructs in organizational research. Organizational Research Methods , 1, 45-87.
2.
Bagozzi RP and Heatherton TF ( 1994). A general approach to representing multifaceted personality constructs: Application to state self-esteem. Structural Equation Modeling, 1, 35-67.
3.
Bandalos DL and Finney SJ ( 2001). Item parceling issues in structural equation modeling . In Marcoulides, G. A. and Schumacker, R. E., editors, Advanced structural equation modeling: New developments and techniques. Mahwah, NJ: Lawrence Erlbaum Associates.
4.
Bentler PM ( 2002). EQS for Windows 6.0. Encino, CA: Multivariate Software.
5.
Bentler P., Bagozzi R. and Cudeck R. ( 2001). SEM using correlation or covariance matrices. Journal of Consumer Psychology, 10, 85-87.
6.
Brown JD, Hudson T., Norris JM and Bonk W. ( 2002). Investigating second language performance assessments . Honolulu, HI: Second Language Teaching and Curriculum Center, University of Hawai‘i Press.
7.
Carswell CM, Bates JR, Pregliasco NR, Lonon A. and Urban J. ( 1998). Finding graphs useful: Linking preference to performance for one cognitive tool. Cognitive Technology, 3, 4-18.
8.
Carswell CM, Emery C. and Lonon A. ( 1993). Stimulus complexity and information integration in the spontaneous interpretation of line graphs. Applied Cognitive Psychology, 7, 341-357.
9.
Crick JE and Brennan RL ( 1983). Manual for GENOVA: A generalized analysis of variance system. (ACT Technical Bulletin No. 43). Iowa City, IA: ACT.
10.
Crookes G. ( 1989). Planning and interlanguage variation. Studies in Second Language Acquisition, 11, 367-383.
11.
Ellis R. ( 1987). Interlanguage variability in narrative discourse: Style shifting in the use of the past tense. Studies in Second Language Acquisition, 9, 1-20.
12.
ETS (Educational Testing Service) (2001-02). TSE and SPEAK band descriptor chart: TSE and SPEAK score user guide, 2001-02 edition. Princeton, NJ: Educational Testing Service. Available online at http://www.ets.org/media/tests/TSE/008659.pdf .
13.
Foster P. and Skehan P. ( 1996). The influence of planning and task type on second language performances. Studies in Second Language Acquisition , 18, 299-323.
14.
Iwashita N., McNamara T. and Elder C. ( 2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51, 401-436.
15.
Katz IR, Xi X., Kim H. and Cheng PC-H (2004). Elicited speech from graph Items on the Test of Spoken English. TOEFL Research Report No. 74. Princeton, NJ: Educational Testing Service.
16.
Leinhardt G., Zaslavsky O. and Stein MK ( 1990). Functions, graphs, and graphing: Tasks, learning, and teaching. Review of Educational Research, 60, 1-64.
17.
Little TD, Cunningham WA, Shahar G. and Widaman KF ( 2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9, 151-173.
18.
MacCallum RC,Widaman KF, Zhang S. and Hong S. ( 1999). Sample size in factor analysis. Psychological Methods, 4, 84-99.
19.
Mehnert U. ( 1998). The effects of different lengths of time for planning on second language performance. Studies in Second Language Acquisition, 20, 83-106.
20.
Ortega L. ( 1995). The effect of planning in L2 Spanish oral narratives. Research Note No. 15. Honolulu: University of Hawai’i, Second Language Teaching and Curriculum Center .
21.
Ortega L. ( 1999). Planning and focus on form in L2 oral performance . Studies in Second Language Acquisition, 21, 109-148.
22.
Peebles D., Cheng Pc-H and Shadbolt N. ( 1999). Multiple processes in graph-based reasoning. In Hahn, M. and Stoness, S. S. (Eds), Proceedings of the Twenty First Annual Conference of the Cognitive Science Society. Vancouver, British Columbia.
23.
Pinker S. ( 1990). A theory of graph comprehension. In Freedle, R. (Ed), Artificial intelligence and the future of testing (pp. 73-126). Mahwah, NJ: Erlbaum.
24.
Robinson P. ( 1995). Task complexity and second language narrative discourse . Language Learning, 45, 141-175.
25.
Robinson P. ( 2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22, 27-57.
26.
Robinson P. ( 2003). Attention and memory during SLA. In Doughty, C., and Long, M. (Eds) Handbook of research in second language acquisition (pp. 631-678). Oxford: Blackwell .
27.
Shah P., Hegarty M. and Mayer RE ( 1999). Graphs as aids to knowledge construction: Signaling techniques for guiding the process of graph comprehension. Journal of Educational Psychology, 91, 690-702.
28.
Simkin D. and Hastie R. ( 1987). An information-processing analysis of graph perception . Journal of the American Statistical Association, 82, 454-465.
29.
Skehan P. ( 1998). A cognitive approach to language learning. Oxford: Oxford University Press.
30.
Skehan P. and Foster P. ( 1997). Task type and task processing conditions as influence on foreign language performance. Language Teaching Research , 1, 185-211.
31.
Skehan P. and Foster P. ( 1999). The influence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93-120.
32.
Skehan P. and Foster P. ( 2001). Cognition and tasks. In Robinson, P. (Ed), Cognition and second language acquisition . Cambridge: Cambridge University Press.
33.
Wigglesworth G. ( 1995). The effect of planning time on oral test discourse . Paper presented at the 17th Language Testing Research Colloquium , March, Baltimore, MD.
34.
Wigglesworth G. ( 1997). An investigation of planning time and proficiency level on oral test disourse. Language Testing, 14, 85-106.
35.
Wigglesworth G. ( 2001). Influences on performance in task-based oral assessments . In Bygate, M., Skehan, P., and Swain, M. (Eds), Researching pedagogical tasks: Second language learning, teaching and testing (pp. 186-209). Harlow: Pearson Education.
36.
Xi X. ( 2005). Do visual chunks and planning impact the overall quality of oral descriptions of graphs? Language Testing, 22, 463-508.
