Do test design and uses influence test preparation? Testing a model of washback with Structural Equation Modeling

Abstract

This study introduces Expectancy-value motivation theory to explain the paths of influences from perceptions of test design and uses to test preparation as a special case of washback on learning. Based on this theory, two conceptual models were proposed and tested via Structural Equation Modeling. Data collection involved over 870 test takers of College English Test Band 4 in China. A perception of assessment questionnaire was given at the beginning of a 10-week preparation period; a test preparation questionnaire was given eight weeks later. Test takers who endorsed high-stakes, instrumental test uses as the primary purpose for taking the test tended to value test taking; test takers who perceived test design positively tended to attach high importance to test taking and appeared more confident. Furthermore, higher endorsed task value and higher expectation of test success jointly contributed to greater engagement in test preparation. Knowledge of the test was also related to increased self-regulation in test preparation and more practice of test-taking skills.

Keywords

Expectancy-value theory perceptions of assessment Structural Equation Modeling test use washback

Get full access to this article

View all access options for this article.

References

Alderson

J. C.

Hamp-Lyons

(1996). TOEFL preparation courses: A study of washback. Language Testing, 13(3), 280–297.

Alderson

J. C.

Wall

(1993). Does washback exist? Applied Linguistics, 14(2), 115–129.

Alwin

D. F.

Hauser

R. M.

(1975). The decomposition of effects in path analysis. American Sociological Review, 40(1), 37–47.

Andrews

Fullilove

Wong

(2002). Targeting washback: A case study. System, 30(2), 207–223.

Arbuckle

(2006). Amos 7.0 users’ guide. Chicago: SPSS.

Atkinson

J. W.

(1964). An introduction to motivation. Oxford: Van Nostrand.

Bandura

(1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122–147.

Baron

Kenny

(1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182.

Bentler

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.

10.

Bong

(2001). Role of self-efficacy and task-value in predicting college students’ course performance and future enrollment intentions. Contemporary Educational Psychology, 26(4), 553–570.

11.

Cheng

L. Y.

(2005). Changing language teaching through language testing: A washback study (Vol. 21). Cambridge: Cambridge University Press.

12.

Eccles

J. S.

Adler

T. F.

Futterman

Goff

S. B.

Kaczala

C. M.

Meece

Midgley

(1983). Expectancies, values, and academic behaviors. In Spence

J. T.

(Ed.), Achievement and achievement motives (pp. 75–146). San Francisco: W. H. Freeman.

13.

Eccles

J. S.

Wigfield

(2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132.

14.

Entwistle

N. J.

(1992). Experiences of understanding in revising for degree examinations. Learning and Instruction, 2(1), 1–22.

15.

Fredericksen

J. R.

Collins

(1989). A systems approach to educational testing. Educational Researcher, 18(9), 27–32.

16.

Frederiksen

(1984). The real test bias: Influences of testing on teaching and learning. American Psychologist, 39(3), 193–202.

17.

Fulcher

(2000). The ‘communicative’ legacy in language testing. System, 28(4), 483–497.

18.

Gist

M. E.

Mitchell

T. R.

(1992). Self-efficacy: A theoretical analysis of its determinants and malleability. The Academy of Management Review, 17(2), 183–211.

19.

Gosa

C. M. C.

(2004). Investigating washback: A case study using student diaries. Unpublished PhD thesis, Lancaster University, Lancaster.

20.

Graham

J. W.

(2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576.

21.

Green

(2007). Washback to learning outcomes: A comparative study of IELTS preparation and university pre-sessional language courses. Assessment in Education: Principles, Policy & Practice, 14(1), 75–97.

22.

X. D.

(2005). Positive or negative? An empirical study of CET washback on College English teaching and learning in China. Unpublished PhD thesis, Shanghai Jiao Tong University, Shanghai.

23.

Hoyle

R. H.

(1995). Structural equation modeling: Concepts, issues and applications (1st ed.). Thousand Oaks, CA: SAGE Publications.

24.

Jacobs

J. E.

Eccles

J. S.

(2000). Parents, task values, and real-life achievement-related choices. In Sansone

Harackiewicz

J. M.

(Eds.), Intrinsic Motivation (pp. 405–439). San Diego, CA: Academic Press.

25.

Jin

(2006). Improving test validity and washback: Proposed washback study on CET4/6. Foreign Language World, 6, 65–73.

26.

Jöreskog

Sörbom

(1981). LISREL V user’s guide. Chicago: National Educational Resources.

27.

Karabenick

S. A.

(2004). Perceived achievement goal structure and college student help seeking. Journal of Educational Psychology, 96(3), 569–581.

28.

MacKinnon

D. P.

Fairchild

A. J.

Fritz

M. S.

(2007). Mediation analysis. Annual Review of Psychology, 58, 593–615.

29.

Messick

(1998). Test validity: A matter of consequence. Social Indicators Research, 45(1), 35–44.

30.

Pajares

(1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543–578.

31.

Pintrich

P. R.

(1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (Technical Report 91-B-004). Michigan: The Regents of the University of Michigan.

32.

Pintrich

P. R.

(1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459–470.

33.

Pintrich

P. R.

(2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92(3), 544–555.

34.

Preacher

K. J.

Kelley

(2011). Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods, 16(2), 1–23.

35.

Prodromou

(1995). The backwash effect: From testing to teaching. ELT Journal, 49(1), 13–25.

36.

L. X.

(2005). Stakeholders’ conflicting aims undermine the washback function of a high-stakes test. Language Testing, 22(2), 142–173.

37.

Schunk

D. H.

(1991). Self-efficacy and academic motivation. Educational Psychologist, 26(3), 207–231.

38.

Scouller

(1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(4), 453–472.

39.

Shohamy

(2001). The power of test: A critical perspective of the use of language tests. London: Pearson.

40.

Steiger

(1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180.

41.

Struyven

Dochy

Janssens

(2005). Students’ perceptions about evaluation and assessment in higher education: A review. Assessment & Evaluation in Higher Education, 30(4), 325–341.

42.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). Boston: Pearson, Allyn & Bacon.

43.

Tucker

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.

44.

Wall

Alderson

J. C.

(1993). Examining washback: The Sri Lanka impact study. Language Testing, 10(1), 41–69.

45.

Watanabe

(1996). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing 13, 318–333.

46.

Wenden

(1998). Metacognitive knowledge and language learning. Applied Linguistics, 19(4), 515–537.

47.

Wigfield

(1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational Psychology Review, 6(1), 49–78.

48.

Wigfield

Eccles

J. S.

(2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81.

49.

Xie

(2011). Is test takers’ perception of assessment demand related to construct validity? International Journal of Testing, 11(4), 324–348.