Reconciling accountability and development needs in heritage language education: A communication challenge for the evaluation consultant

Abstract

The paper offers a retrospective evaluation of recent evaluative studies of bilingual programs in the Australian state of Victoria, in an attempt to determine how successfully the evaluation process met the dual criteria of external accountability and development. The programs in question were located in primary or secondary government schools and involved partial immersion in a heritage language. Data for the paper are drawn from the following: (a) the consultant's recollections of the evaluation context and process, and (b) the evaluative reports relating to three different programs (Vietnamese—English, Chinese—English and Arabic—English respectively). In hindsight it appears that the effectiveness of each evaluation may have depended in part on the degree of fit between the school and the consultant's views about the function of the evaluation initiative, as well as on her ability to communicate findings in terms which were both academically defensible and meaningful for teachers and program administrators. While the task of bridging the gap between the accountability and ameliorative functions of each evaluation was challenging for all parties (and possibly exacerbated by linguistic and cultural divides), it is argued that the former is not necessarily at the expense of the latter. The requirement that outcomes be reported objectively to an external stakeholder can, if appropriately handled, generate insights among program participants which can be harnessed for program improvement. The paper concludes with an account of the lessons learned from the evaluations, in the hope that these help will evaluation consultants in forging more productive relationships and better communications with program participants.

Get full access to this article

View all access options for this article.

References

Alderson, C. , & Scott, M. (Eds.). (1992). Insiders, outsiders and participatory evaluation. In C. Alderson and A. Beretta (Eds.), Evaluating second language education (pp. 25-58). Cambridge: Cambridge University Press.

Berthold, M. (Ed.). (1995). Pioneer and lighthouse: The Benowa experience. Canberra : National Language and Literacy Institute of Australia .

Chelmsky, E. , & Shadish, W. (1997). Evaluation for the 21st century. Newbury Park, CA: SAGE Publications.

Clyne, M. (Ed.) (1986). An early start: Second language at primary school. Melbourne : River Seine Publications.

Clyne, M. , Jenkins, C. , Chen, I.Y. , Tsokaldou, R. , & Wallner, T. (1995). Developing second language from primary school: Models and outcomes. Deakin, ACT: National Languages and Literacy Institute of Australia.

Cronbach, L.J. , Ambron, S.R. , Dorubush, S.M. , Hess, R.D. , Hornik, R.C. , Phillips, D.C. , Walker, D.F. , & Weiner, S.S. (1980). Towards reform of program evaluation. San Francisco, CA: Jossey Bass.

Cummins, J. (2000). Language power and pedagogy: Bilingual children in the crossfire. Buffalo, NY: Multilingual Matters.

de Courcy, M.C. (2002). Learners' experiences of immersion education: Case studies of French and Chinese. Clevedon, UK: Multilingual Matters.

de Courcy, M.C. , Burston, M. , & Warren, J. (1999). Language development in an Australian French early partial immersion program. Babel: Journal of the Australian Federation of Modern Language Teachers Associations, 34(2), 14-20, 38.

10.

de Courcy, M.C. (1997). Benowa High: A decade of French immersion in Australia . In R. K. Johnson & M. Swain (Eds.), Immersion: International perspectives (pp. 44-62). Cambridge: Cambridge Press.

11.

Dellora, M. (1989). One model of bilingual education in Australia at Richmond West Primary School. Melbourne Papers in Applied Linguistics, 1(1), 1-7.

12.

Devlin, B. (1997). Links between first and second language instruction in Northern Territory bilingual programs: Evolving policies, theories and practice. In P. McKay et al. (Eds.), The Bilingual Interface Project Report (pp. 75-90). Canberra : DEETYA.

13.

Elder, C. (1989). Drowning or waving? An evaluation of an Italian partial immersion program at a Victorian primary school. Melbourne Papers in Applied Linguistics, 1(2), 9-17.

14.

Elder, C. (1998). `Luxton' Primary School comparative literacy study. Unpublished report. Melbourne: Language Testing Research Centre, University of Melbourne.

15.

Elder, C. (2005). Evaluating the effectiveness of heritage language education. What role for testing? International Journal of Bilingual Education and Bilingualism, 8, 196-212.

16.

Elder, C. , & Liem, I. (1999). The bilingual Chinese/English program at `Luxton' primary school, July 1998-June 1999 (External evaluator's report). Melbourne: Language Testing Research Centre, University of Melbourne.

17.

Elder, C. , & Mayer Attenbourgh, C. (2000). `Seaview' Secondary College Arabic-English Bilingual Program (External evaluator's report). Melbourne: Language Testing Research Centre, University of Melbourne.

18.

Fernandez, S. (1996). Room for two: A study of bilingual education at Bayswater South Primary School (2nd ed.). Belconnen, ACT: National Languages and Literacy Institute of Australia.

19.

House, E. , & Howe, R. (1998). The issue of advocacy in education. American Journal of Evaluation in Education, 19, 233-236.

20.

Joint Committee on Standards for Educational Evaluation. (1994). The Program Evaluation Standards (2nd ed.). Newbury Park, CA: SAGE Publications.

21.

Kiely, R. , & Rea-Dickins, P. (2005). Program evaluation in language education. New York: Palgrave Macmillan .

22.

Lorch, S. , McNamara, T. , & Eisikovits, E. (1992). Late Hebrew immersion at Mount Scopus College, Melbourne: Towards complete Hebrew fluency for Jewish day school students. Language and Language Education, 2(1), 1-29.

23.

Lotherington, H. (2001). A tale of four teachers: A study of an Australian late-entry content-based program in two Asian languages. International Journal of Bilingual Education and Bilingualism, 4(2), 97-106.

24.

Lynch, B. (1996). Language program evaluation: Theory and practice . Cambridge: Cambridge University Press.

25.

Mackay, R. , Wellesley, S. , & Bazergan, E. (1995). Participatory evaluation. ELT Journal, 49(4), 308-317.

26.

Molyneux P. (2004). Bilingually educated children reflect on their learning. Australian Language & Literacy Matters, 1(2), 4-10.

27.

Norris, J. (2004). Validity evaluation in foreign language assessment. Unpublished doctoral dissertation. Honolulu: University of Hawai`i.

28.

Norris, J. (2006). The why (and how) of assessing student learning outcomes in college foreign language programs. The Modern Language Journal, 90, 577-583.

29.

Oller, K. , & Eilers, R. (Eds.). (2002). Language and literacy in bilingual children. Clevedon, UK: Multilingual Matters.

30.

Owen, J. (1996). Program evaluation: Forms and approaches. New South Wales: Allen and Unwin.

31.

Patton, M.Q. (1996). Utilization-focused evaluation: The new century text (3rd edition). Thousand Oaks, CA: SAGE Publications.

32.

Passow, A.H. (1990). Reporting the results of evaluation studies. In H. J. Walberg & G. D. Haertel (Eds.), The international encyclopedia of educational evaluation (pp. 745-750). Oxford: Pergamon Press.

33.

Rea-Dickins, P. and Germaine, K. (Eds.). (1998). Managing evaluation and innovation in language teaching: Building bridges. Harlow , Essex: Longman.

34.

Reid, J. (1996). Recent developments in Australian late immersion language education. Journal of Multilingual and Multicultural Development, 17, 469-485.

35.

Stake, R. E. (ed.). (1975). Evaluating the arts in education: A responsive approach. Colombus, OH: Merrill.

36.

Stake, R.E. (2001). A problematic heading. American Journal of Evaluation, 22, 349-354.

37.

Stake, R.E. (2004). Standards-based & responsive evaluation. Thousand Oaks, CA: SAGE Publications.

38.

Stufflebeam, D.L. (1981). Meta-evaluation: Concepts, standards, and uses . In R.A. Berk (Ed.), Educational evaluation methodology: The state of the art (pp. 146-163). Baltimore, OH: The Johns Hopkins University Press.

39.

Weir, C. , & Roberts, J. (1994). Evaluation in ELT. Oxford: Blackwell.

40.

Zbar, V. , Elder, C. , McNamara, T. , & Rossi, C. (1998). `Seaview' Secondary College Bilingual Program evaluation. Melbourne, Victoria: Zbar & Schapper Consulting.