Teaching Summative Assessment: A Curriculum Analysis of Pre-Service Language Teacher Education in Sweden and Finland

Abstract

The purpose of this study is to investigate the curricular manifestation of summative assessment literacy in language pre-service teacher education at three universities in Sweden and Finland through multiple case studies. Data sources included program guidelines, course curricula, and study guides. A thematic approach was used to analyze the data based on a theoretical framework involving conceptual understandings, skills, and dispositions in summative assessment. The results indicate that all programs emphasize the basic assessment concepts such as validity and alignment and using assessment to inform teaching and learning. However, compared with formative assessment, summative assessment receives less attention in the curriculum. There are differences in addressing summative assessment through stand-alone and embedded courses and in the literacy areas covered. Implications for teacher education to address summative aspects of assessment in line with teachers’ tasks in schools are offered.

Introduction

The purpose of this study is to investigate summative assessment (SA) literacy targeted in language teacher education (TE) at three universities in Sweden and Finland through a document analysis. It is part of a broader research project that aims to investigate SA literacy of pre-service and novice teachers longitudinally in the last year of TE and the initial years of teaching in schools. The interest in this curriculum analysis arises from the need for aligning the TE curriculum with the critical SA tasks teachers carry out in schools, such as interpreting grading criteria and national tests, constructing classroom tests, and informing students and their guardians.

Teacher assessment literacy has become an established term for the construct of teacher knowledge, skills, and dispositions as a central aspect of the teaching profession (Fulcher, 2012; Inbar-Lourie, 2017; Pastore & Andrade, 2019; Xu & Brown, 2016). As a sub-dimension of this, SA literacy refers to the competencies teachers should possess in relation to determining, interpreting, and communicating the level of students′ performance in a certain area. In many contexts, the most tangible form of SA is grading, conceptualized as a technique of SA with a specific purpose to place a student on a continuum of quality or proficiency by delivering a numeric or verbal judgment on the attainment of learning objectives (Anderson, 2018). Typical instances of this form of SA can be end-of-unit, course or school term grades, national testing for core subjects such as Swedish and English in Sweden, and school leaving exams (e.g., Finnish Matriculation Exam). These evaluations are expected to sum up academic performance and frequently are high stakes with a strong impact on the future lives of students.

The critical nature of SA requires teachers to be competent and professional assessors to be able to interpret knowledge requirements and make valid and reliable assessment decisions. Pre-service teachers are expected to develop this competency initially during TE and nurture it further through experience in schools. Yet several studies reported that TE programs fall short in helping pre-service teachers establish a firm foundation for their SA tasks in schools (Malone, 2013; Stiggins, 1999; Vogt & Tsagari, 2014). Moreover, DeLuca and Klinger (2010) identified several areas that were not addressed in sufficient depth in TE such as modifying and reporting assessment results and constructing certain task types. Although TE is only one of the sources of teachers’ SA literacy and deploying SA practices embodies multiple kinds of knowledge that develop overtime, these apparent needs raise questions on what teachers learn during TE and its relevance for SAs they carry out in schools. Therefore, the alignment between the assessment knowledge and skills addressed in TE courses and the assessment tasks of teachers is a critical issue to address (Poth, 2013).

The legal framework for TE in Sweden and Finland outlines only the basic requirements regarding assessment and grading, which means that autonomy prevails on the local level. For Swedish universities, the Higher Education Ordinance regulations require TE programs to offer pre-service teachers the opportunity to “demonstrate specialized knowledge of assessment and grading” and “to demonstrate the capacity to observe, document and analyze their students’ development and learning in relation to educational objectives and to inform and cooperate with students and their guardians” as part of professional qualifications for secondary schools (Swedish Council for Higher Education, 1993, Annex 2). The Finnish legislation provides broad terms for teacher qualifications at various levels of education (Finlex, 1998), and a decree on TE and teacher training schools (Finlex, 2009). However, the 13 universities providing TE for basic and general upper secondary education are relatively free to arrange their programs in accordance with the legislated qualifications. Coherence and collaboration are promoted by national initiatives such as “Teacher Education Forum” (Lavonen, 2018), but the content and focus vary between universities.

This autonomy enjoyed by the universities may result in variation in the extent and depth of the coverage of SA in the curriculum in line with the explicit or implicit culture of assessment informed by broader socio-cultural, political, and pedagogical agenda for each TE program. As Inbar-Lourie (2008) defines it, a “testing culture” focuses on systematic-reliable testing, skills such as test construction and statistical analysis while a “learning culture” paves the way for grading by introducing formative strategies. The SA undertaken by teachers with these differing orientations may lead to issues of equity and equality of assessment at the school level. For example, several studies in Sweden and Finland attest that school grades are based on a highly variable set of components and their interpretations (Jönsson & Klapp, 2020; Skolinspektionen, 2019; Skolverket, 2019) and that with the same evidenced learning outcomes, a range of vastly different grades may have been assigned to students at the end of basic education (Hilden et al., 2016). Although it may not be as apparent, this variation might also be observed in other SA tasks teachers carry out in schools.

Another area relevant to teachers’ SA literacy involves the national tests which are implemented both in Sweden and in Finland but of different types. National tests in Sweden are implemented at the end of Year 9 (compulsory schooling) as well as in language courses including Swedish, Swedish as a second language, and English at the upper secondary school level. In the Finnish lower secondary school, there are no final national exams in languages but at the end of upper secondary school, all students take a high-stakes test, the Finnish Matriculation Exam, which is external and decisive to their future life choices. Given this framework, teachers in Sweden and Finland need to be able to interpret the national tests and inform the students and their guardians about their implications.

Furthermore, as one of the broader SA skills, the criterion-based grading in language instruction in Sweden and Finland is closely aligned with the Common European Framework of Reference for Languages (CEFR) (Council of Europe, 2001). The CEFR defines communicative language proficiency as a compilation of listening, speaking, reading, and writing skills with descriptive scales that can be adapted for national use. Using the CEFR as a reference point, teachers need to master tasks and procedures for assessing students’ development in these proficiency areas validly and consistently.

Within this context, a comparative analysis of curriculum decisions at the program and course level might provide insights into how SA literacy is addressed in their respective national and educational context. Accordingly, this study investigates the curricular manifestation of SA literacy in language TE programs at three universities in Sweden and Finland through document analysis. Specific research questions are:

• Research Question 1: What learning objectives and content areas does language teacher education at three universities in Sweden and Finland focus on in relation to knowledge, skills and dispositions of summative assessment?

• Research Question 2: How does language teacher education at three universities in Sweden and Finland compare in relation to their learning objectives and content areas for summative assessment?

Previous Research

Research on TE curriculum analysis for second and foreign language teachers’ SA literacy is rare, but there are studies on assessment literacy in general and language assessment literacy in specific. In one of the earlier curriculum analysis studies on assessment literacy, Rogers (1991, cited in Poth, 2013) collected descriptions of measurement and evaluation components in Canadian TE programs through an oral interview with teacher educators and found that many programs did not have a particular assessment focus and a required assessment course at the beginning of 1990s. As this was not a curriculum analysis, it did not offer details into assessment literacy areas and priorities in these programs. Later, Poth (2013) carried out such a study analyzing 57 assessment course syllabi offered by 14 Western-Canadian TE programs. She focused on learning objectives, content, methods of teaching, and assessment strategies in these syllabi. The results revealed that most TE programs in Western Canada (12 out of 14) required the completion of at least one assessment course reflecting a shift from none required in most TE programs in the early 1990s. In addition, Poth detected shared attention to assessment purposes among the 14 TE programs with a “learning-focused assessment culture” (Poth, 2013, p. 652).

DeLuca and Bellara (2013) also carried out a document analysis study in the United States to investigate the alignment between TE accreditation policies, professional standards for teachers assessment practices, and 10 pre-service assessment course syllabi across content focus, depth of knowledge, and range of knowledge. They identified alignment for the content themes of “assessment processes,” “assessment fairness,” and “measurement theory” and misalignments for “assessment purposes,” “assessment for learning,” “communication of assessment information,” and “classroom environment and assessment” across the three sets of data sources. These results indicated that the courses focused on conventional assessment literacy while leaving out the dimensions identified with contemporary assessment approaches such as assessment for learning.

Another study carried out by Lam (2015) investigated language assessment literacy through the analysis of course and program documents in five TE programs in Hong Kong as well as interviews with pre-service teachers (N = 43) and teacher educators (N = 9). The results indicated that pre-service teachers did not receive sufficient training in assessment as TE programs rarely offered stand-alone courses on assessment and mostly addressed limited aspects of assessment knowledge, skills, and principles within larger curriculum and teaching courses. The curriculum analysis revealed that courses focused on assessment with an “examination-oriented” approach and overlooked theory-practice connection and “assessment for learning” dimensions.

In addition to curriculum analysis studies, research on assessment competencies and needs of teachers also provides insights into the present study. For example, DeLuca and Klinger (2010) used a survey to investigate perceived confidence levels and further needs of pre-service teachers at a mid-size university in Ontario in assessment practice, theory, and philosophy. The responses of 288 participants showed that pre-service teachers developed their confidence in assessment mostly through school practice and would need more direct instruction in specific areas of assessment such as developing items, item reliability and validity, and reporting achievement.

In a similar study, Giraldo and Murcia (2018) explored pre-service teachers’ needs and expectations from a language assessment course at a state university in Colombia through questionnaires for pre-service teachers and teacher educators, an expert interview, and reflective journals. The results indicated that pre-service teachers expected to develop practical assessment skills but also focus on broader issues such as theory-practice connection, and consideration of local policies for alignment.

Both curriculum analysis and descriptive studies above indicate that TE programs address assessment literacy in different course structures with varied content, but mostly focus on the conventional aspects of assessment and ignore “assessment for learning” orientation. They mostly focus on assessment literacy and language assessment literacy but fall short in investigating SA literacy addressed by TE programs. Although SA accounts for a significant amount of time and effort in teachers’ work in schools, our understanding of their preparation and proficiency for this work is still insufficient to provide guidelines for improving their practices (Vetenskapsrådet, 2015). Particularly, research on novice teachers’ assessment skills is limited or almost nonexistent (Grainger & Adie, 2014). This gap in the literature makes examining SA priorities, focuses, and approaches in TE programs within their unique national and local context critical.

Theoretical Framework

Multiple assessment literacy frameworks are presented in the literature (e.g., DeLuca & Klinger, 2010; Edwards, 2017; Fulcher, 2012; Inbar-Lourie, 2008; Pastore & Andrade, 2019; Stiggins, 1999; Taylor, 2013; Xu & Brown, 2016). An earlier one proposed by Stiggins (1999) included seven competency areas to prepare pre-service teachers for the assessment work in schools:

connecting assessments to clear purposes, clarifying achievement expectations, applying proper assessment methods, developing quality assessment exercises and scoring criteria and sampling appropriately, avoiding bias in assessment, communicating effectively about student achievement, and using assessment as an instructional intervention” (pp. 25–6).

Taylor (2013) offered another detailed construct for assessment literacy under eight thematic areas: “knowledge of theory, technical skills, principles and concepts, language pedagogy, sociocultural values, local practices, personal beliefs/attitudes, scores and decision-making” (p. 410).

Other frameworks characterize assessment literacy under fewer but broader categories. Fulcher (2012), for example, offered a model involving “practices” which refers to knowledge, skills, and abilities, for the practice of language testing, “principles” involving processes, principles, and concepts to guide the practice, and “contexts” referring to historical, social, political and philosophical frameworks to address origins, reasons, and impacts of assessment. Similarly, DeLuca and Klinger (2010) offered three categories: “assessment theory” which refers to established knowledge such as principles of validity, “assessment practice” referring to skills and practices for assessment, and “assessment philosophy” which focuses on rationale for assessment and implications for teaching and learning. Recently Pastore and Andrade (2019) also offered a three-dimensional framework involving “conceptual, praxeology, social-affective” aspects of assessment. Conceptual dimension refers to the understanding of assessment, whereas praxeology concerns using different forms of assessment in practice. Socio-affective dimension focuses on broader assessment concerns such as communicating assessment results.

We established our analytical framework based on the three dimensions referred to in various models repeatedly with different terms and groupings: knowledge and understandings, skills and practices, and dispositions. Knowledge and understanding refers to conceptual understanding of assessment literacy such as theories, research, principles, purposes, models, history, and approaches. This competency establishes the foundation for skills and practices a teacher would need to carry out the assessment task, feeds into the development of values, attitudes, and principles to make rational, ethical, and fair decisions. Skills and practices cover competencies such as item writing, constructing a test, grading, statistical analysis, and practices such as carrying out a grading conference, observing a classroom, and writing an assessment report. A good understanding of essential aspects of assessment is critical in sharpening skills and enhancing practices but not sufficient. An explicit focus on skills and practices is critical for pre-service teachers to develop these competencies. Finally, dispositions refer to beliefs, philosophical perspectives, values, attitudes, and professional principles that are needed to make ethical, objective, fair, and relevant decisions in relation to assessment. Questioning and reflecting on one’s own belief system, understanding the impact of various kinds of assessment on students, guardians, and other individuals and society, considering social and emotional aspects of assessment, and ethical principles guiding assessment decisions are some of the examples in this dimension. Dispositions are critical for teachers’ motivation and integrity in planning and implementing assessment in their daily classroom practice and making informed decisions on assessment. We found these three dimensions relevant to represent the construct of SA in relation to both the general assessment literacy and to analyze the goals, content, and examination criteria we located in the TE curriculum for languages.

Method

The design of this study involves a multiple case study of language TE at three universities in Sweden and Finland in relation to SA literacy through document analysis. The first case in Sweden, University A, offers language TE through a 5-year degree program as well as a 1.5-year certificate program. Both programs have several compulsory “stand-alone” courses designed to teach exclusively assessment knowledge, skills, and dispositions, but the 5-year program also offers “embedded courses” in which assessment is only one of the several major content areas in curriculum and teaching. School practice courses (N = 3-4) also provide opportunities for pre-service teachers to develop their assessment literacy in both programs.

The second case in Sweden, University B, also offers language TE through a long (4.5- to 5.5-year degree) and a short (1.5-year certificate) program. The short program offers one optional module on SA while the long program has two courses exclusively on assessment. Especially during the second and third school-practice courses, attention is given to pre-service teachers’ formative and SA literacy.

The third case, University C in Finland, gives 1 year of pedagogical studies at the master’s level after 4-year undergraduate studies in the subject area. Assessment is mainly addressed in two embedded courses on curriculum development and teaching, the first focusing on formative assessment, and the latter on summative decisions.

These universities were selected because they house large TE programs in Sweden and Finland, and differ in their curricular emphasis on assessment in terms of scope and detail. For example, University A has a strong focus on assessment and grading as it has been responsible for various types of language tests since the early 1960s. Therefore, it has lifted assessment literacy as a particular area of study in TE programs. University B has embedded it primarily within courses focused on teaching and expected mentor teachers in schools to guide pre-service teachers about assessment and grading. Similarly, University C addresses assessment literacy in relation to broader areas of teaching competence such as curriculum and teaching. University C also differs from Universities A and B in offering a 1-year program. The two Swedish universities have both short and long programs in TE.

Data sources primarily include program guidelines (or regulations), course curriculum (or syllabi), and study guides (or course plans) as they were available in the 2018-2019 academic year. “Program guidelines” reflect the national and local policies for TE and outline the general focus, requirements, and goals or recommendations for the courses. A “course curriculum” typically includes short and general descriptions of the entry requirements, content focus and description, learning outcomes, course implementation, examination, and the course literature. A “study guide” refers to specific instructions for pre-service teachers regarding the implementation of the course such as the schedule of sessions, preparations, assignments, and examination. Although study guides operationalize the curriculum through more practical information as stated above, they also contain essential information on expectations from students about assessment literacy. National documents do this at a more general level by setting the overall expectations about teacher competence, and providing guidelines for addressing assessment competences in TE programs.

Among these data sources, the course curriculum was the most significant data source in our study as it often included relevant data on the assessment content focus and targets in course descriptions and learning objectives (i.e., learning outcomes). We view course curriculum as a valid and reliable source for understanding the general aspects of education pre-service teachers receive, but we also found both program guidelines and study guides useful in analyzing the SA focus in the programs.

Data collection involved the selection of relevant documents at the program and course levels in the 2018-2019 academic year. We included program guidelines in our analysis if they included references to assessment as a goal or content. We collected syllabi both for stand-alone and embedded courses addressing assessment literacy fully or partially. We included a course syllabus in our document analysis only if the course description and/or the learning objectives involved assessment as a key area. As a result, we analyzed 28 course syllabi at University A, 12 at University B, and 5 at University C including school practice courses.

In addition, we analyzed 40 study guides (28 at University A, 12 at University B) as they also provided additional information on assessment learning objectives and content areas. If the study guide represented a stand-alone course on assessment, it was included in the analysis as a whole whereas only relevant sections of these documents for assessment learning objectives and content areas in the embedded courses were considered in data analysis.

Data analysis involved a thematic content analysis (Wolcott, 1994) of the data drawn from the documents in line with the research questions. For this purpose, we first established a common set of thematic categories based on the three aspects of SA described in the theory section: knowledge and understanding, skills and practices, and dispositions. Then we carried out a case-based analysis focusing on content areas and learning objectives addressed in various sections of these documents such as program goals, course descriptions, and examination criteria. Using a template involving the analytical framework, we coded the data from the documents into thematic boxes as short statements of goals and content areas covered in relation to SA (see Appendix). The thematic consistency in coding across the three dimensions of SA was established through discussion among the four researchers and checking each other′s coding. In cases where the learning objectives and the content areas combined more than one theme in the syllabi or other documents, data were double-coded to represent more than one thematic area. Based on the case-based codes, we generated descriptions of SA literacy offered in each university. A comparative analysis followed to identify similar and different aspects of SA targeted in the language TE programs of the three universities.

Results

The results of our analysis of the SA areas targeted in the TE courses offered by the three universities are presented below under three themes: knowledge and understanding, skills and practices, and dispositions (see Appendix for a detailed listing of SA areas targeted).

University A (Sweden)

Knowledge and Understanding

The three didactic courses in the 5-year program on language teaching, learning, and assessment primarily emphasize formative assessment. Learning objectives refer to describing the differences between formative and SA and their uses from ethical and various theoretical perspectives, understanding “formative and SA in relation to language learning and curriculum goals” and “how formative and SA inform learning.”

The first of the three courses covers a historical account of language learning, teaching, and assessment, and how it has developed, interacted, and changed over time until present practice. This includes the elements of a communicative language test, the aims and purposes of the CEFR, and what we term “traditional language testing.” The overall purpose is to develop decisions based on this understanding in line with pedagogical goals. Pre-service teachers are expected to account for different types of assessment and their uses in a written test and their theoretical bases in the second course. Assessment is emphasized as a broader term before grading. The third course aims to develop an “understanding of the research behind language assessment and theory of language test construction,” advantages and disadvantages of different test techniques, their implications for teaching and practice in the classroom as well as how the Swedish national test differs from these and international tests.

The two pedagogical courses aim to develop an understanding of and to problematize concepts such as validity, reliability; norm- and criterion-grading systems; and how different learning theories might influence assessment practice. Understanding the aims and purposes of national tests and large-scale international tests is also an important goal. They also cover a national angle through research studies.

The four school practice courses where pre-service teachers work with a mentor teacher between 3 and 8 weeks aim to develop pre-service teachers′ competency to understand different ways to document and assess students’ progress in their subject through observations and reflections on students’ learning in relation to the course goals.

In all these course syllabi and guides, SA is rarely stated explicitly as a goal or content. Assessment is mentioned as the overarching concept and in general terms, and often includes both summative and formative practices. Validity and reliability are the major concepts focused on in these courses.

The 1.5-year certificate program courses include similar statements on assessment. The two stand-alone courses address assessment primarily in relation to theoretical perspectives with a general orientation rather than focusing on a specific subject area. SA is addressed in relation to formative assessment through statements like “summative and formative assessment,” “different models of assessment,” and “purposes of assessment and grading.”

Skills and Practices

Pre-service teachers in the 5-year program are expected to apply different types of assessment, considering “when and why” questions in the didactic courses. They should be able to assess a piece of written material and problematize inherent difficulties as well as prove their skills by grading an essay. They should be able to show that they understand the principles but also the difficulties involved in test construction through practice. They are expected to construct a language test, pilot the test, and receive and give peer assessment on it.

In the pedagogical courses, the aim is to emphasize assessment skills further through hands-on activities, that is, co-operative assessment, giving feedback and grading tests, constructing a summative test as well as “mock grading” discussions with other pre-service teachers. These courses require pre-service teachers to construct and critically evaluate assessment activities, and to evaluate and grade students’ development in relation to national syllabus and criteria. In school practice courses, they are also expected to construct and grade sample tests, and to observe classroom practice of assessment.

In the 1.5-year certificate program, assessment skills and practices are mentioned in a general manner. In one of the pedagogical courses, for example, pre-service teachers plan modules for teaching assessment aligned with course criteria with the aim of demonstrating what types of tests are appropriate and how assessment can help and develop students′ learning. There is an aim to develop practical assessment skills such as test construction and how to plan teacher/student conferences regarding grades, giving feedback, cooperative assessment, using digital tools. They are also expected to work individually and in groups, practicing assessing and grading authentic student responses from tests and assignments brought in from their teaching practice.

Dispositions

In the 5-year program, all three didactic courses include a critical discussion of different uses of SA from an ethical point of view as well as when they are appropriate to use. Concepts such as validity and reliability include ethical considerations of equality, objectivity, and fairness.

The two pedagogical courses address questions of how validity and fairness may influence teacher assessment practice, and how formative and SA can be used to help improve students’ learning while at the same time evaluating their learning outcomes. Course objectives indicate that pre-service teachers are to specifically problematize assessment and grading in relation to ethical aspects that teachers should adhere to when assessing as to understand the consequences of assessment activities for individuals, schools and society. Concepts of fairness and objectivity are emphasized.

In two of the four school practice courses, pre-service teachers are expected to relate their observations to and reflect on a particular skill to be assessed, the purpose or type of assessment (formative or summative), how the assessment was aligned with the criteria, how the assessment was experienced by the teacher and students, and what theories the pre-service teachers could see were behind the assessment. The third course includes writing a blog on how students’ learning and development in the subject is documented and assessed. Through these different tasks, pre-service teachers are expected to consider the implications of assessment and testing on individual students as well as learning.

The 1.5-year certificate program’s pedagogical course includes targets in relation to auditing tests and other SA activities to give the pre-service teachers a chance to show what they know, and reflect on how to establish an ethical assessment practice when using students’ results. The course also presents concepts such as validity, reliability, fairness, and objectivity. Pre-service teachers are expected to critically discuss the consequences of different types of assessment. In addition, school practice courses require them to select a moment in their practice and to relate it to how assessment or grading is performed.

In summary, the learning objectives and content areas at University A in general reflect contemporary assessment perspectives, and can thereby be characterized as “assessment oriented” as opposed to traditional “testing oriented.” They aim at a “learning culture” rather than a “testing culture” as this is evident, particularly in the treatment of SA from a formative assessment perspective and the emphasis on pedagogical implications of various types of assessment activities and outcomes. A basic difference between the two programs is that the 5-year program integrates specific language aspects (e.g., teaching, learning, and assessment of writing skills) in addressing assessment whereas this integration does not occur in the 1.5-year program. In both, there is little explicit attention to test construction and grading practices. SA literacy is treated through conceptual perspectives, and less emphasis is given to practicing language assessment and testing.

University B (Sweden)

Knowledge and Understanding

In the long program (4.5-5.5 years), pre-service teachers take on-campus courses in language education that focus on approaches to teaching methods, assessing, grading, lesson planning, and curricular knowledge. The coverage of assessment in these embedded courses depends on the instructor, but their content scope and sequence are generally consistent with recent research on assessment as well as national and European guidelines and recommendations in school policy documents. Two courses, deal specifically with formative and SA in terms of terminology, theory, and method in the sense of how to reach targets for language development and portfolio assessment, testing, and self-assessment.

The first school practice is offered during the third semester, the second one during the fifth semester, and, finally, the third course is then offered during the ninth term. In all these courses, regulations and recommendations are covered regarding assessment and grading in light of the school policy documents and the CEFR (Council of Europe, 2001).

Moreover, during the fifth term, there is a campus course on the advanced level that addresses the assessment of language acquisition in theory and practice. Pre-service teachers are expected to give accounts of how to analyze and assess students’ acquisition of languages as well as of speaking and writing considering various methods of assessment. This course on language education, formative and SA, and curriculum theory forms a “sandwich course” with the second school practice, in which the pre-service teachers attend seven seminars on campus and then return for the examination when the second practicum is completed.

There are similar counterparts for the short program (1.5 years). Theories about assessment approaches are introduced and form part of the examination on campus. Pre-service teachers in English language education are asked to interview school practice mentors and other teachers about assessing speaking and how formative and summative approaches are used for respective tasks and tests, and then present their findings during the examination on campus. The paper along with comments from peer- and self-assessment is handed in to the teacher for formative assessment and, subsequently, for grading.

In all these courses, specific focus is placed on how formative assessment can be conducted in light of summative concerns but also to some extent on how formative and summative aspects can and should interact. Furthermore, how to manage and grade national tests and how to engage in co-operative assessment are on the agenda mainly for the sake of preparing the pre-service teachers for school practice. Hence, the theoretical and regulatory aspects are addressed, or what to assess and why as well as to a limited extent how to assess and grade language tests. The practical aspects of assessing and grading are covered by several learning objectives during school practice. Still, the knowledge base regarding assessment literacy for the various mentors in teaching practice cannot be guaranteed, even if visits and formative three-party conversations are arranged and assessed by faculty members and the grading is conducted by course teachers on campus. The three-party conversation refers to a formative dialogue during practicum in which the university instructor and the school practice instructor assess and scaffold pre-service teachers in their development.

In addition, the short program offers an optional module of five credits that focuses on test construction and grading at the advanced level. For all the above-mentioned courses, terminology such as alignment, validity, transparency, authenticity, cooperative assessment, and reliability as well as various aspects of language testing and test production are considered in tasks and for deliberation during seminars. The examination consists of a paper in which the pre-service teachers are expected to argue for their individual teaching unit plans in which assessing and testing are aligned with teaching and learning. In the ensuing practicum, they are expected to implement these plans.

Skills and Practices

Assessment and grading with a criterion-referenced approach to teaching are briefly mentioned in the embedded courses at the basic level, focusing on the school curricula and planning for communicative language teaching. This approach relies on concepts like alignment, validity, transparency, and authenticity. Pre-service teachers are expected to “design and motivate lesson plans with clear targets in light of relevant national school policy documents as well as to conduct and evaluate the lessons in light of the targets” in the first school practice. In the second and third school practice, the intended learning outcomes (ILO) indicate that pre-service teachers should be able to guide students in self- and peer assessment, which may involve formative and summative aspects. During the third school practice, they are expected to assess and comment on students’ achievements.

Furthermore, pre-service teachers both in long and in short programs become acquainted with and practice assessment and grading processes at the local level during the second and third school practice. This is clear in the ILOs and reflected in the grading criteria for these courses. For example, one ILO indicates the expectation that “pre-service teacher can document, analyze and assess students’ learning, as well as constructively communicate assessments to students under supervision, and with help of theories of teaching and knowledge of grading and assessment.” Another states, “under supervision, the pre-service teacher can create conditions where students can evaluate and assess their own learning in different activities.” This ILO reflects the national regulations in the curricula regarding assessment and grading.

Dispositions

On the one hand, University B refrains from listing a variety of explicit requirements for dispositions about pre-service teachers′ beliefs and attitudes. On the other hand, the learning objectives include international and European (such as the CEFR), national and local regulations, guidelines, and recommendations implicitly in both on-campus and school practice courses. These circumstances enable university instructors to opt for individual strategies regarding detail, but there are widespread expectations on pre-service teachers to reflect on, and ultimately establish, teaching personas that both function in light of regulatory frameworks and suit their individual perspectives on language education. As for demands in the national school policy documents on ethical aspects, such as fairness, objectivity, and equity that form part of the school curricula and the support material published by the Swedish National Agency for Education, these dispositions may be developed on campus in the sense that the pre-service teachers are expected to take responsibility for their own learning, but they are mainly implemented during school practice and under the supervision of mentors in schools.

Furthermore, several learning objectives for the formative trialogue and the summative examination also address such demands regarding how the pre-service teachers are supposed to communicate and interact with students, their guardians, and school staff. During the second and third school-practice courses, pre-service teachers work toward certain advanced learning objectives that involve assessing and grading in theory and practice. In general, the implied terminology employed by the faculty members revolves around alignment, transparency, validity, reliability, feedback, feed-forward, peer-, and self-assessment.

In summary, the curriculum at University B in Sweden addresses two general areas of assessment and grading: First, pre-service teachers are offered theoretical and practical knowledge about various steps in formative assessment. Second, they are given the opportunity to gain insight into how SA and grading can be managed with particular attention to national tests for Swedish and English as well as alternative high-stakes tests for further foreign languages. In this context, pre-service teachers practice assessing and grading test items for speaking and writing competencies. In addition, task construction is included in all the courses that deal with lesson- and teaching-unit-planning at the basic and advanced levels, respectively. However, instruction in test construction per se is neither introduced on a broad scale for courses held on campus nor for school practice.

University C (Finland)

Knowledge and Understanding

In principle, assessment issues are merged into the thematic foci of the different courses mostly as formative assessment. The term assessment expressively appears only in one-course label. Even there, the core content revolves around basic planning of lessons and assessment for and as learning, while summative functions of assessment are only attended later in spring, primarily on another course on curriculum and educational development. In the other courses, assessment is touched upon depending on the main content, most typically from the formative angle.

One of the embedded courses offered early in the program states, “pre-service teachers who have completed the course will know the different functions and objectives of assessment and the significance of feedback and assessment for developing the teaching of one’s own subject.” As this course belongs to the subject didactic strand, the CEFR and its basic assumptions including level descriptions, alongside the national core curricula play a focal role in the course literature.

More in-depth knowledge on fundamental aspects of assessment is conveyed on another embedded course on curriculum and educational development. This course aims at the pre-service teachers’ understanding of the dimensions of society and education policy that are related to assessment in developing teaching and education, and the principles and types of multifaceted formative and SA of learning. On the contrary, the duration of the course does not allow for much more than a brief introduction into such complex constructs as validity since the skill component and practical test task design is most prominent.

At a research-oriented course, pre-service teachers are expected to understand didactics as an area of education and as the theoretical basis of teachers’ professional knowledge and skills. They should understand the process of teaching–studying–learning, and the key elements in the planning, implementation, and assessment of teaching as well as their interrelationships as applied to their own subject of study. They should also understand the educational significance and ethical nature of teaching. In practice, however, the provision of the research course greatly depends on the selected theme of the thesis that comprises presentations, giving feedback on others’ theses and reflective and evaluative discussions.

Skills and Practices

The two major subject didactic courses aim at the alignment of theoretical and practical components of assessment literacy, whereas the other on-campus courses tangent the components more occasionally. Pre-service teachers learn to apply various principles and types of assessment purposefully as well as assess one’s own solutions critically. The practices deployed in this course also provide models of assessment pre-service teachers are expected to implement in their future work.

Pre-service teachers should also know the basics of applying information and communication teachnology (ICT) in teaching and studying including various methods for assessing learning and giving feedback during studying, and the potential of ICT in assessment. In accordance with the task-supported language-teaching model, an evaluative and reflective closing sequence is normally built into each task-cycle designed and demonstrated by the pre-service teachers, but SA is usually not discussed at the initial introductory course into language didactics except for designing and piloting small-scale test tasks. The aim of this course is to train pre-service teachers to apply the various forms of SA purposefully as well as to assess one’s own solutions critically.

The most practice-oriented study units are the school practice periods, one in the autumn and one in the spring term. The first consists of the planning, implementation, and assessment of practice lessons, lesson observations, group guidance sessions, and a component focusing on the school as a community. Apart from formative feedback embedded in the process of learning, it is relatively safe to say that assessment plays a secondary role in relation to the elementary constituents of planning and classroom management at this initial stage of TE.

In the second school practice, pre-service teachers are expected to plan, implement, and assess teaching independently and consider the needs and objectives of diverse students and groups as well as the nature of the curriculum and the subjects taught. The practice carried out in spring, seeks to involve pre-service teachers in more demanding sequences of teaching covering the entire cycle from planning to assessment than is commonplace in autumn. Nevertheless, mentor teachers may offer varied opportunities to design test items and try them out in class; therefore, the learning outcomes about SA may be highly variable.

Dispositions

The general philosophical principles and cornerstones of valid assessment are addressed in the two didactic courses. Issues of integrity, equality, and objectivity are discussed in relation to the examples from existing tests, and media or drawn from the pre-service teachers’ own educational experiences. They may gain additional perspectives on these concepts in courses dedicated to general pedagogy, such as social, cultural, and philosophical foundations of education. Moreover, a short course on didactic research aims at promoting pre-service teachers’ understanding of “the educational significance and ethical nature of teaching,” issues evidently pertinent to the validity of SA in a broad sense. Through all these courses, they are expected to problematize discrimination, inequality, and injustice present in education. Yet, profound considerations of SA dispositions typically fall beyond the limited temporal scope of TE.

In addition, pre-service teachers gain additional experiences through active participation in assessment practices executed in the TE program: how the assessment criteria are determined and negotiated, how equally the criteria are deployed, how a course grade is compiled, and what consequences it implies for the students. Therefore, multiple types of assessment are used to maintain the diversity of options to display different kinds of knowledge and skills in respect to individual pre-service teachers. Self and peer assessment is widely adopted in several courses, even as components of a summative course grade. The assessment practices deliberately blur the border between formative and SA as the entire workflow during the course has an impact on the final grade.

In summary, the overall share of SA in University C in Finland is modest. Estimated from the course descriptions and practical experience as a teacher educator, some two credits at the highest are allotted to SA. The skill component seems to prevail in the didactical courses, while the role of knowledge and dispositions tend to be less prominent.

Discussion

In response to the first research question, a major conclusion of this study is that TE curriculum in the three universities addresses SA literacy areas to a limited degree and within the general context of the assessment field. SA is rarely explicit in the learning objectives and the course content although references are made to terms like grading and testing that would indirectly indicate the SA dimension addressed in the courses. They all target the theoretical and practical aspects of assessment, purposes, principles, and functions of formative and SA in general as well as language assessment. Furthermore, they emphasize the relationship between assessment and learning, alignment between curriculum goals and assessment, and communicating the results of assessment.

A major difference between the three universities in response to the second research question is that the TE programs in Sweden target SA literacy through both stand-alone and embedded courses on assessment whereas the one in Finland aims to achieve this goal only within two embedded courses. Accordingly, the University C courses include mostly general objectives in relation to teaching and learning with implicit targets on assessment, while both University A and B courses refer to more specific areas on assessment in the wording of course objectives. When University A and B are compared in this respect, the learning objectives in University A are driven by specific assessment terms while University B objectives reflect an integrated focus on assessment theory and practice. However, the objectives at all three universities present an emphasis on communicative language teaching and assessment.

In the knowledge and understanding category, University C distinctively focuses on ICT in SA, CEFR, and National Core Curriculum in terms of their basic assumptions and level descriptions while University A emphasizes the research dimension of SA as well as the purposes of national tests, and University B highlights assessment theory, the national regulations (e.g., Higher Education Ordinance) and school policy guidelines.

At the skills and practices level, University A focuses on test construction, grading, giving feedback, co-operative assessment practices, self-assessment, and applying national syllabus and criteria. University B and C mostly address these aspects within the areas of lesson planning, implementation, observing, and assessing these processes. For University B, designing tasks for formative and SA, assessing students’ learning, and communicating the results to students and guardians as well as reflecting on teaching and learning based on assessment results are embedded in the courses. In other words, they are taken for granted considering research results and the national and European policy documents, but they are not specified in detail for the university instructors to follow. The course syllabi merely state that pre-service teachers are expected to learn about assessment and grading in a progression. The objectives at University C on the contrary target general areas of teaching and learning, thereby specific SA skills and practices are less apparent in the course documents.

When it comes to dispositions, understanding the importance of assessment for students, schools and society, ethical, social, emotional consequences of assessment, critical reflection on assessment activities are important areas of target at University A. University B highlights validity, alignment, accountability, transparency, feedback, fairness, self-, peer- and co-assessment, and critical reflection on assessment processes and consequences. University C’s learning objectives are general under this theme as well reflecting a curriculum and teaching emphasis, which implicitly addresses the dispositions regarding assessment. Taking into consideration the fact that Swedish universities primarily offer longer programs than Finnish ones, it is possible that pre-service teachers have more opportunities to develop a wider repertoire of assessment competencies than in a shorter 1-year program. Furthermore, Swedish universities are engaged in national test construction. This may lead to comparatively more attention to the circumstances of SA.

Another major finding in our study is that all three universities address SA in relation to formative assessment reflecting an “assessment for learning” (Inbar-Lourie, 2008) approach in the courses. This approach is evident in the emphasis on the connection between assessment and learning, using SA with a formative perspective, communicating the results of assessment, considering various principles and values in making assessment decisions, and aligning assessment with curriculum goals and national policies. This learning-oriented assessment is also observed in Western Canadian TE programs (Poth, 2013) while TE programs in the United States (DeLuca & Bellara, 2013), Hong Kong (Lam, 2015), Columbia (Giraldo & Murcia, 2018), and several European countries (Vogt & Tsagari, 2014) were criticized for ignoring this aspect of assessment. This is a major difference in relation to SA literacy the three universities offer in the TE curriculum in Sweden and Finland.

The greater focus on formative strategies rather than SA is in line with learning-oriented assessment. However, this may also result in a lack of sufficient SA skills and techniques that teachers would need in schools. Moreover, formative practices such as teacher feedback, self-, and peer assessment may also entail the use of homework and classroom tests. This is evident in University A’s focus on general assessment, which could be at the expense of testing theory and practice. Including structured and systematic instruction about test construction and techniques may eventually help teachers gain insights about and appreciate the contributions of SA and the end-of-year/course higher stake test results provided by external bodies such as national agencies (Jönsson & Klapp, 2020). Teachers need to be able to develop expertise in different forms of assessment, methods, and instruments but at the same time pose critical questions on their suitability for different learning situations. The assumption that, for example, multiple-choice items (which often appear in national and international tests) do not give valid information about language skills and only formative assessment tools are of any value might result in a situation where teachers do not have a wide enough variety of assessment instruments at their disposal. Both types of assessment forms may be valid and/or misused, but it is important that teachers have access to knowledge and skills in both areas.

Testing as well as grading and appraisal of achievement are addressed at all three universities but due to the general nature of the learning objectives and content labels, it is difficult to judge the level of skills developed in relation to these areas through an analysis of curriculum documents. Still, University A and B through stand-alone courses have more opportunities to address these specific skills and practices than University C, which does not offer any such course. Considering teachers’ SA literacy needs in schools, a more direct focus on specific areas of assessment such as developing items, establishing item reliability and validity, reporting achievement, and modifying assessments (DeLuca & Klinger, 2010) might be needed in TE programs.

Dispositions such as ethics, fairness, transparency, equality/equivalence, as well as the consequences of assessment for students, teachers, schools, and society in general, is a priority area in all programs, but it is not as explicit in the learning objectives at University C. Our coding was more interpretive in this category to draw meaning out of general learning objectives based on the experiences of the researchers as teachers of some of these courses. Such an implicit focus on dispositions may run the risk of not attending these critical aspects of assessment due to different understandings and approaches of the teachers. Therefore, one of the implications of this study is that learning objectives should be more explicit so that they provide clearer rules and guidelines to teachers and pre-service teachers. This is particularly important since some of the earlier studies in other contexts indicated that there was less interest in assessment dispositions among pre-service teachers even though they attached significance to critical areas of assessment in the knowledge and skills dimensions (DeLuca & Klinger, 2010; Fulcher, 2012; Vogt & Tsagari, 2014).

Stand-alone compulsory or optional courses on assessment (University A and B) allow covering detailed aspects of assessment while embedded courses include more general objectives and content to address assessment in relation to other areas of education (University C). Universities A and B offer assessment literacy throughout the programs, whereby certain learning objectives are repeated across several courses. As our analysis does not involve the impact of courses, we cannot speculate on the consequences of these approaches for pre-service teachers’ development of SA literacy. However, this distributed approach might provide pre-service teachers with more opportunities to reinforce and deepen their understanding and perhaps sharpen the skills they develop.

Courses with a specific focus on SA will be essential as long as school systems continue to employ tests and grades. TE needs to be sustainable in educating pre-service teachers to work in a profession in which changes occur often in both school policy documents and grading practices. It is therefore imperative that language teachers can manage assessment theory and practice about SA literacy, as they are both expected to construct a variety of tests and understand national test results as well as inform students and their guardians about final grades. Future teachers need to be confident in the high-stakes decisions they are expected to make, and which can determine the basis for students′ future studies and career opportunities.

In summary, this document analysis shows that SA receives less attention in the TE curriculum in these three universities, which might be due to the current emphasis on formative assessment in both countries. Assessment literacy as an overarching concept requires teachers to align formative and SA to serve the overall gain of student learning (Broadbent et al., 2018). It is therefore important for TE programs to explicitly address formative and summative aspects of assessment in a balanced way. The subdomains of knowledge, skills, and dispositions of both strands should be considered in planning and implementing instructional activities. In addition, mentor teachers should work with pre-service teachers in line with the competencies they are expected to use in schools.

We acknowledge that content analysis of program and course documents provides a limited understanding of what takes place in TE. These documents are often general, and their implementation might present variety in the depth, breadth of the concepts covered, the extent of skills emphasized and practiced, the sensitivity to assessment values and principles. Methods of instruction, activities, assignments, and examinations could also have a large impact on pre-service teachers′ learning. Furthermore, we recognize that the contextual characteristics of the programs pose a potential threat to our analysis as they present varied opportunities and processes for pre-service teachers such as available time and pressure from other studies. In Finland, for example, all courses are packed into one study year, while more time is available to University A and B students to digest the content and practice with skills, particularly in the longer programs. Although limited, this curriculum analysis enhances our understanding of the current state of the scope and nature of SA literacy offered as intended learning experiences for pre-service teachers.