Abstract
This paper focuses on the influences and changes of recent Taiwan teacher preparation program evaluation (TTPPE) as one of the national evaluation projects conducted by the Higher Education Evaluation and Accreditation Council of Taiwan. The main concerns are what kind of ideology is transformed through the policy by means of evaluation, and what influence TTPPE has as well as its usage. The first section provides a brief introduction of the development of TTPPE. The second section illustrates how market ideology (mainly neo-liberalism) influences and transforms into policy agenda. The third section critiques original usage of TTPPE outcomes. The last section shows the visions and challenges for TTPPE. The research methods include policy analysis of official documents and interviewing four stakeholders and faculty members to understand their experience and reflection. The paper is expected to provide critical insights for future teacher preparation program evaluation projects as well as for other teacher programs evaluation.
Introduction
While teacher quality influences education quality, education policy makers around the world are often concerned about the quality of teacher preparation programs. Evaluation provides a mean to understand the quality of the education program and even improves program quality. This paper provides some critical perspectives on the Taiwan teacher preparation program evaluation (TTPPE) by conducting document analysis and interviewing stakeholders who are involved in TTPPE. TTPPE has been carried out for around a decade influencing Taiwan teacher education. Stakeholders provided some insights into not only understanding TTPPE but, further, providing critiques for future evaluations.
Taiwan has been shifting its teacher education system from a restricted system toward open multiple form since 1994. In the past, only students who had graduated from normal universities or teacher colleges were able to obtain a teaching certificate to teach in elementary or secondary schools. However, new teacher education law allows every higher education institution to provide teacher preparation programs under supervision from the Ministry of Education, Taiwan (MET). Starting from 1995, MET further conducts teacher program supervisions and evaluations to assist each teacher program either to improve or maintain certain quality. These teacher program evaluations were originally designed for higher education to self-evaluate for improvement. This policy was changed around the early 2000s. Accountability ideology was further introduced in evaluation policy during that time. Ever since, the policy makers have strengthened the usage of evaluation outcomes influencing the development of teacher education institutions. Negative evaluation outcomes can result in closing teacher education programs. Supporters said that TTPPE evaluation can improve efficiency while opponents claimed evaluation should encourage improvement instead of only punishing. In fact, in this study, stakeholders even stated political influences in TTPPE which may restrict the usage of evaluation. The following sections are going to offer some literature reviews relating to this study, analysis of interviews, and visions.
Taiwan teacher program evaluation and policy
Global neo-liberalism influences and accountability
Global neo-liberalism extends its influence to educational policies recognizing the connection between education and economic growth (Marginson, 1997; Marginson, 2007). This ideology influences policy makers and the public to adopt management ideas into public policies to embrace market-oriented policy concerns. Shifting to market-oriented ideology led to corporatization, privatization, commercialization, and demands for accountability (Lipman, 2004). These public management policies based on neo-liberal ideas have emphasized accountability and efficiency (Biesta, 2009; Kehm and Lanzendorf, 2006). There are also increasing measuring activities and methods that increase efficiency while reducing costs as “managerialism” for institutions’ performances (Trow, 1994). Therefore, the neo-liberalism policies often consider management as a proper means to achieve accountability to manage resources achieving certain goals.
Managerialism can be a way of maintaining appropriate levels of quality control at a reasonable cost (Harvey and Knight, 1996). Some countries have used a quality control system to evaluate and control higher education quality. For example, the United Kingdom government developed a quality assurance system during the 1980s partly to justify government expenditure on higher education (Kells, 1999). This also either supported or gave pressure to develop metrics and performance indicators to monitor higher education institutions (Harvey and Knight, 1996). Furthermore, the United Kingdom formed the Higher Education Funding Council for England (HEFCE) and the Higher Education Quality Council (HEQC) to ensure value for funding and quality management for higher education (HEFCE, 1993; HEQC, 1996). In most studies, higher education is quite often restricted to measuring the effectiveness or efficiency of institutional management while discussing performance indicators (Clare, 2012).
In Taiwan, a similar organization, the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT), was established in 2005 to be responsible for evaluating Taiwan’s higher education institutions as a kind of quality assurance. As for the teacher education program, the MET directed the setting office of the National Academy for Education Research (NAER, legislation established in 2008) to conduct full teacher education preparation programs evaluation in 2006. By 2009, HEEACT had taken over the responsibility of conducting TTPPE. The MET is still involved in TTPPE through legislated government power. The evaluation outcomes may influence funding and administrative affairs. TTPPE offers accountability for the Taiwan government to determine the teacher preparation programs quality and answer to the public concerns about the teacher quality.
Evaluation and accountability
Evaluation and assessment are intended to record education quality and institutional effectiveness, conduct program improvement, and demonstrate accountability (Baker, 2004). However, without clarifying the goals of evaluation or assessment, evaluation and assessment may become an end in themselves instead of quality improvement or accountability (Baker, 2004). This may lead to misuse of evaluation.
Evaluations can focus on key issues and offer information for decision makers on accountability usage (Alkin and Christie, 2004; Stufflebeam, 2003). Manage-oriented evaluation often concerns who will use the evaluation outcomes (mostly decision makers) in addition to evaluation subjects, process, and results (Fitzpatrick et al., 2004). A program evaluation may emphasize and improve the accountability while offering important information for decision makers (Wholey, 2004). However, not everyone is in favor of accountability. The argument over whether it is proper to use accountability in higher education continues, especially while working out how to balance professional autonomy and accountability (Burke, 2005). When evaluation highly links to accountability, some evaluators argue evaluation should not only concern outcomes.
Professional concerns are not the only influential elements in evaluation. State priorities (political), market force, and academic concerns (professional) serve as the accountability triangle (Burke, 2005). Neo-liberalism influences policy makers to focus on accountability in which policy makers transform neo-liberal ideology though political power. Political concerns may play a crucial role while conducting evaluation. Patton (2003) stated that the misuse of evaluation mostly related to political issues. Weiss (1991) claimed that evaluation is influenced by politics and verse via. Political power may favor certain people and ideology influencing evaluation activities. Although the main purpose of evaluation is to offer information for policy makers to make decisions and manage resources, the voices of disadvantaged stakeholders should also be heard so that evaluation can serve social justice (House, 1991). Evaluation can provide more than one standardized perspective. Evaluation should work on improving communication among stakeholders to be more inclusive (Stake, 2000). There are multiple perspectives among stakeholders, and evaluators should consider the importance of democratic evaluations (McDonald, 1971). While evaluators should not be the main source for determining the values, evaluators communicate with main stakeholders to include multiple value concerns (Guba and Lincoln, 1981). Moreover, evaluation may play a more positive role to improve program quality and stakeholders’ ability. Fetterman (1996) supported the belief that those who have been evaluated should have the right to self-decided and empowered improvement. Therefore, evaluation may offer opportunity for program improvement and encourage stakeholders to express and communicate to each other. Furthermore, evaluation is not merely just comparing achievement of standard reaching but involves improvement and empowerment.
Taiwan teacher education evaluation policy
In the past, anyone in Taiwan who wished to teach in schools had to first enroll in or attend the teacher preparation program education offered by three normal universities or a few teacher colleges. While the Taiwan society became more open in the late 1980s, scholars and the public called for openness of teacher education programs. They critiqued teacher education which at that time trained teachers within limited institutions and resulted in a lack of diversity. Education reformers at the time believed that the more institutions that offered teacher education programs the greater the diversity of teachers who would be trained. Teachers from more diverse backgrounds can bring different insights into classrooms. In 1994, there were more higher education institutions offered teacher education. The MET in the past also controlled the total amount of number of qualified teachers each year to fulfill the needs of the market (schools). After 1994, a growing number of qualified teachers has overwhelmed the positions needed in schools causing challenging market competition and quality concerns.
In 2004, the MET carried out the “Planning for teacher education account project” which introduced accountability ideology into teacher evaluation policy. In 2006, MET announced the legalistic “Practice of Higher Education Teacher Education Program Evaluation” which linked evaluation outcomes to the enrollment number of each teacher program. In other words, only those institutions that pass the evaluation may maintain their number of enrollments of students into their teacher preparation programs. This law also clearly defined the standards, ways of grading, and grades. Institutions that pass the evaluation and are graded first class will maintain the number of enrollments. Those graded as second class will have a reduction of 20% in the number of enrollments the following year. If an institution has been graded as third class, it can no longer provide teacher program education if the program does not pass the follow-up evaluation. This policy influences teacher preparation program development directly. For institutions graded as second class, reducing the number of enrollments may also lead to new challenges. These challenges include cutting funding, less students enrolled, and further restricted faculty recruitment. Taiwan’s TPPE has a strong influence on program and institution development so that it gains a lot of attention.
Critiques and concerns
Interviewee coding and information.
Although interviewees critiqued Taiwan’s TPPE, they held relevantly positive perspectives. In their perspective, TPPE in Taiwan can encourage and force each teacher program to review their past and plan for future development. Evaluation can help to examine the past performance and encourage an institution to keep developing. (T1) Conducting evaluation can encourage us (teacher preparation program institutions) to develop. (T3)
Through literature and interviews, some critical issues appear in TTPPE, namely the evaluation indicators and usage, evaluator’s judgments, and political concerns.
Evaluation indicators and usage
In the TTPPE there is a set of evaluation indicators established by HEEACT. The original was designed to provide some guidelines for both the evaluator and the institutions being evaluated. However, in some cases, evaluation indicators become so influential that they restrict the perspectives of both the evaluators and the institutions. … evaluation indicators are not bad things. However, people seem to be restricted by these indicators. There should be other important things and issues need to be included in teacher program evaluation. (T1) Evaluation today can’t show the dialogue within the institution … the direction of a teacher program development should open the discussions opportunity for all faculty members in the university so that each teacher program can develop diversity according to their characteristic. (T3)
The content of evaluation indicators can also be re-considered. Some indicators may not apply to the real condition or the meaning is hard to define. These evaluation indicators do not necessarily fit into our (domestic) teacher program. They are adjusting indicators from other countries, especially from USA. (T2) Another important issue is how these indicators were designed and selected. (T1) They are putting more details into the (evaluation) indicators … also referring to the practical outcomes, some empirical data may not necessary reflect the indicators … (For example) does the rate of passing the teacher certification exam really reflect the learning outcome? (T2)
Institutions are forced to follow these unifying indicators instead of developing their own characteristics. Along with policies and legalistic involvement, evaluation indicators may become a means to restrict multiple developments among institutions. … (we) sometimes got lost in indicators … If failing to meet the indicators leads to punishment, tension would increase. (T3) … if the higher level administration (MET) requests to follow, too many evaluation indicators may harm the organic development of an organization. (T3)
The MET has legalistic processes to govern higher education institutions through funding and resources control. Using evaluation indicators on one hand can provide guidelines for the evaluation process, but on the other hand may restrict diverse development of each teacher program.
Evaluators’ judgments
TTPPE invites many professors as member of evaluation committee to conduct evaluations. Although these professors are experts in their academic field, some are not professional in evaluation. This may influence the outcomes of evaluation with critiques from the broader audience. Evaluation involves value judgment in which evaluators may play a crucial role. However, some evaluators arrived at the evaluation filed and showed their prejudices. If evaluators’ judgments were too subjective to influence evaluation outcomes, they may reduce the justifications of TTPPE outcomes. The evaluators committee will try to maintain their professional judgment to achieve high quality TTPPE. He (an evaluator) just arrived and saw me. He said that he doesn’t need to look around anymore. He already knew what to write down. But he had not even read the data yet. (T1) There are some issues about the evaluators … sometimes they are not familiar with the teacher program, so how can they provide reliable evaluation? (T2) Once there was one evaluator who insisted on his option … he is a notable professor … we took some time to persuade him to change his mind. However, most of the time the committee try to maintain an objective attitude. (T4)
The outcome of TTPPE is quite influential so that judgments from evaluators play an important role. According to the MET, those failing to meet evaluation indicators would face the consequence of reduced student enrollment and funding. Therefore, evaluators’ judgments and evaluation outcomes can influence an institution and the program. Although many evaluators offer useful feedback and professional judgment, some evaluators with a lack of evaluation training may influence the quality of TTPPE.
Political concerns
Moreover, the means of using TTPPE to achieve other political goals is a critical issue. Using evaluation indicators to evaluate and to give some kind of “punishment,” such as reduced funding, reduced student enrollment, restricted faculty recruitment (number of faculty in teacher program is based on the number of student enrollment) may lead to the closing of a teacher education program. Sometimes, political issues aroused, people from the higher level may influence. (T1) We can’t speak for them (people from MET), but there seem to be unspoken words from the MET … it seems that they (MET) might be considering the number of teacher programs. (T4)
There has been a shift of TTPPE usage and how to conduct evaluation in the past decade. During 1995–2002 the MET conducted supervisions which were used to understand how each teacher program was performing. The reports were only kept in official documents for institutions to read and to improve their performances. After 2003, the reduction in the birth rate also somehow influenced the teacher unemployment rate in Taiwan. Fewer newborn children meant fewer schools were needed indicating fewer teaching positions. As was stated in the introduction, teacher programs have changed from the government controlling the number of trained teachers to an open market. In such an open market, more qualified teachers are educated to compete for limited positions. While the qualified teacher unemployment rate rises, the MET seem to use TTPPE as a means to control the total number of graduated teachers. While TTPPE also shifted to the accredited system in 2012, TTPPE reminds us that the MET’s original design was as a means to control number of teacher program graduates. The evaluation outcome in each year is open to the public and follows the accountability system. TTPPE outcomes are so influential that people debated the usage of evaluation. Some argue for using TTPPE as a system of accountability while others argue against it. Some content of TTPPE should belong to administration affairs. MET should take responsibility and take over these affairs instead of using TTPPE to do so. (T1) It is okay if the MET uses TTPPE for accountability or to solve the problem of teacher numbers … (T2) Evaluation is a good thing to see how each program performs. (T4)
In political concerns, TTPPE also brings some positive influences on teacher education other than just from the aspect of evaluation. TTPPE may increase the interactive relationship among institutions and attentions from university administrators. In the past, each teacher program developed their way. Now with TTPPE, many institutions start to interact with each other to understand how to prepare for TTPPE and other relevant issues which can be a nice thing. (T1) The outcome of TTPPE is so influential that they catch university administrators’ attention … administrators of teacher programs can go to university level administrators to ask for faculty number, space, and resources to fulfill TTPPE’s legal requirements. (T2)
There are political concerns in TTPPE and it may increase if TTPPE emphasize the link between evaluation outcomes and punishment instead of quality improvement. However, the present TTPPE seems to strengthen the usage of the outcomes for determining student enrollment number and funding rather than supporting quality improving. Evaluation, like Stufflebeam said, is to improve, not to prove. But now (TTPPE) is to prove, and it also links to punishment. (T4)
Evaluation similar to TTPPE that influences a program’s long term development should not only concentrate on meeting objective standards but be concerned about other issues. Evaluation can further respond to the interests of more stakeholders and program participants. Moreover, evaluation can focus on improvement rather than just proving the program outcome achievement. The political influence of TTPPE can encourage but also restrict the quality of evaluation.
Vision and challenges
Beyond critiques and concerns, TTPPE is an important way to manage teacher preparation programs’ quality for the Taiwan government. Both government and the public need evaluation for accountability. However, the use of TTPPE and how to increase positive use of TTPPE are both crucial for Taiwan teacher education programs in the future.
Continuous needs for TTPPE
Evaluation is an important way to understand the strength and weakness of a program. While the Taiwan government has been conducting education reform in recent years, teacher quality plays a crucial role in the success of education reform. Although there are some critiques on TTPPE, TTPPE still remains an important role in managing teacher program quality. TTPPE can achieve the goal of understanding how a teacher program performs and can analyze a teacher program’s weakness. Therefore, the need for TTPPE will remain for both the government and the public.
As the neo-liberalism influence with increasing demand of public management and accountability, evaluation becomes a means of response to accountability. Whether evaluation should just become a means for government to conduct accountability is another issue for future discussions. One can be sure that evaluation will still play the core role for Taiwan teacher education development. The second round of teacher program evaluation in Taiwan started in 2012. This round of evaluation will end in 2016, and many institutions are facing serious consequences if they cannot achieve the standards. Although there are some critiques on TTPPE, especially connecting the evaluation outcome to program survival, using evaluation to understand and improve the quality of teacher programs is the main policy in Taiwan. This will also support and justify the needs of TTPPE.
Uses, quality and operation of TTPPE
TTPPE plays an important role not only in evaluating program performances, but also in determining a program development through the MET legal approach. The MET can cut the student enrollment number of a teacher program based on TTPPE outcomes. As stated before, the reduction in the number of students may result in closing the teacher program. There was a shift from “grading” to “quality accredit” in the TTPPE system in the most recent round of evaluation in Taiwan. However, this does not change the connection between unsatisfied performance and cutting student enrollment. Therefore, each teacher program still faces the pressure of proving their performance and justifying their existence.
From interviewees’ responses, there might be some ways to improve TTPPE. First, raising the evaluation profession of the committee is an important part. Although many committee members provide solid evaluation outcomes, some evaluators can improve evaluation quality with some knowledge of the evaluation profession. This may be done through more evaluation training and the awareness of political concerns of TTPPE which lead to the next paragraph.
Second, what TTPPE stands for can still be discussed. As an evaluation, if the use of TTPPE outcomes were only to prove the value or justify the program, it may reduce the function of program improving. This may also lead to more political concerns while conducting evaluation. Evaluation should offer the opportunity for a program to really find its disadvantages to improve. Too much emphasis on proving a program value means that sometimes people tend to only show their best part instead of showing their weakness. Clarifying the weakness may be more important for long term development. In addition, some administration affairs may not necessary need TTPPE to do the work. The MET can take responsibility to administrational work, such as a survey about the number of faculty members, or graduates earning teacher certification. Some data can be collected without using TTPPE to investigate.
Third, indicators and criteria used in TTPPE may restrict the diversity of teacher programs. Although the original policy of open teacher programs is to provide diverse teacher training, the MET and TTPPE are now involving and directing the way each teacher program develops. More legal documents and policy agendas from the government with regard to using TTPPE may reduce the freedom and diverse development among each institution. The MET can take control over teacher programs by using restricted requirements for the curriculum and using TTPPE. Evaluators and teacher program administrators are following and depending on TTPPE indicators. This method of evaluation shows that only things related to indicators are being emphasized or valued. Some indicators may not necessarily reflect the performance of a program. Some other important elements excluded from indicators are ignored.
The usage of TTPPE may be further discussed in the future. In what degree TTPPE outcomes should relate to program funding, student enrollment and development are all important issues. TTPPE may emphasize improvement rather than just judging the current value of a program. The quality of TTPPE is also a concern. Evaluators’ evaluation profession and the method used in the TTPPE, especially highly depending on standard indicators, may influence the overall quality of TTPPE. Policy makers and stakeholders can further discuss together to transform TTPPE toward the improvement of the program.
Conclusion
This study conducted interviews to not only understand TTPPE but also gather critical perspectives. There are three parts that impact the validity or at least the trust of the outcomes of TTPPE. They are: evaluation indicators and usage, evaluators’ judgments, and political concerns. There are some people questioning the content of evaluation indicators, saying they may not fit into the reality of Taiwan education. There are also some people who disagree with the method of using evaluation indicators that restricts the diversity of the teacher program and determines the existence of a program. Some evaluators in TTPPE are being critiqued as having academic profession but lacking in evaluation profession experience. Interviewees and literature both illustrated that political concerns may influence the TTPPE.
This study further suggested that TTPPE still retains, and should continue to retain, an important role in improving Taiwan teacher education. There will be a continuous need for evaluation. However, the uses, quality and operation of TTPPE can be further discussed. The outcomes of TTPPE can serve for improvement instead of only proving the present performance of a teacher program. Evaluation indicators in TTPPE may provide guidelines but not restrict the diverse development among teacher programs. The quality of TTPPE can also increase through adjusting TTPPE content to raise validity. As to the practice of TTPPE, more evaluation training for evaluators and different forms of evaluation can be introduced.
Although there are critiques on TTPPE, TTPPE can supervise the quality of a teacher program. As long as more people care about evaluation, it can develop and improve. TTPPE is influential to all teacher programs in Taiwan so that the uses and quality of TTPPE should also continuously improve. There are only four interviews conducted in this study. More interviews and data can be gathered in future studies to understand TTPPE and to provide more insights.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.