Faculty Perceived Barriers and Attitudes Toward Peer Review of Classroom Teaching in Higher Education Settings

Abstract

Peer review of teaching (PRT) is one of the various assessment methods employed to assess teaching effectiveness. The current meta-synthesis aims to summarize the faculty perceived barriers and attitudes toward PRT process. A systematic search was done across a range of databases, with 26 studies being found acceptable for data extraction. Because all the data were presented in narrative form only, no statistical test was applied to assess the significance, difference, or association between the variables. A thematic analysis was performed, which identified the following significant issues: the importance of the involvement of teaching experts in the PRT process, the content of PRT and quality of feedback, objectives associated with PRT, and faculty perceived barriers to PRT. Overall, it is shown that academic culture, time-related issues, and the need for restructured PRT guidelines are common barriers to a willingness to participate in PRT.

Introduction

Peer review of teaching (PRT) is one of the various assessment methods employed to assess teaching effectiveness (Henderson, Turpen, Dancy, & Chapman, 2014). British dictionary defines “peer” as a person who is an equal in social standing, rank, age, and so forth, and “review” as to look at or examine again or to inspect formally or officially. The importance of quality and effectiveness in teaching has resulted in the widespread utilization of PRT. In general, PRT has been utilized in its two forms, namely summative and formative evaluation (Keig & Waggoner, 1994). Summative assessment often serves as a tool for academic promotions/appointments, whereas formative evaluation is well recognized as an assessment specifically designed to improve teaching quality (Keig & Waggoner, 1994). Given their distinct rationales, Keig and Waggoner (1994) recommended the combined use of both summative and formative assessments. In today’s education system, PRT is used for a wide range of purposes (O’Leary, 2012). In university, PRT is recognized as a means of helping academics to meet all of their institution’s standards in terms of professional services and development in the context of tenure and promotion (Osborne & Purkey, 1995). In schools and colleges, however, PRT is widely used for performance management as part of a centralized quality assurance system (O’Leary, 2012). Moreover, in today’s education system student evaluations are also often employed alongside PRT to assess an instructor’s teaching effectiveness (Henderson et al., 2014).

PRT involves colleagues giving and receiving feedback on each other’s teaching practices (Burrows et al., 2011). There are three main models by which this takes place, namely, the evaluation model, the developmental model, and the peer review model (Gosling, 2002). Gosling (2002) compared these three models and found explicit differences between them in terms of the status and role of the reviewer, what is reviewed, and the purpose of the PRT exercise. The evaluation model emphasizes the involvement of senior staff who serve as observers to evaluate teaching performance for quality assurance and assessment. The developmental model, meanwhile, prefers the use of educational developers or expert teachers to conduct classroom observations and to evaluate teaching competencies. Last but not least, the peer review model employs teachers to observe each other, to encourage mutual reflection via engagement in discussion. Regardless of which model is used, PRT has significant benefits for teaching quality, and this is why it is incorporated as an essential part of many academic programs. For example, PRT is considered as an integral part of lecturers’ continuing professional development (Lomas & Nicholls, 2005) and, more recently, as part of postgraduate awards in learning and teaching, which are a compulsory qualification for new staff in many universities. Indeed, PRT is seen as an integral part of the on-the-job training provided as part of the Postgraduate Certificate in Higher Education (PGCHE) to offer on-the-job training (Elton & Partington, 1991; Hardman, 2007).

Generally, in today’s practice, the “peer review” model is more applicable compared with other two. The main reason for this is the higher reliability of evaluation and involvement of peers rather than senior staff or teaching experts or students (Greguras, Robie, & Born, 2001). Most universities have established committees to ensure that PRT is up to mark (Wellein, Ragucci, & Lapointe, 2009). PRT helps to develop one’s teaching capabilities and enhance students’ learning through constructive feedback (Harris et al., 2008). In addition, Piggot-Irvine (2003) found that PRT serves as a mutual process that benefits both the reviewer and the teacher being reviewed to identify areas of improvement for both. Good practices identified during the PRT can be disseminated across the faculty (Cox & Ingleby, 2014).

In spite of all these benefits, it is never easy to implement PRT in a teaching setup. It is possible that there may be some criticism and barriers to the PRT process. Brent and Felder (2004), for example, argued against PRT on the basis of the inconsistent perceptions of reviewers and different interpretations of teaching effectiveness. In addition, it is difficult to have an unbiased assessment when observing just one or two teaching sessions (Brent & Felder, 2004). Bingham and Ottewill (2001), meanwhile, criticized self-congratulatory feedback from peers, and Cox and Ingleby (2014) challenged the capability of lecturers to assess someone else’s teaching. Despite these various challenges, PRT is widely used but there remains a lack of any concise compilation that addresses the attitude of teachers/instructors to PRT after they were assessed. Keeping in view the importance of such exploration in teaching, the current review aims to compile perspectives and attitude of lecturers toward PRT.

Method

A systematic search was conducted to identify potential papers for inclusion. Studies published in English were searched through the Education Resources Information Center (ERIC). To ensure an effective search, keywords and search strings were defined as follows. Common search terms that were used were peer review, perception, teaching, barrier, summative review, formative review. Furthermore, PubMed, Medline, Embase, EBSCO, and Google scholar search (Google Scholar, Google) were also searched to ensure that no article was missed. Upon completion of the initial search, a list of studies was generated, and duplications were removed. Selection of studies for data extraction was done based on the following criteria:

• The studies must address the PRT process in a higher education setting.

• Any studies addressing students’ views about teaching quality was excluded from the study.

• Any studies found to present data about students’ peer assessments was excluded from the study.

• Studies addressing peer review issues in research were also excluded.

• Last, any study presenting data about teachers’ self-assessment or evaluation of teaching was also excluded.

• Studies must be published in peer-reviewed journals.

The period of time covered by the search depends on the availability of access to electronic databases from the Monash University library during March 2015 to June 2015 which the searches were performed. Details about the search strategy implemented using “Preferred Reporting Items for Systematic Reviews and Meta-Analyses” is shown in Figure 1. For effective data extraction, an electronic form was generated in an Excel® spreadsheet into which were input the perspectives and attitude of faculty toward PRT as extracted from each study. A total of 26 articles were found to be eligible for data extraction. Because all the data were presented in narrative form only, no statistical test was applied to assess the significance, difference, or association between the variables.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) search strategy.

Results

Once the data extraction was completed, the entire Excel® spreadsheet was reviewed for similarities and differences in the outcomes reported in each research report/studies. Keeping in view the similarities, four main issues were generated to present the data in such a way as to address the objectives of the study. The issues devised are shown below:

• Importance of the involvement of a teaching expert in the PRT process

• Contents of PRT and quality of feedback

• Objectives associated with PRT

• Faculty perceived barriers to PRT

Importance of the Involvement of a Teaching Expert in the PRT Process

Teaching experts are the peer review panel members invited to assess teaching quality. The role that studies assign to a teaching expert varies: Some prefer inviting them to provide PRT training for faculty members (DiVall et al., 2012; Wellein et al., 2009), while others insist that the expert should review the faculty in a class observation (Pattison, Sherwood, Lumsden, Gale, & Markides, 2012). Despite their different roles, faculty members unanimously support the involvement of teaching experts (Pattison et al., 2012). Yiend, Weller, and Kinchin (2014) stated that the involvement of teaching experts in PRT enables lecturers to receive informed and critical feedback regarding their teaching quality.

In some cases, it was noticed that the presence of teaching experts could cause lecturers anxiety and apprehension prior to or during PRT, although initial meetings and prior discussions may assist in reducing such problems (Pattison et al., 2012). Goldberg et al. (2010) reported that lack of information and prior training about PRT also caused a substantial degree of uncertainty, due to the teachers being evaluated not being familiar with the contents of the assessment. Those with previous experience of PRT, however, were found to be satisfied with the structure of the PRT (Bornmann, Mittag, & Danie, 2006; Wellein et al., 2009), and the presence of observers during class observations did not bother the instructors (Pattison et al., 2012). Yiend et al. (2014) reported that experienced teachers are reluctant to volunteer for PRT assessment, believing that since they are experienced they have already refined their teaching to an optimum level. DiVall et al. (2012), however, reported that this attitude may change after participation. They explained that the lack of understanding about PRT results in misconceptions, which improve dramatically after a couple of sessions (DiVall et al., 2012; Pattison et al., 2012).

Contents of PRT and Quality of Feedback

The perception of faculty members concerning the components that constitute an ideal framework for PRT is identified as a main issue in most of the studies. Generally, faculty members want the reviewer to examine a full range of their peer’s teaching, covering most of the activities in which students commonly engage as well as instructional scholarship (Burrows et al., 2011) including appropriateness of course content, and their own presentation and competence to deliver this content (Osborne & Purkey, 1995). Most teachers went through PRT with an intention to identify areas of improvement and enhance quality teaching through expert feedback and reflection about their teaching (Iqbal, 2014). In cases when the PRT panel was found not to be comprised of teaching experts, however, faculty members were disappointed with the feedback received. Iqbal (2014) reported constructive feedback as a key part of the PRT process, and noted that most senior members participating in his study were concerned with “what happened” after PRT that only involved class observation. It is very important, therefore, that there should be a verbal or written feedback identifying the strengths and weaknesses in one’s teaching. Iqbal (2014) explained that, where feedback was missing, the opportunity to learn and improve was missed. Overall, the presence of experts and importance of critical feedback were reported to be the main concerns by most of the researchers. Some studies highlighted that participants commented on the vagueness of feedback: that it was non-specific and unable to identify critical areas for improvement (Iqbal, 2014; O’Leary, 2012; Yiend et al., 2014). Moreover, some argued that feedback focused on the positive aspects of the teacher, thus reflecting no need for improvement. Referring back to the first issue, the involvement of teaching experts in PRT remains a core element for constructive feedback. It is noticed that when expert members participated in the PRT session most were satisfied with the assessment of their teaching session and the feedback they received (Bornmann et al., 2006; DiVall et al., 2012; Pattison et al., 2012; Wellein et al., 2009). The majority gave a high level of recognition to the evaluation work of the reviewers and they agreed that the received feedback was balanced, with positive and constructive suggestions. Many felt that the feedback was valuable and appropriate and that it resulted in improved teaching with critical reflection and innovation. Importantly, O’Leary (2012) pointed out that effective PRT requires reflection to be focused on wider issues in the teaching and learning process and not just on that observed lesson(s). He also stressed that quality enhancement rather than a quality assurance instrument is a more powerful way of encouraging continuous professional development.

Objectives Associated With PRT

The reviewed literature provides various purposes that are often associated with PRT: It can be institutionally oriented or can be the personal objective of the individual lecturer. O’Leary (2012) found that, in the U.K. education system, PRT is utilized to improve academic performance. Irby (1983), however, reported PRT to be a tool to govern academic promotion. In some cases, PRT has an elective but punitive nature where participants appear for PRT if they are willing to teach at another campus as sessional or part-time teaching staff (Osborne & Purkey, 1995). They found that the system of PRT was useful for ensuring the counselor education faculty were fully engaged in activities relatively unique to the field of counseling while meeting their college or university responsibilities.

Addressing the personal objectives associated with PRT, many faculty members perceived PRT as contributing to continuous improvement via collegial and collaborative assessment (Blackmore, 2005; Kumrow & Dahlen, 2002; Lomas & Nicholls, 2005; O’Leary, 2012). Moreover, many believed that PRT is a potential way to help each other to reflect on the quality of teaching and to identify areas for improvement (Blackmore, 2005; O’Leary, 2012). It also assists in learning new techniques to enhance students’ learning and teacher’s confidence in their ability to teach (Blackmore, 2005; Keig, 2000). In addition, Blackmore (2005)’s review emphasized the benefits of PRT for new and part-time faculty members. Her findings revealed that PRT served as developmental process for new faculty members. As for part-time staff, their inclusion in PRT enhanced their sense of belonging to the institution. Moreover, there were some benefits for the existing staff as well, in particular in terms of enabling them to learn new ways of teaching and presenting information during lectures. It is also believed to promote collegiality within the department (Kumrow & Dahlen, 2002).

Faculty Perceived Barriers to PRT

Identifying the barriers to PRT is perhaps the main theme that is directly associated with the objectives of this narrative review. It was quite common to observe anxiety in some faculty members about their peers, apparently, passing judgment on them (Blackmore, 2005). In addition, it was noticed that many lecturers consider PRT as a burden (Blackmore, 2005; Bornmann et al., 2006; Fleak, Romine, & Gilchrist, 2003; Hutchings, 1996; Lomas & Nicholls, 2005) In cases where PRT aimed to achieve quality assurance–related goals, some perceived this as an attempt to keep staff on their toes, while others criticized it as a disguise and that, in fact, PRT was just a paper exercise to tick boxes in order to provide paperwork evidence (Blackmore, 2005). Faculty members were willing to participate in PRT insofar as they thought it to be expedient for the university to implement its own internal systems to ensure quality of teaching, but a minority of staff members were openly hostile about such an imposition (Lomas & Nicholls, 2005). In some cases, the resistance to PRT was found to be associated with bad experiences of evaluation systems that the participants had previously gone through (Kumrow & Dahlen, 2002). It is observed that if the assessment is more quality assurance oriented or conducted by the administration, rather than the peers, participants were more likely to be dissatisfied with the management of the PRT system (Kumrow & Dahlen, 2002). O’Leary (2012) suggested the dual purpose of PRT, that is, for performance management and developmental needs, made participants perceive PRT negatively as a “quality assurance in the guise of tutor support.” Moreover, uncertainty regarding the components of PRT contributed to resistance (Kell & Annetts, 2009; O’Leary, 2012). O’Leary (2012) reported that most of the respondents were uncertain regarding the expectation of their roles, whether as an observer or as the one being observed. A lack of shared understanding of the term critical feedback and a lack of experience made them uncomfortable to engage in the culture of providing constructive feedback (O’Leary, 2012). Overall, it was noticed that there was a lack of consensus and clarity in terms of purpose of the PRT process, despite their appearing to be a consensus about the benefits PRT could provide (Kell & Annetts, 2009).

Iqbal (2014) reported similar concerns. Most interviewees in this study admitted that they conducted PRT blindly without any standard guide or check list. Thus, poor understanding about the contents and outcomes of PRT led to a subjective rather than objective assessment. It is possible that the subjective nature of the assessment made them less confident in sharing their critical views about the teaching quality and areas of improvement (Iqbal, 2014). Moreover, in most cases when colleagues are assessing a colleague, it was preferred not to give negative feedback which may undermine the colleague’s confidence (Blackmore, 2005). To facilitate appropriate feedback, many believed a standardized tool (Burrows et al., 2011), instruction in assessment methods, and the standards that were being assessed should be provided (Keig, 2000). To facilitate the perception of PRT as non-threatening, reviewers and the ones to be reviewed should meet prior to class observation and discuss (Burrows et al., 2011). To ensure an unbiased PRT, several independent class observations should be conducted (Eddy, Converse, & Wenderoth, 2015).

A lack of inclination to invest large amounts of time in PRT was found in Iqbal’s (2014) study. The interviewed faculty members revealed that universities often focus on research; hence, there was doubt as to whether PRT would actually contribute to decisions about career advancement. In contrast, the majority of the participants in DiVall et al.’s (2012) study felt that the benefits attained from PRT outweighed the effort and provided an opportunity for colleagues to discuss substantive issues that are highly stimulating (Bernstein, Jonson, & Smith, 2000). In addition, respondents from some studies reported their preference to be peer reviewed by a colleague rather than an external reviewer from another discipline or areas of expertise (Quinlan, 1995).

Discussion

The above assessment of the literature has identified three main issues that the authors would like to discuss to improve the willingness of lecturers to participate in PRT. These issues are

• academic culture and feasibility of PRT,

• consensus over the type of assessment for PRT,

• time-related issues for involvement in PRT, and

• content of the PRT process.

Academic Culture and Feasibility of PRT

Elements of academic culture among faculty members stood out as the crucial factor affecting the attitudes and perceptions of faculty members toward PRT, which consequently affects its feasibility and implementation (Blackmore, 2005; Iqbal, 2014; Kell & Annetts, 2009). The value placed on collegiality (Iqbal, 2014), beliefs, and teaching norms indirectly affects whether there is a culture among faculty members to accept constructive criticism, and one that sees such criticism as an opportunity for improvement rather than as a threat (Blackmore, 2005). To understand the academic culture, however, it is essential to understand the concept of organized skepticism and universalism (O’Meara, 2011). Merton (1942, 1957, 1973) explained organized skepticism as the phenomenon where we critically scrutinize scientific claims, methods, and findings according to accepted standards and criteria via a structured community scrutiny. The norm of universalism, meanwhile, explains the evaluation of claims based on universal criteria rather than personal criteria such as gender, personality traits, and nationality (Merton, 1942, 1957, 1973). We can apply the knowledge of organized skepticism to explain the phenomenon that when faculty members are not trained about PRT beforehand, they do not understand the purpose or the components of PRT; hence, they will not be confident enough to criticize someone else’s teaching. When they are asked to judge on someone’s teaching without any standard guidelines which they agree on, it is against the norm of universalism to judge blindly on someone’s teaching. Hence, this explains why training by teaching experts, or the provision of standard guidelines, is important in assisting an effective review process.

Moreover, tenure is another force we found to be embedded in the culture which drives the academic norm. As mentioned in the introduction, PRT is utilized as a summative evaluation for making personnel decisions or award nominations. (Brent & Felder, 2004) Tenure is highly valued by faculty members as it confers job security, power, and prestige (Chait, 2002; DaCosta, 2012). Feedback, especially in written form, and even if intended to be constructive, which focuses on areas for improvement carries the risk of being misconstrued and consequently might impede colleagues’ tenure decisions. This is why lecturers often despise the dual PRT purpose of formative and summative evaluation, because this creates suspicion and distrust as one can risk losing one’s job or promotion chances by participating in PRT. Lomas and Nicholls (2005) suggested embedding PRT as part of departmental culture, one needs to understand organizational culture and sub-cultures of a university and its departments with its particular historical and political issues and one should challenge the culture if necessary. A new concept on teaching and student learning is stressed by Hutchings (1996) for a better prospect of PRT. Hutchings (1996) mentioned faculty should turn classrooms into communities of scholarly inquiry in which students can be authentic participants and that teaching should be seen as an aspect of scholarship rather than a separate technique. When such a concept is embedded, choices about course design, assignments given, criteria for evaluating student learning are all reflections of the way the teacher thinks about his or her field and what it means to know it deeply.

Consensus Over the Type of Assessment for PRT

Lomas and Nicholls (2005) consider PRT to be a complex process that requires careful and sensitive management. Bernstein et al. (2000) recommended the non-normative approach to evaluating teaching practice. In a non-normative approach, the reviewee sets goals individually that will be assessed by the reviewers, and upon completion of PRT, written comments are only exchanged privately between peers for formative use. Burrows et al. (2011), however, supported the use of the nominal group technique (NGT) that engages academics in discussion to develop a PRT framework from the initial stage, therefore assisting the establishment of a collegial network and promoting a peer review culture in the faculty (Osborne & Purkey, 1995).

It is essential, however, to get the faculty’s consensus about the assessment method. Obtaining consensus is a sensitive issue, and during the process, it must be ensured that the opinions from faculty members are heard. Such an initiative consolidates collegiality and shared understanding. It encourages the building of trust among peers and in the PRT system, and this enhances the willingness of lecturers to volunteer. Once the framework of PRT is decided and criteria for assessment are outlined in consultation with faculty members, there was a greater degree of satisfaction with the PRT process (Burrows et al., 2011; Osborne & Purkey, 1995).

Time-Related Issues for Involvement in PRT

Time is another critical issue that detracts from the willingness to take part in PRT, because PRT is perceived as a time-intensive process (Keig, 2000; Lomas & Nicholls, 2005). The need to ensure minimal disruption in faculty members’ time was stressed; however, for an effective and unbiased assessment the reviewer and the reviewee may need several independent observations (Eddy et al., 2015). Experts recommend that 45 to 60 min need to be set aside for pre- and post-class observation discussions (Siddiqui, Jonas-Dwyer, & Carr, 2007). More importantly, the effort staff members take in setting aside time for PRT should be regarded and valued, especially in university cultures where research productivity has equivalent or greater importance than teaching (Fleak et al., 2003; Iqbal, 2014; Keig, 2000).

There has been a lack of evidence to demonstrate the extent to which PRT really has produced quantifiable outcomes. Certain groups of faculty members perceived lack of time as a factor in low participation and completion rates of evaluative processes (Fleak et al., 2003; Schultz & Latif, 2006). Those motivated to be peer reviewed were, however, found to report improvements in their teaching practice (Eddy et al., 2015).

Blackmore (2005), meanwhile, stressed the importance of quantifiable and objective outcomes to encourage faculty members, especially experienced academics, to embrace PRT. In addition, this measure assists faculty members to explain if the outcome justifies their effort in participating in PRT. In addition, Keig (2000) suggested a rewards structure to enhance faculty members’ willingness to participate in PRT. The implementation of rewards structures would not be possible without the support of the faculty, however. For example, extra pay can be allocated to teachers in participating PRT. A cautious approach should be taken to implement a balanced pay system, however (Kumrow & Dahlen, 2002). An example of a reward structure can be seen in Osborne and Purkey (1995)’s study in which merit pay was forfeited if faculty members opted not to participate in PRT. It was suggested by Osborne and Purkey (1995) that such an approach was successful in encouraging participation. Wellein et al. (2009) recommended decreasing the number of evaluators and increasing the interval of evaluations to every 2 to 3 years, to help reduce the time and workload of faculty members with respect to PRT.

Contents of the PRT Process

From the results, we conclude that the essential contents of PRT are the appropriateness of course content, how essential content is presented, general professionalism of colleagues, and scholarly competence (Osborne & Purkey, 1995). It is desirable to perform these assessments though a combination of indirect methods, in which course materials are evaluated, and direct methods, such as class observations (Burrows et al., 2011; Keig, 2000). The possibility of biased PRT has been stressed and can be countered by conducting several independent class observations or by involving multiple reviewers (Eddy et al., 2015). The involvement of multiple reviewers, apart from increasing the reliability of PRT, also helps to gain new insights in teaching effectiveness by gathering varying viewpoints (Keig, 2000; Sealey, 2013). Triangulation with student feedback is also worthwhile to provide an additional perspective on teaching effectiveness.

In a nutshell, a consensus was identified among lecturers in that they believe a standardized tool is crucial to facilitate appropriate feedback (Burrows et al., 2011). Such a standardized tool to assist PRT should include guidelines for instruction in the assessment methods and standards to which teaching is assessed (Keig, 2000). Discussion pre- and post-PRT is essential for a smooth and non-threatening PRT process (Burrows et al., 2011). Establishing a committee to assist, monitor, and assess PRT has been shown to be useful in guiding a smooth PRT (Bernstein et al., 2000; Fleak et al., 2003; Osborne & Purkey, 1995). Above all, the results have shown the importance of involving teaching experts in PRT. O’Leary (2012) mentioned training in PRT by teaching experts as a way of exposing faculty members to the “new” kind of culture of providing constructive feedback. This exposure helps lecturers to feel more comfortable in participating in PRT, which is important in ensuring a suitable PRT implementation (Kell & Annetts, 2009). A hybrid model has also been suggested by Yiend et al. (2014) which combines peer review and developmental models (Gosling, 2002) as interventions: Involving teaching experts has in particular resulted in improvements in critical reflection of teaching, essential in PRT (Bornmann et al., 2006; DiVall et al., 2012; Goldberg et al., 2010; Pattison et al., 2012; Wellein et al., 2009; Yiend et al., 2014).

Conclusion

It is identified that time and culture-related issues are the main barriers to PRT. In addition, faculty’s concerns regarding reward and recognition were identified as other factors hindering faculty members from volunteer for the PRT process. Institutions willing to adapt PRT to assess the quality assurance in teaching must consider culture and time-related issues in advance to avoid complications later on. In addition, for an effective PRT, it is essential that teaching experts are involved and that there are structured guidelines outlining the process to the reviewers, and objectives and outcomes to the reviewee. The absence of structured guidelines leads to mistrust in PRT and the reviewers and reviewee will consider PRT as a documented formality rather than a way to learn and improve teaching practices.