Assessing Mindfulness-Based Teaching Competence: Good Practice Guidance

Abstract

Inclusion of assessment of teaching competence in Mindfulness-Based Program (MBP) teacher training enables international benchmarking of standards, which in turn underpins the integrity of this emerging field and the potential to deliver effective, transformative interventions. However, there is a risk that the inclusion of competence assessment could lead to reductionism and undermining of the pedagogical features that make mindfulness-based teaching distinct. It can also make the costs of training prohibitive. The science underpinning the integrity of competence assessment is not yet robust enough to justify wide scale implementation, but when feasible, including the option for assessment enables trainees to engage in rigorous and effective training processes. When assessment is included, it is critically important that the process is held with awareness and sensitivity, and is implemented by experienced assessors with thoughtful governance. Navigating these issues involves balancing rigour with accessibility and pragmatism. This paper lays out some guidelines for good practice for MBP teaching assessment, and raises unresolved dilemmas and questions.

Introduction

This paper offers a synthesis of understanding from three university-based training centers in the UK and the US, on good practice for integrating teaching competence assessment into mindfulness-based program (MBP) teacher training and research. It draws on practice-based evidence (of integrating assessment into MBP teacher training pathways), and evidenced-based practice. ¹ It focuses on the main competence assessment tool in the MBP field – the Mindfulness-Based Intervention: Teaching Assessment Criteria (MBI:TAC). ²

Good Practice in Using the MBI:TAC to Conduct Assessments

Research indicates that inclusion of assessment of competence in MBP teacher training develops more effective teachers. ³ Assessment can be conducted in ways that support growth and learning, and are sensitive to the inherent vulnerability of the process; or in ways that undermine learning and damage confidence – the impact of which can last for years. It is critical therefore that the necessary ethical basis, governance and good practice for conducting MBP teacher assessment are considered. Embedding understanding of these principles into practice is necessary for individual assessors and for the organization holding the process.

Why assess? First, it is important to be clear about the rationale for assessing, and what is assessed. Within the context of university validated (e.g. Masters) program delivery, assessment is a necessary and embedded part of student monitoring, awarding qualifications and evaluating the program’s effectiveness. The choice points, therefore relate to the focus of assessments and methodologies employed. However, most MBP teacher training programs internationally are delivered as professional development, so are not surrounded by established university governance and frameworks. In these contexts, assessment is not a necessity, but is increasingly used to enable trainees and the training program to point to substantive evidence of attainment and rigor.

Within research contexts, it is necessary to report on intervention integrity.^4,5 From a wider field perspective, transparent and consistent systems for ensuring standards support the integrity and credibility of the work. Ultimately, it is only by exposing their teaching to external review that a teacher can validly represent themselves as qualified or competent. There are therefore a number of reasons to build assessment into training processes, but as outlined below there are also a number of counterpoint cautions. These include the importance of good governance informing the conduct of assessments, and the emergent status of the science on MBP teaching competence. It is wiser not to include formal assessments in training processes unless they are implemented with good practice and thoughtful governance. The MBI:TAC can—and we argue should–still be implemented informally in a range of ways within training and supervision.^6,7

It is important also to remember that an observational tool such as the MBI:TAC only assesses one element – i.e. the skills of the teacher observed during a teaching session. The tool is not designed to assess theoretical understanding, professional ethics and practice, or reflective skills. If assessment is included, it is important therefore that varied methods are employed (e.g. written assignments testing theoretical and professional knowledge, and reflective skills).

How are assessments governed and organised? Within the context of Master’s assessment there are existing and established governance and organizational structures that surround the conduct of assessment. These include: learning outcomes aligned to assessment tasks; timelines for submission/return; trained assessors conducting assessments with reference to marking criteria and learning outcomes; internal procedures for double marking (two assessors marking the same piece of work, blind to each other’s assessments), and moderation (overarching checks to ensure consistency); procedures for external scrutiny of assessment processes; and processes for responding to complaints or student challenges to grading which protect both student and markers. These processes provide a reassuring context within which to embed the use of the MBI:TAC as an assessment tool, as part of a multi-faceted approach to assessment.

When moving towards building assessments into contexts that do not have these established processes, we recommend modelling university structures to ensure that the assessments are conducted fairly and with integrity. If governance is poor, there is a risk of bringing the wider field into disrepute by awarding qualifications inappropriately; of giving students a poor learning experience; and of offering students an inadequate holding environment for the potential vulnerability of the assessment process. Most of these necessary governance structures are internal to the organisation: for example, establishing clear aims and organizational structures; ensuring that assessors are well trained and up to date with developments in assessment protocol; and building in internal moderation.

Future processes potentially require wider collaborative design of systems beyond each training organisation. Over time, it would strengthen integrity if the field moved towards setting in place systems for external moderation of teaching competence assessments. There is the potential to develop on a field level, a central organised body of practitioners who are expert in conducting assessments held by national or regional associations beyond the auspices of individual training organisations. This would enable organisations to build assessment into their training processes with linkage to the wider context; provide the organisation and their trainees reassurance that appropriate governance is in place; provide checks to ensure that assessments are aligned to central benchmarks; and strengthen the integrity of the wider field by establishing and communicating consistency about teaching standards. Graduation with a certificate of competence from one program would have equivalence with other programs. This is one path that could support global recognition of training level. However, in our view this development is contingent on the psychometric properties of assessment strategies being supported by the science. Research is in progress, but we have not yet reached this stage of development. In addition, aspects of diversity and inclusion in the field are only beginning to be researched and addressed.

Scaling up the inclusion of assessment within MBP teacher training brings the potential of increasing the quality of program delivery in regions around the world. However, the challenge of scale-ability and accessibility is significant. The two main challenges are training assessors and financial viability for trainees. Ideally, trainees are assessed in their own language by someone from their own cultural context. In countries where MBP training programmes have been more recently established, it can take years for teachers to gain the experience required to be eligible for training as an assessor. The costs involved in this training may result in assessment not being accessible to trainees from less economically advantaged areas and backgrounds.

How are assessments conducted? The MBI:TAC manual ² and the level 2 ⁸ in person training detail the process of conducting assessments, and assessors need training prior to assessing others. It is good practice that practitioners who assess others have had their own teaching practice assessed – both to ensure that they are familiar with the vulnerability inherent within the process, and to ensure that their teaching is at least at ‘Proficient’ level. The assessor also needs to be an experienced teacher of the particular MBP curriculum that they are assessing, to ensure that they are able to adequately assess adherence to curriculum.

The quantity of teaching practice material that is reviewed will depend on context and the time point of the assessment within the training process. Assessment can be staged by lowering the expected pass level for early trainees (i.e. ‘Advanced Beginner’ is an appropriate level for trainees in their first year of training whilst, ‘Competent’, is a necessary level for full graduation from the program). Staging of assessments can also be implemented by assessing earlier students teaching small elements of the curriculum (e.g. half hour clip of meditation practice guidance followed by inquiry). For final competence certificate review, it is recommended that trainees submit video recordings of the whole course. Written permissions from participants for the recording and its submission are needed. Carefully crafted written feedback ⁹ is given in addition to a competence level. The following procedure is employed:

• Markers randomly select two sessions to assess in their entirety. One session is selected from sessions 1–4, and one from sessions 5–8.

• Markers sample up to an hour of additional material to enable completion of the assessment. The following questions guide the sampling:

• Perhaps review how week 1 was started and week 8 was ended to assess management of beginning and ending (domain 6)?

• Was there a domain of competence that was particularly skilful/unskilful? Sample other examples of this to see whether this was consistently demonstrated or not.

• Have you formed a hypothesis about the teaching that needs testing by viewing more material? (I.e. the first clips of teaching may give the impression that the teacher moves prematurely to make a teaching point. To decide whether this is a pattern which replicates itself view more inquiry).

Risks and cautions: Clearly, the intention behind embedding assessment into training processes is to strengthen standards and support public access to high quality teaching. However there are some inherent risks. Introducing assessment into the training process can shift the focus from immersion in learning, to focusing on outcome. For example, trainees may have a sense of needing to ‘teach to the test’ to get a good grade. The process of teaching could thus become a performance or playing a role rather than teaching through an embodied connectivity to immediacy.

It is important therefore that the inclusion of assessment within a training process is an integrated part of an overall approach to cultivating trainee’s capacity to teach authentically.^7,10 The MBI:TAC was designed through close up observation of the actuality of the teaching process, and its implementation within training should not be shaping the teaching process, but rather should enable fidelity to its original intentions.