Guidance on the assessment of learning outcomes for those designing procedures and projects – Report of an ETPLAS Working Group

Abstract

This document provides assessment criteria for evaluation of each of the Learning Outcomes of the Modules specified (in addition to the Core Modules) for those designing procedures and projects in the Education and Training Framework guidance document by the European Commission and endorsed by the Member States Competent Authorities. This Working Group was tasked to produce these criteria by the Education & Training Platform for Laboratory Animal Science, which was funded by the European Commission to this aim. The assessment criteria address knowledge and skills (including critical thinking) expected to be acquired during education and training of persons preparing to design research procedures and projects using animals under the scope of Directive 2010/63/EU. Recognizing the diversity of expertise and experiences of those being educated and trained, we provide two levels of attainment, an ideal response and one that would be acceptable for each Learning Outcome. The balance between ideal and acceptable could be decided by the particular course providers and/or assessors, according to their local requirements. We envisage that the use of these assessment criteria by training providers and accrediting or approving bodies will help harmonize the education and training for those who will design procedures and projects using animals for scientific purposes. In Europe, this may also contribute to mutual recognition of training, and facilitate free movement of scientists.

Introduction

The purpose of the present document is to provide accessible guidance on the assessment criteria for the Learning Outcomes specified in the European Commission Education & Training (E&T) Framework guidance document ¹ for the training of personnel designing procedures and projects (Function B staff) ^a

a

The authors have provided assessment criteria for the Function B specific Modules 7, 9, 10 and 11 in this current document. Function B staff are required to also complete the Core Modules 1, 2, 3.1, 4, 5 and 6.1.

. Article 23 of Directive 2010/63/EU ² requires that staff undertaking this function are adequately educated and trained.

The Education & Training Platform for Laboratory Animal Science (ETPLAS) ³ was tasked by the European Commission funded Preparatory Action ‘Promoting alternatives to animal testing through accessible and harmonized education and training’ (grant agreement no. 09.200200.A092004/2021/864077/SUB/ENV.B.2) to provide assessment criteria for the Learning Outcomes stated in the E&T Framework guidance document. ¹ The authors of this assessment criteria document constituted the Working Group. The intention was to develop assessment criteria for the Learning Outcomes in accordance with good educational practice and to agreed quality standards. Assessment criteria are statements which specify (and itemize) the standards that must be met and what evidence will be taken to show achievement of the respective Learning Outcomes. ⁴

The assessment criteria developed seek to evaluate knowledge and intellectual skills acquired, and critical thinking. They follow a knowledge, skills, and attitudes (or behaviours) framework widely used (instead of Bloom’s taxonomy ⁵ ) in professional assessments,^{4,6 –9} and are forward-thinking in outlook. The aim was that they should be available, accessible, and understandable to educators and learners, and transferrable across the European Union. For those participating in these educational activities, which may include individuals undertaking professional life-long learning, we consider them as ‘learners’ rather than as ‘trainees’. Recognizing the diversity of expertise and experiences of the learner community, we provide two levels of attainment: an ‘ideal’ (I) response and one that would be ‘acceptable’ (A) as a pass level for a Learning Outcome (see Discussion). Course providers and/or assessors may decide that ideal is more suitable for assessment of some or all of the Learning Outcomes in their particular situations.

To allow for the flexibility appropriate for the document to be useful in different establishments and systems, these assessment criteria do not indicate the way in which assessment should be carried out, but instead have been written so that all would be amenable to different formats of written assessment. The wording provides for different types of content. Thus for ‘listed’ a simple listing would be sufficient but ‘related’ is looking for some extra text for each item on a list; ‘illustrated’ requires that there is an example for each of the elements of a topic, while for ‘illustrative’ one general example of the topic would be sufficient. ‘Provided’ or ‘given’ allow for a variety of appropriate content, with the judgement on whether the response meets the criterion made by the assessor. Course assessors may decide on whether the variety and quantity of the responses provided are sufficient for an ‘ideal’ or ‘acceptable’ response, which is indicated in the wording of the assessment criteria such as variety, several, few, or one.

In the following Tables of the Modules for persons who will design procedures and projects, namely Modules 7, 9, 10 and 11 according to the E&T Framework guidance document, ¹ the assessment criteria are placed to the right of the Learning Outcomes (LOs) column. Underneath the assessment criteria for each Module, for some LOs, additional details or links to other material have been provided, to allow all those with an interest in the educational process to see a range of content which may be included.

All tables are preceded by the original text from the E&T Framework guidance document. ¹

Module 7: Minimally invasive procedures without anaesthesia – species specific (theory)

[Function Specific for Functions A and B]

This module provides an introduction to the theory relating to minor procedures. It provides information about appropriate methods of handling and restraint and describes appropriate techniques for injection, dosing and sampling relevant to the species. It should provide information sufficient for individuals to understand what will be required of them before they go on to be ^b

b

Addition to the original E&T Framework guidance document.

trained in the practical aspects of these skills whilst under supervision.

c

Change of sequence to the original E&T Framework guidance document, as principles have to be understood before being applied in practice.

[Abbreviations: I = Ideal, A = Acceptable]

The trainees should be able to: The learners should have:
Learning Outcome 7.1.Describe appropriate principles and methodsc to be followed when handling animals (including methods of manual restraint and use of restricted environments).	Assessment criteria pertaining to LO 7.1:I and A – Related good practice in animal handling indicating its impact on the welfare of the animals involved. Illustrated how the method used for handling, or any environmental restriction, is appropriate for the scientific purpose.
Learning Outcome 7.2.Describe the biological impact of procedures and restraint on physiology.	Assessment criteria pertaining to LO 7.2:I – Provided several different examples and related how they show the effect of procedures and restraint on an animal’s physiology.A – Provided an example and related how it shows the effect of procedures and restraint on an animal’s physiology.
Learning Outcome 7.3.Describe refinement opportunities for procedures and restraint e.g. through training (using positive reinforcement)d, habituation and socialization of animals.	Assessment criteria pertaining to LO 7.3:I – Provided several different examples and related how they show the refinement of scientific procedures and methods of restraint.A – Provided an example and related how it shows the refinement of scientific procedures and methods of restraint.
Learning Outcome 7.4.Describe techniques/procedures including, for example, injection, sampling and dosing techniques (routes/volumes/frequency), dietary modification, gavage, tissue biopsy, behavioural tests, use of metabolic cages.	Assessment criteria pertaining to LO 7.4:I – Related, with a variety of illustrative examples, minimally invasive procedures without anaesthesia appropriate to the species and stage of development.A – Related, with a few illustrative examples, minimally invasive procedures without anaesthesia appropriate to the species and stage of development.
Learning Outcome 7.5.Describe how to perform minor techniques and relate appropriate sample volumes and sampling frequencies for the relevant species.	Assessment criteria pertaining to LO 7.5:I – Illustrated how to perform a variety of minimally invasive procedures without anaesthesia appropriate to the species and stage of development. This should have included information regarding volumes and sampling frequencies appropriate for the animal’s age and weight, and the site used.A – Illustrated how to perform a few minimally invasive procedures without anaesthesia appropriate to the species and stage of development. This should have included information regarding volumes and sampling frequencies appropriate for the animal’s age and weight, and the site used.
Learning Outcome 7.6.Describe the need for rigour and consistency in conducting scientific procedures and the correct recording and handling of samples.	Assessment criteria pertaining to LO 7.6:I – Related a variety of factors which may affect the reproducibility of scientific experiments; provided several different examples illustrating the importance of accurate recording of data acquired and handling of samples.A – Related a few factors which may affect the reproducibility of scientific experiments; provided a few examples illustrating the importance of accurate recording of data acquired and handling of samples.
Learning Outcome 7.7.Describe appropriate methods for the assessment of the welfare of animals with respect to the severity of procedures and know what appropriate action to take.	Assessment criteria pertaining to LO 7.7:I – Listed several ways of assessing animal welfare throughout a procedure and provided an example of each; provided an example of a system for assessing the severity of a procedure; provided several examples of taking appropriate action to minimize severity.A – Listed a few ways of assessing animal welfare throughout a procedure and provided an example of each; provided an example of a system for assessing the severity of a procedure; provided an example of taking appropriate action to minimize severity.
Learning Outcome 7.8.Recognize that refinement is an on-going process and know where to find relevant, up-to-date, information.	Assessment criteria pertaining to LO 7.8:I and A – Stated that refinement is an on-going process and listed appropriate sources of relevant and up-to-date information.
Learning Outcome 7.9.Describe the biological consequences of transport, acclimatization and husbandry conditions and experimental procedurese on the species concerned and describe how these can be minimized.	Assessment criteria pertaining to LO 7.9:I – Related several biological consequences, relevant to the species, of transport, movement, new environments, and husbandry conditions; provided for each an effective way of minimizing the impact.A – Related a few biological consequences, relevant to the species, of transport, movement, new environments, and husbandry conditions; provided for each an effective way of minimizing the impact.

Module 9: Ethics, animal

d

Typo corrected.

welfare and the Three Rs (level 2)

[Function Specific for Function B]

This module provides guidance

e

Deleted here from the original E&T Framework document as biological consequences of experimental procedures were covered previously.

and information to enable individuals designing procedures and projects (Function B of Article 23) to look, in detail, at different aspects of ethics and the Three Rs and to apply the principles learned to the ethical and welfare issues raised by the use of animals in scientific procedures within their own programme of work.

The purpose of this module is to address the fact that those designing procedures should command a deeper and broader understanding of the general issues. Thus, the main difference between level 1 and level 2 Modules on ‘Ethics, animal welfare that f

f

The word ‘that’ in the original E&T Framework guidance document has been replaced by ‘and’.

and the Three Rs’ is not necessarily the topics to be covered (which have not been repeated here) but rather that some of them are addressed in more detail and with a greater expectation for the Learning Outcome itself. For example at level 1 there are elements the trainee should know and be able to describe, which at level 2 the trainee should have a more detailed understanding and be able to discuss. This module also prepares individuals so that they are able to keep themselves informed in order to continuously apply the Three Rs to their work as new methods and approaches evolve.

[Abbreviations: I = Ideal, A = Acceptable]

The trainees should be able to:	The learners should have:
Learning Outcome 9.1.Understand that there is a broad range of ethical, welfare and scientific perspectives on the use of animals in scientific procedures, and that thinking on all of these matters evolves over time and is influenced by culture and context.	Assessment criteria pertaining to LO 9.1:I and A – Listed at least three views on the use of animals in scientific procedures; commented on a scientific procedure from a general ethical, animal welfare and scientific perspective; outlined how societal attitudes on the use of animals have evolved over time and provided examples; outlined, using examples, how the attitudes on scientific procedures using animals are influenced by culture and context.
Learning Outcome 9.2.Understand that this means there is need for on-going critical evaluation of the justification for using animals and of implementation of the Three Rs at all stages of the life of a project.	Assessment criteria pertaining to LO 9.2:I – Discussed why it is necessary to: a) justify the use of animals and conduct a critical evaluation throughout the lifetime of a project, b) implement the Three Rs continuously; identified key animal welfare individuals who should be involved in this on-going critical evaluation.A – As above, but with limited discussion.
Learning Outcome 9.3.Recognize that there are ethical limits to what it is considered permissible to do under the Directive and that even within these legal constraints, there are also likely to be national and institutional differences in this respect.	Assessment criteria pertaining to LO 9.3:I – Listed the key limits on the permissible use of animals in scientific procedures under the Directive; stated that there could be additional constraints above what may be permissible under the Directive in national legislation and institutional policy.A – Listed at least four key limits on the permissible use of animals in scientific procedures under the Directive
Learning Outcome 9.4.Explain that legislation requires that the justification for programmes of work is assessed by weighing potential adverse effects on the animals against the likely benefits; that harms to animals must be minimized, and benefits maximized.	Assessment criteria pertaining to LO 9.4:I – Stated the legal requirements to justify programmes of work by weighing likely adverse effects on the animals against the expected benefits (harm–benefit analysis); stated that harms to animals must be minimized, and benefits maximized; summarized an example of a harm/benefit analysis.A – Stated the legal requirements to justify programmes of work by weighing likely adverse effects on the animals against the expected benefits (harm–benefit analysis); stated that harms to animals must be minimized, and benefits maximized.
Learning Outcome 9.5.Understand and provide the information necessary to enable a robust harm/benefit assessment to be performed; and explain why they personally consider that the potential benefits outweigh the likely adverse effects.	Assessment criteria pertaining to LO 9.5:I – Provided the critical information required for a robust harm/benefit assessment of a project, including the purpose, expected outcomes, potential benefits, likely adverse effects and animal welfare considerations; provided an outline of a complex project and the procedures in it and a personal evaluation of why the expected benefits could be considered to outweigh the likely adverse effects.A – Provided the purpose, expected benefits and likely adverse effects relevant to a specific small project and the procedures in it; provided an assessment of how the likely harm to the animals involved in that project would be balanced against the expected benefits of the project.
Learning Outcome 9.6.Understand the need to communicate appropriate information to a wider public audience, and be able to prepare an appropriate non-technical summary to facilitate this.	Assessment criteria pertaining to LO 9.6:I – Justified with evident understanding why there is a need for transparency and communication of appropriate information of scientific projects to a wider public audience; prepared a non-technical project summary (NTS) of a specific scientific project to this end, including a discussion of purpose and benefits, and information on animal uses including species, procedures, numbers and prospective severity classification, and information on the application of the Three Rs.A – Created a non-technical project summary for the public as above for a scientific project specified by the assessor.
Learning Outcome 9.7.Describe the importance of disseminating information that will promote understanding of ethical issues, good animal welfare, good science and application of the Three Rs.	Assessment criteria pertaining to LO 9.7:I – Discussed the considerations why it is important to disseminate information regarding the conduct of scientific projects; provided examples of how ethical considerations, good animal welfare practices, good science and application of the Three Rs combined are necessary for disseminating information on scientific projects using animals.A – Described the importance of disseminating information regarding the conduct of scientific projects including ethical considerations, good animal welfare practices, good science and application of the Three Rs.

Note:

Assessment criteria pertaining to LO 9.6. A format for an NTS has been provided by the European Commission implementing decision 2020/569/EU. This can be retrieved at https://eur-lex.europa.eu/legal-content/EN/TXT/?toc=OJ%3AL%3A2020%3A129%3ATOC&uri=uriserv%3AOJ.L_.2020.129.01.0016.01.ENG.

The EU public database of NTSs on project licences issued since 2021 may be consulted at https://environment.ec.europa.eu/topics/chemicals/animals-science/statistics-and-non-technical-project-summaries_en#project-summaries-database or at

European Commission, Directorate-General for Environment, Non-technical project summaries under Directive 2010/63/EU on the protection of animals used for scientific purposes, Publications Office of the European Union, 2022, https://data.europa.eu/doi/10.2779/778680.

Module 10: Design of procedures and projects (level 1)

[Function Specific for Function B and Additional for Function A (as required)] [Function Specific for Functions A and B]

This module is a pre-requisite for people who will be designing projects (Function B) but it is also be ^g

g

The word ‘be’ has been deleted from the original E&T Framework guidance document.

beneficial for scientists who have some involvement in designing the procedures that they carry-out (Function A). The module comprises information about experimental design concepts, possible causes and elimination of bias, statistical analysis and information about where expertise can be found to assist with procedure, design, planning and the interpretation of results.

[Abbreviations: I = Ideal, A = Acceptable]

The trainees should be able to:	The learners should have:
Learning Outcome 10.1.Describe the concepts of fidelity and discrimination (e.g. as discussed by Russell and Burch and others).	Assessment criteria pertaining to LO 10.1:I – Related fidelity and discrimination to the reproducing of the properties of a target by an animal or non-animal model, with fidelity described as how well all of the properties are reproduced, and discrimination as the extent of reproduction of one particular property, and provided an appropriate example of each.A – Provided an adequate description of fidelity and discrimination as above
Learning Outcome 10.2.Explain variability, its causes and methods of reducing it (e.g. uses and limitations of isogenic strains, outbred stocks, genetically modified strains, sourcing, stress and the value of habituation, clinical or sub-clinical infections, and basic biology).	Assessment criteria pertaining to LO 10.2:I – Provided a comprehensive description of biological variability and its causes including but not limited to genetic variability, health status, effect of stress and distress, environmental conditions, handling, housing, and husbandry, and provided an example of how to reduce the variation from each.A – Provided an adequate description of biological variability and given some of its causes with an example of how to reduce the variation from each.
Learning Outcome 10.3.Describe possible causes of bias and ways of alleviating it (e.g. formal randomization, blind trials and possible actions when randomization and blinding are not possible).	Assessment criteria pertaining to LO 10.3:I – Defined bias and listed several ways in which animal-based research is susceptible to bias; defined the terms randomization and blinding as applied to research studies and given an example of each showing how they minimize bias and how they are carried out.A – As above with only examples of two different biases given.
Learning Outcome 10.4.Identify the experimental unit and recognize issues of non-independence (pseudo-replication).	Assessment criteria pertaining to LO 10.4:I – Correctly defined independent replication and the term experimental unit and given some examples of the experimental unit in different types of experiment; defined pseudo-replication and given examples of it.A – As above but without appropriate examples.
Learning Outcome 10.5.Describe the variables affecting significance, including the meaning of statistical power and ‘p-values’.	Assessment criteria pertaining to LO 10.5:I – Defined the term significance and related it to hypothesis testing and the risks of false positives and false negatives; explained the relationship between effect size, variability, and numbers of experimental units needed to have ability to show genuine differences between groups and equated that to statistical power; defined p-value as the probability that a difference as large as that seen in the experiment would have occurred by random variation alone if the treatment groups were in fact not different.A – As above but have not related the terms to hypothesis testing and/or given a definition of p-value that only mentions probability and random variation.
Learning Outcome 10.6.Identify formal ways of determining of sample size (power analysis or the resource equation method).	Assessment criteria pertaining to LO 10.6:I – Described how power analysis is carried out from estimates of effect size and variability and input of figures for level of significance, power, and one or two tail comparison; shown how this can be carried out for block designs; stated the resource equation correctly and explained how the elements of it are used to determine numbers, with examples for block and factorial designs.A – As above but without inclusion of block or factorial designs.
Learning Outcome 10.7.List the different types of formal experimental designs (e.g. completely randomized, randomized block, repeat measures [within subject], Latin square and factorial experimental designs).	Assessment criteria pertaining to LO 10.7:I – Listed the types of design in the learning outcome, giving the principal characteristics of each, and in addition distinguished between cross-over and serial sampling repeat measure designs, and included at least two of these designs: sequential, survival, release-and-recapture, and observational.A – As above but without giving more than the principal characteristics or including additional designs.
Learning Outcome 10.8.Explain how to access expert help in the design of an experiment and the interpretation of experimental results.	Assessment criteria pertaining to LO 10.8:I – Outlined the way in which help can be obtained within the organization or externally from more experienced researchers or statisticians knowledgeable in animal-based research; given what such expert help may wish to know about the work to provide advice (experimental question, comparisons being made, outcome measures and variability); stated that such expert help should be sought during preparation of the proposed work.A – Outlined the way in which help can be obtained within the organization; given what such expert help may wish to know about the work to provide advice (experimental question, comparisons being made, outcome measures and variability).

Module 11: Design of procedures and projects (level 2)

[Function Specific for Function B]

This module provides a relevant level of understanding of the national and international legal and regulatory framework within which projects are constructed and managed, and of their legal responsibilities.

The trainee must be able to identify, understand and respond appropriately to the ethical and welfare issues raised by the use of animals in scientific procedures generally, and specifically within their own programme of work. These have been addressed in Module 2.

The trainee should be able to develop, direct and control a programme of work in order to achieve its stated objectives, while ensuring compliance with the terms and conditions of any regulation governing the project. This includes implementation of the Three Rs throughout the programme of work. Learning outcomes relating to Reduction are addressed in Module 2.

[Abbreviations: I = Ideal, A = Acceptable]

(i) Legal issues

The trainees should be able to:	The learners should have:
Learning Outcome 11.1.Describe in detail the main components of the national legislation regulating the scientific use of animals; in particular, explain the legal responsibilities of those designing procedures and projects (Function B staff) and those of other persons with statutory responsibilities under the national legislation (e.g. the person responsible for compliance, veterinarian, animal care staff, training officers).	Assessment criteria pertaining to LO 11.1:I and A – Related the key components of the national legislation regulating the scientific use of animals, listed and explained the legal responsibilities of those designing procedures and projects, and listed the statutory responsibilities for different staff under the national legislation.
Learning Outcome 11.2.List the key purposes of other relevant EU and international legislation and associated guidelines that impact on the welfare and use of animals. This includes Directive 2010/63/EU and legislation/guidelines relating to: veterinary care, animal health, animal welfare, genetic modification of animals, animal transport, quarantine, Health & Safety, wildlife and conservation	Assessment criteria pertaining to LO 11.2:I and A – Stated that there are legislative instruments other than Directive 2010/63/EU and guidelines that impact on the use of animals for scientific purposes; listed sources of information and advice on these.

Notes:

Assessment criteria pertaining to LO 11.1. Member states are required to implement in their national legislation the legal responsibilities of those designing procedures and projects and those of other persons with statutory responsibilities stated in the Articles 20, 23, 24 and 25. It is not expected that the learners should be required to reproduce the whole of the national legislation.

Assessment criteria pertaining to LO 11.2. Although those designing procedures and projects may not need to specify what other legislation and guidelines could apply to their animal use, they should be aware that there are many regulations other than the Directive 2010/63/EU which apply to their work and that they need to consult on these.

(ii) Good scientific practice

The trainees should be able to:	The learners should have:
Learning Outcome 11.3.Describe the principles of a good scientific strategy that are necessary to achieve robust results, including the need for definition of clear and unambiguous hypotheses, good experimental design, experimental measures and analysis of results. Provide examples of the consequences of failing to implement sound scientific strategy.	Assessment criteria pertaining to LO 11.3:I – Identified all the key elements of a sound scientific strategy, from defining the experimental question to data analysis, necessary to achieve reliable and reproducible results; provided an example for each element of how not conducting it properly leads to unreliable experimental results.A – Identified key elements of a sound scientific strategy, from defining the experimental question to data analysis, necessary to achieve reliable and reproducible results; provided for one of these elements an example of how not conducting it properly leads to unreliable experimental results.
Learning Outcome 11.4.Demonstrate an understanding of the need to take expert advice and use appropriate statistical methods, recognize causes of biological variability, and ensure consistency between experiments.	Assessment criteria pertaining to LO 11.4:I and A – Provided an assessment of his or her personal competence in experimental design and statistical analysis, and the limitations that would make it necessary to seek expert advice at the planning stage; listed several causes of biological variability; outlined steps to provide consistency between experiments.
Learning Outcome 11.5.Discuss the importance of being able to justify on both scientific and ethical grounds, the decision to use living animals, including the choice of models, their origins, estimated numbers and life stages. Describe the scientific, ethical and welfare factors influencing the choice of an appropriate animal or non-animal model.	Assessment criteria pertaining to LO 11.5:I – Given a comprehensive argument for the importance of providing ethical and scientific justification for deciding to use animals in scientific studies and given examples of more than one harm-benefit analysis; considered a range of general scientific ethical and welfare factors influencing the choice of an appropriate animal or non-animal model for scientific research, taking into account the stage of development, and have provided examples of how these led to particular choices.A – Given reasons for considering it important to provide ethical and scientific justification for deciding to use animals in scientific studies and given an example of a harm–benefit analysis; considered key general scientific ethical and welfare factors influencing the choice of an appropriate animal or non-animal model for scientific research, taking into account the stage of development, and have provided an example of how these led to a particular choice.
Learning Outcome 11.6.Describe situations when pilot experiments may be necessary.	Assessment criteria pertaining to LO 11.6:I – Given the features of a pilot experiment (for example those listed in the UK NC3Rs document on pilot experiments) and some examples of when pilot experiments may be necessary, explaining why.A – Listed key features of a pilot experiment and an example, with explanation, of when one may be necessary.
Learning Outcome 11.7.Explain the need to be up to date with developments in laboratory animal science and technology so as to ensure good science and animal welfare.	Assessment criteria pertaining to LO 11.7:I – Given several reasons why researchers need to be up-to-date with developments in laboratory animal science and technology, with examples of how the use of outdated laboratory animal science practices and/or technology has impacted on scientific outcomes or animal welfare.A – Given only key reasons why researchers need to be up-to-date with developments in laboratory animal science and technology, with an example of how the use of outdated laboratory animal science practices and/or technology has impacted on scientific outcomes or animal welfare.
Learning Outcome 11.8.Explain the importance of rigorous scientific technique and the requirements of assured quality standards such as GLP.	Assessment criteria pertaining to LO 11.8:I – Listed what is needed to conduct an experiment that is reproducible, provided several reasons why experiments need to be so conducted, and given some examples of how failure to do so has led to unsafe conclusions; listed the components of a bioscience quality assurance system such as Good Laboratory Practice.A – Listed key features of the conduct of an experiment that is reproducible, provided a reason why experiments have to be so conducted, and given an example of how failure to do so has led to unsafe conclusions; listed key elements of a bioscience quality assurance system such as Good Laboratory Practice
Learning Outcome 11.9.Explain the importance of dissemination of the study results irrespective of the outcome and describe the key issues to be reported when using live animals in research e.g. ARRIVE guidelines.	Assessment criteria pertaining to LO 11.9:I – Provided several reasons why study results, whatever they are, should be published or otherwise made available to appropriate stakeholders; listed the components of good reporting when using live animals either as both the essential and recommended items from the ARRIVE guidelines or a similar set from other guidance.A – Provided a reason why study results, whatever they are, should be published or otherwise made available to appropriate stakeholders; listed the key components of good reporting when using live animals either as the essential items from the ARRIVE guidelines or a similar set from other guidance.

Notes:

Assessment criteria pertaining to LO 11.3. This LO refers to a ‘good scientific strategy’ and ‘robust results’. A good scientific approach that would yield robust results requires a clear objective (why the experiment is being performed) for each study and, where appropriate, the testable hypothesis that comes from it. It includes obtaining valid, unbiased results by the proper conduct of an experiment and consideration, with evident understanding, of the use of different experimental designs, in particular fully randomized, randomized block, factorial and cross-over arrangements, and how each of these designs could be statistically analysed.

Assessment criteria pertaining to LO 11.4. It is expected that the experts to be consulted will be those knowledgeable in the design, conduct and analysis of animal experiments. The causes of biological variability to be considered could include inter-individual differences, age, sex, genetic status, environment and handling.

Assessment criteria pertaining to LO 11.5. Harm–benefit analysis is discussed with examples in the FELASA Working Group reports (https://felasa.eu/Portals/1/Reports/Bronstad_2016_FELASA_AALAS_HBA_P1.pdf?ver=eos0kLpCTV1BrUN8N0T5Rg%3d%3d, and https://felasa.eu/Portals/1/WorkingGroupsPublic/Laber_2016_FELASA_AALAS_HBA_P2%20(1).pdf?ver=MsIzEOgiUkc4NhHJaOV4eg%3d%3d).

Assessment criteria pertaining to LO 11.6. The NC3Rs link on pilot studies (https://nc3rs.org.uk/3rs-resources/conducting-pilot-study) explains what they are and when they are valuable.

Assessment criteria pertaining to LO 11.8. Good Laboratory Practice (https://one.oecd.org/document/env/mc/chem(98)17/en/pdf) governs the documentation of how the experiment was planned, conducted and analysed.

Assessment criteria pertaining to LO 11.9. The Animal Research Reporting In Vivo Experiments (ARRIVE) guidelines vs. 2.0 are available at https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000410.

(iii) Implementing the Three Rs

The trainees should be able to:	The learners should have:
Learning Outcome 11.10.Demonstrate a comprehensive understanding of the principles of replacement, reduction and refinement, and of how these ensure good science and good animal welfare.	Assessment criteria pertaining to LO 11.10:I – Defined the Three Rs principles; provided multiple examples of the operation of each and of how replacement, reduction and refinement assist good science and animal welfare in the context of an experiment.A – Defined the Three Rs principles, and considered how replacement, reduction and refinement assist good science and animal welfare in the context of an experiment.
Learning Outcome 11.11.Explain the importance of literature and internet searches, discussion with colleagues and with relevant professional bodies in identifying opportunities for applying each ‘R’.	Assessment criteria pertaining to LO 11.11:I – Provided multiple examples illustrating the value of literature and internet searches, and of discussion with colleagues and with relevant professional bodies in identifying opportunities for applying each ‘R’; provided three examples for each ‘R’.A – Provided an example illustrating the value of literature and internet searches, and of discussion with colleagues and with relevant professional bodies in identifying opportunities for applying each ‘R’.
Learning Outcome 11.12.Describe relevant sources of information relating to ethics, animal welfare and the implementation of the Three Rs.	Assessment criteria pertaining to LO 11.12:I – Listed relevant sources of information relating to ethics, animal welfare and the implementation of the Three Rs and given examples of how each may be used.A – Listed relevant sources of information relating to ethics, animal welfare and the implementation of the Three Rs.
Learning Outcome 11.13.Explain how to use different search tools (e.g. EURL ECVAM Search Guide, Go3Rs) and methods of search (e.g. systematic reviews, meta-analysis).	Assessment criteria pertaining to LO 11.13:I – Listed different search tools and provided several examples of each, and how each is used.A – Listed different search tools and provided one example and how it is used.
Learning Outcome 11.14.Describe examples of alternative methods and research strategies that replace, avoid or complement the use of animals in different types of research programme.	Assessment criteria pertaining to LO 11.14:I – Provided illustrated examples of research strategies and alternative approaches that replace, avoid, or complement the use of animals in different types of research programmes.A – Provided an illustrated example of a research strategy and alternative approaches that replace, avoid, or complement the use of animals in different types of research programmes.
Learning Outcome 11.15.Identify, assess and minimize all of the welfare costs to animals throughout the animals’ lifetime (including adverse effects relating to sourcing, transport, housing, husbandry, handling, procedures and humane killing); explain and give examples of welfare assessment protocols.	Assessment criteria pertaining to LO 11.15:I – Provided examples of negative and positive influences on an animal’s wellbeing at different stages of its life; provided a mitigation strategy for each negative example; provided species-specific examples of assessing animal welfare at different stages of its life.A – Provided at least one example of a negative and a positive influence on an animal’s wellbeing at different stages of its life; provided a species-specific example of assessing animal welfare at different stages of its life.
Learning Outcome 11.16.Define and apply appropriate humane end-points; establish suitable criteria to identify when the humane endpoint has been reached.	Assessment criteria pertaining to LO 11.16:I – Defined the concept of humane endpoints in animal research and provided illustrated examples of the use of humane endpoints.A – Defined a humane endpoint in animal research and provided an illustrated example.
Learning Outcome 11.17.Describe possible conflicts between Refinement and Reduction (e.g. in the case of re-use) and the factors that need to be considered to resolve this conflict.	Assessment criteria pertaining to LO 11.17:I – Provided several illustrated examples of where there is a conflict between the numbers of animals used and the level of severity experienced by individual animals, and how the conflict can be managed.A – Provided an example of where there is a conflict between the numbers of animals used and the level of severity experienced by individual animals, and how the conflict can be managed.
Learning Outcome 11.18.Define the requirements for, and controls on, re-homing of animals; identify any relevant re-homing guidelines.	Assessment criteria pertaining to LO 11.18:I – Stated the requirements for, and controls on, re-homing of animals and provided a variety of examples of when re-homing is appropriate.A – Stated the requirements for, and controls on, re-homing of animals and provided an example of when re-homing is appropriate.

Notes:

Assessment criteria pertaining to LO 11.13. The assessment criteria do not cover the elements in parentheses (e.g. EURL ECVAM Search Guide, Go3Rs) in the E&T Framework document LO 11.13 as these examples are not search tools but repositories and databases. It also does not cover Systematic Reviews and Meta-Analyses as they are primarily methods of analysis, not methods of search.

Assessment criteria pertaining to LO 11.14. The assessment criteria changed the sequence regarding alternative methods and research strategies in the LO 11.14 since ‘replace, avoid or complement’ are related to alternative methods more than to research strategies.

(iv) Responsibilities

The trainees should be able to:	The learners should have:
Learning Outcome 11.19.Explain the need to be aware of local arrangements relating to project licence management, e.g. procedures for ordering animals, accommodation standards, disposal of animals, safe working practices and security, and the actions to take in the event of unexpected problems arising with any of these.	Assessment criteria pertaining to LO 11.19:I – Stated local arrangements for good project licence management may differ between establishments; provided examples of these and the actions to take in the event of problems.A – Stated the local arrangements within their establishment; provided an example of good project licence management and the actions to take in the event of problems.

Discussion

This document is aimed at all stakeholders (learners, educators and accrediting bodies) involved in the education of learners who will go on to design procedures and projects, written so all know what is expected to be examined and the assessment approach. Given the roles of those being educated and the knowledge they have gained from this education, we have moved beyond just testing knowledge recall to a broader assessment that includes intellectual skills acquired and critical thinking, and appropriate attitudes and behaviours,^{4,6 –9} embedding these in the wording of the assessment criteria. So, for instance, an ‘illustrated example’ requires both explanation and application, not just stating the example.

The objective is that the overall assessment of the learning activity is to move from an end-of-course assessment that can be a pass or fail, to a stage in the learner’s life-long learning journey. Educators may set the balance of acceptable (A) versus ideal (I) depending on the cohort mix, experience and needs of their learner community. As they progress through their careers, learners should transition from the acceptable (A) to the higher level ideal (I), based on their individual needs, roles in the workplace and life-long learning requirements.

Educators who are delivering content that will be assessed using these criteria are recommended to adopt a student centred, active learning educational approach in their delivery,^10,11 as student active learning should develop the appropriate skills and attitudes/behaviours.^12,13

Module 7 assessment criteria have been written for those designing procedures and projects, giving ideal and acceptable levels. A previous publication ¹⁴ has provided assessable learning outcomes with a level of detail appropriate for teachers and assessors dealing with those performing procedures. Assessment of those designing procedures and projects requires assessment criteria appropriate to the extent of reflection and critical thinking expected from them, and the text here is broadly written to both encompass and go beyond the assessment elements given previously. Ideally, reassessment of Module 7 should be considered for personnel moving from performing procedures to designing procedures and projects.

Question setters may find the notes related to the assessment criteria of the modules helpful in preparing questions. Particularly for Module 11, there is considerable guidance on content in the notes provided.

We anticipate that different types of questions will be needed to fully assess the particular Learning Outcomes according to these criteria. For example, a simple multiple choice question format may be appropriate where ‘listed’ has been specified, whereas where the assessment criterion has specified ‘related’, a more complex question format could well be needed.

We hope, in accordance with ETPLAS’ wider aspirations, ¹⁵ this will provide a common framework to assist the harmonization of the assessment after training of individuals designing procedures and projects. We also hope that our identifying some points that could be modified in subsequent versions of the E&T Framework guidance document will be helpful.