Face validity of a new aquatic skill assessment tool: RAEE (refuse,assist,effective,efficient)

Abstract

Drowning is recognized globally as a neglected public health issue. Learning to swim is one important component of a multi-sectoral approach to drowning prevention. The assessment of aquatic skills can be complicated due to the variety of disparate approaches and programs found within learn-to-swim. Establishing the validity of any assessment tools is necessary. A descriptive qualitative design with face-to-face focus groups of AUSTSWIM trainers was used to establish the face validity of an assessment tool known as RAEE (Refuse, Assisted, Effective, Efficient), against the Australian Water Safety Council's 2016 benchmarked aquatic skills. Three assessment related themes were found: Issues with the competent/not yet competent approach; more wide-ranging concerns within the learn-to-swim industry in general; and the usefulness of the RAEE tool. This study confirmed the face validity of the RAEE assessment tool and was found to be a useful tool to implement in aquatic skills assessment.

Introduction

Since 2014, drowning has been increasingly recognized around the globe to be a neglected public health issue.^1,2 In response there has been a concomitant increase in research into the prevention, treatment and rescue methods associated with drowning prevention.^3–12 Drowning is a complex issue that requires a multiple ‘layers of protection’ approach to reduce its overall burden.^1,13 One of the identified layers is the facilitation of teaching basic swimming, water safety and self-rescue skills, pragmatically referred to as ‘learning to swim’. Learning to swim, in the sense of gaining aquatic motor skills is axiomatically seen as a practical solution to help prevent drowning. ¹⁴ The Australian Water Safety Council's (AWSC) National Water Safety Competency Framework (2016) includes a benchmark level of swimming, water safety and self-rescue skills, knowledge and understanding (from herein referred to as aquatic skills, a component of aquatic competency). Aquatic competency is a collective term that refers to the overarching goal of attaining not just the requisite skills but also the associated knowledge and understanding that is required for drowning prevention above and beyond any individual aquatic skill(s). The framework recommends that irrespective of which learn-to-swim program is utilized, all children finishing primary school should be meeting this level of aquatic skills, represented by the benchmark as the minimum best practice recommended for drowning prevention.^15,16 This is an ambitious goal, however assessing a learn-to-swim program's outcome in ensuring that component of aquatic competency is more complex than simple observation of a set of swimming skills. ¹⁷

To adequately assess aquatic skill performance, requires an established and clearly communicated set of guidelines that indicate which skills are assessed, assessment criteria of each skill, and assessment tool(s) to be used and their validity of use.^17–19 The requirement for a reliable and valid assessment approach has not been a frequent consideration in the development of the common tools that are used in the learn-to-swim industry. ¹⁹ Therefore, the need to establish the validity of existing and new assessment tools used to assess learn-to-swim skills is warranted.

Validity ensures an assessment tool provides an accurate reflection of the intended measure; many assessment tools fail to measure what they have been designed to.^20,21 There are several types of validity (e.g., construct, criterion, content), however the overall purpose of establishing validity is to ensure the accuracy of the tool being used. ²¹ Face validity is a specific type of content validity and involves a panel of experienced professionals coming together to formally explore opinions of the tool and determine whether the tool appears ‘at face value’ to meet its intended purpose.^21–23 Face validity does not provide the strongest type of validation, yet in situations where there is an absence of other types of validity, there is sufficient value in establishing the face validity of a tool. ²¹

Developing aquatic competency involves acquiring the broad set of psychomotor skills, cognitive knowledge, and affective behaviors that enable safe interaction in, on and around water. ¹⁴ This dynamic process should also match the capabilities of the learner (personal characteristics, task demands and environmental context).^14,17,19,24 Psychomotor skills are foundational yet the current use of dichotomous assessment outcome of either competent or not yet competent (C/NYC), when assessing a given aquatic skill has limitations. These include the inability to identify the capabilities of the learner and the inability to provide constructive feedback.^19,25 These considerations led to the development of a more comprehensive assessment tool in this study, the RAEE (Refuse, Assisted, Effective, Efficient) (Table 1). The RAEE was designed to enhance the scope of information when assessing swimming, water safety and self-rescue skills in the context of aquatic competency. This tool employs a seven-point Likert scale and incorporates the influence of a learner's capabilities (developmental stage; confidence in environment; and efficiency of motor skill) on their ability to demonstrate aquatic skills. The demonstrated ability of any aquatic skill can be placed on a continuum of outcomes (i.e., refusal to attempt, attempt requiring assistance, effective attempt and efficient attempt). The RAEE therefore provides an ordinal scale offering options to categorize a learner's abilities and/or confidence (e.g., willingness to attempt skill in different aquatic environment). The RAEE tool also uses a ‘stop light’ color coded table, in descending order of the tool's name (R = red, A = amber and EE = green), with each colored section including descriptions and definitions for use of the tool. The provision of scoring options aims to improve the validity of the assessment and allow feedback opportunities beyond the default dichotomous ‘can’ or ‘cannot’ accomplish skill given this type of assessment outcome does not provide information to individuals of where they can improve their task performance.^26–28

Table 1.

Assessment tool – RAEE (refuse, assisted, effective, efficient).

1	Refused to attempt task.
2	Attempted task – did not complete.
3	Attempted task – completed with assistance.	Assistance is defined as kickboards, noodles, floats or physical assistance from another person. Breathing is controlled throughout task – i.e., no choking, gasping or reliance on aids to breathe comfortably.
4	Completed task with difficulty, little or no confidence and/or inefficiently.	Difficulty is defined as the inability to coordinate the individual components of the stroke as a whole (i.e., body position, arm action, leg action and breathing).
5	Completed task with moderate ease and lacking confidence and/or efficiency (Correct stroke technique as described in the AUSTSWIM manual).	Moderate ease is defined as the ability to coordinate the individual components of the stroke as a whole, but not having the ability to maintain that coordination over distances greater than 10 m. Confidence is demonstrated through non-reliance on aids such as goggles, pool edge, shallow water and/or lack of anxiety in the water.
6	Completed task with ease, but lacking confidence and/or efficiency (Correct stroke technique as described in the AUSTSWIM manual).	Ease is defined as the ability to coordinate the individual components of the stroke as a whole and maintaining that coordination over distances greater than 10 m.
7	Completed task with ease, confidence and with moderate or high efficiency (Correct stroke technique as described in the AUSTSWIM manual).	Efficiency is demonstrated through reduction of frontal and eddy resistance affecting the body while moving through water – i.e., body position, maintaining streamline, reducing time/distance from streamline position.

The RAEE is designed to apply to movement skills, water safety knowledge and understanding with the rating system broadly broken into four main areas: Refusal – Assisted – Effective – Efficient.

The importance of acquiring aquatic competency for effective drowning prevention is now being recognized, impacting and influencing the validity of the aquatic skills chosen by learn-to-swim programs focused on reducing drowning.^14,24,29 Currently in Australia, learn-to-swim teachers are introduced to the principles associated with aquatic motor skills and the importance of water safety through initial teacher training. ²⁷ Over time, they will incorporate this into their own instructional approach through personal experience and mentoring with other teachers, which may align with swimming for safety, recreation, or competition. ³⁰ In the absence of a learn-to-swim governing body, the setting of curricula, standards and criterion of how learn-to-swim teachers assesses aquatic skills is determined by stakeholders such as government departments of education, swim school operators and/or individual teachers.^9,29,31 As such learn-to-swim teachers may incorporate any combination of swimming and water safety skills, knowledge and strategies into their lessons. ²⁷ This also means that a consistent approach to assessment practices across different providers is lacking, with minimal formal education about assessment methods.^28,32–34

Establishing a consensus within the learn-to-swim industry on a singular preferred approach to curriculum and what skills should be assessed, despite the recommended benchmark is therefore unlikely.^32,35 However, the RAEE assessment tool is applicable to any demonstration of swimming, water safety, and/or self-rescue skills. Providing a more in-depth picture of a student's capabilities, it may provide a step towards ensuring aquatic competency is met through learn-to-swim instruction. In theory it should provide a more nuanced tool for swim teachers, yet before any further development and implementation, this approach needs to be validated amongst the intended users of the tool.

The Australian Council for the Teaching of Swimming and Water Safety (AUSTSWIM), is the national industry standard for how teachers are trained, accredited and licensed (ISO/IEC 17024) and is internationally recognized as the first aquatic education organization in the world to be accredited.^36,37 The purpose of this study was to engage experienced AUSTSWIM trainers (i.e., swim teacher trainers qualified in vocational education as Trainers and Assessors) as experts to determine face validity of the RAEE assessment tool in assessing AWSC (2016) benchmarked aquatic skills.

Method

A descriptive qualitative study using face-to-face focus groups of AUSTSWIM trainers was undertaken. An AUSTSWIM Presenter (also known as a ‘trainer’) is an accredited AUSTSWIM teacher who has undertaken further training to meet accreditation requirements to deliver the knowledge and skills in the delivery and assessment of the AUSTSWIM Teacher of Swimming and Water Safety™ course. Focus groups involve assembling groups of individuals to discuss and comment on particular topics. ³⁸ This method was chosen because it allowed for open interaction between the AUSTSWIM trainers to engage in dialogue to explore experiences of current approaches to swimming assessment and the efficacy of the RAEE rating system to assess an individual's swimming ability.

Participants

Three focus groups were sourced from a pool of AUSTSWIM trainers attending a professional development workshop at a series of ‘Soak it up 19’ AUSTSWIM State Conferences in either New South Wales (n = 2) or Western Australia (n = 1). Participant Information sheets and consent forms were distributed by email prior to the workshops and consent forms provided prior to the commencement of the focus groups.

The focus group participants were classified as trainers, who instruct potential candidates at a minimum of the entry level qualification of Teacher of Swimming and Water Safety.^39,40 They may also have been involved in the delivery of extension courses that include the following specialties: Infant and Preschool; Adults; Towards Competitive Strokes and Access and Inclusion. ²⁷ Access and Inclusion focuses on teaching non-mainstream students such as; people with a disability or chronic illnesses, people from a culturally and linguistically diverse backgrounds and/or older adults. ³⁹

Procedures

Within each focus group, participants were divided into groups of no more than eight participants with two facilitators (members of the research team with experience in conducting focus groups; CJ, SE and NN). Audio recordings of each group were made using an iPhone X (Voice Memos app, Apple Inc., CA USA). Focus groups were facilitated using an interview schedule that was informed and developed following a review of the literature, ⁴¹ see Supplementary 1. The schedule provided a guide of topics and was designed to allow the facilitator to prompt discourse between participants about current approaches to swimming assessment, and to prompt to clarify and provide meaning to responses. The topics covered in the focus group schedule included the use of a C/NYC scale and the applicability of the RAEE assessment tool as an assessment approach. Ethical approval for the study was granted by The University of Newcastle Human Research Ethics Committee (H-2019-0069).

Analysis

The focus group discussions were transcribed verbatim using www.trint.com and imported into a qualitative data analysis software (NVIVO 12, QSR International, Cambridge, MA USA) for analysis. A ‘thick analysis’ was then undertaken, involving several cycles of data collection and analysis, combining several analysis methods together creatively, allowing for a more comprehensive ‘thick description’. ⁴² To do this, participants were de-identified via numerical coding and then transcripts were inductively thematically analyzed. ‘Peer debriefing’ was employed as a strategy by two researchers (NN, CJ) who independently read transcripts and analyzed the data into key themes, codes and categories. These inductively derived codes and categories were then collectively synthesized to best report the experience of the participants. ⁴³ This iterative and ongoing dialogic process led to intersubjective agreement on the final codes and categories used for analysis. ‘Thick description’ was used to increase potential transferability. ⁴² Methods to promote trustworthiness within this qualitative study included dependability, credibility and confirmability and were implemented during data analysis. ⁴³

Results

A total of 23 participants (n = 20 female, n = 3 male), ranging from 25 years through to >65 years, consented to be involved in the focus groups (Table 2). Most participants had held an AUSTSWIM qualification for 20 years or more, with the others reporting either 15 to 20 years or 10 to 15 years of experience. All participants were qualified as Teachers of Swimming and Water safety, with the next most common qualifications being Infants and Preschool, followed by Access & Inclusion.

Table 2.

Focus group participant demographics.

									Type of Qualification(s) held
Gender	n	%	Age	n	%	Years Qualified	n	%	Teacher of:	n	%
Male	3	13.0	<18	0	0.0	<1	1	4.3	Swimming-and Water Safety	23	100.0
Female	20	87.0	18–24	0	0.0	1 to 3	0	0.0	Access-& Inclusion	16	69.6
Other	0	0.0	25–34	1	4.3	4 to 6	1	4.3	Towards Competitive Strokes	14	60.9
			35–44	7	30.4	7 to 9	0	0.0	Infants-& Preschool	18	78.3
			45–54	8	34.8	10 to 15	4	17.4	Adults	12	52.2
			55–64	6	26.1	15 to 20	6	26.1	Aqua Teacher	1	4.3
			>65	1	4.3	>20	11	47.8	Trainer	17	73.9
									Trainer-in training	3	13.0

Three key themes about aquatic skills assessment generally emerged within the learn-to-swim industry as well as the use of the RAEE as an assessment tool emerged. These interconnected themes related to: 1. C/NYC approach to assessment; 2. the Aquatics Industry as a whole; and 3. the usefullness of the RAEE assessment tool (Figure 1).

Figure 1.

Focus group common themes.

Theme 1. Competent or not yet competent approach to assessment

There are different learn-to-swim strategies and approaches to assessment reported to be employed within the industry. Participants discussed the challenges associated with the current default method of aquatic skill assessment in the industry, the C/NYC rating with i) the aquatic skills assessment criteria across the industry, ii) the inconsistent application due to subjective bias, and iii) altering the specific environment and/or situation could alter the C/NYC rating of a aquatic skill.

Parts of the assessment criteria of the aquatic skills viewed to be critical (i.e., ‘must sees’ aquatic skills) by the swim teachers, programs and between private and public providers used to inform such assessment ratings were described as overly broad, vague and interpreted differently between swim teachers, programs and between private and public providers:

The assessment criteria of the aquatic skills used to inform such assessment ratings were described as overly broad and vague and thus what is considered critical components of the aquatic skills (i.e., ‘must sees’) are interpreted differently the between swim teachers, programs and between private and public providers:

‘if you look through a whole range of different swim schools, and going through the Education Department right through to private, here in Western Australia, the ‘must sees’ vary.’ Responder 2-3

Another perceived issue with the commonly employed C/NYC assessment method and its inconsistent application due to subjective bias, related to the teacher basing their assessment on a personal interpretation of how the aquatic skills should be completed.

‘I think that a competent and not yet competent does really rely way too much on personal opinions…’ Responder 2-2

The specific environment that learn-to-swim assessment is performed in, and that this environmental context is not adequately reflected within the C/NYC assessment method was also raised. Participants mentioned that a swimmer could be deemed competent in a specific swimming skill in one situation and/or environment but could be deemed not yet competent in performing the same skill in another situation and/or environment; yet this was not reflected in any of the C/NYC assessment criteria.

‘Your environment also plays a big part in using that as an example because not all facilities have that deep water. So we have to allow, interpret what they understand just verbally as opposed to allowing them to actually, do it before you even get to the assessment of it.’ Responder 4-2

Theme 2. Aquatic Industry

The second theme on the aquatic industry raised several points including: the variability in learn-to-swim staff teaching, the need for improved aquatic training education, the consistency of aquatic skill assessments, and how the aquatic skill assessment process can be potentially influenced by parents of children enrolled in learn-to-swim classes.

In the context of discussing the RAEE tool, participants identified some issues related to the learn-to-swim industry, particularly around industry standards and the lack of existing assessment criteria.

‘Even within the own swim school environment. Every single teacher is different. … I think we need to go down that path because if you want to develop the quality of the industry, then we need to have those high standards.’ Responder 8-1

Discussion about the industry also included the need for improved training education for swim teachers to develop their aquatic assessment skills and the need to establish a common understanding of their application for existing and future teachers and raters.

‘If you have this and the teachers of the swim school, and an in-course training on how to use this, I think it would be really, really good.’ Responder 4-2

The need for consistency of aquatic skill assessment within the industry emerged as a major topic of discussion. Participants spoke of the need for consistency of assessments in order to meet identified standards.

‘…it's a step in the right direction for standardizing assessment across an industry that doesn’t have a standardised assessment.’ Responder 4-1

Participants also identified the problems that parents of children enrolled in learn-to-swim classes can pose within the aquatic skill assessment process. It was agreed that the parent's ideals and perspectives often constrained the assessment of aquatic skills, such that teachers were restricted to considering certain practices based on what parents want or don’t want for their children.

‘Parents don’t view that (assessment) as value. They want their kids swimming laps and distances and that sort of stuff.’ Responder 8-1

Theme 3. Usefulness of the RAEE assessment tool

Participants acknowledged the potential usefulness of the RAEE as an aquatic skill assessment tool that would improve the current practice of defaulting to the basic C/NYC approach used within teaching and for swim teacher education. This third theme revealed i) consistent and relevant reference criteria, ii) adaptability of the RAEE across aquatic skills, iii) improve communication between aquatic staff and parents, and participants within this study shared their ideas of ways to improve the functionality of the RAEE assessment tool.

The need to have an ability to give a consistent and relevant reference criteria for teachers and raters across the industry, regardless of program or approach was highlighted as a specific need for the industry, and something the RAEE would help provide.

‘This would provide a common tool to be used between centres to then be applied to any program really, which would be useful for the industry on the whole.’ Responder 5-1

The adaptability of the RAEE was also discussed. Specifically, its usefulness across a range of aquatic skills assessment categories (including swimming, water safety and self-rescue) could enable better communication with respect to the situational context and a swimmer's actual abilities when the wider environment is considered.

‘Benchmark consistency across no matter what the curriculum is, where the curriculum is, who delivers it. That give you a consistency that can be a, I don’t know, a central point of reference.’ Responder 6-3

Participants noted that the RAEE could help facilitate better communication with both students and parents, and also between teachers and raters as it provides a more detailed assessment of a student's actual capabilities in different contexts.

‘When you’re giving feedback to participants, or parents of participants, having that ability to go, where we are in that scale, how close do I get to, because for some kids, it's going to take terms or months or weeks or years.’ – Responder 3-2

Participants within this study shared their ideas of ways to improve the functionality of the RAEE assessment tool. Suggestions from participants in this study included improvements about specific language used in the descriptions of the criteria, the formatting and layout of the tool, and methods to implement such a tool within the industry.

‘Where, if the language was just, “as described in the AUSTSWIM manual”, you could adapt it to either because if it's a stroke and technical or if survival and, so I think because it refers to stroke, that's implying that its’ stroke and technical.’ Responder 4-1

Discussion

This study used a focus group approach with an expert panel consisting of AUSTSWIM trainers to explore assessment practices within the learn-to-swim industry and establish the face validity of the RAEE assessment tool for aquatic skills assessment that was developed as a part of this study. Three themes relating to assessment were identified from the focus group data: the C/NYC approach that is prevalent within the industry; concerns within the learn-to-swim and aquatics industry in general, and the applicability of the RAEE assessment tool.

There was a consensus that the assessment of aquatic skills within the learn-to-swim industry lacked consistency to communicate outcomes accurately. A range of issues thought to contribute to this situation were also identified. The lack of consistency described by the panel in this study might relate to how the aquatics industry in Australia has evolved over time, reflecting its beginnings where teachers would rely on incidental expertise. ¹⁹ According to Langendorfer, ¹⁹ a reliance on incidental expertise meant that those who traditionally taught learn-to-swim classes did so because they were already successful at swimming in the narrow sense of being experienced in competitive swimming and/or lifesaving. The learn-to-swim programs that ensued, emphasized skills and techniques developed for the competitive swimming strokes, based on the teacher's personal beliefs and observations, and therefore traditionally showed a wide variation.^19,29,44,45 The learn-to-swim industry has evolved from these beginnings into a commercial enterprise that comprises qualified teachers who have gained experience within the aquatics industry and not necessarily through their own competitive swimming careers. Accordingly, the expectations about what teaching ‘swimming’ should involve have slowly changed since the mid-1990s, to include a broader range of water safety knowledge and skills. The requirements for compliance with standardized government regulations have also become increasingly important.^19,29,32,45 Regulation encompassing areas such as qualifications, insurance, risk management and health and safety has increased the accountability of the professional aquatic teachers.^36,46 However, the retrospective application of these accountability measures to industry that include a wide range of learn-to-swim programs, (with different skills and techniques being emphasized), has led to problems with consistency with respect to what are essential and/or foundational aquatic skills that need to be taught.

This lack of consensus about what constitutes the fundamental set of aquatic skills that are most appropriate to include in learn-to-swim programs, and what level of proficiency of these aquatic skills is required for the beginner (entry level) swimmer, constrains the industry's ability to produce recognized standards. ⁴⁷ A related problem within the industry is the use of the same generic term of ‘swimming’ to refer to any set of skills, programs and/or standards that is created by individual teachers or swim schools under the banner of ‘learn-to-swim’. This has created a situation where entry or ‘beginner’ level participants in learn-to-swim are referred to by many names (for example stages, levels, colors and/or various aquatic animal names), and these can include different combinations of water familiarization, water safety and swimming skills. The use of the vague term ‘swimming’, combined with a lack of recognized standards to narrow down what should be taught, means that there has been a breakdown in the ability to convey what skills the learner is or is not yet proficient in, and what aquatic skills have not yet been taught to a particular learner. Therefore, what is communicated through the assessment outcome of ‘competent in beginner or foundational skills’ does not reflect any established set standards nor focus (i.e., competition, water safety, recreation and/or health) of the learn-to-swim activities included within the program. Even if a set of skills can be identified (e.g., swim school mission statements), the standards that the student should be measured against is left to individual assessor's interpretation, which is influenced by their qualification, personal perspectives and experience.^19,48 Inconsistency in assessment practices within the industry may also be related to raters lack of specific training in skills assessment, which is a wider issue for the aquatic industry as a whole. ³²

The lack of consistency in what is taught and assessed within the aquatic industry has been linked to the absence of an acknowledged assessment criteria for aquatic skills taught by teachers^14,45,48 and teachers not using existing benchmarks for assessments.^27,29 This may be a consequence of the commercial nature of the industry and related to the transitory nature of the workforce, with commercial swim schools having a high teacher turnover rate^32,48 and the impact of COVID-19 leading to acute shortage of teachers. ⁴⁹ The occupation of swim teacher is generally seen as a short term vocation (5 years or less), with the majority employed casually, and nearly half holding secondary employment and/or undertaking studies for a full time career in another industry.^32,50 This high staff turnover causes additional burdens of time, costs associated with recruiting and training new staff, and leads to situations where teacher's training, experience and exposure to best practice and/or standards can be limited.

Perceived parental expectations about aquatic assessment was identified by the AUSTSWIM trainers in this study as restricting the scope of assessment procedures incorporated into learn-to-swim programs. A failure to appreciate the need for the assessment of a wider set of aquatic skills (i.e., aquatic competency) was noted in this current study, with parents exerting their influence on the assessment practices used and frequently valuing only assessments that describe the capability to swim specific distances (i.e., ‘laps’ of competitive strokes). This idea is supported by a survey of teachers of swimming and water safety, that found teachers believed parents valued competitive swim strokes more than teachers. ²⁷ However the majority of surveyed parents cited they valued personal survival skills and water safety knowledge, ²⁷ revealing a mismatch in the perceived parental biases between what is valued and what is actually taught. ²⁹ This disparity of opinion may be that parents with a competitive focus are in the minority, yet are more vocal in expressing how assessment outcomes are communicated, and therefore exert undue influence on the wider assessment practices across the industry. ⁵¹

A persons practices can be influenced by past experiences, strongly held beliefs and the input from vocal minorities.^19,51 These factors can contribute to inconsistencies in aquatic skills and the criteria raters choose to focus their assessments upon. Furthermore, assessment outcomes that appeal to a simple C/NYC dichotomy do not clearly communicate the influence of these important factors in making that judgement (e.g., assessor's past experience, training and familiarity of skills, environment and student).^18,52 For example, an assessment of ‘competent’ does not communicate if this competency is tied to a specific ‘situational context’ (i.e., if competence is dependent on the depth of water or familiarity of the aquatic environment such as knowing where the ladder or step is etc.). This current study has identified that the inability to communicate the existence or extent of these factors and situational contexts is considered to be a limitation of the current practices that deem a learner ‘competent’, one that requires further acknowledgement and action. This establishes a need for an improved capacity for assessing beyond the C/NYC that appears to be the default approach in the absence of any other relevant tools.

The final theme to emerge from the focus group in this study pertained to the usefulness of the proposed RAEE assessment tool that was designed to acknowledge the situational context surrounding aquatic skill performance and provide an improved way to identify and communicate different levels of aquatic skill competence. The AUSTSWIM trainers in this study reported that the increased number of potential assessment outcomes provided by the RAEE would help facilitate a more accurate assessment of a student's aquatic skills. Overall, this study supports the face validity of the RAEE assessment tool and its ability to measure aquatic skills in general, but especially for its ability to better establish the aquatic competency objectives of the AWSC's (2016) benchmark. The RAEE was acknowledged as being able to improve consistency in assessment within the industry (i.e., reliability), a result consistent with other studies that have found most reliable scores come from scales. ²⁸

The AUSTSWIM trainers’ opinion in this current study suggested that the proposed seven levels of the RAEE would be too unwieldy and time consuming for use in real time assessment of aquatic skills and that a four-level tool would likely be more practical to implement. This viewpoint encompasses several issues related to assessment in general, that consistent application of the RAEE would be useful in addressing. For example, the current state of assessment within the aquatics industry, similar to the situation in other physical education assessments in general, lacks sufficient training for how to conduct assessments in the field.^53–55 Common issues cited in the literature include the lack of familiarity with assessment criteria; limitations in reliable assessments in real time due to staffing restrictions; concurrent large group assessment and lack of time or resources.^53,54,56,57 The use of the C/NYC approach, which is common in the aquatics industry, can obscure the influence that these issues have had in determining assessment outcomes. The RAEE was seen by the AUSTSWIM trainers in this study as a useful tool due to its adaptability in being relevant for a range of aquatic skills and its ability to improve communication. The role of teacher or environmental assistance in the performance of aquatic skills is recognized in the RAEE and as such it highlights situational competence (i.e., can indicate where someone is competent but only with external assistance). This means that this assessment tool can create a standardized reference point that can be clearly communicated to other raters, teachers, students and/or parents, regardless of the program, skills or criterion being used. The benefit of being able to give consistent, constructive feedback, using a recognized framework, is acknowledged as an important aspect of student's progress and motivation in the physical education sphere.^34,53 The RAEE was reported as showing potential to provide welcome and a positive ability to provide critical feedback of an individuals’ aquatic competency within the time constrained format of current assessments.

Conclusion

This study supported the face validity of the RAEE assessment tool in relation to the AWSC (2016) recommended benchmark level of swimming and water safety skills, as a useful tool to implement in aquatic skills assessment. The RAEE addresses limitations associated with the current C/NYC approach to assessment in the learn-to-swim industry. Firstly, it was seen to overcome the inability to accurately identify the full capabilities of the learner, taking into account their personal developmental characteristics, the task demands and environmental context. Secondly, it was shown to have the potential to provide constructive feedback about how to improve aquatic skills, allowing the teacher to convey the importance of the content and to easily communicate the context of the assessment outcomes, which is beneficial to both raters and parents. This study has shown the potential for the RAEE assessment tool to provide a valid framework to better communicate assessment outcomes and warrants further research to investigate refining this tool for use within the aquatics industry, including further validation and testing.

Supplemental Material

sj-docx-1-spo-10.1177_17479541251334018 - Supplemental material for Face validity of a new aquatic skill assessment tool: RAEE (refuse, assist, effective, efficient)

Supplemental material, sj-docx-1-spo-10.1177_17479541251334018 for Face validity of a new aquatic skill assessment tool: RAEE (refuse, assist, effective, efficient) by N. Nyitrai, C. James, M. Brunton and S. Edwards in International Journal of Sports Science & Coaching