Content validity of a new aquatic skill assessment tool: RAEE (refuse,assist,effective,efficient)

Abstract

The assessment of ‘swimming’ in the context of preventing drowning is complex. Programs may/may not emphasize water safety, survival or aquatic competency. Therefore, establishing content validity of any assessment tool is an essential first step for a more consistent approach for assessments in learn-to-swim. The content validity of the RAEE (Refuse, Assist, Effective, Efficient) assessment tool was investigated using AUSTSWIM experts (n = 23) as participants, the Australian Water Safety Council's benchmark program and a cross-sectional questionnaire on demographics and evaluation of relevance and usefulness of the RAEE assessment tool in assessing these aquatic skills. Opinions from industry standard experts supported the content validity of the RAEE assessment tool for use in assessing the benchmark level for swimming and water safety, finding it relevant (77%±8) and useful (73%±5). A range of issues requiring ongoing research were highlighted, including communication (terminology surrounding swimming and water safety), assessment (pedagogical approach, program design/use, assessment methods/tools), and raters knowledge and understanding of water safety skills. There is an apparent disconnect between awareness of water safety (expert awareness of the Australian Water Safety Council 96%) and how that is translated into practical knowledge and understanding, regardless of the program being taught. Increasing the usage and knowledge of valid and reliable assessment tools can potentially help establish the link between drowning prevention and the aquatic skills, knowledge and understanding are required in an aquatic environment.

Keywords

Drowning personal flotation device swimming water safety

Introduction

The World Health Organisation (WHO) guided by research into global drowning prevention has identified swimming, water safety and self-rescue skills, knowledge and understanding (collectively to be referred to as aquatic skills) to be key factor in drowning prevention.^1–3 Following the 2016 National Water Safety Education Competency Framework, an integral part of Australia's Water Safety Strategy (AWSC), it is recommended for learn-to-swim programs to ensure that all children within their programs attain a benchmark level of age appropriate aquatic skills.² The instruction and assessment of aquatic skills (aspects of the more holistic construct of aquatic competency), is mostly conducted within learn-to-swim programs, delivered nationally by a range of independent commercial and school-based providers.^4–7 The range of learn-to-swim providers makes it difficult to establish the overall effectiveness of any particular program in meeting the goals of AWSC as the aquatic skills taught and assessed by each school varies.⁸

The lack of a consistent approach across learn-to-swim providers may in part stem from the term ‘swimming and water safety’, terminology intended to emphasize the importance of including water safety in lessons.^9,10 This term refers to the separate elements of swimming, in the sense of a physical motor action performed to propel one's body through the water and to a series of more safety-oriented actions (and the associated knowledge) required to successfully interact in various aquatic environments.³ This general term of ‘swimming and water safety’ is open for interpretation in relation to what specific skills should be included in the design and delivery of swim programs, with different perspectives prevalent within the industry, about which are most essential in drowning prevention.^6,11 In addition to the lack of consensus about what aquatic skills should be taught within a learn-to-swim program, it also remains unclear which of those aquatic skills should be assessed, and how this should be accomplished.^4,12 This is exacerbated by a lack of recognized tools to assess these aquatic skills.^6,8,13,14 While assessment tools have been developed for specific aquatic programs (e.g., assessment for students with disabilities and infants/pre-schoolers), their recognition and general use across the mainstream learn-to-swim industry is limited.^6,15 ^#3728

The term ‘swimming’, is frequently used in a colloquial sense to denote any grouping of aquatic skills needed within the ‘learn-to-swim’ industry, which may or may not emphasize water safety and survival skills or the broader notion of aquatic competency.^7,12,16 Establishing a valid tool to assess the attainment of aquatic skills requires differentiating between assessing ‘swimming’ skills and the more general water safety and self-rescue skills.¹⁷ This is because the intent of application for specific aquatic skills and knowledge such as for competition, recreation and/or survival interacts with the interpretation of relevant assessment criteria. Such ambiguity about what it means to be proficient at ‘swimming’, confounds attempts to establish valid and reliable assessments.⁶ a problem acknowledged by various sources.^18–22

Existing aquatic skills assessment methods have evolved from physical education practices in general.^4,23–25 Formal assessments of movement skills in physical education are considered time consuming, of little value and usually include only a dichotomous nominal scale assessment methods, a situation that is echoed in learn-to-swim.²⁶ However, in an aquatic environment, where the goal is water safety, survival and drowning prevention, a dichotomous scale that identifies whether someone can or cannot accomplish a specific skill, might not appropriately indicate the extent of skill acquisition. Nor can it accommodate the situational context of the aquatic skill performance. This refers to the variety of environmental factors such as changes to water depth, water temperature or current,²⁷ which can potentially change the outcome of an individual's assessed performance from seemingly ‘expert’ to ‘only just’ or ‘cannot’.^28,29 Given assessment outcomes for aquatic skills are used as an indicator of personal safety, an assessment approach that lacks the ability to differentiate discrepancies across conditions in proficiency is not ideal.^2,12 There is a need for assessment tools that are specific to learn-to-swim, designed to incorporate evidence-based measurement approaches that acknowledge and address these limitations.

One approach to address the limitations of aquatic skill assessments is to incorporate an ordinal scale, which allows for the qualitative aspects of the movement patterns being assessed to be identified and corrected (i.e., process oriented assessment).^30–32 This would better relate to the ultimate intended outcome of attaining aquatic skills. Prevention of drowning is complex and includes the ability to recognize and understand interactions (both intentional and unintentional) with water and how to apply appropriate actions and decision making to mitigate potential risks in an aquatic environment.¹⁶ This goes beyond the ability to swim using a recognized stroke in a controlled environment, and it is these type of actions in an aquatic environment that needs to be included in ‘learn-to-swim’ assessments if they are to be valid strategy in the prevention of drowning.^3,19,33,34 In addition to requiring a focus on the qualitative aspects of the skills being measured, valid assessments should align with the goals of the program (i.e., which combination of swimming, water safety and self-rescue skills were used and with what intent). In response to these identified needs, a process oriented, ordinal scale assessment tool, referred to as RAEE (Refuse, Assist, Effective, Efficient; Table 1),³⁵ that used the Australian Water Safety Council's benchmark program – Royal Life Saving Society Australia (RLSSA) Swim & Survive (level - Active Award 4–2016 version), was developed. We have established the face validity of the RAEE as a useful tool to implement in aquatic skills assessment.³⁵

Table 1.

Assessment tool – RAEE (refuse, assisted, effective, efficient). The RAEE is designed to apply to movement skills, water safety knowledge and understanding with the rating system broadly broken into four main areas: Refusal – Assisted – Effective – Efficient.

1	Refused to attempt task.
2	Attempted task – did not complete.
3	Attempted task – completed with assistance.	Assistance is defined as kickboards, noodles, floats or physical assistance from another person. Breathing is controlled throughout task – i.e., no choking, gasping or reliance on aids to breathe comfortably.
4	Completed task with difficulty, little or no confidence and/or inefficiently.	Difficulty is defined as the inability to coordinate the individual components of the stroke as a whole (i.e., body position, arm action, leg action and breathing).
5	Completed task with moderate ease and lacking confidence and/or efficiency (Correct stroke technique as described in the AUSTSWIM manual).	Moderate ease is defined as the ability to coordinate the individual components of the stroke as a whole, but not having the ability to maintain that coordination over distances greater than 10 m. Confidence is demonstrated through non-reliance on aids such as goggles, pool edge, shallow water and/or lack of anxiety in the water.
6	Completed task with ease, but lacking confidence and/or efficiency (Correct stroke technique as described in the AUSTSWIM manual).	Ease is defined as the ability to coordinate the individual components of the stroke as a whole and maintaining that coordination over distances greater than 10 m.
7	Completed task with ease, confidence and with moderate or high efficiency (Correct stroke technique as described in the AUSTSWIM manual).	Efficiency is demonstrated through reduction of frontal and eddy resistance affecting the body while moving through water – i.e., body position, maintaining streamline, reducing time/distance from streamline position.

Establishing content validity of a movement assessment tool is commonly achieved through exploring the opinions of experts.³⁶ An expert panel is the simplest and most direct method in this respect and provides a means for gauging whether an assessment tool is an accurate reflection of the content the tool is meant to measure. For this content validity study, the organization recognized as providing the national industry standard for training swim instructors was incorporated. The Australian Council for the Teaching of Swimming and Water Safety (AUSTSWIM), is a national governing body including representatives of swim, surf and water safety organizations from all states and territories of Australia.⁴ AUSTSWIM is internationally recognized as the first aquatic education organization in the world to be accredited for how teachers are trained, accredited and licensed (ISO/IEC 17024), and therefore recognized as industry leaders in learn-to-swim.³⁷

This study aimed to explore the opinions of a panel of AUSTSWIM trainers, known as ‘presenters’ within the industry, to establish the content validity of the newly developed nominal, process-oriented assessment tool RAEE, in assessing the ‘must see’ criteria of skills in the RLSSA Swim & Survive (Active Award 4–2016 Version).

Method

Participants

Establishing the content validity of this assessment tool is an important first step to bring a more consistent approach for aquatic skills assessment to the learn-to-swim industry. Specifically developing a valid tool to assess the full range of aquatic competencies necessary to aid with drowning prevention. Therefore, a panel of experts in aquatic competencies (n = 23) were recruited from a cohort of AUSTSWIM presenters attending one of two professional development workshops at the ‘Soak it up 19’ AUSTSWIM State Conferences in New South Wales and Western Australia. An AUSTSWIM Presenter (also known as a ‘trainer’) is an accredited AUSTSWIM teacher who has undertaken further training to meet accreditation requirements to deliver the knowledge and skills in the delivery and assessment of the AUSTSWIM Teacher of Swimming and Water Safety™ course. Participant information sheet and consent forms were sent via email by the AUSTSWIM conference organizers to the registered participants prior to the workshop and re-distributed in hard copy at each workshop to obtain consent form prior to commencement of the questionnaire.

Protocol

A cross-sectional questionnaire approved by The University of Newcastle Human Research Ethics Committee (H-2019-0069) was disseminated to a cohort of AUSTSWIM presenters recruited. These paper-based aquatic skill questionnaire included two main sections and included questions 1) relating to demographics and background, and 2) questions designed to evaluate the relevance and usefulness of the RAEE assessment tool in assessing the ‘must see’ criteria of the RLSSA Swim & Survive Active Award 4. The AUSTSWIM experts watched a video compilation of Active Award 4 skills being performed prior to answering the relevance and usefulness questions. When summarizing the results to establish content validity a criterion level needs to be established.³⁸ In this case, agreement from 7 out of 10 experts (70%) was recognized as the minimum level of agreement amongst the experts in order to find the assessment tool to be valid.³⁸

Aquatic skill questionnaire

The paper-based aquatic skill questionnaire included two main sections. Section one involved demographic and presenter background questions that were divided into four categories: 1) basic information of age, sex, years of experience; 2) information about experience relating to teaching environments, including programs and assessment methods utilized; 3) Information about qualifications, the maintenance of currency and awareness of benchmark criteria; and 4) awareness of the AWSC, benchmark and associated programs/skills.

The second section of the questionnaire offered a series of questions asking each participant's opinion of the RAEE assessment tool. This section included a color coded table that presented the RAEE Assessment tool including the criteria for use of the aquatic skills tool (Table 1). The questions response options in this section were a three-point Likert scale (not relevant, neutral, relevant) and presented against an itemized list of each ‘must see’ criteria from the RLSSA Swim & Survive Active Award 4 Assessment guide. Participants were asked to rate how relevant the RAEE assessment tool was for assessing each criterion, using the Likert scale, then to repeat the same rating process to gauge how useful the RAEE was for assessing each criterion.

Procedures

After completing the first two sections of the questionnaire, participants were given five minutes to review the RAEE assessment tool material provided and had any queries answered, prior to viewing the video compilation. The participants then viewed, as a group, the three-minute video compilation that demonstrated the performance of all 14 aquatic skills from the RLSSA Swim and Survive: Active Award 4. After viewing the video clips, participants completed the third section of the questionnaire that related to the relevance of the RAEE tool. Following this, the same video compilation was re-played before participants completed the final section of the questionnaire relating to usefulness of the tool. All completed questionnaires were then collected.

Video clip compilation

The video recording of the 14 RLSSA Swim & Survive Active Award 4 aquatic skills shown to the panel included adult volunteers filmed completing these aquatic skills in a 50 m heated pool as part of a larger study (H-2015-0272). These swimmers (n = 28) were aged between 18 and 34 years (self-reported) and were assessed by a licensed AUSTSWIM trainer as competent swimmers. Each aquatic skill performance was filmed using a hand-held video camera (Neos Wi-Fi Model No: HN-TOP1). All skills were performed in accordance with RLSSA Swim & Survive Active Award 4, with the exception of modifications to clothing worn (long- sleeved shirt, long sleeved jumper and long pants) and exit (into an inflatable boat). The video of each aquatic skill was edited to approximately 10 s and a 3-min compilation video of these skills was created.³⁹

Analysis

The questionnaire data were collated and imported into SPSS (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp) and descriptive statistics calculated for all demographic questions. Any unanswered questions were allocated to ‘no answer’ and included in the descriptive statistics. In addition, percentage agreement was calculated for the question relating to the usefulness and relevance of the RAEE, and the criterion for the minimum level of agreement in order to find the assessment tool to be valid set at 7 out of 10 experts (70%).³⁸

Results

Demographics

Twenty-three participants (n = 20 female, n = 3 male), with a median age group of 45–54 years consented to be involved in the study (Table 2). Almost half (48%, n = 11) of participants had held an AUSTSWIM qualification for more than 20 years, and a further quarter for between 15–20 years (26%, n = 6). All participants were AUSTSWIM qualified as Teachers of Swimming and Water Safety, with approximately three-quarters of experts also being qualified to teach Infants and Preschool (78%, n = 18), Presenters (74%, n = 17), and Access and Inclusion (70%, n = 16). Maintaining currency of qualification was achieved mostly by attendance of state conferences (87%, n = 20) and professional development workshops (87%, n = 20).

Table 2.

Participant demographics.

Sex			Age (Years)			Years Qualified			Qualification(s)			Continuing Education
	n	%		n	%		n	%	Teacher of:	n	%		n	%
Male	3	13	<18	0	0	<1	1	4	Swimming and Water Safety	23	100	AUSTSWIM elective	14	70
Female	20	87	18–24	0	0	1 to 3	0	0	Access & Inclusion	16	70	Regional Conference	8	35
Other	0	0	25–34	1	4	4 to 6	1	4	Towards Competitive Strokes	14	61	State Conference	20	87
			35–44	7	30	7 to 9	0	0	Infants & Preschool	18	78	National Conference	9	39
			45–54	8	35	10 to 15	4	17	Adults	12	52	Professional Development workshop	20	87
			55–64	6	26	15 to 20	6	26	Aqua Instructor	1	4	AUSTSWIM e-learning	12	52
			>65	1	4	>20	11	48	Presenter	17	74	Other	0	0
									Presenter in Training	3	13

Most participants worked with commercial swim schools employing more than six instructors (78%, n = 18), followed by school-based swim schools (70%, n = 16; Table 3). Individually designed programs run by swim schools were the most commonly reported program (57%, n = 13), followed by school-based swim schools, Swim and Survive programs (39%, n = 9) and individually designed programs (35%, n = 8). The assessment approach undertaken to identify the appropriate placement into a learn-to-swim program, for both new students and for progression within an existing program, was primarily conducted informally by an individual instructor (83%; 78%) or single assessor (74%, 57%), respectively.

Table 3.

Program demographics.

Type of swim school			Type of aquatic programs utilized							Who completes Aquatic Skill assessment
	n	%		n	%	n	%	n	%		n	%	n	%
Private	12	52	Program	Private		Swim School		School Based			New students		Progressing students
Commercial* < 5 instructors	13	57	Swim & Survive	2	9	6	26	9	39	Single assessor	17	74	13	57
Commercial* 6 + instructors	18	78	Swim Safe	0	0	2	9	1	4	Instructor assessor	19	83	18	78
School based	16	70	Franchise	1	4	3	13	0	0	Written	6	26	9	39
Other	2	9	Individual	7	30	13	57	8	35	Informal	15	65	4	17
			Other	0	0	1	4	7	30	Other	1	4	2	9

* - Number of staff employed in Commercial Swim School.

Almost all participants were aware of the existence of the AWSC (96%, n = 22), however only just over half were aware of the benchmark (61%, n = 14; Table 4). While nearly three quarters of the participants used the Swim and Survive program (74%, n = 17), only just over half used the assessment guide included within this program (52%, n = 12). Even fewer participants (48%, n = 11) reported being familiar with all aquatic skills included within the Swim and Survive program.

Table 4.

Awareness and use of benchmark.

Aware of AWSC			Aware of benchmark			Familiar with Swim & Survive skills			Use Swim & Survive			Use Swim & Survive assessment guide
	n	%		n	%		n	%		n	%		n	%
Yes	22	96	Yes	14	61	Yes	11	48	Yes	17	74	Yes	12	52
No	1	4	No	8	35	Most but not all	2	9	No	6	26	No	10	44
			No answer	1	4	Some but not all	6	26				No answer	1	4
						No	4	17

Relevance

Each ‘must see’ criteria of RLSSA Swim and Survive: Active Award 4, as listed in the accompanying assessment guide to this program, was ranked on a Likert scale. When responding to questions about the relevance of the RAEE for each of the listed items, over three quarters of participants (77% ± 8 SD) found the tool to be relevant across all items (Table 5). However, the individual levels of agreement dropped below the 70% criterion level³⁸ for three items (one from sculling and body orientation and two from survival and personal flotation device skills). When responses were considered according to the experts’ use of the RLSSA Swim and Survive program in practice, the relevant response increased to a 71% amongst those who actually used the program.

Table 5.

Content validity results: relevant and useful percentage agreement – RLSSA swim & survive ‘must see’ assessment criteria.

RLSSA swim & survive ‘must see’ assessment criteria	Relevant %	Useful %
Entry and Exit – Demonstrated skills - Compact jump & deep-water exit
Safe and confident entry	83	65
Body vertical, streamlined and protected	78	74
On entry, body tucked or piked to slow downward movement	78	74
Exit unassisted from deep water using the edge	78	65
Section average	79	70
Sculling and Body Orientation - Feet First Scull and tucked body rotation
Attempted controlled hand sculling action	78	78
Movement in a feet first direction	74	70
Rotation about vertical axis	74	70
Body in tucked position	74	74
Face maintained above the surface of the water	65	74
Section average	73	73
Movement and Swimming Strokes – Above-water and underwater arm recovery
Continuous swim for 75 metres	87	74
50 m swum using above water arm recover stroke(s) (e.g., FS, BS, Bfly)ⱡ	100	83
25 m swum using an underwater arm recovery stroke(s)(e.g., BrS, SBS,SS)ⱡ	96	83
Body position, breathing and arm and leg action should be recognisable as the chosen strokes	78	78
Section average	90	79
Survival and personal flotation device skills - clothed survival swim
Scull, float or tread water confidently	83	70
Stationary position	70	70
Face clear of the water	65	70
Swim slowly using survival strokes	78	74
Change stroke after every minute	65	61
Collect aid when thrown	70	70
Hold aid securely for support	70	70
Section average	71	69
Underwater skills - surface dive
Submerge head or feet first confidently	74	78
The body should be completely submerged	74	74
Effective underwater swimming technique	87	83
Open eyes underwater to locate object	78	70
Object brought to surface	70	70
Section average	77	75
Rescue Skills - Throw rescue flotation aid 5 m
Provide reassurance and effective instructions to partner	78	74
Ensure self- preservation positioning safely and securely	78	74
Throw accurately	83	74
Partner instructed to safety and secured	83	74
Section average	80	74
Water safety knowledge - dangers in aquatic environment
Answered question confidently	74	78
Extension skills - butterfly arms 5 m
Attempt simultaneous and symmetrical action	78	83
Overall Average	77	73

ⱡ FS (freestyle); BS (backstroke); Bfly (butterfly); BrS (breaststroke); SBS (survival backstroke) & SS (sidestroke).

Usefulness

Usefulness of the RAEE was determined by selecting from a scale of useful, neutral, or not useful against the same ‘must see’ criteria. The responses for how useful the RAEE assessment tool was similar to its relevance findings, with just under three quarters of participants finding the tool to be useful (73% ± 5 SD). There was an increase in ‘no answer’ (11%) responses, as well as three items that didn’t meet the 70% criterion level (two from entry and exit skills, and one from survival and personal flotation device skills). Again, when use of the program was taken into account, all but one item reached a minimum of 71% agreement that the RAEE tool was useful. The only item that did not meet the criterion, regardless of use of the RLSS Swim and Survive program was part of the survival and personal flotation device skills (change stroke after every minute).

Discussion

This study explored the content validity of the newly developed RAEE aquatic skills assessment tool. Establishing a valid aquatic skill assessment tool specific to the learn-to-swim industry is important given the need to ensure the aquatic skills assessments being conducted are not only an accurate reflection of the intended content, but also relate to the proposed outcome of drowning prevention.⁶ Expert opinion from industry standard experts (i.e., AUSTSWIM trainers) was sought in this content validity study and the results indicated that the RAEE aquatic skills assessment tool met the criterion level agreement for the overall items for both relevance and usefulness, with 77% and 73% respectively. This would indicate that this aquatic skill assessment tool is valid for both face validity³⁵ and content validity (relevant and useful) in this study when assessing the aquatic skills with the RLSSA Swim and Survive – Active Award 4.

It was identified that assessments of aquatic skills within the learn-to-swim industry are based most frequently on subjective opinion based on the raters’ observation. The validity of a subjective qualitative assessment is often criticized as it is susceptible to being prone to personal bias, not being judged against well-defined criteria, and/or not being recorded.⁴⁰ This study found variability between individuals in levels of agreement for relevance for the individual assessment items (grouped into eight sub-sets of similar skills) suggesting subjective bias.

Specifically, the results within the sub-sets illustrated a disparity between the levels of agreement for items assessing swimming skills (movement and swimming strokes, 90% agreement as relevant) when compared with the survival and water safety skills (survival and personal flotation device skills, 71% agreement as relevant). It is likely that this disparity in the opinions about the relevance of using a tool to assess these skills reflect a bias towards teaching locomotive and specific swimming strokes rather than the full range of aquatic skills (including survival and personal flotation device skills) in the learn-to-swim industry.⁷ In Australia, up to 80% of swim schools do not provide essential water safety and survival oriented aquatic skills and knowledge in their programs, and are more likely to teach swimming skills (e.g., freestyle 87%, backstroke 83%) than water safety skills (e.g., treading water 53%, survival backstroke 46%).⁷ The lower reported level of agreement for the relevance of the RAEE for assessing survival skills may be indicative of either a lack of familiarity with water safety skills in general and/or the specifically defined assessment criteria. This is supported by the demographic results that showed less than half of the experts were familiar with all of the skills of the Swim and Survive program (47%), which specifically incorporates water safety and self-rescue items.

The usefulness of the RAEE assessment tool when considering the level of agreement for each of the subsets, showed that 79% identified it was useful for movement and swimming strokes and 69% for survival and personal flotation device skills. The level of agreement for survival and personal flotation device skills was similar to the findings relating to the relevance of the RAEE, with the exception of movement and swimming strokes where 79% indicated it was useful yet 90% indicated it was relevant. As the majority of swim schools have a focus on teaching swimming strokes over water safety or survival skills, these results may reflect a greater familiarity for teaching and assessing these skills amongst instructors.^7,12 A possible explanation for rating the usefulness of the RAEE lower for the movement and swimming strokes could, as with the findings for relevance, be the higher level of familiarity with these items compared to familiarity of the survival and self-rescue items. This is supported by the demographic results that indicate less than half (48%) were familiar with all of the Swim and Survive tasks. There is an apparent disconnect between awareness of water safety (expert awareness of the Australian Water Safety Council 96%) and how that is translated into practical knowledge and understanding, regardless whether the program taught is private, swim or school based. This disconnect warrants further investigation as there appears to be multiple aspects regarding the teaching of water safety skills that could be impacted. Aspects such as the adequacy of existing training and ongoing support within the aquatics industry, the inconsistent inclusion within learn-to-swim programs, as well as the lack of familiarity amongst teachers of swimming and water safety.

Practical findings for swim instructor educators

The minimum standard of skills attainable for all Australian children is the AWSC's 2016 benchmark level of swimming and water safety. In the aquatic industry there is a general lack of consensus relating to what constitutes proficiency in performing these aquatic skills,^21,35 inconsistency to communicate outcomes accurately as well as in assessment practices³⁵; a finding reaffirmed by this current study. For swim instructor educators it is important to ensure that their knowledge and understanding of aquatic skills are consistent with other fellow swim instructors and/or swim school(s) they work with by maintaining clear communication in teaching and assessment. To do this, the RAEE offers swim instructor educators a valid framework to better communicate assessment outcomes, which is likely beneficial to both other swim instructors and parents.

Conclusion

While the results of this study supported the content validity of the RAEE assessment tool for use in assessing the benchmark level for swimming and water safety, it has highlighted a range of issues requiring ongoing research. These issues include communication (terminology surrounding swimming and water safety), assessment (pedagogical approach, program design/use, assessment methods/tools) as well as knowledge and understanding of aquatic skills, specifically those relating to water safety skills amongst AUSTSWIM trainers. This exploration into the content validity of an assessment tool for swimming and water safety highlights the need for valid and reliable assessments within the learn-to-swim industry. Increasing the usage and knowledge of valid and reliable assessment tools can potentially help establish the link between drowning prevention and the skills, knowledge and understanding are required in an aquatic environment.

Footnotes

Acknowledgements

The authors of this study would like to acknowledge statistician Kim Colyvas for his invaluable assistance in providing statistical support and assistance with using the JMP Pro 14 and IBM SPSS Statistics 25 software. This assistance was provided as part of the Higher Degree Research provisions by The University of Newcastle Australia.

Ethical considerations

This study was approved by The University of Newcastle Human Research Ethics Committee (H-2019-0069).

Consent to participate

Participant information sheet and consent forms were sent via email by the AUSTSWIM conference organizers to the registered participants prior to the workshop and re-distributed in hard copy at each workshop to obtain consent form prior to commencement of the questionnaire.

Declaration of conflicting interests

Suzi Edwards, Carole James and Michaela Bruton has no interest to declare. Nina Nyitrai is a AUSTSWIM Trainer Assessor, a contract trainer for Royal Life Saving Society Australia and is the owner of Swim4Surivial that provides in-service training and education for swimming pool and swim school centres.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

S Edwards

References

World Health Organisation. Preventing drowning: an implementation guide. Geneva, Switzerland: World Health Organisation, 2017, pp. 1–97.

Australian Water Safety Council. Australian water safety strategy 2016–2020. Report no. https://www.royallifesaving.com.au/__data/assets/pdf_file/0020/37253/RLS_AWSS2016_Report_2016LR.pdf. 2016.

Langendorfer

Moran

Stallman

. Guiding principles: applying water competence to drowning prevention. Int J Aquatic Res Educ 2018; 11: 22.

AUSTSWIM. AUSTSWIM teacher of swimming and water safety™ (TSW). 3rd ed. Sydney, Australia: AUSTSWIM, 2020.

Hulteen

Morgan

Barnett

, et al. Development of foundational movement skills: a conceptual model for physical activity across the lifespan. Sports Med 2018; 48: 1533–1540.

Langendorfer

. Aquatic readiness: developing water competence in young children. Champaign, IL: Human Kinetics Publishers, 1995, pp. 1–224.

Willcox-Pidgeon

Peden

Scarr

. Exploring children's participation in commercial swimming lessons through the social determinants of health. Health Promot J Austr 2021; 32: 172–181.

Peters

Moncrieff

. Swim lesson models: effectiveness and impact study. Report no. https://www.researchgate.net/publication/336372312_Swim_Lesson_Models_Effectiveness_and_Impact_Study. 2019.

Royal Life Saving Society – Australia. National swimming and water safety education symposium summary report. Report no. https://www.royallifesaving.com.au/__data/assets/pdf_file/0006/73743/2023-RLSSA-National-Aquatics-Symposium-Report-FINAL.pdf. 2017. Sydney.

10.

Royal Life Saving Society – Australia. National swimming and water safety framework. https://www.royallifesaving.com.au/__data/assets/pdf_file/0018/32157/National-Swimming-and-Water-Safety-Framework_FINAL-2020.pdf (2020, accessed 8 July 2025).

11.

Petrass

Blitvich

McElroy

, et al. Can you swim? Self-report and actual swimming competence among young adults in Ballarat, Australia. Int J Aquatic Res Educ 2012; 6: 5.

12.

Australia RLSS. Royal Life Saving – National Drowning Report 2019. https://www.royallifesaving.com.au/__data/assets/pdf_file/0005/32684/rlssa-ndr-2019-digital.pdf (2019, accessed 8 July 2025).

13.

Grosse

. Assessment of swimming in physical education. Strategies 2005; 19: 35–36.

14.

Sršen

Vidmar

Pikl

, et al. Content validity and inter-rater reliability of the halliwick-concept-based instrument ‘swimming with independent measure’. Int J Rehabil Res 2012; 35: 116–123.

15.

Tirosh

Katz-Leurer

Getz

. Halliwick-based aquatic assessments: reliability and validity. Int J Aquatic Res Educ 2008; 2: 4.

16.

Guignard

Button

Davids

, et al. Education and transfer of water competencies: an ecological dynamics approach. Eur Phys Educ Rev 2020; 26: 938–953.

17.

Creswell

. Educational research: Planning, conducting, and evaluating quantitative and qualitative research. 4th ed. Boston, MA: Pearson, 2012.

18.

Peden

Franklin

Larsen

. Survey of primary schools across Australia: an examination of key water safety issues. Int J Aquatic Res Educ 2009; 3: 10.

19.

Stallman

Moran

Quan

, et al. From swimming skill to water competence: towards a more inclusive drowning prevention future. Int J Aquatic Res Educ 2017; 10: 1–35.

20.

Langendorfer

. What's in a name? Int J Aquatic Res Educ 2013; 7: 2.

21.

Stallman

. Which stroke first? No stroke first!. Int J Aquatic Res Educ 2014; 8: 5–8.

22.

Royal Life Saving Society – Australia. No Child To Miss Out: Basic Swimming & Water Safety Education – The right of all Australian children. 2012. Sydney, Australia.

23.

Lynch

. Swimming and water safety: reaching all children in Australian primary schools. Int J Aquatic Res Educ 2012; 6: 267–278.

24.

Lynch

. Australian Aquatic curriculum reform: treading water carefully? Int J Aquatic Res Educ 2015; 9: 201.

25.

Barker

Bergentoft

Nyberg

. What would physical educators know about movement education? A review of literature, 2006–2016. Quest 2017; 69: 419–435.

26.

van der Mars

McNamee

Timken

. Physical education meets teacher evaluation: supporting physical educators in formal assessment of student learning outcomes. Phys Edu 2018; 75: 582–616.

27.

Cho

Koh

Chian

. Subjective knowledge in open water activities: scale development and validation. Int J Sports Sci Coach 2024; 19: 723–734.

28.

Durden-Myers

Green

Whitehead

. Implications for promoting physical literacy. J Teach Phys Edu 2018; 37: 262–271.

29.

Baker

. Are children as safe in open water as I think they are? Int J Aquatic Res Educ 2018; 11: 4.

30.

Preston

Colman

. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol 2000; 104: 1–15.

31.

Hulteen

Lander

Morgan

, et al. Validity and reliability of field-based measures for assessing movement skill competency in lifelong physical activities: a systematic review. Sports Med 2015; 45: 1443–1454.

32.

Stodden

Goodway

Langendorfer

, et al. A developmental perspective on the role of motor skill competence in physical activity: an emergent relationship. Quest 2008; 60: 290–306.

33.

Langendorfer

. Changing learn-to-swim and drowning prevention using aquatic readiness and water competence. Int J Aquatic Res Educ 2015; 9: 4–11.

34.

Guignard

Rouard

Chollet

, et al.

Individual–environment interactions in swimming: the smallest unit for analysing the emergence of coordination dynamics in performance?

Sports Med 2017; 47: 1543–1554.

35.

Nyitrai

James

Bruton

, et al. Face validity of a new aquatic skill assessment tool: RAEE (refuse, assist, effective, efficient). Int J Sports Sci Coach 2025. https://doi.org/10.1177/17479541251334018

36.

James

MacKenzie

Capra

. Content validity of the WorkHab functional capacity evaluation. Aust Occup Ther J 2019; 66: 380–392.

37.

AUSTSWIM. AUSTSWIM ISO accredited. https://austswim.com.au/about-us (2013, accessed 25 Nov 2024).

38.

Sireci

Faulkner-Bond

. Validity evidence based on test content. Psicothema 2014: 100–107.

39.

Royal Life Saving Society – Australia. Swim and Survive Aquapak: A Guide for Swimming and Water Safety Teachers. Report no. https://www.royallifesaving.com.au/educate-participate/swimming/swim-and-survive. 2011. Sydney.

40.

Morrow Jr

Jackson

Disch

, et al. Measurement and evaluation in human performance. 3rd ed. Portland: Copyright Clearance Center, Inc, 2005, p. 60.