Electronic data capture on athletes’ pre-participation health and in-competition injury and illness at major sports championships: An extended usability study in Athletics

Abstract

This study set out to identify factors critical for the usability of electronic data collection in association with championships in individual sports. A qualitative analysis of electronic data collection system usability for collection of data on pre-participation health from athletes and in-competition injury and illness from team physicians was performed during the 2013 European Athletics Indoor Championships. A total of 15 athletes and team physicians participated. Athletes were found to experience few problems interacting with the electronic data collection system, but reported concerns about having to reflect on injury and illness before competitions and the medical terminology used. Team physicians encountered problems when first navigating through the module for clinical reporting, but they were not subjected to motivational problems. We conclude that athletes’ motivation to self-report health data and the design of the human–computer interface for team physicians are key issues for the usability of electronic data collection systems in association with championships in individual sports.

Introduction

Major international championships in individual sports, such as the International Association of Athletics Federation (IAAF) World Championships in Athletics or the International Aquatics Federation (FINA) World Aquatics Championships, attract thousands of participants and millions of spectators from all over the world. Despite the development of injury surveillance systems for use at this type of large sports events, ¹ their effective implementation is still a methodological and practical challenge. It has been pointed out that the context of major sports championships features unique constraints, such as a limited window for data collection and large amounts of data to be recorded and rapidly validated.^2,3 To manage such logistical issues, electronic data collection (EDC) methods have been proposed^4–6 and introduced for use at large multi-sport events. ⁷ In many individual sports, overuse conditions, or tissue damage that results from repetitive demand over the course of time, are the most common health problem.^2,3,8,9 For surveillance of overuse conditions, athlete self-reporting of data on pain and other symptoms has been reported to be superior to reports from coaches or medical practitioners. ¹⁰ EDC systems have also been applied for such longitudinal self-reporting of injury data, both in Athletics and in other individual sports.^9,11

There are ample reasons, ranging from increased data quality to efficacy, for investigating the use of EDC systems for collection of self-reported pre-participation health data from athletes and clinical data on injuries and illnesses from medical teams in association with championships in individual sports. Availability of these data sets is essential for planning, implementation, and evaluation of preventive measures. This study sets out to identify factors associated with the usability of EDC methods for gathering of data from athletes and team physicians in association with major championships in individual sports. Usability is defined in this research as the extent to which a system can be used by specified users to achieve detailed goals with effectiveness, efficiency, and satisfaction for an indicated context of use. ¹² The starting point for the study focused on the users of the intended system. It assumes that the short duration of a championship differs from the longitudinal athlete surveillance setting and that the design of the EDC system has to be adjusted accordingly. ¹³ In system design for championship use, it also must be taken into consideration that little or no time is available for learning how to use the EDC system before starting to enter data and that data for the athlete population is culturally heterogeneous.

Methodology

This is an observational study that uses data collected from a population sample at a specific point in time. Qualitative data were collected from athletes and team medical staff using interviews and think-aloud usability evaluation methods.^14,15 Data collection was performed over a 3-day period at the European Athletics Indoor Championships during 1–3 March 2013 in Gothenburg, Sweden.

EDC for Athletics injury and illness data collection

Two demonstration versions of the online EDC systems were developed. The first was for collection of data from athletes and the second for collection of data from team medical staff. An assumption was made that there was no possibility of providing the intended users with on-site training in the EDC systems. The data collection forms in both systems were conceptually identical to the corresponding paper-based forms previously used in the injury and illness surveillance studies conducted during Athletics championships. The demonstration system for athletes (Figure 1) was developed to exhibit collection of data on pre-participation health. It was programmed for the purpose of this study and consisted of a human–computer interface for data collection without underlying database functions.

Figure 1.

EDC system for athletes. Buttons are used (1) to allow selection between three general categories of health problems and (2) to display lists, different for each category. The athletes can select response items from two lists for the categories injury and illness, and from one list for pain. These selections are communicated with (3) ‘OR’ or ‘AND’. (4) In a drop-down menu, the athletes can select the duration of health issue. The reported health issues are shown as (5) list items, which can be removed by clicking the delete icon.

The demonstration system for use by team medical staff (Figure 2) was developed to collect data on injury and illness sustained during the championship according to standardised classifications and vocabularies. ¹⁶ This demonstration system was also programmed for the purpose of this study only, and consisted of a graphical human–computer interface without underlying database functions.

Figure 2.

EDC system for team physicians. From (a) an initial display, the physician navigates to select (b) an injury or illness to report and finishes the report by navigation through a menu system. The application is intended to resemble a simple electronic medical record system.

Study applications

For study of EDC on pre-participation health problems among athletes, a partial EDC system was developed using the open-source tools Node.js (https://nodejs.org/en/) and MongoDB (http://www.mongodb.org) (Figure 1). The central design goal was to provide an interface for data entry that is self-explanatory and simple to use. Evaluations of EDC systems have showed that single select questions have lower error rates compared to free text and date fields. ¹⁷ The intention was therefore to allow the athletes to complete the pre-participation form only using the mouse, that is, without having to use the keyboard. For EDC on injury and illness from team medical staff, an open-source version of the OpenClinica software package (http://www.openclinica.com) was used (Figure 2). This package is created specifically for development of EDC applications and it complies with good clinical practice (GCP) requirements. The motivation for the choice of the package was that the software both has recognised back-end security and is open-source, and thus can be progressively expanded.

Data collection

Participants were recruited by an announcement at the team medical meeting before the championships, asking athletes and team medical staff to participate. The recruitment of participants was based on the saturation principle, that is, that additional participants were invited as long as new phenomena were revealed in the data collected. Before each session, which took place at the championships venue, the participants were informed that the purpose of the systems was data collection in injury and illness surveillance studies (pre-participation and newly acquired injury and illness data) during international athletic competitions, and that no data were recorded, used and/or diffused. Participants were then asked to complete the EDC form while thinking aloud during the process. Qualitative data were gathered from the evaluations through observations, audio recordings and field notes. ¹⁸ Each session was complemented with questions about the difficulty of the task, the usefulness of the system and opinions about reporting data online. Data saturation was found to have been achieved among athletes after eight in-depth interviews, and among team medical staff after seven interviews.

Data analysis

The evaluation data were transcribed and analysed using thematic analysis methods. ¹⁹ The focus was on identifying opinions and judgements about system use and detecting whether any features prevented task completion, caused delays, confusion or generated suggestions for improvements. Meaning units were defined as sentences containing aspects of relevance to usability and the aim of the study through their content and context. The meaning units were then coded by assigning them one or two keywords. Manifest interviewee statements (using the direct meaning without interpretation) as well as latent interpretations of statements (trying to understand to intended meaning of what was stated) about factors related to system usability were used. The codes could be abstract or concrete and were used to facilitate understanding and to compare meaning units in the different system evaluation aspects that were used as categories. The categorization was first analysed by two researchers (D. K. and T. T.). Discrepancies were debated until agreement on what categories to use was reached. The categories were then organized (by D. K.) into a synthesized structure and this structure was then presented to the entire research group. In addition, the original data set was made available to the individual research group members, who were asked to make notes on suggestions for adjustments. In the final step, an online discussion was continued until consensus on the categories and category descriptions was reached.

Results

Athletes showed proficiency when completing the EDC forms for collection of data on pre-participation injury and illness but reported issues with motivation and difficulties with interpreting the survey questions and terminology. The physicians initially faced problems when navigating the multi-layered forms for injury and illness reporting. After a short learning period, they deemed completing the forms fairly easy and were generally of the opinion that EDC systems would be adequate for collecting data during Athletics championships.

EDC on athlete pre-participation health

The athletes completed the form without any human–computer interaction errors. However, the survey design posed problems for the athletes. For example, regarding the question ‘How much fatigue have you experienced during the last month?’ the athletes explained that they could have experienced psychological fatigue, but almost no physical fatigue or vice versa. Reporting the mean time spent on training was also found to be difficult because the athlete did not count time spent on training in weeks (but in days), which required re-calculation to get the average training time. But even those athletes who could account for hours per week also encountered problems. Some participants suggested that it would be easier to just report the time spent training for specified weeks without having to calculate the mean over a longer period. Another problem was knowing what counted as training hours. One participant said that athletes basically train 24 h a day. It was also believed that if the survey also accounted for psychological issues, then mental training should also be included in their answers. When the athletes entered data on health issues, they communicated a good understanding of previous or current injuries and illnesses, that is, the location and type of injury. The main problem with the health reporting was understanding the medical terminology used in the electronic form. Most of the athletes were of the opinion that the terminology needed to be simpler or that explanatory information was necessary. Thus, the athletes found the questionnaire and the formulation of the questions more challenging than the EDC form per se.

The main overall perception regarding the usability of the EDC system communicated by the athletes concerned their motivation to provide injury and illness data in association with championships. Even though the electronic form was found to be easy to complete, the athletes stated that they might not complete it anyway because they wanted to avoid reflecting on injuries and illnesses that might affect their performance during competition. Therefore, if they were not mandated to complete a survey before a competition, they would most likely ignore it. Some participants communicated that a positive motivational factor could be to recognise a tangible benefit for themselves. The athletes also reported that they would have preferred to fill in the form after competition.

EDC on injury and illness from team physicians

When first using the EDC system, the physicians reported concerns about navigating between the forms in the injury and illness recording system. All participants were hesitant at some point on how to continue the report. The prototype EDC system had more than 10 functions for recording injury and illness data, which meant that physicians were required to make many choices when completing tasks. For example, to initiate the report, four options were available, and to continue to the next step, it was necessary to click at ‘Save and assign injury category’. All participants reported insecurity on what button to use, and one participant also clicked ‘Save and finish’. The main reason communicated for this confusion was that the physicians were not familiar with the human–computer interaction terms and phrases used in the interface. It was thought that non-technical or even graphical instructions adapted to the busy context of Athletics championships would make the buttons more understandable.

For a few physicians, the navigation problems led to errors that stopped them from completing the task: that is, they clicked the wrong button and became confused about how to continue. The mechanism in these cases was clicking the wrong button while confused about how to continue to next step. However, when the physicians had received guidance and completed the task, the majority of them perceived that finishing the form had been ‘fairly easy’. They reported that it was similar to the systems they usually work with in their home country. But it was also thought that too much time between reporting would make it difficult to complete due to lack of guidance in the system. Thus, contrary to the athletes, the EDC format posed more problems for the medical teams than the questionnaire itself and the terminology used.

The main overall perception communicated by the physicians was that online EDC systems would increase the efficiency of data collection by shortening the time needed for entering data. EDC was also expected to increase accessibility because paper forms sometimes were not available when an injury or illness needed to be reported. One experienced participant explained that paper forms were often unavailable at the end of championships, which could cause under-reporting of injuries and illnesses.

It was believed that an online reporting system would encourage team physicians to report injury and illness data after the championships. It was also explained that an electronic system, if available on a personal device, that is, computer, tablet or smartphone, would be beneficial. One participant suggested that a computer with Internet connection made available in the warm-up area would improve the procedure of data collection during Athletics championships. However, concerns about Internet connectivity were communicated from the participants. It was pointed out that during championships, especially if it is outdoors, Internet access is often limited and it was believed that the EDC system also needed to be available offline.

Discussion

This study set out to investigate factors critical for the usability of EDC on pre-participation health and newly acquired injury and illness in association with championships in individual sports. We observed major differences between the main user groups. Athletes reported concerns about their motivation to reflect on injury and illness before competitions, query formulations, and the terminologies used. The physicians, on the other hand, experienced problems when navigating complicated forms for recording data on injury and illness. The physicians did, however, eventually recognise EDC systems as an adequate solution for data collection during championships.

Athlete motivation was thus found to be a critical factor for success in the collection of pre-participation injury and illness data. Contemporary theories of motivation assume that people initiate and persist at behaviours to the extent that they believe the behaviours will lead to some desired outcomes or goals, for example, when they can identify a gain in personal health and well-being. ²⁰ In this study, the athletes pointed out that they would be willing to complete the electronic form if they could identify a benefit for themselves, for example, feedback on their relative health status. Another possibility communicated by the athletes was that the coaches and medical teams demand all their athletes to complete the pre-participation forms. In other studies where athletes have been asked to provide data for registration of overuse conditions, ¹¹ the procedure has been to first approach team coaches and ask about interest in participating in the study. Only if the coaches consent, the athletes are asked about participation. A similar approach can be tried during championships in individual sports, that is, approaching team representatives first and asking them to communicate the purposes and procedures of the study to the athletes. Nonetheless, further investigations are needed to understand how the athletes can be motivated to complete pre-participation forms, and both intrinsic and extrinsic motivation should be investigated. The second main issue associated with reporting pre-participation injury and illness data identified by athletes was the construction of the query and the terminology. These problems most likely had their origin in translations of controlled scientific notions into lay expressions, and could be at least partially addressed technically by providing online explanations. ²¹ Similarly, the athletes found that calculating mean training hours was difficult because the hours differed considerably between weeks, especially before competitions. They thought it would be easier to report factual training hours for defined periods of time. Such reporting could be supported technically using a drop-down menu for each period in question.

With regard to EDC on injury and illness from physicians and medical teams, we found that the data-entering task required a complex human–computer interface design, involving several levels, which made the task of entering data without previous training difficult. None of the team physicians managed to complete their tasks without receiving help and instructions. It has been shown that an EDC questionnaire that is initially too difficult to use may discourage responses and lower the quality of the data. ²² These observations suggest that the challenge of allowing medical teams to report data easily and accurately needs to be handled carefully at some level in the overall design of the EDC system. Measures can be taken either at the level of championships organisation, by scheduling instructional sessions on system use for team physicians before competitions, or at the human–computer interaction level, by ensuring that the EDC system is sufficiently self-evident in its design to allow use without separate educational interventions.

This study has both limitations and strengths that need to be considered. It addressed usability issues, and does not provide complete information for the design of an EDC system for use during championships in individual sports. However, it highlights general information system development issues that are important to consider before initiating a large-scale EDC effort. The motivational concerns among athletes and the human–computer interaction issues encountered by physicians are two examples of this. However, it is also important to acknowledge that collecting data through an EDC system is just one step in a complex workflow. Moreover, an assumption was made in the study design that the possibilities for personal on-site training on system use for athletes and team medical staff were limited. Individual training sessions on system use would probably have prevented several of the human–computer interaction issues. Although a qualitative approach based on usability tests and interviews was used, with the purpose of eliciting rich accounts of the experiences of athletes and medical teams, some important aspects and areas may have been omitted in the data. However, the similarity of the accounts collected from observed behaviours and interviews suggests that the findings can be regarded as trustworthy.

Conclusion

In this study of features critical to the usability of EDC systems for gathering data on pre-participation health and newly acquired injury and illness in association with major sports championships, we found major differences between the main user groups. Athletes’ motivation to provide pre-participation data before entering the competition and the team physicians’ ability to rapidly learn the human–computer interface design were key issues. We contend that a successful implementation of EDC during major sports championships is likely to require parallel adjustment of administrative and organisational processes and reallocation of resources. The results of this study can be used as a basis for implementation and evaluation of prototype systems at future championships.