Discriminating capacities of physical performance testing in male youth soccer academy programmes

Abstract

Objective assessment and monitoring of physical performance in soccer players are mandated features of practice in English soccer academies. Based on these approaches, our interest was to investigate and compare the longitudinal physical performance profiles of those players that had eventually been successful and unsuccessful at the two highest levels, becoming male professional English Premier League and Championship level soccer players. The specific research objective was to investigate the between player variability and predictive validity of physical assessment measures against eventual playing outcome. A total of 297 male youth participants (born between 1999 to 2006) affiliated to three category one academies were tracked for eight years across six objective physical tests. Data were analysed by a series of mixed method ANOVAs, comparing outcome with age. Canonical Discriminant Function Coefficients and the TRIPOD Statement were applied for transparent reporting of prediction efficacy. Results suggest that physical characteristics offer little or no discriminatory power. We conclude by offering guidance for academies’ interpretation and usage of physical data.

Introduction

Across sports, talent development is well acknowledged to be a multifaceted and complex phenomenon, requiring integrated, holistic and systematic development.^1,2 Supportive of these developmental needs, research interest in the notion of the Talent Development Environments (TDEs) ³ has grown in recent years, yet there remains little evidence that informs the design of tracking and monitoring measures in talent systems. Research in male soccer has previously suggested key characteristics such as skill, ⁴ sprinting and agility ⁵ and psychological resilience. ⁶ Work in other Olympic sports (e.g., skiing, wrestling, swimming, ice hockey, speed skating, track and field) have highlighted mental toughness (7) and within other talent development systems, creativity, spontaneity, flexibility and commitment (8) for performers selected into TDEs. These suggested characteristics potentially offering early discrimination between performers^7,8 and informing measures and processes in talent systems. ⁹

Within male youth soccer, the Lewis ¹⁰ review into player development in English professional soccer led the English Premier League (EPL) to invest £1.94 billion in youth academies and implement the Elite Player Performance Plan (EPPP). A key aspect included the creation of a measurement system designed to track and monitor players progress, ¹¹ thus operationalising the profiling and tracking of player development. Most recently, research has highlighted various elements across technical, tactical, physiological and psychological factors that influence this complex and multidimensional talent development process.^12–14 Yet, despite studies considering these multidimensional factors being focussed during shorter time periods,^8,15,16 there appears to be limited data that captures youth players’ longitudinal progression or offers potential discrimination and prediction of ultimate outcome.^4,17,18 This begs the question; in male youth academy soccer, what factors are discriminatory and hence potentially most predictive of future progress. ¹⁹

Although viewed as a mostly reliable tool for assessing and profiling soccer players’ current physical competency,^20–22 there is limited evidence that physical measures mandated by EPPP offer validity in discriminating between the future potential of youth players. This raises questions as to the real-world utility or potential pitfalls of implementing such methods for elite youth football. ²³

Reflecting these ideas, previous research across sporting domains has considered approaches to physical profiling, measuring, benchmarking and integration within player development structures.^24–27 Despite research showing field-based fitness tests to be reliable measures of physical performance monitoring in youth soccer players,^22,28 research suggests the limitations of isolated assessment measures in discriminating between male youth players.^25,29

In practice, seeking means of conceptualising differences between factors that are genuinely discriminatory of future performance, Hoyes and Collins ³⁰ distinguished between two categories of characteristics. The first; ‘hygiene factors’ were identified as protecting athlete wellbeing and the amelioration of developmental risk factors but only being required to reach a set standard after which further development had no positive impact. The second category, ‘performance factors’, were conceptualised as being directly related to the performer's ability to perform. In other words, more ‘performance factor’ skills were better.

Physical data collected by academies are clearly important, directly impacting players’ careers (retain, release or professional contract). ²⁴ If the highly promoted and operationalised collection of physical data are used for the purpose of discriminating between male youth players and predicting future long-term potential, however, there is a need to understand the validity of these measures for this purpose. Consequently, the aim of this study was to critically consider data from mandated physical testing measures ¹¹ that assess and track the progression of a group of male youth academy players. Specifically, we were interested in the cross-sectional and longitudinal profiles of those that had eventually been successful and unsuccessful in becoming professional players at EPL and Championship level. The specific research objective was to investigate the between player variability and predictive validity of physical measures. The objective was considered on the basis of what factors were most predictive and discriminatory across the eventual playing outcomes for the sample. To specifically examine the predictive validity of these measures, we employed the TRIPOD (2015) methodology across the study.

Methods

This study tracked the longitudinal physical progression of male youth academy players at three EPPP category one academies. As is common practice within English youth academies, physical characteristics were measured using standardised EPPP procedures.

Participants

Following ethical approval from the University of Edinburgh ethics committee, a longitudinal design was used. Written club and coach consent were gained alongside comprehensive written and oral explanation with club representatives. Data (already collected and stored by academies) were collected on 297 male youth academy players aged under 9 - under 16 (U9-U16) (physical tests) with years of birth ranging from 1999 to 2006. Participants were affiliated to three EPPP category one academies (20% of all the highest category academies at the time of data collection) and tracked across six physical tests. Players were categorised by their subsequent career progression (as confirmed by each club), with specific examination applied to the two extremes of outcome; namely, English Premier League (EPL) (n = 15) and Championship (n = 67) against Released (n = 215).

Procedures

As part of their normal club criteria, participants completed the prescribed standardised physical fitness testing battery three times per year (circa July, December, and May; age 9–16 years (U9-U16) as mandated by the EPPP. Scores were collected objectively for all players against their completion of each physical test. If a player did not undertake a test (i.e., due to illness, injury, or release), their score was not reflected in the data.

Measures

Participants completed the prescribed standardised physical tests as part of their normal club criteria. Data were collected using the EPPP standardised physical fitness testing battery ¹¹ including (a) Counter movement jump (CMJ) (b) 5 m,10 m,20 m speed and (c) asymmetric agility (505L,505R). As required, and to ensure standardisation of physical testing, each academy tested in the same environment (indoor 3G surface) at the same time and administered as a one-off session. The parameters of the test used were consistent, providing a summary consistent statistic related to the same individual subject, thus enabling the potential for reliability of measures. The nationally recognised measures purport to accurately and reliably track progress at consistent time periods. Physical data for all players were listed with accompanying performance scores correlating with their completion of each test. CMJ data were collected from two clubs using Opto Jump System (Micro Gate, NY,) ³¹ and one club using Bosco Ergo Jump System (Barcelona, Spain). ³² Data were recorded to the nearest 0.01 m on both systems. Both tests are considered a valid and reliable method for assessing anaerobic power showing strong concurrent validity with force plates, ³³ particularly for jump height and ground contact time suggesting it is a valid tool for these parameters. ³⁴ However, some differences exist, especially in power and velocity measurements.^34,35 In conclusion, both systems were used alternately.^36,37 Linear speed and acceleration were measured over a 5 m/10 m/20 m distance. Data were collected via the Brower TC Timing System (Brower Timing Systems, Draper, UT) and recorded to the nearest 0.01 s. This measuring tool is considered reliable and accurate for measuring sprint times to assess youth academy level players. However, some studies have found limitations in their validity when compared to laboratory-based systems.^38,39 Starting points were standardised with participants starting in a standing start 0.3 m before the first infrared photoelectric gate. ⁴⁰ Data were read and manually recorded. Change of direction (CoD) were assessed using the agility 505 ⁴¹ test. Data were recorded using Brower TC Timing System (Brower Timing Systems, Draper, UT) with scores collated to the nearest 0.01 s. The 5-0-5 CoD test uses one set of timing gates to assess an individual's ability to sprint 5 m, perform a 180-degree CoD and sprint 5 m back through the timing gates. ⁴² Across each physical test data were read and manually recorded. Verbal encouragement was offered to players and consistent for each test and across all clubs (39).

Statistical analysis

Descriptive data from the data set were transferred into a longitudinal line chart, which tracked the estimated marginal mean scores for each test. Outcome categories presented EPL/ Championship and Released (the two extremes) for physical tests. These outcomes were selected to present a clear picture on each graph between players making it at the highest level of the game (i.e., EPL/Championship) or not (i.e., Released).

Statistical Package for the Social Sciences (SPSS v26) was used to analyse the data. Mauchly's Test of Sphericity was used to test for assumptions of sphericity and, if significant, Greenhouse-Geisser adjustments were applied to degrees of freedom within subject effects. Data sets were analysed by a series of mixed method ANOVAs, comparing Outcome as a between subject factor with Age as a repeated measure. The number of levels varied depending on the data available. However, against the objectives of the study, we focused on comparing the two extremes; namely, players who made appearances in the EPL and Championship and those who were released from the academy process. Where necessary, post-hoc tests were conducted using Tukey tests.

Finally, to offer a test of the predictive validity, we applied the TRIPOD (Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis – 2015) guidelines. Developed for use in clinical settings, this offers a checklist of procedures which should be followed and reported. All aspects of these recommendations were applied. As per the statistical recommendations, we completed a Canonical Discriminant Function analysis on the physical performance scores of the two groups (EPL/Champ versus released) at the U16 stage. This is the age group at which players are identified for subsequent contracts (known as scholar status) or released.

Results

Table 1 shows the outcomes of the mixed model ANOVAs on specific groupings of physical tests.

Table 1.

Outcomes of physical tests.

Test	Age	Age * Outcome	Outcome
CMJ	F(7364) = 40.6***	F(7364) = 0.89 ns	F(1,52) = 0.01 ns
5m	F(7378) = 15.9***	F(7378) = 0.55 ns	F(1,54) = 0.36 ns
10m	F(7, 364) = 62.2***	F(7, 364) = 0.61 ns	F(1,52) = 0.14 ns
20m	F(7378) = 91.7***	F(7378) = 0.78 ns	F(1,54) = 0.35 ns
505L	F(7350) = 42.2 ***	F(7350) = 1.06 ns	F(1,50) = 0.37 ns
505R	F(7, 371) = 38.0***	F(7371) = 1.1 ns	F(1,53) = 0.09 ns

*signifies p < .05

**signifies p < .01

***signifies p < .001

Figures 1(a) to 1(f) show estimated marginal mean scores for the two outcome groups across ages for each test.

Figure 1.

(a) countermovement jumps (CMJ), (b) 5m, (c) 10m, (d) 20m, (e) 505L, (f) 505R.

The results of the prescribed physical tests show that unsurprisingly all physical tests demonstrate significant improvement with age (see table 1). Relevant to our purpose, however, all Age * outcome interactions and main effects of outcome were non-significant. There were no significant differences between EPL and Championship against Released players based upon the physical parameters at each age group (please refer to Figure 1(a) – 1(f)). Put simply, the predictive potential of the physical tests is negligible, as no differences are apparent relating to eventual outcome.

Finally, the discriminant function analysis showed an eigen value of .023 and a weak canonical correlation of .15. The Wilks Lamda of .978 (Chi square (4) = 3.292) was non-significant at .51. The results suggest that the physical tests do not significantly predict eventual outcome in this sample.

Discussion

Based upon the differential approaches to player assessment in place within English youth academies, the aim of this study was to critically consider data from prescribed (physical) measures ¹¹ that assess and track the progression of a group of male youth academy players. Specifically, we were interested in the cross-sectional and longitudinal profiles of those that had eventually been successful and unsuccessful in becoming professional players at EPL and Championship level. The specific research objective was to investigate the between player variability and predictive validity of physical measures. The objective was considered on the basis of what factors were most predictive and discriminatory across the eventual playing outcomes for the sample.

Our results (age*outcome on ANOVA and discriminant function analysis) suggest that the battery of physical measures promoted by the EPPP, lack validity in discriminating between later career outcomes. These findings mirroring research on progression at the youth level. ¹⁹ It may be the case that physical factors are better considered as hygiene factors, essential thresholds that players must meet either to avoid deselection, or as baseline requirements to reach a certain performance level. ³⁰

Of course, this is not to suggest that objective measures of all factors cannot be developed within an academy setting, nor that this wouldn’t be a worthy aim. This is especially the case given that many of the attempts to develop more objective measures of technical and tactical competence seem to lack ecological validity. ⁴³ At a minimum, our results should lead us to consider the epistemological basis for what we seek to measure and by what means we build our assessment framework.

At earlier stages of physical assessment, it could be that these players are benefiting from early biological maturation,^44,45 or other early advantages ⁴⁶ and thus stand out more, perhaps distorting the accuracy of physical efficacy. Future research should consider the relative long-term advantage or disadvantage presented by earlier biological maturation, something evident in research from ice hockey ⁴⁷ and rugby league. ⁴⁸ These studies suggest that populations initially advantaged by advanced biological maturation may experience a later reversal of this advantage, similar to the distinct concept of relative age advantage.^49,50 Whilst further research is necessary, it would seem that the essence of physical advantage seems unlikely when comparing between players, because more mature players would benefit from enhanced physical capacities at the same age. ⁵¹ These concerns notwithstanding, there appears to be a perception that adult driven physical fitness assessments are overly utilised for the purpose of benchmarking and if used incorrectly, might provide an invalid benchmark for effective developmental comparison, ⁹ potentially due to a greater weighting of objectivity. Despite academies spending significant time testing, measuring, and discussing these elements, our results suggest that physical factors seem to lack discriminatory capability thus can be considered hygiene factors. In this regard, there is a need for clubs to be clear on the purpose of testing physical characteristics. In addition, that academies account for the limitations of physical testing ⁵² by contextualising longitudinal changes resulting from confounding factors such as maturation, injury and detraining. ²⁶

In practice, this is not to disregard the role of physical factors which will interact with a player's capacity to develop across other factors. However, we suggest that the framing of hygiene and indeed, potential performance factors present an appropriate developmental weighting for soccer academies, coaches, and players. To achieve this, key performance indicators and individual development planning appear to form the framework from which objective and subjective assessment of performance is interpreted. Using individual player case review, we suggest that physical data is interpreted with formative rather than summative intent, thus enabling appropriate deployment.

Of course, we understand that some would take a contrary position, for example, EPPP promotes physicality as a component, which seems to be face valid whether grounded on an opinion or evidence based or not that these factors are important. Furthermore, the work of Dugdale, Sanders ²⁹ shows that the physical metrics do seem to be associated with who makes it, albeit towards the end of the academy player's career. Clearly this merits further investigation. It would thus be important for any future studies trying to isolate the physical metrics to do these epidemiologically. In other words, to look back on players who made it and where they scored due to any self-fulfilling prophecy or Pygmalion effect. In effect, the players who coaches think are good will be scored higher. Of course, however in this study it is the player's second or final chance, ⁴ therefore they may well be trying harder! Additionally, the study is a new trial and does show some variation. Similar contentions have been made by Kelly, Wilson ⁵³ and contrary to our data, highlighting physical discriminatory function. Likewise, Kelly and Williams ⁵ point to physical characteristics seeming to be more favourable predictors at youth academy level for achieving future senior professional status. From a holistic perspective, Höner, Murr ⁵⁴ highlighted players who progressed had faster sprint times, better agility, superior ball skills, as well as higher coach ratings in technical, tactical, and psychosocial domains. Once again, there is a necessity to consider.

Clearly this work merits more attention. We would have to say from our experience and reflecting the findings of Fuhre, Øygard, ⁵⁵ coaches tend to prefer other performance characteristics over physical in terms of making judgements on players.

Limitations

Clearly, the present study is not without limitations. Outcome categories presented EPL/Championship and Released categories for physical tests reflecting the detail provided by each club. Other outcome categories such as League 1/2 and semi-professional were omitted due to a lack of sufficient data availability across all clubs.

Differences in equipment used across clubs may question the standardisation of physical testing scores. ⁵⁶ Poor data collection practice is difficult to account for, whilst some data may not be represented due to player injury or illness at the time of physical testing. ⁵⁷ Similarly, we need to distinguish between cross sectional ⁵⁸ and longitudinal assessment. ⁵⁹ If a range of heuristics are in operation across academy soccer related to the timing of development, this will clearly impact on coach decision making, selection and development processes. For instance, players released due to later biological maturation may distort the data, as they are denied the chance to progress and are consequently excluded from the dataset. Data do not consider the players biological maturity status which could provide some meaning when comparing test data. Importantly, however, this dataset does not support this explanation as systematic advantage or disadvantage in physical performance would surely add to the discriminant impact of the tests. In other words, better performance of early maturers (or weaker of later developers) would add to rather than lower between outcome groups effects. Furthermore, these should be apparent around the Peak Height Velocity (PHV) stages where such effects would be assumed to be most marked.

Of course, future investigations should check this contention through more detailed data sets. For the moment, however, our findings suggest a negligible impact of fitness data across ages with respect to outcome, excepting the hygiene related ideas supporting injury-free progression mentioned earlier.

The present sample size dataset aims to detect clinically relevant differences ⁶⁰ between male youth players across three academy settings. Even though the number of players is satisfactory, not too small or excessive ⁶⁰ (n = 297), given different approaches to selection, the clubs contributing to the study may not represent the full academy picture. Therefore, the findings cannot be taken as wholly representative of every domestic or international academy setting. In addition, the data presented represent male soccer academies and cannot be considered to be representative of female soccer. ⁶¹ To this end, we ask the reader to judge methodological validity indicators including the assessment of objective scores (i.e., reliable, and valid measurements) and their transferability within the context of male youth players within the youth academy system.

Conclusion

Using a high-level sample of male youth academy soccer players, this study critically considered data from prescribed (physical) measures ¹¹ that assess and track the progression of a group of male youth academy players. Specifically, we were interested in the cross-sectional and longitudinal profiles of those that had eventually been successful and unsuccessful in becoming professional players at EPL and Championship level. The specific research objective was to investigate the between player variability and predictive validity of physical measures. The objective was considered on the basis of what factors were most predictive and discriminatory across the eventual playing outcomes for the sample.

Physical tests did not appear to predict or discriminate later performance trajectory, holding little predictive and discriminatory validity. Physical factors do not appear to be the most valid discriminators and predictors of future ability at the highest level of the game. Therefore, we recommend that academies adjust their weighting based on the apparent distinction between hygiene and potential performance factors.

Supplemental Material

sj-xlsx-1-spo-10.1177_17479541251405477 - Supplemental material for Discriminating capacities of physical performance testing in male youth soccer academy programmes

Supplemental material, sj-xlsx-1-spo-10.1177_17479541251405477 for Discriminating capacities of physical performance testing in male youth soccer academy programmes by Matthew Layton, Dave Collins and Jamie Taylor in International Journal of Sports Science & Coaching