Identifying the characteristics,constraints and enablers to creating value in applied performance analysis: A commentary

Introduction

The use of Performance Analysis (henceforth PA) in elite team sports has increased in recent years with a rising number of analysts employed and the use of statistics/video to support player and team development commonplace across different standards of participation.^1,2 Alongside the applied growth of PA, the amount of research investigating different aspects of PA has escalated. ³ Historically, research in PA has principally focused on the identification of the key performance indicators that are associated with success; however, more recently, researchers have made significant and varied attempts to understand the process of PA ⁴ and the role/responsibilities of the performance analyst. ⁵ These attempts have culminated in research that has investigated the sociological aspects of PA ⁶ as well as advancements into the broader aspects of the applied use of PA. ⁷ Such research has strived to reflect the complexity of sporting performance and the increasing use of sport science to aid coaches’ decision-making and develop team/player performance.

The recent paper by Martin et al. looks to build on research which proposed a framework for professional practice in applied PA ⁷ and is welcome in its attempt to further knowledge on the increasing employment of the performance analyst in applied sports environments. There are several interesting findings for both researchers and applied practitioners including a discussion on the poor professional infrastructure and lack of role clarity for analysts. Such results highlight aspects of the role of the analyst which have received limited attention and emphasise the relatively undocumented and somewhat adverse elements of PA present in sporting environments. However, the paper also sparks several areas for critique and various questions for contemplation.

Critical realism

Martin et al. highlight they utilised a critical realist research philosophy to underpin the research project; however, limited information is provided as to how the researchers interpreted and used critical realism within the research project. Time is spent discussing the method of data collection and analysis, yet the broader methodology is given only a token mention. Although the researchers were limited by word count, calls for detail to be provided in articles on the ontological/epistemological understanding of the underpinning research philosophy used within a research project are growing. ⁸ As such, further discussing critical realism would allow the method of data collection and analysis, to be rationalised and critiqued considering the underpinning research philosophy.

Within PA, research has historically used a reductionist approach which has focused on establishing the relationship between specific performance variables and outcomes. ⁹ Such an approach views behaviour as quantifiable and measurable, resulting from an unambiguous trigger or source. ¹⁰ However, sport is a social phenomenon and the performance, role and development of individuals, such as performance analysts, cannot be understood through a simple cause-and-effect approach. ¹¹ As such the use of critical realism, which considers the world by drawing upon both social theory and seeking explanations for social phenomena, as well as using analytical tools to aid data collection and data analysis, ¹² to underpin the Martin et al. paper appears promising. However, according, to Scott (p. 9) ¹³ ‘though the philosophy of critical realism is well developed, its application to the collection and analysis of data at an empirical level is manifestly under-developed’. Consequently, further detail needs to be provided by Martin et al. on how they used critical realism to underpin the method of data collection and analysis used in this project. After all, if a theory is deemed of value enough to underpin a research project then its use should be explored, rationalised and critiqued, rather than merely touched upon. Furthermore, despite critical realism's apparent suitability for the research project, warning signs have been sounded about researchers’ unredacted use of a theory which has been highlighted as conflating the philosophy of science with sociological theories. ¹⁴ As such, research projects underpinned by critical realism have been emphasised as requiring the rigorous analysis of a limited number of cases, coupled with the systematic analysis of the interplay of ontological layers. ¹⁵ This has resulted in it being common for research underpinned by a critical realist theory to be small-scale ethnographic studies ¹⁶ or adopt a case study design. ¹⁷ Such projects tend to use various data collection methods such as interviews, observations, field notes and document analysis. Therefore, it would have been interesting to understand more explicitly how critical realism underpinned the data collection method of focus groups and an online survey, particularly given that critical realist interviews should be both theory-driven ¹⁸ and constructivist. ¹⁹ As such the question needs to be asked of how these principles were applied to the focus group data collection, particularly given the diversity of the research participants working in different environments with different roles and responsibilities.

Martin et al. explain that they analysed their data through reflective thematic analysis which resulted in themes linked to; the why, what and how of professional practice as well as the broader issues and constraints which may have impacted professional practice. However, it is unclear how the fundamental tenants of data analysis in a critical-realist paradigm; developing an understanding of what works, for whom and in what circumstance ²⁰ underpinned the data analysis process followed. Furthermore, it is unclear how the data analysis process was theory-driven and utilised abduction to identify the underlying causal mechanisms which explained the experiences of the research participants. Frequently the method of data analysis used in a project underpinned by a critical realist philosophy is tailored to the project that has taken place, for example, the six-stage process of analysis used by Stutchbury ¹² highlights the nuance of the data analysis in a project underpinned by critical realism. Consequently, it is not anticipated that the six stages of analysis from Stutchbury ¹² should/could be used as a framework for data analysis for any other research project, certainly not in the same manner as Braun and Clarke's ²¹ six stages of reflective thematic analysis are frequently used as a framework for qualitative data analysis. As such, it is significant to note that Martin et al. choose to use the more formulaic reflective thematic analysis from Braun and Clarke, rather than create their nuanced analysis framework structured around a critical realist research philosophy.

The context of performance analysis

In addition to the critique of the Martin et al. paper which has focused on the use of critical realism is a query relating to the success by which the findings can be (or should be) contextualised across the multitude of sporting environments where PA is used, and analysts are employed. The importance of taking into account the context in which PA takes place has been highlighted in criticism of the ‘one-size-fits-all’ models of the process of PA.^22,23 These models fail to consider the separate effects of the individual personnel concerned with PA and the dynamic/unpredictable nature of sporting performance. ²⁴ Additionally, inadequate attention is given by such models to the environment in which PA takes place ⁷ and the specific socio-political influences on PA, such as individual self-interests and an organisation's priorities,^25,26 despite the impact these can have on PA. Similar questions must be asked about Martin et al.'s proposed delineation of the roles which fall under the discipline of PA with a particular critique of ‘Role 1’. ‘Role 1’ is categorised as ‘Sports Performance Focused Role Applied PA’ and includes individuals with roles focused on collecting and analysing data who are embedded within a performance ecosystem. The researchers acknowledge the challenges with attempting to break down the different, often overlapping roles within PA and highlight the diversity of practitioners/roles that fall within ‘Role 1’. Yet when they suggest that a conceptual model of the profession of PA must include the range of practitioners from novice to expert and, therefore, potentially include the range of roles from intern analyst to departmental lead it is unclear how this could be achieved with enough detail to be of value or enough ambiguity to be generalisable.

Discussions on role definition are taking place across various sporting roles with calls for clarity to be made on the role of the Sporting Director, the Sport Psychologist and the Sports Coach. However, it is unclear how such conceptual clarity could provide enough flex for the individual circumstance in which analysts work, particularly as PA is increasingly taking place across different sports, contexts and participation levels. Would it, therefore, be possible or even desirable to attempt to create a purpose for each role as posited by the researchers? Would it be possible for example to create conceptual clarity, regulation and purpose for an applied analyst working full time in the English Premier League as part of a team of performance analysts and an individual working part time in academy football with various age groups, teams and coaches? Or would such descriptors likely fail to offer the detail needed to be of use by both those employed as analysts and those employing analysts? Perhaps it is not the intention of the researchers to progress towards such one-size-fits-all descriptors and certainly more research is required into the role of the analyst. Yet as an increasing amount of analysis takes place it can be argued further understanding needs to be developed of what works for a specific analyst in a certain context. This echoes recent advancements that have proposed the importance of context in the role of the sports coach, the delivery of coach education and the evolving engagement of coaches with sport science. As such, perhaps the challenge of a lack of role clarity for analysts identified by Martin et al. could be addressed through the construction of detailed roles and responsibilities of all individuals involved in PA by those who are involved, for example, analyst, coach and player. As such, a context-based process of PA, which takes into account the different influences (contextual, structural and social including macro-, meso- and micro-contexts), there are on those involved in PA within a specific club could be developed meaning each analyst has nuanced role bespoke to the context in which they work.

Conclusion

This commentary has sought to highlight the need for researchers who use a critical-realist methodology to emphasise how it influences the data collection and analysis process used. Without such detail, it becomes unclear how a research project can claim to have a critical-realist theoretical underpinning. Furthermore, the commentary has sought to question the way in which the findings can be contextualised to the wide range of performance analysts employed in sporting environments. Such critique of the Martin et al. paper is notwithstanding the value that can be seen in the contribution the paper makes to advancing knowledge of the role of the performance analyst. Furthermore, given that the authors acknowledge that their paper is part of a wider project, it would be interesting to note if some of the queries highlighted here are addressed if the project is viewed as a whole rather than through the lens of this one paper. Finally, this commentary has only managed to delve in a limited manner into the complex debate regarding the philosophical and scientific merits of critical realism. ¹⁴ Despite the increasing popularity of critical realism in sport research, this nuanced debate should be further engaged in and acknowledged by researchers who use the methodology within their research.