Thinking big – here comes the sun

Abstract

In 2000, Pincott and Branthwaite published ‘Nothing New Under the Sun?’ which discussed the role of the internet in research both in terms of the hazards and opportunities this confers. Specifically, Pincott and Branthwaite argued that whilst the advent of the internet offers up new approaches and techniques, this must not be at the detriment of methodological rigour and standards we have grown accustomed to in the industry. In this paper, it is argued that the advent of big data, big qual and the emerging potential of the metaverse present new opportunities in research but pose the same questions originally conceived by Pincott & Branthwaite, 2000. Moreover, methodological rigour and standards are framed by three interconnected themes, namely, the role of data science and big data, the importance of softer skills of interpretation and narration and finally, the role of theory in elevating and grounding research.

Looking back to make sense of the future

Market researchers are storied in the art of pontificating about the challenges facing the industry. Indeed, if we look back at The International Journal of Market Research alone, in the last 2 years there have been no fewer than five articles that do exactly that (Mytton et al., 2021; Nunan, 2021; Poynter, 2021; Yallop et al., 2022). Such an occupation is not without its merits, ruminating helps the industry develop new skills and tilt to the challenges of the day. In 2000, Pincott and Branthwaite published ‘Nothing New Under the Sun?’ which discussed the evolution of technology and the hazards and opportunities this confers. As alluded to in the title of the paper, the authors argued that whilst the advent of the internet offers up new approaches and techniques, this must not be at the detriment of methodological rigour and exacting standards we have grown accustomed to in the industry. It is argued that the advent of big data, big qual and the emerging potential of the metaverse present new opportunities in research but pose the same questions originally conceived by Pincott & Branthwaite, 2000. In the discussion below, methodological rigour and standards are framed by three interconnected themes, namely, the role of data science and big data, the importance of softer skills of interpretation and narration and finally, the role of theory in elevating and grounding research.

Data science as curator

Fast forward to the present day and big data has been transformative, bringing about data and modelling efficiencies that would have been unimaginable in 2000. Big data is about bringing together data to form a holistic customer view. It is typically automatically collected and mostly passive in nature, examples being, activity on a brand’s website or measurement of media consumption where data is combined to establish a customer centric view. To maximise the big data opportunity, AI and Machine Learning programmes are applied to the data. Typically, these programmes seek to predict consumer behaviour; organisations use these models to predict what brands consumers identify with, what products they buy, what services they use and so on. Big data enables businesses to identify efficiencies, improve sales, and reduce churn. As such, the case for big data is irrefutable. However, whether big data has delivered against the promise of business value is debateable. This apparent shortcoming is attributed to data quality and a lack of resources to interrogate the data. There is much merit in this argument, inevitably greater resources will improve the quality of data and subsequent modelling, albeit there is disconnect in what constitutes data quality in data science and market research. For instance, in data science there is the suggestion that more data results in a proportionate increase in accuracy (Bosch, 2016). In market research we seek to make generalisable statements based on statistical robustness, where more data results in diminishing returns; this misconception might go some way to explain the heightened expectations of big data. There is also the issue of missing data, for example, if sociodemographic data is missing, it will be difficult to make observations about the profile of customers. There are issues of representativeness; if we take data from social media, we are essentially taking snippets of comments and behaviours from a social media site, inevitably this will result in sample bias where the silent majority are conspicuous by their absence. In marketing science, the phrase ‘garbage in, garbage out’ has come to personify the challenge of poor data quality and questionable modelling outcomes. Perhaps, within the context of this discussion, a more pertinent phrase is ‘garbage in, gospel out’, where it is acknowledged that with poor data, the value of modelling can be overstated. In qualifying this discussion, the apparent disparity in what constitutes data quality and the impact on modelling therein should not be an impediment to progress but rather, underlie the importance of ensuring data scientists and research practitioners work closely together in adhering to the principles of generalisability and representativeness.

Interpretation in a big data world

Big data is dumb without interpretation and narration. If we take an example of a big data project to help improve sales for an engineering company, the company models both transactional and operational data to predict churn at an overall and individual account level. Important though this data is as a baseline, the model outcomes need to be set within the context of on the ground knowledge about individual accounts, wider knowledge of the market, challenges facing customers, price sensitivity, service expectations, and so on. This knowledge will be instrumental in not only deciding what questions to ask of the data in the first place, but in giving context and perspective to the model outcomes. In other words, applying the research skills of interpretation and narration to big data will result in a more nuanced understanding of churn that will help the engineering company develop an effective strategy to improve customer retention.

A similar argument can be made regarding big qual, and as the name implies, this refers to qualitative research at scale. Although what constitutes scale is debateable, we are looking in the magnitude of 100 participants or more (Brower et al., 2019). Both video and voice are analysed as a composite, akin to big data but looking at story arcs rather than data patterns. The process starts with text and sentiment analysis programmes that identify emergent themes which ‘dig shallow’ pits to find areas that merit ‘digging deeper’. In-depth analysis/traditional qualitative analysis is then utilised to understand the qualitative narrative. Prima facia, this is a logical way of managing qualitative data at scale that would otherwise not be possible without significant research resources. There are inevitably issues with this, ergo, when using text and sentiment analysis for the shallow dig, it is easy to miss or under emphasise abstract concepts and nuances in the data. To mitigate against this, the outcomes of the text and sentiment analysis should not be taken at face value but viewed within the lens of softer, interpretive research skills.

As noted, with data we seek to make generalisable statements based on statistical robustness; in qualitative research we use purposeful sampling and seek to establish transferability based on a detailed understanding of behaviour (Peshkin, 2001). Notwithstanding this obvious distinction, it is suggested that with time, the principles of generalisability will logically be applied to big qual (Maxwell & Chmiel, 2014). However, this will challenge the very notion of what constitutes qualitative research, not least there is the question about the nature of reality, qualitative researchers assume multiple realities and perspectives, whilst coding qualitative data assumes one underlying reality (Brower et al., 2019).

No discussion about big data would be replete without contemplating the nascent opportunity of the metaverse. Once considered the preserve of science fiction, the metaverse is beginning to take shape, albeit largely within the domain of gaming where global platforms such as, Roblox, Fortnite and Minecraft bring players together in a shared player environment. One could be forgiven for asking, “what is the metaverse?” There is no exact definition, although the metaverse could be described as the internet but in 3D. It harnesses the power of blockchain technology and social media to create rich and immersive environments - where people can interact in virtual and augmented realities - in digitally interconnected communities. Notwithstanding the hyperbole, the metaverse will result in a new frontier in research that will be an important addition to the research tool kit. In the wake of the metaverse, big data and modelling will be important. However, the metaverse will be much more than simply a passive data pipeline, researchers will create virtual environments to test new products and services, user avatars will navigate environments and respond to simulations using real time rendering. Interoperable networks will allow users to carry their identity from one virtual world to another. Researchers will observe how users socially interact with each other, and brands will be able to walk in the shoes of their customers by adopting customer avatars. Whether active or passive data, researchers will need to make sense of this new virtual world; it is argued that this necessitates the soft skills of interpretation and narration.

Research bounded by theory

Making sense of human behaviour is at best challenging. Indeed, we can all attest to the often complex and puzzling nature of human behaviour. Although there is no panacea to this problem, drawing upon social science theory about human behaviour offers one possible solution. At its simplest, theory gives meaning to observed patterns of behaviour. As we know, theory operates as a primer, where researchers can formulate hypotheses to test against data. Theory provides guidance on the relationship between different influencers of behaviour. Reviewing data through the lens of theory helps give saliency and perspective, rather than existing in isolation. Theory allows us to not only make sense of what data we have, but to make clear what is missing and address possible data inconsistencies. As such, it can help focus finite resources on data and allow researchers to make informed decisions about the generalisability of data, which in turn will inform strategies and action plans to take to the business. Theory is written in academic language for academics and thus can feel unwieldy and opaque. Consequently, theory tends to be applied selectively, inappropriately, or not at all; there are exceptions to the rule, where theory has been embraced by the industry. A case in point is behavioural economics, which suggests that human decision making is an imperfect science and that individuals can be ‘nudged’ to make better decisions. The theory has transcended its academic origins and emerged as the zeitgeist of contemporary economics policy and research. The success of behavioural economics is likely a product of its simplicity and internal logic. Further, heuristics, framing, loss aversion, and mental accounting offer a practical tool kit in which to foster behavioural change. Testament to the success of behavioural economics is the emergence of nudge units/behavioural science teams in government and business alike. Moreover, there are many examples of the application of the theory to practical problems, whether that be tax compliance or making smart investment decisions; this goes some way to counter the claim that ‘theory is for theory’s sake’. Arguably, theory is a useful tool in which to make sense of passive and active data. In the metaverse, users will be able to move in and out of virtual communities and socialise with other users. Therefore, it will be important to draw upon theory that looks not only at the mind and behaviour but also at the role of social structures and how that influences behaviour. To put it another way, behaviour in the metaverse is recursive, whereby, social structures within virtual communities will influence user behaviour and users will enact change in the virtual communities they inhabit.

In conclusion

In short, the central tenant of Pincott and Branthwaite’s paper holds as true today as it did back in 2000. That is, technological innovation offers up new approaches and techniques but this must not be at the detriment of methodological rigour and exacting standards. Whilst the issues discussed in 2000 centred on the internet and data collection, in today’s big data world, data is drawn from many different sources and is considerably more complex in nature.

Technological innovation in big data and big qual have challenged the very notion of what constitutes ‘the art of possible’ when analysing large quantitative and qualitative data sets. That said, research practitioners should be mindful of the shortcomings of such innovations, not least regarding methodological rigour and the need for interpretation and narration.

The virtue of grounding research in theory has been presented, where it is suggested that theory that is accessible and underpinned by practical application will help researchers to make sense of the complex nature of human behaviour.

Amongst industry leaders, the vernacular of the day is the democratisation of research, whilst the rhetoric is compelling, the industry must adapt to support this changed reality. Indeed, research is increasingly being conducted by individuals that are not research practitioners, but part of a wider user ecosystem of marketeers, sales agents, operational staff and so on. This offers up an opportunity for research practitioners to act as ‘experts’ to non-researchers, by promoting methodological principles and theory in a manner that is accessible and practical to the uninitiated.