Qualitative Inquiry: A Key Element for Assessing the Social Impact of Research

Abstract

The need to develop adequate methodologies to comprehensively assess the impact of research, especially the social impact of European Union (EU)-funded research, is one of the main concerns within the European Commission as well as for EU citizens, who are more active than ever. This article discusses the rationale behind using a qualitative approach to better address these concerns. Drawing on the FP7 IMPACT-EV research project, the present article discusses how to overcome a positivist approach that evaluates the social impact of research conducted only for its economic objectives and using only quantitative data. The focus on what is needed and what research is expected to bring to society are emphasized and made possible through qualitative inquiry of the social impact of the EU social sciences and the humanities (SSH) research. Thus, the development of qualitative-based analysis of the social impact of research is increasingly required to be conducted in dialogue with citizens.

Introduction

European society is increasingly demanding research that effectively contributes to overcome the great challenges it faces, such as poverty, unemployment, climate change, migration, or the rise of xenophobia. Amid this context, the social usefulness of the social sciences and the humanities (SSH) has been strongly questioned (European Commission [EC], 2013). While in some scientific domains, such as medicine, many European-funded investigations have been linked to subsequent improvements in social conditions, it is less common to find SSH research that achieves positive outcomes beyond the realm of academia. This debate has reached the highest political levels in Europe to the point that the continuity of the SSH as part of the EC’s research funding program Horizon 2020 was questioned within the Commission itself (Flecha et al., 2015). In times “of tighter budgets and more public attention to the effectiveness of public funding and EU-funded research, there is a need to demonstrate the performance, impact and added value of EU programs” (EC, 2015), and in this context, the European SSH was given the opportunity to remain a part of the future EC research funding program, provided they demonstrate and show the large benefits that they bring to society in different ways and domains. Thus, assessing the extent to which their scientific endeavor improves the lives of citizens has become a priority for European SSH and is increasingly being discussed in academic forums across the continent. ¹

However, researchers still face several difficulties when asked to demonstrate the social impact of their investigations, such as the problem of attributing an impact to a specific research study of theirs (Bornmann, 2013) or the time scale needed to observe such impacts. Furthermore, at present most European countries’ funding agencies still do not have sound systems for evaluating the social and political impacts of SSH (IMPACT-EV Consortium, 2015), and as a result, researchers are usually not immersed in a tradition of collecting the evidence of such impacts. In this sense, until very recently a particularly relevant situation has been the lack of unified criteria and tools for comprehensively assessing the social benefits directly derived from SSH in contrast to the more developed tools and systems for assessing the impact of research in academia and scientific knowledge itself (Bornmann, 2014). A step forward in this regards has been the effort led by the European Commission in defining the monitoring and evaluation system of Horizon Europe, the ninth European Framework Programme (FP) for research and innovation, where the introduction of Key impact pathways, and related key impact pathway indicators allow a better tracking of progress towards the programme objectives over time (Besselaar, Flecha & Radauer, 2018).

In terms of methodology, there is now a broad consensus that quantitative methodologies and tools by themselves, such as statistical analyses, mathematical modeling, or the use of existing citations-based metrics (Ravenscroft et al., 2017), are not enough to grasp the complexity of the impacts of SSH research. Instead, the addition of qualitative methods together with quantitative methods, conducted so that the different dimensions of the research and its interactions with the context can be comprehensively grasped, is increasingly becoming the most used and recommended approach (Donovan, 2011). From a qualitative perspective, peer reviewing social impacts (Holbrook & Frodeman, 2011) and conducting case studies (Bornmann, 2013) are considered two of the most solid strategies for assessing social impact, particularly in the SSH (Ernø-Kjølhede & Hansson, 2011). In addition, while not an exclusively qualitative tool, surveys stand as valuable tools for measuring perceptions and grasping social opinions regarding specific investigations and their outcomes. ²

In this article, we argue the need to develop qualitative tools and systems for assessing the social impact of SSH research, with an emphasis on those approaches that integrate citizens’ voices into it. After introducing the main features of the communicative methodology (CM) of research, the IMPACT-EV project is briefly presented in relation to the mainstream debate of the social impact evaluation of EU-funded SSH research projects. In this line of thinking, the authors reflect on the multifaceted transformations that the communicative approach of evaluation caused in some European researchers’ perspectives regarding the social impact of their investigations. This discussion is presented in three primary interconnected parts. First, we present the controversy regarding the evaluation of the impact of EU-funded SSH research. In addition, a set of dimensions to deeply respond to the social impact issue and the improvement of society and its social meaning are discussed. Second, the origins of social science and the recovery of its founding meaning are discussed. In doing so, social sciences need to take into consideration bottom-up approaches which reclaim the dialogue and a thorough analysis of the consequences of social actors’ actions. Third, the dialogic and democratic reprocessing of social sciences is approached from the view of assuring equal access for all in terms of participating in all processes which might shape their lives. Finally, the article ends with final remarks about future research in these areas.

Transforming Social Realities Using the CM of Research

The CM is a methodological approach that not only seeks to describe but also to contribute to the transformation of any given social reality under study (Puigvert et al., 2012). The CM achieves this objective thanks to the communicative perspective on which it is grounded. This communicative perspective considers that knowledge useful for transforming the social realities that emerge from the egalitarian dialogue established between the researchers (who bring to the debate the existing scientific knowledge) and the people participating in the research (who provide their knowledge and experience from their everyday lives). Throughout the entire process of research, the reflections and points of view of all of the actors involved regarding their social situations become the focus of the study, enabling the researchers to identify exclusionary elements (those that reproduce the existing inequalities) and transformative elements (those that contribute to overcoming them) (Puigvert et al., 2012). Thus, the outcome of the CM is not only the new scientific knowledge itself but also how its implementation can lead to the transformation of the lives and contexts of the participants involved, who reinterpret their situations in light of the dialogues established in the research.

On evaluating the scientific, political, and social impacts of research on social sciences and humanities, there is an international debate on how such impacts of research should be measured. Which tools, methods, mechanisms, and approaches should be implemented to that extent? While this is a very new spectrum of research, the debates produced within the scientific community also focus on the relationship between science and society. In their research, Emanuela Reale and her colleagues (2017) highlight the bottom-up demand from civil society but also from academia, to researchers and research institutions, to enhance research and its value, for society.

For more than two decades, the CM has proved to be greatly useful for the analysis and transformation of social inequalities. For that reason, CM has been used in several research projects of the highest scientific ranking in Europe coordinated by the research group Community of Researchers on Excellence for All (CREA ³ ). This was the case for the FP5 project WORKALÓ (CREA, 2001–2004), which identified strategies to promote social inclusion of minority groups in areas such as education and employment, particularly for Roma communities. Another example was the FP6 INCLUD-ED (CREA, 2006–2011) project, which identified actions to promote social cohesion and inclusion in education and was selected by the EC as one of the 10 success stories of the Framework Programs due to its added value to society. Both projects achieved great political impact and contributed to improving the lives of the people involved in the projects thanks to the communicative approach used. ⁴

Evaluating the Impact of EU-Funded SSH Research

In the context of the international debate around the social impact of research, the project IMPACT-EV differentiates the concept of social impact into disseminating results (making the published scientific knowledge known by the society regardless of whether it is implemented or not) and knowledge transfer (using the scientific knowledge as the basis for policies or interventions regardless of whether or not they lead to social improvements). Instead, social impact is defined as when the scientific knowledge that has been previously published, disseminated, and transferred leads to social improvements in relation to the social objectives set by the society, including social organizations. In this sense, the new approach to social impact goes one step further when compared with previous impact assessment approaches that only evaluated SSH research in relationship to economic objectives or scientific productivity.

The social impact assessment has to be approached from very different perspectives and fields. Approaching this assessment from a singular perspective impoverishes the analysis of that impact and the social sciences itself. It is not necessary to replicate the error of the homo economicus, which already draws from the neoclassical economy and was used as the concept that describes a type of human who acts rationally by reflecting and according to the information they possess, primarily concerning economic decisions. Thus, the impact of research cannot only be analyzed regarding economic objectives and measured just in quantitative terms. Among the different theories tackling human behavior in society, the theory of rational choice is reductionist by limiting the human being to its individual utilitarianism, excluding other dimensions which also explain behavior, such as emotional preferences and feelings, and the behavioral foundations of economic theory (Sen, 1977).

Social impact and the improvement of society cannot be achieved without people’s contribution. People are the ones willing to have a say and demanding for improvements in their lives. The reductionism approach is overcome through a radical democracy which involves the inclusion of citizens’ voices in the process of evaluating the social impact of research. The egalitarian emphasis provided by the CM brings an adequate frame under which people are taking the lead to participate in the creation of a path for achieving impact. The open peer review system is an example of the democratic orientation of impact and science evaluation. A deliberative democracy builds people’s capacity for choice based on dialogue, argument, and consensus. This theory goes one step further by maintaining that by arguing and covenanting with others those individuals participate in a process which not only adds to their individual preferences but also transforms them. The same deliberation is defined as normatively regulated negotiation to reach consensus (Elster, 1998).

As the theories of dialogic democracy and deliberative democracy argue, the decisions concerning our actions are influenced by the process of intersubjectivity. This process includes dialogue according to the principles of communicative action (Habermas, 1987). The communicative action approach, based on the validity claims, is also the basis of the CM of research and aims to do science with civil society not only for them but also with them (Flecha & Soler, 2014). Elster (1998) himself refers to Habermas when recognizing its influence as one of the main positions of the democratic theory, which is based on the idea that democracy revolves around transformation and is not based on the mere accumulation of ideas. Pursuing the same aim of impacting society through science, other methodological approaches lead to social transformation. The action research methodology and its participatory branch, similar to the CM, involves conducting research with a problem-solving focus (Reason & Bradbury, 2001).

International markets put a great deal of pressure on society and the system (Cole, 2016) to not be research endeavors but an exception to them. Research commodification influences a way of elaborating certain knowledge in response to client dynamics. This wave of global market pressure influences many sectors of society and also falls on scientists who are pressured to carry out their work by directing them to produce more. On the contrary, the market leads scientists to do a series of activities without taking into account the social impact of these activities on society or its consequences. The market challenges scientists’ predisposition to improve society and creates important contributions as it makes client dynamics imperative.

The capitalist market takes on what democracy offers to its citizens. The evaluation of social impact becomes an opportunity to return meaning to scientists and professionals. The analysis of Max Weber (1922) focused on how the modern state and the capitalist market lead to the bureaucratization of society, which will not only influence the way companies function but also on the whole social structure founded on the power managed by institutions. This bureaucratization and its consequences affect not only the capitalist system but also the social sciences. In this way, aspects such as the loss of autonomy, the establishment of power relations, the adherence to rules, the preestablished authority, and the routinization of life disrupt the human freedom and the human capacities to overcome the resistance to change.

The capitalist system, along with the process of rationalization, produces disenchantment, power struggles, pressure groups, and bureaucratization processes which, as Weber stated, leads to the loss of meaning, in line with what he called the “iron cage” when referring to the increase in the rationalization of the social life. The bureaucratization of the social order takes away everything that democracy provides to people. Individuals thus act on a rational basis, bureaucratizing their actions, and as a result, they lose sight of the meaning. This process has a very direct impact on the organization and production of sciences, which are increasingly being routinized at the institutional level, without safeguarding the goals under which they were created. Schütz (1973) analyzed the concept of the lifeworld as such spaces that inspired and are based on the social experience of any community. The lifeworld was constituted by the already lived experience and by the envisioned experience, which may be understood as a community in which people live, share, and transform. Systems and lifeworld are linked in this way. The transformation produced within the lifeworld also impacts the institutional level, again empowering people and making a bottom-up influence on the system.

In this context, the development of a communicative system of social impact measurement responds to a reality that seeks to give importance to the qualitative approach and research carried out using qualitative methodologies, which have been proven successful at including the voices of all people. As Habermas (1987) wrote, there is no methodologically relevant inequality between the interpretations of the researchers and those of the social actors. Indeed, the current moment of our science places researchers not only as observers of reality but also as receptors of the whole society’s views. This perspective improves the specific analysis of reality and changes it. This approach to qualitative research (Denzin & Lincoln, 2005) is based on people’s ability to denounce social problems and thus has the dual goals of transforming people’s lives as well as using that analysis to impact social policies. The effects of bureaucratization can also be overcome through democratizing knowledge, making it more egalitarian, thereby also responding to a political need in a participatory manner (Álvarez-Jiménez & Padrós-Cuxart, 2017).

At a time when the social sciences have become bureaucratized to the point that they no longer serve to improve society and when bureaucracies have established such high levels of resistance to change, the social sciences lose the initial goal under which they were created, and a very profound loss of meaning occurs. The social sciences and sociology recognized this loss of meaning when they analyzed the consequences of the process of bureaucratization on the social structure without solving the problem that it created. We believe that better forms of academic and business organization, in which people are considered crucial actors, exist and can improve this reality.

The influence of the bureaucratic apparatuses of the state and the market results in the loss of meaning in the social sciences work. Interests that often deviated from people’s interests and only served to improve researchers’ résumés without having any impact on the agents they studied have generated a way of carrying out the social sciences that no longer has the support from the groups studied for social improvement. In this same way, social movements and active organizations that developed their activities based on the values of the social transformation also begin to lose the meaning by which they fight.

However, it is the people, not the systems, who are the protagonists of social actions (Habermas, 1987; Schütz, 1973) and who give meaning to their actions. Individuals are the basis of social actions, and therefore, individuals or groups generate social facts depending on the meaning of those actions (Beck, 1992; Giddens, 1991). The elements that lead to losing that meaning not only have a negative impact on the progress of society but also lead to disenchantment with everyday life. It is also precisely from the lifeworld where Habermas proposes reenactment with everyday life.

Social Sciences Origin and Meaning Recovery

When were the social sciences created? Why were the social sciences were created and how did their meaning come about? The systematic study of human societies begins at the end of the 18th century and the beginning of the 19th century, in a context in which the changes and transformations occurring were remarkable. The French Revolution and the Industrial Revolution in Europe produced several of the best changes during this time. This time period reflects a context of Illustration, in which reasoning, universalism, and freedom predominate. It is also a context of modernity, in which reason prevails to govern society.

This historical moment also represents the rupture of traditional ways of life. From the changes generated by these revolutions, great thinkers such as Rousseau, Comte, and Adam Smith attempted to develop a new way of understanding the social world and the natural world. Humanity finds itself at a time when science is beginning to be considered more important than religion in understanding the reality of the world. In this context, the analyses of society have some type of impact according to the way in which they are carried out. For authors such as Comte and Marx, societies are more than the sum of individual decisions, but on the contrary, individuals and their lives are influenced by society. Social phenomena cannot always be simply perceived by the people. One of the problems found in the analysis of human phenomena is that they are also studied by humans. As a result of this human component, emotions may lead to bias in analysis. However, at the same time, there is a need to include emotions in our research.

The major issues of social theory become those scientifically understood by society and the actions that reproduce and transform it. There is a need to scientifically analyze the consequences of the different possible actions and to provide elements of analysis to the social actors. Thus, beyond describing why social realities occur, the social sciences are created with the objective of analyzing which actions promote the improvement of this reality and which actions hinder them. The emergence of social sciences such as sociology occurs within a context of change and transformation of the different spheres of social life, such as industrial economics, new cities, political changes, and progress in the fields of science and technology. What differentiates these scientists from earlier thinkers is their ability to understand and explain how society really works.

The role of social scientists in this regard is to analyze the consequences of different actions and choose those that are most likely to be the most successful paths in a constant egalitarian dialogue with stakeholders. It is possible to conduct global social science from below, starting with the people. Weber also sets forth a process of globalization through democracy, which must be built. Recovering the orientation of Weber provides the meaning through which the social sciences were created.

For a society to trust in the social sciences, we (as scientists) have to prove its validity, the potential impact it may achieve, and its improvement on people’s lives. Thus, we must first believe in ourselves and our skills. We cannot demand that society defends or funds science in which scientists themselves often do not believe in. Science and social impact have a direct relationship when people are inquired about this desired impact and when scientists are asked to do it. Through “dialogue,” we can reclaim the original meaning of the social sciences. In fact, the social impact of science finds itself currently in an historical moment. This moment is similar to how the 1970s and 1980s symbolize the turning point for qualitative researchers, ethnographers, and participant observers, who found themselves a place to make their voices heard; the qualitative program emerged as a result (Denzin, 2010). Currently, social sciences need to embrace this dialogue to flourish on a more egalitarian basis.

Dialogic and Democratic Reprocessing of Social Sciences

In a context where society calls for social sciences with impact that is useful due to improved functioning, the democratic social sciences can contribute to the systematic elaboration of scientific knowledge on how institutions and social structures are being developed and how these can be improved. The dream of a better society is obtained from social transformation. This transformation is not only the responsibility of scientists but also the entire society to the extent that it puts pressure on having scientific knowledge useful for social transformation. Action research shares this aim and has been implemented in a broad variety of contexts, such as with minority issues and educational problems, and has been shown to be successful.

Indeed, society no longer asks for the social impact but also the extent to which such impact achieves the maximum possible benefit for society. This principle is completely related to social and political justice. In a fair society, everyone would have equal access and extensive social media and materials to progress in society. If we speak of an egalitarian society, all people would have equal and wide access to participate significantly in decision-making processes that affect their lives. A fair society means freedom for people to choose those options which will not affect them; it also enables people to participate in decisions that affect them directly. Thus, the social sciences can contribute to eradicating fatalism with the systematic creation of scientific knowledge on viable alternatives (Wright, 2010).

Responding to a way of doing things differently, the IMPACT-EV project offers indicators to assess the social impact of projects and thus collects indicators to measure such impact (Reale et al., 2017). Therefore, the open repository SIOR (Social Impact Open Repository) has been developed as a tool that gathers research from all disciplines with the highest impact from around the world. Qualitative research is at this point and has been shown to be useful for indicators and evaluations which are usually linked to the quantitative sphere. IMPACT-EV (CREA, 2014–2017) includes this qualitative dimension for collecting people’s voices and adding to other already existing systems.

The democratic participation of society in science is achieved through the principles of dialogue, for example, the egalitarian dialogue that becomes a common effort to achieve equal participation of all people and social actors, giving equal importance to the voices and contributions of each one of them. The way to achieve equal intervention is also through increasing that participation. Thus, the dialogic participation is based on a communicative conception of the social sciences that poses a reality built by the interactions between agents, researchers, and society (Tellado, 2017). Dialogical modernity (Beck et al., 1994) questions traditional authorities and is dedicated to reorienting the objectives and methodologies of the social sciences to direct them toward successful results. Thus, the possibility and convenience of egalitarian transformations is defended as a result of the dialogue, surpassing the interpretation and understood as a negotiation of definitions susceptible to consensus.

Weber, when seeing social actions as those where different actors offer the same meaning, opened the door to Habermas’s suggestions to use our capability of language and action to recover the meaning that had been lost. Thus, social actions, those human behaviors to which subjects attribute a subjective meaning, are endowed within a development-oriented meaning (Weber, 1922). When social actions and human behavior are transformed, this does not only contribute to achieving social impact, but also leads to managing science in a way that is appreciated by citizens. The transformative paradigm of authors such as Donna Mertens (2009) provides manners of meaningfully understanding our own practice to conduct research and evaluation for everyone and specifically within ethnically diverse and marginalized communities, contributing to the fulfillment of social justice and human rights goals.

Final Remarks

The development of qualitative methodologies in the analysis of social sciences and their impacts not only complements quantitative analysis but also increases the visibility of how science can be at the service of citizens. The qualitative methodologies that are contributing to better capturing the social impact of research are those that integrate equal dialogue and democratic participation as one of their key features.

As science gets better recognition, there will be more and better science produced, leading to more funding for research and increased demand and support for science from citizens. By developing their tasks in such a manner, researchers will achieve increased impact and will also more effectively defend social sciences.

The communicative approach is transforming the researcher’s view. Thanks to its dialogic emphasis, the PIs contacted by the IMPACT-EV researchers are raising awareness of the social impact of their own projects. Furthermore, many of them are rethinking the way they conduct research and collect evidence. Society as a whole is asking us researchers to focus our commitment on responding to major social issues. Realities such as increases in employment, access to higher education, reduction of poverty, social exclusion, and dropout rates are aspects which need to be approached from an in-depth orientation of research toward science. The qualitative methodologies are breaking the skepticism among citizens regarding the social impact of research to solve problems directly affecting their everyday lives.