Three principles of pragmatism for research on organizational processes

Abstract

This article explicates pragmatism as a relevant and useful paradigm for qualitative research on organizational processes. The article focuses on three core methodological principles that underlie a pragmatic approach to inquiry: (1) an emphasis on actionable knowledge, (2) recognition of the interconnectedness between experience, knowing and acting and (3) inquiry as an experiential process. The authors’ doctoral projects on non-government organizations are used as examples to examine how the application of these principles strengthen each stage of the research process from project design and data collection to data analysis, conclusions and dissemination. This investigation suggests that pragmatism, which provides a guiding epistemological framework anchored in the inquiry process and research practicality, is a worthy paradigm for researching organizational processes. Pragmatism’s focus on the production of actionable knowledge is of particular benefit to research with non-government organizations, ensuring that research is contextually relevant as well as informed by theory.

Introduction

By exploring the use of pragmatism for research on organizational processes, this article works towards ameliorating the lack of practical case studies regarding the application of epistemological frameworks in social research. This article highlights three principles of pragmatism from the literature and demonstrates application of these at each step of the research process using examples from the authors’ qualitative doctoral projects. In essence, this investigation is driven by a pragmatic desire to evaluate the pragmatist paradigm in relation to the success of its practical application as a framework for research.

Epistemologically, pragmatism is premised on the idea that research can steer clear of metaphysical debates about the nature of truth and reality and focus instead on ‘practical understandings’ of concrete, real-world issues (Patton, 2005: 153). While this approach is compatible with qualitative-dominant interpretivist understandings of socially constructed reality, the emphasis is on interrogating the value and meaning of research data through examination of its practical consequences (Morgan, 2014b). This is particularly helpful in organizational settings where practice is closely intertwined with the ways in which knowledge is produced, causing several classical pragmatists to actually move away from the using nouns, to focus on ‘know-ing’ and ‘learn-ing’. Thus, using pragmatism, researchers working in organizational settings can move beyond objectivist conceptualizations, which have dominated research in the organizational sciences, to exploring and understanding the connections between knowledge and action in context. ‘Knowing’ in this sense, has the potential to transform practice (Biesta, 2010).

Methodologically, the implication is that researchers are better equipped to deal with complex, dynamic organizational processes where action, even if carefully planned, can have varied spatial or temporal qualities. Pragmatic inquiry recognizes that individuals within social settings (including organizations) can experience action and change differently, and this encourages them to be flexible in their investigative techniques (Onwuegbuzie and Leech, 2005). Moreover, rather than questioning the validity and intrinsic value of certain methods and methodologies, pragmatism encourages researchers to base choices on the relevance of these methods and methodologies ‘in terms of carrying us from the world of practice to the world of theory and vice-versa’ (Kelemen and Rumens, 2012: 1).

Pragmatism guides the project examples used in this article. These example projects examine organizational processes, including performance measurement and evaluation, in international and community development non-government organization (NGO) settings. The study contexts are complex, requiring investigation of multiple perspectives and building variation into techniques and analytical schemes. The processes under investigation, while tangible activities, have variable qualities once implemented. A pragmatist approach, with its epistemological focus on the inquiry process and practicality, was found to be more useful than research philosophies that lay a sole emphasis on abstraction or philosophical theory-generation. As Feilzer (2010) states, ‘a pragmatic approach to problem solving in the social world offers an alternative, flexible, and more reflexive guide to research design and grounded research’ (p. 7).

While a detailed discussion of pragmatism’s historical, epistemological and definitional roots is beyond the scope of this article, a concise overview is presented in the ‘Background’ section below to clarify the understanding of pragmatism adopted in the project examples. This is strengthened through presentation and analysis of three methodological principles of pragmatism, used as a framework to detail the project examples. These principles, when applied to research on organizational processes, such as performance measurement and evaluation, provide the rationale for pragmatism as a worthy and useful paradigm to guide research.

The results and discussion section examines how the three selected principles can be incorporated throughout the course of research from design and data collection to data analysis, conclusions and dissemination. This discussion links to practice through reflexive and critical analysis of how each of these steps evolved in the two project examples. Although this article is organized linearly for structural simplicity, the authors recognize that research, particularly qualitative research, is an emergent and iterative process. The intention is to provide other researchers with a clear outline of pragmatism’s utility in research, including as a useful paradigm to navigate qualitative applied social research on NGO processes.

Background

Historically, pragmatism developed in the late 19th and early 20th century as a philosophical movement that focused on the practical consequences of social reality. Its roots can be traced back to academic scepticism over the possibility of achieving perfect knowledge or truth through positivist scientific practice (Ormerod, 2006) The first wave of pragmatism, often referred to as ‘classical pragmatism’, is attributed to the work of Charles Peirce, William James and John Dewey. For Peirce (2014), the meaning of any concept was inextricably linked with its experiential consequences when investigated. In 1878, he outlined what became known as the ‘pragmatic maxim’, a rule for clarifying the content of concepts and hypothesis by tracing their practical consequences and thereby, grounding inquiry in issues of human significance (Peirce, 1878). James (2010) built on Peirce’s work by emphasizing the personal and subjective aspects of meaning. He argued that, when applied to philosophical debates, the pragmatic maxim does not typically show that these lack content; but instead the maxim reveals hidden and substantial issues that are at stake. For James, the pragmatic maxim was a window into the way people think, formulate ideas, experiment and construct new habits (Ormerod, 2006). With Dewey (2008 [1922]), pragmatism advanced beyond the individual, psychological realm towards an emphasis on individual as well as shared human experience. Dewey suggested that all human experience involves some amount of interpretation – interpreting knowledge and beliefs leads to action and reflecting on actions leads to new ways of knowing and acting (Morgan, 2014a). In Dewey’s view, this dialectic interpretation process, if examined thoughtfully and systematically, could uncover social realities in clearer fashion than philosophical approaches that assumed human behaviour and action existed apart from understanding. Conceptualizing epistemology as a theory of inquiry, he contended that understanding inquiry as experiencing, knowing and acting provided for a more dynamic view of social life: ‘Because we live in a world in process, the future, although continuous with the past is not its bare repetition’ (Dewey, 1929: 40). Thus, it was through Dewey’s work that the classical pragmatism movement reached its peak and came to be applied in all areas of philosophy as well as in psychology, education and politics (Morgan, 2014b). Following Dewey, pragmatism has contributed to movements such as symbolic interactionism, action research and grounded theory. More recently, sub-schools of pragmatists have emerged with subtle variations to classical pragmatism such as linguistic pragmatists who differ from classical pragmatists by emphasizing a linguistic focus in social processes (Rorty et al., 2004).

Some theorists have discredited pragmatism arguing that popular versions of pragmatism that focus exclusively on ‘what works’ can delink pragmatism from its important philosophical roots (Hesse-Biber, 2015). Others contend that the practical, real-world emphasis can encourage researchers to take a ‘soft’ approach avoiding the significance of distinctions, such as between quantitative and qualitative research, at an epistemological level (Denzin, 2010: 422). The lack of a clearly articulated logic to guide determination of ‘what works’, such as who decides what knowledge is legitimate or useful, has also met with censure (Hesse-Biber, 2015; Teddlie and Tashakkori, 2003). Thus, adopting an authentic and coherent pragmatist stance requires careful study and understanding of the underlying principles of this approach. The epistemological framework guiding the doctoral project examples discussed in this article most closely aligns with classical pragmatism, particularly the approach taken by Dewey and latter-day pragmatists that emphasize pragmatism as a unique, guiding philosophy for research in the social sciences. This nuanced approach to pragmatism has been comprehensively defined and discussed in more recent literature by David Morgan (2007, 2014a, 2014b).

For the purpose of this article, we broadly define our approach to pragmatism as a philosophical and epistemological framework for interrogating and evaluating ideas and beliefs in terms of their practical functioning. Pragmatism holds that the value and meaning of opinions and ‘facts’ captured in research data are assessed through examination of their practical consequences; this gives them a ‘warranted assertibility’ (Dewey, 1938). This epistemological stance provides scope for the pluralist understanding of multiple truths accepted by interpretivist theoretical frameworks heavily utilized in qualitative research. Contrary to the critique of pragmatism discussed earlier, we find that the emphasis on ‘what works’ actually helps validate research questions and focus inquiry processes at the onset according to how and what respondents identify as working or not working. In addition, we argue that pragmatism is of instrumental value to research on organizational processes as viewing people’s ideas and beliefs as tools for problem solving and acting in new ways involve research subjects as active participants rather than as passive descriptors of the world as it exists. Moreover, as the research progresses, the initial emphasis on ‘what works’ has potential to expand, enabling researchers to map the consequences or meanings of social action for different individuals in an organization.

This ecological understanding of the interlinkages between inquiry, experience, knowing and acting addresses many of the weakness in existing organizational research. As noted by Farjoun et al. (2015), ‘Pragmatism provides a richer and more realistic view of human behaviour than the ones used by rationalist and structuralist accounts’ (p. 4). The use of rational choice models by many organizational theorists has focused research on individual, rational agents and limited the scope to understand complexity and diversity in organizations. Whereas structural models, which are also popular with organizational theorists, provide for a more static view of organizational life. As discussed below, pragmatism contributes to a dynamic paradigm that corresponds with the complex and fluctuating context in which organizations operate.

Principles for pragmatic inquiry

Building on the definition above, we outline three methodological principles for pragmatic inquiry which serve as a critical lens for researchers to conceptualize and manage research projects in line with classical pragmatism. The principles are drawn from existing literature and similar categorizations of pragmatist methodology developed by theorists such as Friedrichs and Kratochwil (2009) and Morgan (2014b). Through our inquiry, we have narrowed down our selection to three principles of critical relevance to research on organizational processes. These principles are (1) an emphasis on actionable knowledge, (2) recognition of the interconnectedness between experience, knowing and acting and (3) a view of inquiry as an experiential process. As discussed below, these principles have particular relevance to research on NGOs but also have a wider application in other organizations.

Emphasis on actionable knowledge

A central tenet in pragmatic inquiry is the view that all research should emanate from a desire to produce useful and actionable knowledge, solve existential problems or re-determine indeterminate situations, drawn from examination of effective habits or ways of acting (Corbin and Strauss, 2008; Feilzer, 2010). In fact, as noted above, the classical pragmatism movement originated to focus inquiry on issues of human significance rather than on metaphysical debates about the nature of truth and reality (Patton, 2005). Classical pragmatists were concerned with knowledge and forms of knowing that had practical consequences. Latter day pragmatists also emphasize the importance of acting on problems in the ‘real world’ (Creswell and Plano Clark, 2003; Rorty et al., 2004). While the term ‘actionable knowledge’ is used more in organizational studies (Argyris and Schon, 1978; Elkjaer and Simpson, 2011), it served as a useful shorthand for the application of pragmatist thinking in our research.

By emphasizing this principle of actionable knowledge as a starting point for research, researchers can develop research agendas anchored in respondent experiences and, hence, ensure the research is of practical relevance. In regard to research on organizations, an iterative focus on actionable knowledge maintains the connectivity of the inquiry process to evolving organizational practice and emergent problems. This methodological principle enables researchers to engage with multiple experiences of the same phenomena and orient the inquiry towards problem solving through the reconstruction of habits and the continuation of vital and social experience. Morgan (2014a) explains this as, ‘In contrast to philosophies that emphasize the nature of reality, pragmatists emphasize the nature of experience’ (p. 27). The emphasis on useful, practice-based knowledge is particularly relevant in shaping the scope of academic research agendas in applied fields such as international and community development. This is due to considerable diversity within and between organizations working in these fields, rendering esoteric theoretical models of limited value to practice (Friedrichs and Kratochwil, 2009).

Recognition of the interconnectedness of experience, knowing and acting

A second principle which serves to strengthen the pragmatist inquiry process is the opportunity to explore the interconnectedness of experience, knowing and acting in the research process within respondent organizations. By seeking a better understanding of the organizational processes under investigation, and documenting actions and experiences of staff, whether well-planned or haphazard, pragmatist researchers are able to surface complex themes and issues hidden in formal documentation or rhetoric. This can be achieved through triangulation of what respondents say and what can be observed. This feature is particularly relevant to NGOs as, often, organizational processes are not properly documented and, to a large extent, rely on the knowledge and interpretations of implementing staff.

Pragmatism can be more suitable than other major philosophical approaches to probe this ‘inner world’ of organizational processes as first, it emphasizes that knowledge is based on experience; and second, it encourages the researchers to analyse organizational practices through experience as well as action. Although pragmatist researchers are sometimes criticized for over emphasizing the practical, in fact, pragmatism enables researchers to overcome the dichotomy between theory and action and give voice to those impacted by organizational process (McKenna et al., 2011). In other words, by emphasizing the principle of actionable knowledge right through the research process, researchers can holistically address the knowing, acting and experience occurring in NGOs.

Inquiry as an experiential process

The final methodological principle distilled from the literature on pragmatism is informed by the Deweyan principle of inquiry, which links beliefs and action through a process of decision-making (Morgan, 2014b). According to Dewey (2008 [1910]), all conscious human actions involve some amount of inquiry or assessment in response to a problem or obstacle. This inquiry or assessment is followed by adaptation and altered behaviour in response to the problem. In Dewey’s approach, there is no distinct boundary between everyday life and research (Morgan, 2014b). Instead, he views research as a form of inquiry that is performed more carefully and more self-consciously than most other human responses to problematic situations to the external environment (Dewey, 2008 [1910]). This embeddedness of inquiry in practical, everyday situations makes classical pragmatism relevant to theoreticians and practitioners alike. It also addresses a key challenge in organizational research which is for the researcher to develop a ‘mediated’ understanding of complex organizational processes (Lorino et al., 2010: 778).

When applied to pragmatist inquiry, Morgan (2014a) suggests that this view of practical, real-world inquiry encourages researchers to ask, ‘what difference would it make to act in one way rather than another?’ (p. 28). By encouraging organizational stakeholders to trace out the likely consequences of different lines of action, researchers are able to gain a more detailed understanding of the phenomena under investigation. This experiential process helps researchers with combining macro- and micro-level perspectives within an organization and encourages a more inclusive research process as different stakeholders are able to situate themselves and their actions within a larger context. Moreover, as detailed further below, the research agenda shifts from merely focusing on ‘what works’ to mapping the consequences, experiences or meanings of social action for different individuals in an organization.

Thus, pragmatism, anchored in these three methodological principles, presents numerous possibilities for developing a contextualized understanding of organizational practices, particularly in NGOs. Furthermore, grounding these principles in an epistemological stance that recognizes the deep interconnections between experience, knowing and acting can help combat the perceived limitations of overly ‘practical’ inquiry. The following section examines how pragmatism can be incorporated throughout the research process in relation to these three principles using our doctoral research as worked examples.

Outline of the project examples

The remainder of this article utilizes our experiences conducting doctoral studies underpinned by a pragmatist approach, to discuss the appropriateness of using pragmatism for qualitative research on organizational processes. Both doctoral projects examine organizational processes related to performance measurement and evaluation in smaller sized NGOs that deliver international and community development programmes. We have used auto-narrative to unpack how pragmatism was enacted at each stage of the research to add clarity to the often-abstract discourse surrounding paradigms and epistemology. These two doctoral projects offer complementary and practical examples of how pragmatism can be used to strengthen research processes. They also shed light on the value of pragmatist inquiry in unpacking the tensions involved in NGO programme evaluation. The many similarities between the two projects lend consistency to the findings, while the differences highlight potential diverse applications for pragmatism.

Table 1 below provides a brief overview of the two doctoral projects. For the purpose of this article, this concise understanding of the projects will suffice to provide context.

Table 1.

Overview of the research project examples.

	Project example 1	Project example 2
Research question	How and why do small NGOs conduct and use programme evaluation	How does evaluative information influence decision-making processes in NGOs
Methodology	Qualitative	Qualitative
Scope	20 case study NGOs	50 case study NGOs
Methods	Interviews, document analysis, participant observation	Interviews, document analysis, online survey
Analysis	Thematic analysis	Thematic analysis, cross-case comparison

NGO: non-government organization.

Applying the principles of pragmatism

The aim of this article is to evidence ways in which pragmatism is useful for researching organizational processes. The following section unpacks this notion by demonstrating how applying three principles of pragmatism enriched key decisions at every stage of the research process for each of our doctoral theses. The sub-headings are set out in a linear fashion, despite the research processes being iterative and more complex than this reductive layout would suggest. The sub-sections below apply the principles of pragmatism to decisions at each research stage in a general way, while drawing strongly on the experiences of our research projects to augment this discussion.

Research design

For most doctoral projects, the research design stage involves significant decision-making around establishing the research objective, framing the research problem and choosing a methodology. As indicated above, the choice of pragmatism as an overarching philosophical orientation was strongly influenced by our desire to contribute useful and actionable knowledge anchored in respondent experience and, hence, of practical relevance to the case study organizations. In both project examples, the adoption of a pragmatist approach was closely intertwined with establishing research objectives. These objectives were enacted by uncovering staff interests and perceived benefits of the research at the very onset. By scanning both theoretical and grey literature for knowledge gaps of practical use to the case study organizations, we were able to inform and refine the research objectives. In this way, pragmatism helped us unpack the research problem and identify elements of the problem considered the most useful by and for respondents. This iterative prioritization process was important for respondents but also important for enhancing the relevance and potential transferability of the research findings, an important aspect of pragmatism (Kelly, 2019c; Morgan, 2007).

Pragmatism’s inherent focus on experience and action helped refine the research problems under investigation. For instance, initial inquiry into programme evaluation (Project example 1) revealed distinctions between how head office and field office staff enact evaluation and the consequences of these actions. Likewise, the process of framing of research questions based on practical understanding of organizational decision-making processes (Project example 2) revealed the diversity of views between board members and staff in NGOs. By building in scope to explore the diverse experiences of different individuals at the level of the research questions, we were able to identify more facets of the research problem than if these were solely framed around the theoretical literature, formal NGO documentation or our existing understanding of NGO processes.

Pragmatism was instrumental in directing us towards making appropriate methodological choices by unpacking different aspects of the research problem at the design stage. Both projects required methodologies that looked beyond organizational documentation to capture the lived experiences of practitioners. In Project example 1, pragmatism helped identify the need to engage deeply with staff at the project implementation level. This was due to the, often subtle, informal and everyday evaluative practices that were taking place without due recognition (Kelly, 2019c). This necessitated a qualitative approach with interviews and participant observation of programme staff, to supplement the limited formal documentation.

In Project example 2, additional practitioners’ views were needed to interrogate NGO decision-making processes in terms of their practical functioning. Hence, an online survey was included in the research design. This survey focused on practitioners’ experiences of NGO performance measurement processes and decision-making processes, enabling comparison with formal documentation, as well as the views of NGO leaders and boards.

A diversity of methods offered a framework to map, triangulate and sequence the research problem against diverse NGO contexts. Simultaneously, this approach highlighted the multiple interpretations of organizational realities experienced by practitioners within these NGOs. As noted by Lincoln (2005), ‘the impact of efforts to improve organizations is always a task left unfinished, because it cannot account for many things unseen’ (p. 222). The pragmatist principles provided guidance to investigate these ‘unseen’ processes and evaluate the success of their practical application.

In addition, pragmatism guided our sampling strategies by helping us identify information rich respondents most likely to provide useful practice-based knowledge, as well as ensuring the sampling process uncovered a range of perspectives. Revisiting our three key principles (an emphasis on actionable knowledge; recognition of the interconnectedness of experience, knowing and acting; and inquiry as an experiential process), helped ensure that sampling decisions made would add towards knowledge generation and provide useful information that could be invested immediately into practice. For Project example 1, this involved seeking out the perspectives of the primary intended users of evaluation. These are staff who exhibit what Patton (2008) calls ‘the personal factor’, a trait recognizable in ‘people who personally care about . . . evaluation and the findings it generates’ (p. 66). This included executives, board members, programme implementing staff and volunteers. For Project example 2, this meant having a balance of NGO leaders and operating staff respondents as this research on decision-making processes required that perspectives of decision-makers were juxtaposed and interrogated by implementing staff – an example of the macro-micro linkages that pragmatism helps establish. Adherence to pragmatism’s key principles led us to ensure that we could provide strong rationale for each decision within our research designs, and that the sampling of both the respondent NGOs and the staff within them fitted within accepted research rigour guidelines.

Data collection

Elements of the data collection stage where we found it valuable to use pragmatist ideas to guide research choices included the scope and sequence of data collection methods, the researchers’ stance and ethical considerations for data collection. Decisions about these elements were made at the research design phase. This section discusses how these were enacted in our research.

A prominent theme in the literature on pragmatism is that it does not dictate choice of methods (Feilzer, 2010; Morgan, 2014a; Teddlie and Tashakkori, 2003). Instead, it provides a framework to help researchers choose which methods will be most appropriate. In both research projects, there was a need for methods that made room for plurality and the interests and agendas of diverse respondents in the research process; a pragmatic approach, with its emphasis on actionable knowledge, enabled us to achieve this. In Project example 1, data were collected using a focused ethnographic methodology which augmented interviews with participant observation. This enabled the researcher to witness different forms of evaluation that staff had not identified in interviews. Focused ethnography was a pragmatic choice as there was no scope for the long-term immersion of conventional ethnographic approaches, yet many aspects of ethnography were considered vital for surfacing consequences of action which would have remained hidden from other forms of inquiry (Kelly, 2019c). In Project example 2, a survey provided additional data to validate initial interviews with managers and better scope the challenges experienced by project implementers. In both research scenarios, the choice of methods was driven by initial scoping of the research question using the principles of pragmatism as a guide. For instance, both project examples surfaced actionable knowledge by asking respondents to describe evaluative practices they consider useful and then triangulating that information to arrive at broad categories of practices for further data collection and analysis. Examples included detailed description of internal evaluation practices, which had not been documented by the case study organizations, and evaluative data or evidence that had been collected but not used in improving NGO projects. Furthermore, we agree with Swanson and Holton (2005) that ‘Enlarging the range of available (and appropriate) methods can only generate better and more complete pictures of phenomena’ (p. 231).

Pragmatist epistemology with its emphasis on the interconnectedness of experience, knowing and acting (a key principle), enabled the sequencing of different data collection methods. At the design stage of Project example 2, it was found that several small NGOs were developing complex, organization-wide performance measurement systems. However, because these systems were in their infancy, descriptions of these systems or reflection on what was working and what was not had not been formally documented. Incorporating a follow-up survey to explore the informal practices of NGO staff, and monitoring the extent to which these were changing over the course of the research, allowed this project to frame the research problems in ways that would remain relevant even as contexts evolved: a key benefit of pragmatist research (Feilzer, 2010). In Project example 1, pragmatism effected sequencing by allowing a level of flexibility unusual in many other frameworks. This meant that the order, dose and duration of data collection among the different case study NGOs could adapt according to NGO requirements without disadvantaging the study’s trustworthiness.

At the data collection stage, pragmatism offers several possibilities for determining a researcher’s stance towards respondents. For both of our doctoral projects, there was a risk that individual interpretations, such as the views of NGO leaders on organizational processes, would skew the data collection process. The benefit of pragmatism is that it creates room for the exploration of how individual experience, knowing and acting are shaped through social interaction. For instance, the way that programme managers in the example projects choose which evaluative tools and methods to use may be shaped by normative beliefs surrounding the nature of evidence. This focus on ‘socially shaped’ (Morgan, 2014b: 1047) behaviour enables a more holistic view of organizational processes as well as determining how researchers engage with different groups of respondents.

In both projects, pragmatism necessitated a reflexive stance at all stages of data collection as respondent perspectives were being shaped by the inquiry processes itself. Reflexivity permits a more nuanced and context-specific understanding of how research processes and relationships are constituted and how they affect theory and its future consequences (Weick, 2002). For instance, early in the data collection process in Project example 2, it became obvious that the link between performance measurement and the decision-making process had not been reflected upon by implementation staff. Similarly, in Project example 1, many respondents highlighted that they had never considered the worth of evaluation. Hence, it was important for the researchers to develop a relationship of openness to surface new perspectives rather than prompt or direct respondents. This was undertaken through the circulation of plain language questions prior to interviews in Project example 2 and by allowing respondents in both project examples to feedback and refine views before, during and after interviews.

Pragmatist inquiry enables researchers to embed ethical considerations in the research process. As noted previously, both projects involved respondents with differing levels of knowledge, experience and control over the process being studied. Ethically, the challenge was to value all different types of knowledge. In addition, as both researchers are practitioners within the NGO sector, we were keenly aware of NGOs’ limited resources and were concerned about our use of NGOs’ time. We found that adhering to pragmatist principles develops researchers’ ‘moral responsibility in presenting knowledge that has consequences for future applications’ (Kelemen and Rumens, 2012: 10). Hence, ambiguities in the data such as contradictions between management and operational staff, or between the formal documentation and informal organizational practices, were easier to approach. Pragmatist inquiry also reminded us to keep the perspective of the research participants in mind. One of the researchers (Project example 2) drew strongly on concepts from other disciplines so she felt she could add value to NGOs by helping them cross a disciplinary border. The pragmatic principles adopted encouraged her to be multi-disciplinary to provide NGOs with knowledge from other disciplines to augment their practice. The other researcher (Project example 1) aimed to compensate NGOs for their time by providing them with ongoing resources, peer review and contacts as applicable.

Analysis, conclusions and dissemination

During the final stages of the doctoral projects, pragmatist ideas influenced the approach and techniques used to analyse the data and draw conclusions. The guiding pragmatist principles are also informing the researchers’ ongoing dissemination strategy and driving an enthusiasm for utilization of the study’s findings.

In both projects, pragmatism helped with the development of an analysis plan by focusing examination of the data on the principle of actionable knowledge. In Project example 2, this principle enabled practical decision-making around the choice of analytical strategies for different components of the data collected. By starting with organizational practices prioritized by case study NGOs (as evidenced by multiple documents from each NGO), the researcher could hone in on those organizational practices that were most likely to have relevance for the sector as a whole. Organizational strategies that were only considered useful in specific NGOs were analysed qualitatively and using case study–based analytical procedures. This data analysis plan, based on useful knowledge as determined by respondents, enabled the researcher to balance the macro- and micro-level investigation required to answer key research questions around NGO performance measurement. For Project example 1, the analysis plan focused on the principle of actionable knowledge by analysing NGOs’ organizational values against widely accepted programme evaluation standards to highlight practices respondents found useful, purposeful and meaningful, as well as identifying practices that could be strengthened.

At this stage, pragmatism enabled the researchers to manage dynamic and iterative analytical processes. This was achieved through acknowledging the interconnectedness between experience, knowing and acting – a key pragmatist principle, throughout the analysis and final write-up stages. Understanding this interconnectedness was enriched in Project example 1 through the cultivation of ongoing relationships with respondents. Keeping respondents in a communication loop enabled respondents to feel invested in the research and send impromptu updates of their evaluation journeys. In both project examples, an important part of the analytical process involved the documentation of field-notes or emergent findings, which included new evidence collected through repeat interviews or updated documentation, as the case study NGOs enacted the practices that were under investigation. In addition to identifying and including new evidence, the field notes also captured elements of the discourse that NGOs were excluding. In Project example 2, this included multi-year comparisons to ascertain changes in the way case study NGOs had positioned performance data on their websites and in their annual reports.

The use of these techniques in both the research projects meant that analytical processes were flexible and adaptive to capture the actual experiences of NGOs as they implemented and amended organizational practices. This contrasts with other organizational research, which may be more static and theory-driven. In pragmatist research, this flexible and adaptive form of data collection and analysis is referred to as abduction. Abduction allows researchers to move from observations to theories, by reasoning at an intermediate level (Friedrichs and Kratochwil, 2009). This involves collecting pertinent observations about the study phenomenon and, at the same time, applying concepts from existing fields of knowledge, instead of either relying solely on abstract concepts (deduction) or developing propositions based solely on observations (induction). By identifying common themes and commonly used performance measurement and programme evaluation in case study NGOs, and then later collecting data on the enactment of associated organizational practices, the researchers were able to develop a comprehensive picture of these study phenomena and assess the emergent theory in terms of the success of its practical application.

A pragmatist view was vital in ensuring the validity of our conclusions and in delivering on practical research outputs for the case study organizations who took part in our research projects. Pragmatism, particularly questions around the usefulness of data from the perspective of diverse respondents, and the fact that respondents were actively involved in the inquiry process, guided an increased depth of analysis and contributed to the validity of findings. As noted earlier, one of the critiques levelled against pragmatism is the lack of a clearly articulated logic about who decides what knowledge is legitimate and useful. The engagement of respondents and the experiential nature of the inquiry process enabled the researchers to address this critique. Moreover, in both the projects, repeated emphasis on the principle of actionable knowledge uncovered, sometimes subconscious, influences of which the researcher and respondents were unaware at the start of the research. For instance, both projects found that many organizational practices had limited internal relevance although resources were being invested in these. This finding was uncovered by asking respondents to think of performance measurement practices and evaluations they had found useful and to describe the value of these practices. By analysing the data across the sample NGOs, it was found that many of the established and documented practices were rarely mentioned in the context of discussions on usefulness. Had the researchers adopted a more theory-testing methodology or one based purely on structured, close-ended methods, it would have been unlikely for the projects to contribute to validate ‘practical understandings’ (Patton, 2005: 153) of the study phenomena.

Finally, a facet of pragmatist research is its focus on research transferability. Rather than becoming mired in discussions regarding generalizability, pragmatism looks more practically at the applicability of the research to other contexts and settings (Morgan, 2007). This tenet was central to both projects as we embarked on these research projects with the aim of finding respondent-driven solutions to real-world problems. While recognizing that our research is highly contextually bound, both projects have scope to share useful findings. Our dissemination strategies pragmatically centre on informing and engaging practitioners, where possible, in discussions on our findings. Both researchers have spent many years working in social programmes and have built extensive networks. These will be particularly effective for Project example 2, which has been working with NGO peak bodies and funders. The dissemination strategy for this project involves the preparation of practice briefs and speaking engagements with NGO decision-makers. As Project example 1 focuses on small NGOs, a different approach is being taken to share the findings directly with these organizations. This dissemination strategy involves roundtable workshopping events and practice-based publications (e.g. Kelly, 2017, 2019a; Kelly and Smith, 2017; Rogers et al., 2019).

In addition, part of both projects’ objectives is to broadcast the work of smaller NGOs to the development sector. This necessitates dissemination to a wider audience through journal articles, blogs and conference presentations which advocate for smaller NGOs and highlight the need for appropriate evaluative processes that correspond with their organizational priorities and operational constraints (e.g. Cordeiro and Kelly, 2019; Kelly, 2016, 2018, 2019b, 2020).

Conclusion

Grounding these research project examples in pragmatism provides a number of benefits as explicated above. Pragmatism, which provides a guiding framework anchored in practicality, is particularly appropriate for these studies examining performance measurement and evaluation in applied NGO settings. It has enabled us to frame organizational processes as social practices and arrive at a dynamic and multi-faceted understanding of practice. By demonstrating the suitability of this approach for research on NGOs, this article contributes to the growing literature supporting pragmatism as a potential paradigm for organizational research (Elkjaer and Simpson, 2011; Farjoun et al., 2015; Lorino, 2018). We suggest that pragmatism is a particularly worthy paradigm to guide research that seeks to provide a public good by improving practice and policy while remaining true to the quality-driven rigours of academic research.

Clarifying the three principles of pragmatism helped us target data collection and strengthen the depth and quality of analysis. The principle of actionable knowledge anchored the research in the experiences of respondents. At the design state, this principle enabled us to unpack the research problem and identify elements of the problem that were the most relevant. As the research progressed, this principle helped us determine our research questions and methods. By emphasizing the principle of actionable knowledge right through the research process, we were able to achieve research outcomes that were relevant, transferrable and contributed to both the theory and practice of NGO performance management and evaluation.

Understanding the interconnectedness of experience, knowing and acting guided our research design. The methods we chose as a result of this principle enabled us to unpack complex problems and contexts. This principle was particularly valuable in guiding our sampling strategies; helping us identify information-rich respondents most likely to provide useful practice-based knowledge, as well as ensuring the sampling process uncovered a range of perspectives. Moreover, by encouraging organizational stakeholders to trace out the likely consequences of different understandings or actions, we were able to gain a more detailed understanding of the phenomena under investigation.

Finally, the understanding of inquiry as an experiential process clarified our roles as researchers. At each stage of the research process, we interrogated and evaluative our ideas and interpretations in terms of their practical functioning. This principle also enabled us to combine macro- and micro-level perspectives within the case study organizations and encouraged a more inclusive research process as different stakeholders were able to situate themselves and their actions within a larger context. Through pragmatism, we could be flexible and adaptive throughout the research process as iterative inquiry, experience, knowing and acting informed ways to improve the research projects’ usefulness and value. Its fluidity allowed for abductive, inductive and deductive reasoning, which supported inclusion of emerging ideas and data.

For us, using pragmatism to underpin our research has strengthened its ability to capture elements of practice that are important to NGOs and their staff, turning practice-based evidence into theory. Thus, rather than being bound by pragmatism’s inherent focus on practice, we have used the focus on practice as a driver for new ways of knowing and understanding. This has reverberations for the relevance and utility of our research findings, not only for NGOs but also for the wider organizational research discourse. Our research projects were driven by intentions to make valuable contributions to NGOs. Pragmatism provided us with the roadmap to guide us there.