Sage Journals: Discover world-class research

Abstract

This article highlights “reflexive integration” as a methodological tool that can facilitate the explicit integration of quantitative or qualitative elements in mixed-methods research. Reflexive integration of research elements (RIRE) is advocated as a mechanism that can be used in any mixed-method study to enhance depth of inquiry and transparency of the steps involved in mixed-method research. An illustrative example is presented to show the step-by-step process of reflexive integration at various stages of a mixed-method study.

Keywords

mixed methods methods in qualitative inquiry ethnography grounded theory observational research

Introduction

The integration of reflexivity in mixed-methods research remains hidden from methodological discussions. The absence of reflexivity obscures knowledge about important decisions researchers make while conducting a study. Some scholars have suggested ways to attend to reflexivity. This may include repositioning paradigmatic approaches to consider positionality, bringing reflexivity to the “foreground” of research endeavors, and understanding that reflexivity facilitates self-awareness about mixing methods along various dimensions of integration, and showing when and how reflexive decisions take place (Biddle & Schafft, 2015; Cain et al., 2019; Cheek et al., 2015; Hesse-Biber, 2010; Popa et al., 2015; Popa & Guillermin, 2017). The reflexive approach advanced in this paper encourages researchers to focus on the elements of research.

Reflexivity involves a researcher engaging in critical thinking about how their location, values, opinions, and worldview may influence decision-making and interpretation during the research process. Reflexivity acknowledges how individual location and biography intersects with the interpretation of research field experiences; it is an ongoing process of explicitly revealing “hidden agendas” in research (Finlay, 2002; Guillemin & Gillam, 2004; Pillow, 2003). Reflexivity enables researchers to display their methodological decisions, for “it is indeed a feature of all research, across the natural, social and cultural disciplines” (Whitaker & Akinson, 2021, p. 14). Reflexivity is a practice and tool that provides insight into how the “how” and “why” of knowledge production (Finlay, 2002; Lumsden & Winter, 2014; Pillow, 2003). The pertinent task is how can researchers achieve reflexivity in their work consistently (Dean, 2017; Finlay, 2002; Morse, 2015). I present a way in which reflexivity can transform ad hoc decision-making into a critically informed selection of research elements and their integration into some particular phase of a research project. I call this research activity the “reflexive integration of research elements” (RIRE).

Reflexive Integration of Research Elements

The reflexive integration of research elements (RIRE) involves integrating qualitative and quantitative “research elements.” Research elements include methods or techniques used in qualitative and quantitative research, respectively (see Johnson et al., 2010 for an overview of methodology and method and Sanscartier, 2018 for a discussion of techne). To further clarify, methodology is the toolbox that contains all the available and epistemologically appropriate tools for each research tradition to examine some reality to generate knowledge; it is the design framework. The research elements are the tools within the toolbox, the methods and specific procedures executed. Methods are the technical tools used to execute research (Kelly, 2009) and what the current article refers to as “elements” or “techniques.” A researcher can draw inspiration from the elements from one toolbox to inform the adaptation of the elements inside the toolbox of the other tradition (Hall & Howard, 2008, p. 253).

Researchers can reflexively integrate elements used in qualitative and quantitative research into a singular research endeavor and mixed-method studies in particular. The main purpose is active engagement of subjective thought processes during the research process and incorporating elements from qualitative and/or quantitative methodologies into a principal methodology of choice (see Palinkas et al., 2011). Furthermore, this purposeful engagement, or exercising reflexivity in the selection of relevant elements aligns well with efforts to problem solve while conducting research from start to finish, but also in a way that is attuned to axiology (Biddle & Schafft, 2015). The nature of the problem may vary and could be external to the research project such as conventions for establishing reliability and validity or for publication (see Finlay, 2002; Pillow, 2003), or reflective of disciplinary conventions, conflicting interests, or issues concerning social justice (Freshwater & Fisher, 2015; Lumsden & Winter, 2014; Popa & Guillermin, 2017). Yet, despite the variety of problems that may arise, we can account for the decision-making that requires some aspect of reflection on preferences for solving issues encountered or for completing research in general.

RIRE helps operationalize subjective thoughts (reflexive moments) into decision-making.

This begins with acknowledging the availability of opportunities to make subjective and selective analytical decisions throughout the research process. These decisions are “decision points” that establish an opportunity for reflexivity and further substantiation of choices, thus transforming decision points into “reflexive moments” (Jootun & McGhee, 2009). One may even be lead to a variety of decision points or “midcourse diversions,” which are also reflexive moments, where researchers make informed decisions about the current step before proceeding to the next decision and reflexive moment (Morse, 2015). Decision points and reflexive moments arise with a researcher’s internal considerations about how to treat data or an issue and proceed. At this juncture, a method for incorporating reflexivity, such as RIRE, can guide the careful selection of qualitative or quantitative elements to address internal considerations and charting a path forward.

RIRE is about drawing inspiration from the broader circle of the research process and the attendant methods either necessary or typically used to carry research through this cycle, whether qualitative or quantitative. For example, qualitative research is an inductive endeavor that may begin with guiding research questions and data collection, followed by analysis, and the aggregation of findings, that may then extend or establish new working theories. The qualitative inductive endeavor informs or can lead into the quantitative deductive process whereby working theories, theory, and theoretical propositions inform hypotheses that are then tested through specifying a research design, developing constructs for measurement, and collecting data that will be analyzed and produce findings. The malleability of qualitative research may even help facilitate this research cycle:

The extent to which a qualitative approach is inductive or deductive varies along a continuum. As evaluation fieldwork begins, the evaluator may be open to whatever emerges from the data, a discovery or inductive approach. Then, as the inquiry reveals patterns and major dimensions of interest, the evaluator will begin to focus on verifying and elucidating what appears to be emerging – a more deductive approach to data collection and analysis (Patton, 1990, p. 194).

In this instance, qualitative research endeavors can produce opportunities for RIRE as the process evolves into a deductive line of inquiry. Qualitative research can also take on a deductive logical process, working from theory to inform and guide the research questions in the similar way a quantitative study would unfold, the obvious difference being the technical tools employed. Yet, along this path, some technical tools and concepts from the quantitative tradition can be integrated into a qualitative methodology and research design (Kelly, 2009; O’Reilly & Parker, 2012). In essence, qualitative and quantitative methodology serves as the building block for the other. “The two paradigms of inquiry should not be seen as dichotomies, but as different positions on each side of a continuum where many different positions are possible” (Foss & Ellefsen, 2002, p. 246; also see Johnson & Onwuegbuzie, 2004B. Johnson & Onwuegbuzie, 2004; Onwuegbuzie, 2003). Along this circular continuum are phases in each respective methodological tradition where respective elements within these phases can be used to complement or enhance the opposite primary research process, be it primarily qualitative or quantitative.

Furthermore, the mixed-method literature supports taking one method from one methodological framework and importing that into the data analysis of another research tradition (Greene, 2007, p. 153). The integration of techniques ranges from randomly sampling to inform an ethnographic study, the development of qualitative effect sizes, to the use of Boolean algebra to analyze qualitative cases for comparison purposes (see Greene, 2007; Onwuegbuzie, 2003; Onwuegbuzie & Leech, 2007; Ragin, 2014). Additional strategies for combining elements for complementarity draws directly from the priority-sequence model of mixed-method procedures. These include using: (1) preliminary qualitative methods to help develop a larger quantitative study, (2) preliminary quantitative methods to contribute to a primarily qualitative study, (3) qualitative methods to complement a principally quantitative project, and (4) quantitative methods to complement a principally qualitative project (also see Creswell & Creswell, 2018; Morgan, 1998; Palinkas et al., 2011). RIRE could also occur in principally quantitative projects such as meta-analyses where during a reflexive stage of that process, coding for various interventions must be categorized to inform meaningful analyses, particularly when there is a low cell count for some interventions. During this stage, consulting associated text data may occur (e.g., descriptions of the intervention) or a form of axial coding where intervention categories are related to each other in order to aggregate and create broader categories that have internal consistency. A few other examples include using the principle of drawing representative samples in quantitative research to inform qualitative techniques to build response variance (Jacobs, 1996; Miles et al., 2014), which also aligns with Patton’s (1990) maximum variation principle. Additionally, using axial coding to establish categories for chi-square measures of association, or using quantitative analysis techniques such as sub-group and sensitivity analyses to inform additional ways for analyzing qualitative data.

Researchers must also take care to balance epistemological assumptions within the context of progressing a study and drawing inspiration from opposite, but complementary sides of the research cycle. Reflexivity can help with this. Researchers can converge reflexivity occurring in the background of their thought processes with element selection in the foreground, the action that translates to decision-making. This background-foreground dynamic produces RIRE. I highlight RIRE of quantitative research elements in an exploratory sequential design study that involved the reflexive integration of quantitative elements during data collection, analysis, and interpretation and discussion. I first begin with a review of literature on reflexive practices. I then review the example study to provide an illustrative example of how others can accomplish RIRE.

Literature Review

Qualitative Practices

A variety of reflexive practices within qualitative research exist. The key is in how reflexive practices are operationalized. These practices include repeated interviews with the same participants, prolonged engagement, member checking, triangulation, peer review, forming of a peer support network, keeping a diary or a research journal, or creating an audit trail of a researcher’s reasoning, judgment, and emotional reactions (Berger, 2015). Reflexive practices are also methodological tools. Reflexivity can be used as a tool for ethical research practice (Cain et al., 2019; Guillemin and Gillam, 2004) or to position oneself in general and in relation to prior scholarship (Dean, 2017). Reflexivity is also a transformative tool to invert subjectivity as a strong standing point for examining and presenting research (Dean, 2017; Harding, 1991; Pillow, 2003).

There are at least four validated reflexive strategies: self-reflexivity, reflexivity to recognize the “other,” reflexivity as truth, and reflexivity as transcendence (Pillow, 2003). Self-reflexivity informs the basis for reflexive integration of research elements (RIRE):

Self-reflexivity acknowledges the researcher’s role(s) in the construction of the research problem, the research setting, and research findings, and highlights the importance of researcher becoming consciously aware of these factors and thinking through the implications of these factors for his/her research. In this way, the problematics of doing fieldwork and representation are no longer viewed as incidental but can become an object of study themselves (p. 179).

RIRE operationalizes the “conscious awareness” of research factors so that they become integrated into the research project and not “incidental.” This facilitates “strong objectivity” (Harding, 1991). Finlay (2002) also offers a similar typology of reflexivity as tool for a variety of means: From introspection and positionality, to revealing motivations, and examining research processes. Reflexivity as a methodological tool and practice to do something in response to “reflexive moments” has included, but is not limited to, constructing arguments or making notations in an interview transcript to interpret silence (Morison & Macleod, 2014; Subramani, 2019). Swaminathan and Mulvihill (2019) frame reflexivity as a methodological tool complete with an explicit acronym, VISION, to facilitate what they term “place-reflexivity.” Place-reflexivity concerns how researchers can “reflect on the location and context and the wider environment” during fieldwork (p. 990). Similar to Subramani (2019), reflexive integration of research elements relies on reflexive moments to activate a deliberate decision-making process to draw inspiration from elements of qualitative or quantitative research into a mixed-method research project.

Quantitative Practices

Few quantitative studies address reflexivity. Yet some scholars recognize reflexivity as important for quantitative research. Quantitative researchers make decisions that inform the research process and work with data generated from specific contexts (Babones, 2016; Gorard, 2006). Johnson and Onwuegbuzie (2004) called attention to subjective decisions in quantitative research: decisions about what to study, development of data collection instruments, choosing tests or instruments for measurement, making score interpretations, and drawing conclusions and interpretations from the data to name a few. Furthermore, Ryan and Golden (2006) and Babones (2016) suggest quantitative research can benefit from embracing reflexivity. Babones advocates for an “interpretive quantitative methods” with reflexivity as an embedded principle. Ryan and Golden (2006) discuss engaging reflexivity during data collection to understand boundaries constructed between them and their study participants. They also reflect on time constraints and other demands. They note the necessity of reflecting on emotional costs of research and that “some research techniques may translate from qualitative to quantitative methods” (p. 1193). Ultimately, they advocate quantitative methods may need to establish protocols that allows for reflexive approaches.

Gorard (2006) calls for quantitative researchers to “take more notice” of judgments often mistaken for statistical significance and precision. The decisions, the judgments, and the models produced require transparency. Transparency encourages revealing subjective approaches; and “the use of open, plain but ultimately subjective judgment … [would] have the effect of making it easier…to adopt mixed methods approaches as routine...” (p. 78). Similarly, Lakew (2017) acknowledges statistical models represent the operationalization of choices. Lakew notes how the research context can shift expectations about generalizability. Furthermore, reflection plays a role in deepening the understanding of context and research realities. Whitaker and Akinson (2021) also call attention to the “reflexivity of methods” which includes the ways in which “quantitative methods [and] positivist epistemologies and the like reflexively constitute the objects of their scrutiny” (p. 14).

Mixed-Method Practices

Hall and Howard (2008) detail a synergistic approach to conducting mixed-method research based a set of core principles. The first principle suggests two or more options can interact so that the combined outcome is greater than the individual contribution of each option. Second, qualitative and quantitative research have equal value despite necessary fluctuations in the emphasis of each throughout the research process. Third, an approach is chosen in consideration of the other to appreciate differences. Fourth, a researcher should consider the skill involved in the particular research design. The process is iterative, nonlinear, and each dimension of quantitative and qualitative research “communicates” with each other through the core principles. This point of “communication” is where reflexive integration of elements can be advanced—at the precipice of some decision that needs to be made and that can harness the permeability between qualitative and quantitative research. Morse (2015) notes the complexity of conducting research leads to adding design components. The additions are not reflective of the original research design, but reflexivity. As mixed-method research unfolds, problems emerge shaping the need for components, thus (re)shaping the design. According to Morse, “This is what reflexivity is about: letting the program lead the design” (p. 14). Engaging in reflexivity facilitates the addition of more design components: “If the researcher identifies an interesting part of the data for which a supplemental measure would assist with clarifying, the researcher can add a supplemental component, a quan or qual at that time…” (p. 216). Morse’s work is probably the closest parallel to reflexive integration of research elements (RIRE).

Similarly, Sanscartier (2018) acknowledges mixed-method research is not a linear process, nor “neat” and is inherently a reflexive practice. The messiness of research requires awareness and adaptability. Sanscartier presents the “craft attitude” as a practice to facilitate this. Three practices guide researchers in using the craft attitude to navigate messiness. The first practice concerns knowledge created from criteria that specifies when, why, and how to collect data and integrate, and analyze data. Sanscartier refers to this as “context-independent knowledge (episteme).” The second practice concerns the application of technical knowledge to execute research; it is the “know-how.” Sanscartier refers to this as “context-dependent craft/art (techne). The third practice is about the ethics that guide research practice (phronesis). Phronesis involves “reflexivity inquiry regarding how we should act in certain contexts, and whether these actions are good in relation to (inter)personal and social contexts.” Dealing with mess requires prudence and “active, conscious reflection on situated behavior” and learning through experience (p. 6). RIRE builds from Sanscartier’s “craft attitude” in that it is another tool to “do something” about the awareness that arises during research.

The Example Study

The reflexive integration of research elements (RIRE) was used in an exploratory sequential design study. The study involved grounded theory and ethnographic techniques to examine street-level drug crimes in Baltimore City, Maryland, USA (Olaghere & Lum, 2018; also see Kraska, et al., 2010; Piza & Sytsma, 2016; St. Jean, 2007). The study involved systematic social observation (SSO) of archived Closed-Circuit Television (CCTV) footage of 100 adjudicated crime events that occurred between 2010 and 2011. The research questions were: what types of behavior routines lead to crimes occurring at high-crime places? What does the process of crime occurring look like at these places and are there any distinctive features? The study generated a classification of behaviors routinely observed in illicit drug transactions on the street (see Olaghere & Lum, 2018). Qualitative and quantitative data was generated using a systematic social observation protocol (SSO protocol) with closed-ended and open-ended answer options. The SSO protocol was piloted twice with a sample CCTV video not included in the study sample. A diversity of information was recorded including background information, temporal characteristics, environmental characteristics, individual actor characteristics, and external activities. The background section of the SSO protocol captured information about where the crime event occurred such as whether the location was a street or intersection, or within a commercial district. In the section covering temporal characteristics of the crime event, information captured included the time, date, and the geographic coordinates of each crime event. The environmental characteristics section included fixed items focusing on the nature and type of physical disorder such as litter and abandoned buildings, and infrastructural features such as the presence of lighting or signs (e.g., stop signs and no loitering signs). The last two sections of the protocol—individual actor characteristics and external activities—focused on behavior and movements of individual actors and activities external to the crime event, but happening either before or during the crime event. A majority of the items in these sections were fixed-item responses with the exception of open-ended fields. These fields allowed for narrative accounts and incorporating the ethnographic feature of “thick description” of the routines, behaviors, and interactions observed among drug sellers and buyers, which translated into multi-page narratives. Using the SSO protocol to systematically observe recorded crime events focused on behaviors leading up to, during, and after a drug transaction. Attention was given to systematically observing the individual actors involved in a drug transaction: the seller, the buyer, and the presence of a potential guardian, either formal (i.e., a police officer) and informal (i.e., a resident or other neighborhood stakeholder).

The SSO format also enabled integrating the quantitative element of reliable measurement. Specifically, intracoder reliability statistics were calculated to measure the consistency of observations to enhance the subjective observations of a single encoder. Additional ethnographic techniques used in the study included prolonged engagement and constant comparison to examine street-level drug transactions unfolding. The protocol was 23 pages long and was used to develop a relational database for direct data entry and record management. The database included 11 tables and a total of 304 fields across the tables. Encoding proceeded over the course of six-and-half months at a police substation with the Baltimore City Police Department, which accounted for a total of 2340 hours of observation, averaging 58.5 hours of observations per a five-day workweek. The average length of recorded events was 45 minutes per recording. Two separate journals were actively updated and maintained through the observation period. One journal was to capture information about crime events for which the SSO protocol did not include fields to capture. This journal elaborated on details about specific people, their behaviors, perceived demographic information, the timing of their comings and goings, and patterns identified between different observations as people interacted within their environment, but in unrelated places. Entries from this journal supplemented the SSO protocol and were entered into the relational database after each observation. The second journal was used to note salient impressions, themes, additional questions, or insights to research questions. This latter journal was akin to preliminary memoing and included themes that highlighted the humanity and striving of behaviors observed and challenged assumptions about the illegality of behaviors involved in street-level drug selling. This journal also included questions and impressions that draw parallels between street-level drug selling and everyday life routines and their innocuity and what is truly “criminal.”

The qualitative data was analyzed across two cycles of encoding involving provisional encoding (a priori and dramaturgical codes) and process encoding. Categories were constructed from themes identified in provisional and processing coding. They were created using the network builder in Atlas.ti, starting with grouping similar codes, linking them to subcodes, and then aggregating the subcodes into categories (Figure 1). Some categories included “adjusting” behaviors whereby sellers would shift their bodies or sidestepping, “(in)conspicuous adaptation” whereby sellers would “perform” or accomplish “normal” behavior to blend into the environment or stand out to engage in a transaction, and finally, “procedural,” which accounted for specific sellers working together taking ownership of different aspects of a transaction (e.g., walking up to a customer, looking out, and retrieval). Axial coding was completed using a coding paradigm (Strauss & Corbin, 1990). This involved assuming the perspective of the drug seller and specifying the consequences, strategies, and interactions within the crime events for each category. Analytic memos were written to expand upon the meaning and significance of observed behaviors, actions, routines, and relationships based on observed behavior routines leading to illicit drug transactions.

Figure 1.

Joint Display of Integration in Example Study. Note. This figure illustrates the sequence of a study with an exploratory sequential design involving grounded theory, and then expanded into quantitative testing of typologies developed during the qualitative phase of the study. The specific typologies are in the lower half of the figure and pointed to the data aggregation stage of qualitative analysis, the phase in which the typologies were developed. The specific activities of the quantitative portion of the study is further illustrated by the arrows from the QUANT box pointing to the right side of the lower half of the figure.

The study results focused on 74 out of the 100 crime events systematically observed and recorded. These 74 crime events represented a homogeneous group of crimes, drug crimes, and provided the most data from SSO. The remaining 26 crime events were a medley of observations for property and violent crime, but without enough data in each category to conduct meaningful analyses. Overall, the study concluded there are a series of micro routines that undergird drug crime events. These micro routines are behaviors and behavioral routines that occur before the commission of a drug crime. The micro routines specified were specific to drug sellers only. These routines included (in)conspicuous adaptation, cooperative selling, hierarchical facilitation, authorization cues, serial resetting, disguised dynamic, controlled aversion, and geoform. The fluid nature of qualitative data collection and analysis that allows back and forth comparisons between data collection and analysis can accommodate the integration of various research techniques to produce innovative methodologies and analyses suitable to the needs of a particular study. In the study, this reflexive approach involved oscillating between the data collected and additional analyses beyond the axial coding and memoing noted above. This reflexive process lead to additional analyses borrowed from the quantitative tradition of analysis. Specifically, we performed various measures of association to examine the convergence of the observed micro routines. We computed Cramer’s V and Fisher’s exact tests to illustrate the overlap and magnitude of the overlap across the five micro routines most prevalent in the drug crimes observed.

Philosophical Approach

While pragmatism was the basis for activating reflexivity, dialectical pluralism is the more appropriate paradigm in which to contextualize the example study. Johnson (2017) articulates dialectical pluralism as a metaparadigm that “[enables] people to continually interact with different ontologies, epistemologies, ethical principles/systems, disciplines, methodologies, and methods in order to produce useful wholes” (p. 158). And as Greene and Hall (2010) suggest, “even pragmatism needs to interact with other paradigms” (p. 170). Dialectical Pluralism counts critical reflexivity as a principle. A goal was to work within existing theoretical frameworks guiding the study. These frameworks noted linear modeling would not suffice in understanding crime as a process. Thus, as Gutterman et al. (2019) assert “researcher transparency regarding decision-making is critical” (p. 191). Reflexive integration was used to expand upon qualitative findings (i.e., test them inferentially) and to advance qualitatively-driven work within an evidence-base and field dominated by quantitative research and positivistic epistemology (see Lumsden & Winter, 2014; Freshwater & Fisher, 2015). This translated to reflexively incorporating quantitative elements to address the needs of building on qualitative findings and reflecting on how to address the dominant empirical standard of an established evidence-base. There were three phases in which reflexive integration of quantitative elements occurred in the example study. These phases were: Data collection, analysis, and interpretation and discussion.

RIRE Approach

The reflexive integration of research elements (RIRE) like reflexivity itself is an ongoing process. It is a tool to operationalize critical thinking or make implicit thoughts explicit. Three steps can aid in the use of RIRE. First, a researcher can activate reflexivity through a variety of maintenance strategies (e.g., member checking, audit trails, journaling, and peer debriefing) that is step one. Second, a research then can focus on insights established in step one (step 2). These insights may be questions, dilemmas, problems, or reflections. Collectively, these are reflexive moments. Third, do something about the reflexive moment(s) (step 3). Below I focus on the data collection and data analysis phases of the example project to show how RIRE occurred. A joint circular display illustrates the occurrence of reflexive moments and RIRE (Figure 2). I constructed the joint display as a circular display to reflect what Johnson et al. (2010) noted can be the circular influence of methods on paradigmatic thinking and knowledge generation.

Figure 2.

Reflective Moments and Reflexive Integration of Research Elements.

RIRE During Data Collection

While designing the SSO protocol and discussing it during peer debriefing, there were two competing insights (step 1). If the SSO protocol was mostly comprised of close-ended questions with limited answer options, I would run the risk of translating complex phenomena into reductive categories and limit descriptive information. Second, I also needed to remain grounded and observe organically without any structure, as close-ended questions would not capture the complexity of behaviors observed; context would be missing. The reflexive moment was the concern about building an insufficient dataset (step 2). This inspired me to do something and engage in problem-solving. I drew inspiration from the quantitative technique of power analysis, but adapted to the epistemological context of qualitative methods (step 3). The “parallels” in qualitative methods already existed: saturation and thick description. This ensured the completion of text fields would involve rich descriptions about what I was observing.

The issue of cognitive drift during encoding also came up during peer debriefing (step 1). The reflexive moment was the concern about meeting multiple benchmarks: the end of data collection, eventually establishing reliability, and building on prior research. I wanted to align the study with prior quantitative research, so the predominantly quantitative discipline in which I work would receive the eventual product well (step 2). These issues informed the decision to draw inspiration from the quantitative technique of interrater reliability as a quantitative element to address the issue of encoder drift over time (step 3). I was thinking ahead to plan for calculating reliability. However, this quantitative technique is typically used between multiple encoders. In the example study, there was a single encoder. This created a new issue (step 2). I re-engaged in problem solving, critically thinking about how to adapt. Still drawing inspiration from interrater reliability, I computed intracoder reliability (step 3) for the fixed-item responses in the protocol (Lipsey & Wilson, 2001). I also created an encoding book to guide the careful selection of answer options. The encoding book also included prompts to stimulate detailed written responses.

Figure 2 below illustrates the RIRE method during data collection. The joint circular display is a depiction of a research circle or cycle, moving in a clockwise direction, from deductive inquiry to inductive inquiry, and then coming full circle. The interior of the joint circular display shows the different qualitative and quantitative elements (i.e., thick description and interrater reliability). The perimeter consists of black arrows pointing clockwise. These black arrows represent two temporal aspects. First, the moment in which reflexive moments occurred (step 2) and where along the research circle the moment occurred. The arrows inside represent step 3 or drawing inspiration from a particular research approach. The arrows show direction, which indicates from where in the research circle, the inspiration to pull and adapt quantitative or quantitative elements, comes, and the destination of its application. The elements are also ordered temporally to indicate when certain elements are engaged during the research process. For example, power analysis would occur before any statistical analyses in a quantitative project. Likewise, thick description would occur during data collection and prior to analytical coding in a qualitative project. The matching shaded triangular areas also indicates step 3.

RIRE During Data Analysis

I converted findings from the qualitative data analysis into quantitative data. The qualitative findings consisted of a typology of drug selling activity. I knew the study would contribute to a largely quantitative knowledge base. I understood in order for the study to be well received, I needed consider whether the typology I constructed had any statistical significance. I needed to assess whether it was possible the findings could be a pattern representative in the broader universe of drug transactions (step 2). The reflexive moment here involved considering whether any attributes that comprised the typology overlapped or were associated with each other (step 2). An overlapping reflexive moment was about whether the qualitative findings would be convincing enough to the positive research community (step 2). Instead of limiting the results to a description of attributes and relationships between attributes and typologies, which would be enough under a qualitative approach, I thought about “what would be enough?” I did so, drawing inspiration from quantitative approaches, converting qualitative data into the appropriate level of quantitative data (e.g., nominal and ordinal). I also decided to assess for statistical significance. I drew from the purpose of inferential statistics to find statistical support for the qualitative data conversion. I decided to assess whether the typology was statistically representative of a broader universe of drug selling activity (step 3). I computed descriptive statistics to evaluate associations between select micro routines. Then I computed Fischer’s exact tests and Cramer’s V to examine statistical significance. The joint circular displays of how RIRE was used in the example study attempts to show how specific techniques from quantitative research can be reflexively integrated in a piecemeal fashion into a qualitatively-driven study (Figure 2).

The creativity of integration in mixed-methods research can be augmented with reflexivity, which helps reveal “unconscious motivations and implicit biases in the researcher’s approach” (Finlay, 2002, p. 225). Reflexivity also encourages ongoing researcher awareness “during the research process which aids in making visible the practice and construction of knowledge within research in order to produce more accurate analyses of our research” (Pillow, 2003, p. 178). Accordingly, the adoption of RIRE and reflexivity may enable a deeper understanding of phenomena and explicitly addressing the limitations of subjective and objective pursuits in the construction of knowledge (Gough, 2003; Jootun & McGhee, 2009; Primeau, 2003).

RIRE During Interpretation and Discussion

Acknowledging the anticipation of how the study would be received by peer reviewers and the broader field occurred during interpretation (see Dean, 2017; Robbins, 2007). Part of my biography, subjectivity, and desire to problem-solve was consciously injected into the research and in a careful way because of an awareness about what research is privileged as legitimate and worthy of publication and recognition (see Dotson, 2012; Dotson, 2014). The reflexive moment during this phase was recognizing the existence of a certain “standard” with which research should not only be performed, but the “objective” and “neutral” language that also needed to be reflected in interpreting results and the discussion of them (step 2). The subsequent action was to include a discussion of limitations from a quantitative perspective: selection bias, nonprobability sample, and the lack of a comparison site (step 3). This reflexive moment also extended to the development of the discussion section to engage reflexivity as it concerned the issue of powerlessness and perpetuating stereotypes and thoughtfully thinking about the implications of the research and its dissemination to the field (step 3). For example, there was a need to discuss the study implications in balance with the equivocal language of social science research (step 1). I also considered the ethical and social justice implications given that the research may be seen as done “on behalf of the powerful” and about the powerless and marginalized groups (step 2) (see Lumsden & Winter, 2014, p. 5; Freshwater & Fisher, 2015; Popa & Guillermin, 2017). What emerged as a result was the following statement featured in the discussion section (step 3):

Examples of bluntly applying interventions on all activity in a space without concern for crime routines have included zero tolerance arrests and blanket stop-question-and-frisk policies. Viewing these micro routines as a part of a longer crime script sequence may help law enforcement reasonably anticipate crime using actual cues related to crime rather than a person’s dress, race, ethnicity, or gender (Olaghere & Lum, 2018, p. 486, p. 486)

Discussion

In the example study, the systematic nature of systematic social observation and the defined categories sets a priori lens upon which to engage the natural world. Second, the ethnographic techniques of prolonged engagement in the field and the use of thick description in the SSO protocol, and the integration of measuring reliability of observations via inter- or intra-coder reliability highlights the functionality of integrating qualitative and qualitative elements. An added benefit was minimizing the work between translating qualitative observations into quantitative values, in the case of using SSO. The reflexive integration of research elements (RIRE), and primarily the integration of quantitative elements, was performed on the basis of responding to reflexive moments. At these moments, critical thinking was engaged to acknowledge and do something about reflexive moments. The need to justify, which can be the initial step in integrating elements (Greene, 2007), informed these reflexive moments. The justification to integrate quantitative elements into the qualitative phase of an exploratory sequential design required reflexivity. Reflexivity was required to recognize the concerns for the research and its viable advancement in a positivist evidence-base. In this regard, RIRE was a useful methodological tool. RIRE relies on many practices for maintaining reflexivity as outlined by Berger (2015), Breuer & Roth (2003), and many others. RIRE continues the tradition of scholars who have identified reflexivity as a tool or strategy, but also called for the clarification of its process (Dean, 2017; Guillemin & Gillam, 2004; Hall & Howard, 2008; Morse, 2015; Pillow, 2003; Sanscartier, 2018; Swaminathan & Mulvihill, 2019). RIRE can also begin with selecting a research methodology from which to work primarily and then using a variety of practices to activate reflexivity (e.g., journaling and peer debriefing). Activating and maintaining reflexivity in this way can stir reflexive moments. These reflexive moments can the benefit from RIRE to draw inspiration from the other methodology to enhance or complement the work undertaken within the primary research methodology selected (see Johnson and Onwuegbuzie, 2004; Palinkas et al., 2011). In effect, researchers will locate themselves within the research process epistemologically, focusing on following the process of the research tradition, and engage in reflexivity and draw inspiration from elements to adapt procedures accordingly.

The reflexive integration of research elements (RIRE) can also be an important methodological tool as researchers attempt to account for the influence of researcher bias, external influence, or prevailing disciplinary conventions on the production of information and knowledge. Lumsden and Winter (2014) note that in the era of evidence-based research, research must be conducted in a reflexive manner, particularly to recognize the role of power and the researcher need to engage in some form of information and knowledge creation (Pillow, 2003; Harding, 1991). Under this current paradigm of knowledge and information generation, researchers co-produce knowledge in collaboration with users and practitioners, some of whom are in positions of power who grant access to data, individuals, and fund research, and thus have a “vested interest in the results and application of the research” (Lumsden & Winter, 2014, p. 3; Popa & Guillermin, 2017). Reflexivity is crucial to provide a layer of “critical distance and engagement,” which Lumsden and Winter (2014) note, “ironically promotes subjectivity as a way of interrogating the un-interrogated hidden biases, conflicts of interest and assumptions of so-called objective scientific research…” that dominates some social science fields (p. 3).

RIRE reinforces the development of creative research approaches that feed into the discovery process of scientific research. Second, RIRE can help illuminate the step-by-step decision-making process in research generally and in mixed-method research. The latter being a well-documented challenge (Alexander et al., 2020; Bryman, 2006; Bustamante, 2019; Gutterman et al., 2019; Johnson et al., 2019). RIRE may also be a bridge that connects reflexivity operating unconsciously, the unconscious predilection for making decisions in research, and activating reflexivity to operate in the foreground as a central tool (see Biddle & Schafft, 2015; Cain et al., 2019; Hesse-Biber, 2010). This requires the conscious acknowledgement of reflexivity operating in the first place. RIRE provides a new approach for being intentional and critically aware while conducting mixed-method research, especially as we consider Sanscartier’s (2018) craft attitude and navigating the messiness of mixed-method research (also see Cheek et al., 2015; Morse, 2008; Morse, 2015). RIRE may also help facilitate replication of quantitative aspects of mixed-method studies, explanatory sequential designs, quantitatively driven mixed-method studies, or illuminate “midcourse diversions” from the planned process where the need for additional design components emerges (see Cheek et al., 2015, p. 756–757; Morse, 2015).

Limitations

The RIRE method discussed is not exhaustive, but a starting point. The RIRE method also does not address how researchers need to position themselves. I focus on the value of what RIRE can provide in advancing a research project. RIRE may produce what Pillow (2003) cautioned as “linear tellings” that center the researcher as knowing better and reflexivity as just a tool to get better data. Similarly, RIRE does not delve deeply into what Pierre Bourdieu referred to as “constant epistemological vigilance” as a researcher engages with technical instruments. The latter should not be a “scientific alibi for blind submission” to the former ((Bourdieu et al., 1991), as cited in Lakew, 2017, p. 234). RIRE also does not address important humanistic and ethical sensitivities during research and which reflexivity can help facilitate (Guillemin & Gillam, 2004). Achieving an equal status mixed-method study is an important feature under dialectical pluralism. While the example study strove for this, given the nature of the reflexive moments, the study was qualitatively driven. RIRE may also require extensive knowledge of different qualitative and quantitative methods. This may be burdensome, reveal knowledge gaps in methods, or require cross-training and/or collaborative research teams (Johnson, 2017). RIRE is not prescriptive but requires a willingness to reveal reasons and motivations for integrative practices.

This paper attempted to outline some basic steps in using RIRE as a tool to reach some objective of a research project. Whatever those objectives may be, RIRE can play a role as long as reflexivity is involved and informs the thoughtful integration of qualitative and/or quantitative elements. Overall, the benefit in harnessing the elements of qualitative and quantitative methodologies lies not in the additive nature of combining methods to increase validity, but integrating elements of one tradition into the other method to complement and add depth and scope of “exploring different aspects of the same problem” (Brannen, 1992, p. 16). The creativity of integration can be augmented with reflexivity, which not only helps “reveal unconscious motivations and implicit biases in the researcher’s approach” (Finlay, 2002, p. 225), but also encourages ongoing researcher awareness “during the research process which aids in making visible the practice and construction of knowledge within research in order produce more accurate analyses of our research” (Pillow, 2003, p. 178). Accordingly, reflexivity presents an opportunity to enable a deeper understanding of phenomena for both qualitative and quantitative research and explicitly addressing the limitations of subjective and objective pursuits in the construction of knowledge (Finlay, 2002; Gough, 2003; Jootun & McGhee, 2009; Primeau, 2003).

Future work in this area should attempt, discuss, and illustrate reflexive integration of qualitative elements into quantitative studies to highlight the full utility of reflexive integration, as reflexive integration should not be limited to qualitative research, but extended to quantitative research as well (see Cain et al., 2019). Indeed, Noaks and Wincup (2004) note that “most research projects in the social sciences are in a general sense multi-method because alongside the main method of choice, subsidiary techniques are used” (p. 9). The key objective to implement is reflexivity at every stage of the research process, coupled with a consideration of elements to incorporate. The goal to work toward is the production of deeper, nuanced knowledge about phenomena. Future work should also explore illustrating reflexive integration as it concerns reflexive decision-making under different paradigms, and among group research teams, particularly research teams with different skillsets and expertise. Additional inquiries should also reveal and discuss reflexive moments and subsequent decision-making as it concerns external influences in the research and publication process, namely epistemological conventions within a field, peer review processes, and publishing standards across a variety of disciplines.

Conclusion

RIRE was discussed and illustrated as a proposed methodological tool to incorporate reflexivity in a mixed-method research project. RIRE challenges the notion of scientific discovery germinating from a logic of discovery or proof, but instead from intentional practices to achieve a variety of research goals. Quantitative and qualitative research traditions are more complementary and connected than diametrically opposed and disconnected. The limitations of one serve as the building block of the other (Foss & Ellefsen, 2002; Johnson and Onwuegbuzie, 2004; Onwuegbuzie, 2003). RIRE illuminates and leverages subjective decision-making inherent in all research as a methodological tool. RIRE can be useful in helping researchers become aware of subjective and external influences that need to be addressed during the research process, deepen their contemplation of this awareness, and transform their awareness into a methodological tool by honing critical thinking skills on how to integrate said awareness and its influence into a productive, explicit tool of research. If practiced regularly, the application of RIRE can become a staple of engaging in scientific inquiry but illustrating the process as well. RIRE can translate the benefit of dynamic reflexivity into intentional practice:

It avoids versions of methods and techniques being held on to and applied at all costs, even if it is clear that a new or different pathway might exist to answer the questions and address data that have arisen during the course of the unfolding of the research. It allows the research question(s) and the researchers to drive the research and enables the researchers to respond to emerging findings and new paths to follow arising from the research [emphasis added] (Cheek et al., 2015, p. 760).

RIRE is an intentional practice of conscientiously problem-solving and doing so at an elemental level by consciously focusing on the process and not solely on the outcomes of research. RIRE can help researchers achieve this mindful practice and a practice that all researchers should engage. Reflexive integration of research elements may help further guard against what Burawoy (1998) called a “binary view of science,” and the subsequent shift from focusing on techniques of qualitative and quantitative research methods to focusing on their explication (p. 29) and allow for the emergence of a pragmatic view of knowledge generation (Brent & Kraska, 2012; Johnson & Onwuegbuzie, 2004B. Johnson & Onwuegbuzie, 2004) and that both quantitative and qualitative research are fundamentally reflexive.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ajima Olaghere

References

Alexander

Eppler

M.J.

Comi

(2020). Data integration: A real-time, participant-driven, and visually supported method. Journal of Mixed Method Research, 15(1), 1–27. https://doi.org/10.1177/1558689820902294

Babones

(2016). Interpretive quantitative methods for the social sciences. Sociology, 50(3), 453–469. https://doi.org/10.1177/0038038515583637

Berger

(2015). Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative research. Qualitative Research, 15(2), 219–234. https://doi.org/10.1177/1468794112468475

Biddle

Schafft

K. A.

(2015). Axiology and anomaly in the practice of mixed method work: Pragmatism, valuation, and the transformative paradigm. Journal of Mixed Method Research, 9(4), 320–334. https://doi.org/10.1177/1558689814533157

Bourdieu

Chamboredon

J.-C.

Passeron

J.-C.

(1991). The craft of sociology: Epistemological preliminaries. Walter de Gruyter.

Brannen

(1992). Mixing methods: Quantitative and qualitative research. Ashgate Publishing.

Brent

J. J.

Kraska

P. B.

(2012). Moving beyond our methodological default: A case for mixed methods. In Copes

(Ed.), Advancing qualitative methods in criminology and criminal justice (pp. 25–43). Routledge.

Breuer

Roth

Wolff-Michael

(2003). Subjectivity and reflexivity in the social sciences: Epistemic windows and methodical consequences. Forum: Qualitative Social Research, 4(2).

Bryman

(2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6(1), 97–113. https://doi.org/10.1177/1468794106058877

10.

Burawoy

(1998). The extended case method. Sociological Theory, 16(1), 4–33. https://doi.org/10.1111/0735-2751.00040

11.

Bustamante

(2019). TPACK and teachers of Spanish: Development of a theory-based join display in mixed methods research case study. Journal of Mixed Method Research, 13(2), 163–178. https://doi.org/10.1177/1558689817712119

12.

Cain

L. K.

MacDonald

A. L.

Coker

J. M.

Velsaco

J. C.

West

G. D.

(2019). Ethics and reflexivity in mixed methods research: An examination of current practices and a call for further discussion. International Journal of Multiple Research Approaches, 11(2), 144–155. https://doi.org/10.29034/ijmra.v11n2a2

13.

Cheek

Lipschitz

D. L.

Abrams

E. M.

Vago

D. R.

Nakamura

(2015). Dynamic reflexivity in action: An armchair walkthrough of a qualitatively driven mixed-method and multiple methods study of mindfulness training in schoolchildren. Qualitative Health Research, 25(6), 751–762. https://doi.org/10.1177/1049732315582022

14.

Creswell

J.W.

Creswell

J. D.

(2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage

15.

Dean

(2017). Doing reflexivity: An introduction. Policy Press

16.

Dotson

(2012). A cautionary tale: On limiting epistemic oppression. Frontiers, 33(1), 24–107. https://doi.org/10.5250/fronjwomestud.33.1.0024

17.

Dotson

(2014). Conceptualizing epistemic oppression. Social Epistemology: A Journal of Knowledge, Culture and Policy, 28(2), 115–138. https://doi.org/10.1080/02691728.2013.782585

18.

Finlay

(2002). Negotiating the swamp: The opportunity and challenge of reflexivity in research practice. Qualitative Research, 2(2), 209–230. https://doi.org/10.1177/146879410200200205

19.

Foss

Ellefsen

(2002). The value of combining qualitative and quantitative approaches in nursing research by means of method triangulation. Journal of Advanced Nursing, 40(2), 242–248. https://doi.org/10.1046/j.1365-2648.2002.02366.x

20.

Freshwater

Fisher

(2015). Mixed methods: Dissonance and values in research with marginalized groups. In Hesse-Biber

Johnson

(Eds.), The Oxford Handbook f multimethod and mixed method research inquiry (pp. 665–676). Oxford University Press

21.

Gorard

(2006). Towards a judgement‐based statistical analysis. British Journal of Sociology of Education, 27(1), 67–80. https://doi.org/10.1080/01425690500376663

22.

Gough

(2003). Deconstructing reflexivity.” Reflexivity: A practical guide for researchers in health and social sciences. In Finlay

Gough

(Eds.), Reflexivity: A Practical Guide for Researchers in Health and Social Sciences (pp. 21–35). John Wiley and Sons.

23.

Greene

J. C.

(2007). Mixed methods in social inquiry. Jossey-Bass.

24.

Greene

J. C.

Hall

J. N.

(2010). Dialectics in pragmatism: Being of consequence. In Tashakkori

Teddlie

(Eds.), Sage handbook of mixed methods in social and behavioral research (pp. 119–144). Sage Publications

25.

Guillemin

Gillam

(2004). Ethics, reflexivity, and “ethically important moments” in research. Qualitative Inquiry, 10(2), 261–280. https://doi.org/10.1177/1077800403262360

26.

Gutterman

T. C.

Babchuk

W. A.

Howell Smith

M. C.

Stevens

(2019). Contemporary approaches to mixed-methods-grounded theory research: A field-based analysis. Journal of Mixed Methods Research, 13(2), 179–195. https://doi.org/10.1177/1558689817710877

27.

Hall

Howard

(2008). A synergistic approach: Conducting mixed method research with typological and systemic design considerations. Journal of Mixed Methods Research, 2(3), 248–269. https://doi.org/10.1177/1558689808314622

28.

Harding

(1991). Whose science? Whose knowledge?: Thinking from women's lives. Cornell University Press.

29.

Hesse-Biber

S. N.

(2010). Feminist approaches to mixed methods research: Linking theory and praxis. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social and behavioral research (pp. 169–192). Sage.

30.

Jacobs

B. A.

(1996). Crack dealers’ apprehension avoidance techniques: A case of restrictive deterrence. Justice Quarterly, 13(3), 359–382. https://doi.org/10.1080/07418829600093011

31.

Johnson

R. B.

(2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156–173. https://doi.org/10.1177/1558689815607692

32.

Johnson

R. E.

Grove

A. L.

Clarke

(2019). Pillar integration process: A joint display technique to integrate data in mixed methods research. Journal of Mixed Methods Research, 13(3), 301–320. https://doi.org/10.1177/1558689817743108

33.

Johnson

R. B.

McGowan

M. W.

Turner

L. A.

(2010). Grounded theory in practice: Is it inherently a mixed method? Research in Schools, 17(2), 65–78

34.

Johnson

Onwuegbuzie

A. J.

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. https://doi.org/10.3102/0013189X033007014

35.

Jootun

McGhee

(2009). Reflexivity: Promoting rigour in qualitative research. Nursing Standard, 23(23), 42–46. https://doi.org/10.7748/ns2009.02.23.23.42.c6800

36.

Kelly

(2009). The role of theory in qualitative health research. Family Practice, 27(3), 285–290. https://doi.org/10.1093/fampra/cmp077

37.

Kraska

P. B.

Bussard

C. R.

Brent

J. J.

(2010). Trafficking in bodily perfection: Examining the late‐modern steroid marketplace and its criminalization. Justice Quarterly, 27(2), 159–185. https://doi.org/10.1080/07418820902814013

38.

Lakew

(2017). Statistical tales: Bringing in reflexivity to make sense of quantitative data. In Tosoni

Carpentier

Murru

M. F.

Kilborn

Kramp

Kunelius

R...

Pruulmann-Vengerfeldt

(Eds.), Present scenarios of media production and engagement (pp. 225–238). Edition Lumière.

39.

Lipsey

Mark

W Wilson

David

B (2001). Practical meta-analysis. Sage.

40.

Lumsden

Winter

(2014). Reflexivity in criminological research. In Lumsden

Winter

(Eds.), Reflexivity in criminological research: Experiences with the powerful and the powerless (pp. 1–19). Palgrave Macmillan. https://doi.org/10.1057/9781137379405_1

41.

Miles

M. B.

Huberman

A. M.

Saldaña

J. J.

(2014). Qualitative data analysis: A methods sourcebook. Sage Publications.

42.

Morgan

D. L.

(1998). Practical strategies for combining qualitative and quantitative methods: Applications to health research. Qualitative Health Research, 8(3), 362–376. https://doi.org/10.1177/104973239800800307

43.

Morison

Macleod

(2014). When veiled silences speak: reflexivity, trouble and repair as methodological tools for interpreting the unspoken in discourse-based data. Qualitative Research, 14(6), 694–711. https://doi.org/10.1177/1468794113488129

44.

Morse

J. M.

(2015). Issues in qualitatively-driven mixed-method designs: Walking through a mixed-method project. In Hesse-Biber

S. N.

Johnson

R. B.

(Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (206–222). Oxford University Press.

45.

Noaks

Wincup

(2004). Criminological research: Understanding qualitative methods. Sage Publications

46.

Olaghere

Lum

(2018). Classifying “Micro” routine activities of street-level drug transactions. Journal of Research in Crime and Delinquency, 55(4), 466–492. https://doi.org/10.1177/0022427818760103

47.

Onwuegbuzie

A. J.

(2003). Effect sizes in qualitative research: A prolegomenon. Quality & Quantity, 37(4), 393–409. https://doi.org/10.1023/A:1027379223537

48.

Onwuegbuzie

A. J.

Leech

N. L.

(2007). A call for qualitative power analyses. Quality & Quantity, 41(1), 105–121. https://doi.org/10.1007/s11135-005-1098-1

49.

O’Reilly

Parker

(2012). ‘Unsatisfactory saturation’: A critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research, 12(2), 190–197. https://doi.org/10.1177/1468794112446106

50.

Palinkas

L. A.

Aarons

G. A.

Horwitz

Chamberlain

Hurlburt

Landsverk

(2011). Mixed method designs in implementation research. Adm Policy Mental Health, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z

51.

Patton

M. Q.

(1990). Qualitative evaluation and research methods. Sage Publications.

52.

Pillow

(2003). Confession, catharsis, or cure? Rethinking the uses of reflexivity as methodological power in qualitative research. International Journal of Qualitative Studies in Education, 16(2), 175–196. https://doi.org/10.1080/0951839032000060635

53.

Piza

E. L.

Sytsma

V. A.

(2016). Exploring the defensive actions of drug sellers in open-air markets: A systematic social observation. Journal of Research in Crime and Delinquency, 53(1), 36–65. https://doi.org/10.1177/0022427815592451

54.

Popa

Guillermin

(2017). Reflexive methodological pluralism: The case of environmental valuation. Journal of Mixed Method Research, 11(1), 19–35. https://doi.org/10.1177/1558689815610250

55.

Popa

Guillermin

Dedeurwaerdere

(2015). A pragmatist approach to transdisciplinarity in sustainability research: From complex systems theory to reflexive science. Futures, 65(1), 45-56. https://doi.org/10.1016/j.futures.2014.02.002.

56.

Primeau

L. A.

(2003). Reflections on self in qualitative research: Stories of family. The American Journal of Occupational Therapy, 57(1), 9–16. https://doi.org/10.5014/ajot.57.1.9

57.

Ragin

C. C.

(2014). The comparative method: Moving beyond qualitative and quantitative strategies. The University of California Press

58.

Robbins

Derek

(2007). Sociology as reflexive science: On Bourdieu’s project. Theory, Culture & Society, 24(5), 77–98. DOI:10.1177/0263276407081284.

59.

Ryan

Golden

(2006). ‘Tick the box please’: A reflexive approach to doing quantitative social research. Sociology, 40(6), 1191–1200. https://doi.org/10.1177/0038038506072287

60.

Sanscartier

M. D.

(2018). The craft attitude: Navigating mess in mixed method research. Journal of Mixed Methods Research, 14(1), 1–16. https://doi.org/10.1177/1558689818816248

61.

St. Jean

P. K. B.

(2007). Pockets of crime: Broken windows, collective efficacy, and the criminal point of view. University of Chicago Press

62.

Strauss

Corbin

(1990). Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications

63.

Subramani

(2019). Practising reflexivity: Ethics, methodology and theory construction. Methodological Innovations, 12(2), 1–11. https://doi.org/10.1177/2059799119863276

64.

Swaminathan

Mulvihill

T. M.

(2019). Learning to ‘site-see’: place-reflexivity as a methodological tool for qualitative researchers. International Journal of Qualitative Studies in Education, 32(8), 982–997. https://doi.org/10.1080/09518398.2019.1635280

65.

Whitaker

E. M.

Akinson

(2021). Reflexivity in social research. Springer Nature

Reflexive Integration of Research Elements in Mixed-Method Research

Abstract

Keywords

Introduction

Reflexive Integration of Research Elements

RIRE helps operationalize subjective thoughts (reflexive moments) into decision-making.

Literature Review

Qualitative Practices

Quantitative Practices

Mixed-Method Practices

The Example Study

Philosophical Approach

RIRE Approach

RIRE During Data Collection

RIRE During Data Analysis

RIRE During Interpretation and Discussion

Discussion

Limitations

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

References