Graphical Retrieval and Analysis of Temporal Information Systems (GRATIS): An Integrative Mixed Methodology and Open-Access Software to Analyze the (Non-)Linear Chronological Evolution of Information Embedded in Textual/Qualitative Data

Abstract

Like a video that reveals much more than a single photo, the incorporation of time to the analysis of qualitative evidence promotes contextualized understandings and allows research participants and readers to interactively review the processes and rationale that researchers followed to craft their findings and conclusions. However, mixed methods and qualitative methodologies available today forfeit the nuances gained by analyzing the chronological/temporal evolution of processes. We contribute to mixed methods research by introducing graphical retrieval and analysis of temporal information systems (GRATIS), a methodology (and open-access software) designed to visualize and analyze the time-based richness embedded in all qualitative/textual data. GRATIS employs dynamic network visualizations and data science mining/retrieval tools to combat the assumption that longitudinal studies require large timespans. We showcase how all qualitatively- or machine-learning-coded textual data may be analyzed with no extra feature engineering (i.e., data cleaning or preparation), rendering fully integrative/interactive outputs that strengthen the transparency of our findings and conclusions and open the “analytic black box” that characterizes most of mixed methods and qualitative studies to date. GRATIS contributes to democratizing data science by removing financial and computer programming barriers to benefit from data science applications. All data and software to replicate the analyses are provided with this submission.

Introduction

Most mixed methods and qualitative methodologies available today capture a snapshot of the world, thus forfeiting the nuances gained by being able to observe and analyze the chronological evolution of processes, events, and information that shape qualitative research findings and conclusions. From this perspective, the incorporation of time as part of our integrative mixed methods frameworks may enhance our perceptions of the social world and strengthen our level of understanding of the processes and the consequences of change or continuity in social settings (Adam, 1990; Neale, 2015, 2019). Accordingly, guided by the idea that a short video can reveal much more than a single photo, and considering that all qualitative datasets inherently contain temporal elements that can be used to recreate the specific contexts ¹ during which participants provided their insights and contributions, the development of methodologies capable of mapping these dynamic processes, represents a relevant contribution to fully integrative mixed methods research. This relevance is justified given that, despite the availability of temporal elements and the analytic benefits that the incorporation of time represents to illuminate processes, trends, and dynamics in our studies (Artale et al., 2007; Neale, 2015), these temporal elements remain very underutilized or foregone in qualitative and mixed methods analyses.

The main reasons behind this underuse of temporal elements in qualitative datasets are (a) the complexity of accounting for the multiplicity of elements evolving simultaneously; (b) the assumption that the incorporation of time as part of the analyses requires long periods of time, when in reality, all qualitative and textual datasets have an inherent time component (Adam, 1990; Neale, 2019) that can be mapped as we discuss below; and (c) the lack of user-friendly software tools capable of effortlessly visualizing the emergence and decay of these chronological elements in qualitative data. Regarding the latter point, as we further illustrate in our “Background” section below, even the most recent advancements in tools for visualizing qualitative data, such as those depicted in the computer-assisted qualitative data analysis software (CAQDAS) Networking Project, do not yet handle temporal information as part of their methodologies. Specifically, although NVivo, ATLAS.ti, Leximancer, MAXqda, and QDA Miner do offer network or relational visualization capabilities (like the capabilities employed in our framework), such software tools stop at presenting an aggregated depiction of all connections, instead of mapping the evolution of information, processes, and knowledge construction over time. Our proposed methodology, and its accompanying software, were designed to capture the evolution of these temporal elements following participants’ discourses linearly, that is, chronologically as their spoke or constructed their dialogues, or non-linearly, which allows their discourses to evolve organically and to even “jump” back and forth in the spatial-temporal narrative continuum of their speeches or episodic memories, as we further elaborate below.

In this study, we offer an integrative mixed methodology that relies on data science, dynamic network visualization, and mining/retrieval tools typically employed in quantitative research, to provide a means to analyze the dynamic processes that shape qualitative and mixed methods research findings and conclusions. Accordingly, based on the fact that all qualitative datasets inherently contain temporal elements ² that can be used (a) to recreate the specific context during which participants provided the insights that eventually informed or will inform research findings and conclusions and (b) to capture their experiences through time (Adam, 1990; Neale, 2019), the purpose of this paper is to contribute to the mixed methods research literature by introducing graphical retrieval and analysis of temporal information systems (GRATIS), an analytic framework and research methodology, and its corresponding open-access software application to visualize and analyze the chronological evolution of processes, events, and information taking place in qualitative and mixed methods research settings. GRATIS’s reliance on data science tools also serves to combat the assumption that longitudinal qualitative studies require large timespans. We showcase how all qualitatively- or machine-learning-coded textual data even those with short timespans (i.e., essays, interviews, and social media posts) may be analyzed with no extra data preparation (i.e., encoding or feature engineering) rendering fully integrative/interactive HTML outputs. Note also that GRATIS may be applied to all labeled or coded data, even if this labeling process did not consider this time element as part of the coding strategy. This means then, that previously coded qualitative data may be re-analyzed with GRATIS, which may lead to new insights and even to different and more nuanced takeaways.

This proposed analytic framework responds to the integration challenge wherein “ $\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}+\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}=\mathrm{m}\mathrm{o}\mathrm{r}\mathrm{e}\hspace{0.17em}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{n}\hspace{0.17em}\mathrm{t}\mathrm{h}\mathrm{e}\hspace{0.17em}\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l}\hspace{0.17em}\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}$ ” (Fetters & Freshwater, 2015a, p. 116). By relying on the analytic power and flexibility of temporal dynamic visualizations and data retrieval, we aim to take data integration to a “new level…, where visual synergy gains enable a ‘value-added’ [and fully integrated] research outcome” (Alexander et al., 2019, p. 34). Procedurally, as shown in Figure 1 (and fully described in the “Procedural Diagram” section), this analytic framework follows an exploratory sequential analytic design (Fetters & Freshwater, 2015b, p. 209) where the qualitative phase is followed by the quantitative one, and it concludes with the integration of the understandings gained through both phases.

Figure 1.

Procedural Diagram.

The paper is structured as follows. We present a background section that reviews visualization strategies currently available in commercial software to highlight structure in qualitatively coded data relying on static network modeling techniques. This background discussion enabled us to start introducing the rationale and data formats employed by GRATIS. Subsequently, we present the conceptual foundation of GRATIS: temporal information systems (TIS) (Artale et al., 2007) and discuss how TIS is operationalized via temporal network analysis methods (Butts et al., 2020; Kolaczyk & Csárdi, 2014; Mitchell, 2006; Wasserman & Faust, 1994). The integration of these concepts and methods enables GRATIS to map and visualize the simultaneous evolution of information provided by research participants, which in turn may strengthen our substantive understanding of the topic of study. In the next section titled “Procedural Diagram, Coding Approaches, and Data Formats,” we bring together all concepts and methods discussed so far to form a coherent and comprehensive depiction of GRATIS. Subsequently, we introduce the software application (freely available for Mac and Windows platforms) that implements GRATIS and conclude with a discussion of next steps and of integration and contribution to mixed methods research.

Background

As briefly discussed above, even the newest advancements in qualitative data visualization that aim to highlight the structure embedded in coded or labeled textual data only present an aggregated snapshot of such structure, rather than the chronological (linear or non-linear ³ ) progression of information and events as they evolved throughout the knowledge generation process. An important takeaway of this opening paragraph is that coding or labeling textual data is fundamental for these visualizations to work. Accordingly, although a comprehensive description of coding data, either qualitatively or quantitatively, is beyond the scope of this study (due to space limitations), before presenting how currently available software tools are visualizing these coded texts, let us describe how this coding process should look like using a short illustration.

Coding or Labeling Textual Data

A code, in qualitative research is a word, a descriptor, or a label that “symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a portion of language-based or visual data” (Saldaña, 2013, p. 3). Language-based or visual data imply that a code may be added to textual data as well as to video or photographs. For clarity, in this study, we focus on coding textual data, but to the extent that visual data may be translated into texts, we could assign codes or labels to such data and apply GRATIS to the resulting codes. Based on this description, coding may be understood as the process of fracturing textual data (Strauss, 1987) into labeled categories or codes to ease their comparison and aid our sense-making efforts (Miles & Huberman, 1994) when we put these data back together for the purpose of pattern detection, explanation, or theorization (Charmaz, 2014; Elliott, 2018, González Canché, 2019, 2022c, 2022d, 2023a, 2023b, 2023c; Saldaña, 2013).

With this brief description in mind, note that Saldaña (2013) documented 32 qualitative coding methods, which, due to space limitations, cannot be covered in this study. Nonetheless, to be as clear and as precise as possible, we next describe the underlying process researchers would need to follow to label or code textual datasets following qualitative and quantitative (via supervised and unsupervised machine learning) methodologies. In short, although the following examples are general or broad by design, they seek to set a common ground for our subsequent presentation of GRATIS.

General Coding Processes

Figure 1 shows the methodological steps upon which GRATIS relies. Although the coding of text data is required for GRATIS to work, such a coding process is not implemented by the software accompanying this study. ⁴ However, note that GRATIS handles qualitatively and quantitatively coded texts, as we illustrate in the following paragraphs.

As indicated in Figure 1, after data collection has concluded, and texts are retrieved or transcribed from interviews, essays, social media posts, websites, or open-ended responses, for example, we may proceed with the coding or labeling process of such databases. For the sake of clarity, let us assume that our textual database is configured by texts retrieved from three participants (A, B, and C), as shown in Table 1. Assume also that each participant wrote a short essay on their reasons to enroll in a graduate data science seminar—we provide access to the complete version of this dataset as part of our distribution of the GRATIS software (i.e., the software comes preloaded with this dataset). However, the examples discussed in this study are based on a subset of this essay dataset containing only participants A, B, and C. This database version is available at González Canché (2023c) and is provided to ease the full replication of our analyses.

Table 1.

Short Essays on the Reasons to Enroll in a Data Science Seminar.

Qualitative Coding Example

Following Crossley (2010) and Saldaña (2013), these texts may be represented (and coded) as shown in Table 1. In this representation, we see that the essays have been coded or labeled so that a short word or sentence is used to summarize the highlighted text (this highlighted text that has the same color as its code or label is also referred to as a quote). At this point of the coding process, we have that each quote is assigned (or linked) to a label and such a label and quote may be both linked or connected to the person (or unit) providing that text. These connections or links may be expressed in a tabular form as shown in Table 2, ⁵ wherein each actor is linked to the code provided as part of their discourse (speech or response), and each code is linked to this quote. Accordingly, then, each actor can be linked to each of their quotes and the emergence of these quotes may be retrieved based on the linear chronological evolution (i.e., moving from top to bottom of each participants’ discourse , which is actually captured by their text transcripts--i.e., from top to bottom of their transcripts). If, instead of following this linear evolution, researchers have a theoretically or empirically relevant reason to re-order these codes (and their respective quotes) following a predetermined analytic rule or strategy, like showing all instances across participants that deal with theme A first, and after exhausting this theme proceed to present the second set of codes or labels associated with theme B (see Braun & Clarke, 2006), for example, then they should rearrange or re-order these codes, and quotes, accordingly. ⁶

Table 2.

Lists of Actor to Codes/Labels Connections and Codes Textual Content.

t	Provider	Code/Label	Quote/Text Content
1	Actor A	Utility_Applicability	I am focusing on an institutional analysis of nonprofit intermediary college access programs using neo-institutional and organizational theories—which is the organizational data I am using for the SSNA exercises
2	Actor A	Research_&_Dissertation	I’d like to use SSNA to study the access organization’s partnerships and reach within the U.S.
1	Actor B	Utility_Applicability	I hope to become more comfortable with quantitative methods for research and to become more proficient using R and other statistical software packages
2	Actor B	Skillset_Building	I also aim to develop a better understanding of the possible methods for modeling social networks
3	Actor B	Utility_Applicability	Potential uses of these models for demonstrating impacts and results to third-party stakeholders such as campus administrators, legislators, and the general public
4	Actor B	Research_&_Dissertation	I believe that this course will aid the research that I am conducting for my assistantship
1	Actor C	Skillset_Building	I am taking this course because I am interested in learning how to analyze networks using R
2	Actor C	Utility_Applicability	I would like to learn how to expedite network analysis when using more than one network via R code
3	Actor C	Skillset_Building	I hope to be able to learn the code and stat packages necessary in order to be able to perform basic descriptive network analyses in R (centrality, density, etc), cluster analyses in R, analyzing/visualizing both two mode and one mode networks in R, key player analysis in R, and ERGMs in R

Unsupervised Quantitative Coding Example

Figure 2 shows the output resulting from applying unsupervised text classification following natural language processing and machine learning methods via Latent Dirichlet allocation and Topic Modeling (see González Canché, 2022a, 2022b, 2023b, and 2023a for software access and examples to apply these machine-learned codes or labels). This unsupervised text classification was applied to these same three essays contained in Table 1. In this classification method, analysts may decide to mirror line-by-line coding (Saldaña, 2013) where the classification will be applied to each of the sentences configuring our texts, or to paragraphs or to complete texts (see González Canché, 2022a and 2023b). Concisely, after thousands of learning iterations, each sentence will be classified into n number of latent codes (or labels) and the learning process links each of these codes to the text that is most likely to represent that latent code based on word to code and text to code probabilistic distributions (Griffiths & Steyvers, 2004). Note that in the output shown in Figure 2, the latent codes are simply referred to as V1, V2, and V3. This is different from the qualitative codes shown in Tables 1 and 2. This means that, with this machine learned output, researchers now have to read the content (quotes) and assign a meaningful label, descriptor, or code (Saldaña, 2013) to such latent codes (González Canché, 2022a, 2022b, 2023b, 2023c). A recent description of this labeling process can be found in Chang et al. (2021), González Canché (2022a, 2022b, 2023b, 2023c), Ho et al. (2021), and Poth et al. (2021). ⁷ Finally, note that the ordering of these latent codes and texts followed the linear evolution based on each participant’s speeches or discourses. As before, we may retain this order or update it based on another criterion.

Figure 2.

Output of Code Detection Based on Machine-learning Text Classification.

Before moving to the next subsection, let us share that the software Leximancer conducts unsupervised text classification, but instead of labeling the codes as V1, V2, …, Vn, it names those codes (referred to as themes in this software) with the most frequent word in a given sentence (or sentences, based on users’ analytic decisions). This “theme” labeling process can be qualitatively analyzed, similar to the output shown in Figure 2, where the most representative texts of latent codes are displayed. ⁸

Supervised Quantitative Coding Example

Supervised text classification implies the conjunction of both, qualitative and quantitative coding or classification methods. That is, as illustrated in software like QDA Miner and its required WordStat add-on module, we may load previously coded texts (likely following qualitative coding methods) to software like QDA Miner, and then apply a machine learning supervised classification to compare text to code probabilistic distributions and classify those texts (i.e., quotes) as those more likely to represent a given pre-determined label/code. When these pre-determined codes are assigned a meaningful label, then the supervised classification stops at this point, that is, researchers do not need to go back and read those learned classes or codes to assign a meaningful label for that label would have been created a priory. Specifically, as shown in the QDA Miner User Guide (p. 242), ⁹ each text (quote) will be assigned to one of the pre-coded labels based on the similarity of word distributions of such a text. In the example shown in Figure 3, the “Economy” label was assigned to the penultimate row in this figure and the bold and blue font indicates that there is a 93.7% chance that this text “correctly” belongs to the label “Economy.” By this same token, since all previous texts were identified as belonging to the “Politic” code with high probabilities, the probabilities of them belonging to the code “Economy” in the column “Economy” is zero or quite close to zero.

Figure 3.

Example of Supervised Learning in QDA Miner.

After this supervised classification is completed, we, once more, will have access to the quote, the label (or code), and the participant who provided such a quote/text. This information can then be exported to build a dataset like the one shown in Table 2, and with that information, we can apply the GRATIS procedures that we will discuss below, also following linear or non-linear temporal elements.

Visualization Tools for Coded Data

Before presenting the GRATIS procedures, let us describe how currently available software tools use the elements shown in Table 2 to render network-based visualizations.

Atlas.ti Networks

The networks tool in Atlas.ti allows users to manually build a visualization with codes and quotes elements, like those contained in Table 2 and Figure 2. In this building process, the connecting lines may be added a meaning by the researcher—that is, a connecting line itself is added a qualifying label. During this process, researchers may also add a “document” to the resulting visual. Considering that documents typically represent individuals (i.e., research participants and interviewees), the inclusion of a document represents adding the participants that researchers are considering as an important actor or unit in the resulting network. Finally, researchers may also incorporate research memos (notes) and groups (or families or codes) in this visualization. ¹⁰ The resulting visualization serves to highlight what researchers considered valuable in explaining their findings. This is different to the subsequent visualization approaches we present, which aim to highlight the resulting structure emerging from connections like those discussed above, where actors are connected to codes and codes are connected to quotes (see González Canché, 2022d and 2019). In Atlas.ti, the resulting network may highlight these actor to code to quote relationships, but when elements are added to the Atlas.ti plotting canvas, their connections are not automatically added; instead, it is the researcher who “manually” includes them.

NVivo Project Maps

Similar to Atlas.ti’s networks, users may select codes, files, memos, and themes to include in the proposed network diagram or representation. A salient difference between Atlas.ti and NVivo is that in the latter, the connections are retrieved from the relationships observed in the data. To illustrate this process, let us build from our example shown in Table 2. Assume we add code “Utility_Applicability” to the visualization canvas. If only this element is added, no connections are drawn. However, if Actor A is added to this map, NVivo will then automatically draw a connecting line between the “Utility_Applicability” code and Actor A. On the other hand, in Atlas.ti, the researcher would need to draw that connecting line manually. In short, when elements are connected, that is, when codes are assigned to a theme or family (using NVivo language), and these elements are added to the map, their connecting lines are automatically added to such a visual map. This implies that, if all codes and actors are added to the map, the result will be an aggregated depiction of the complete set of connections, but no temporal elements are captured in the resulting network aggregated visualization. ¹¹

Leximancer Concept Maps

Concept maps in Leximancer contain the themes (which we have been referring to as labels or codes) that this software detected via unsupervised text classification along with the most highly correlated words associated with those themes; and this concept map is represented in a network form. The size of the nodes (represented as circles or bubbles) is based on their frequency in the transcripts or texts analyzed. Different from Atlas.ti’s and NVivo’s visuals, these concepts maps are automatically generated as a result of the unsupervised text classification and represent an aggregated description of the most frequent connections of themes and configuring words ¹² co-occurring in these texts or quotes. Based on the unsupervised nature of this classification, all sentences across the corpora (all collections of texts) are labeled into one of the resulting themes (i.e., codes as we have described them above). Finally, similar to all our previous discussion of Atlas.ti and NVivo, there are no temporal elements being mapped in these outputs. Moreover, these “themes,” their quotes, and the units that provided those codes may be exported in a tabular form that resembles Table 2 and Figure 2.

MAXQDA MAXMaps

MAXMaps follows the same rationale as NVivo’s project maps. Researchers may select codes, documents, memos and quotes and categories (themes, families) that should be added to the plotting canvas. Once connected elements are added, lines among such connected elements are also automatically drawn. Nonetheless, users may also link objects that are not connected following the rationale shown in Table 2 (i.e., $\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{s}\to \mathrm{c}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{s}\to \mathrm{q}\mathrm{u}\mathrm{o}\mathrm{t}\mathrm{e}\mathrm{s}$ ). That is, researchers may idiosyncratically craft their visualizations as an analytic tool based on their analytic needs or goals. More to the point, the connecting lines may be formatted to allow researchers to highlight attributes they consider relevant.

Another visualization tool available in MAXQDA is called code co-occurrence. This tool highlights the number of documents (interview transcripts or essays, for example) that contain the same pair of codes. Researchers may decide to display these frequencies as another strategy to highlight structure in qualitative data this is also an analytic strategy implemented in GRATIS as detailed below. Again, the end result may be an aggregated depiction of all selected elements to be visualized in a network diagram. ¹³

QDA Miner’s Link Analysis

Finally, the last CAQDAS we have found to integrate network visualization tools is QDA Miner. Link analysis allows the visualization of connections between codes and between codes and cases (i.e., research participants). The goal of this visualization is to highlight patterns and structures of co-occurrences. The resulting visual applies a layout that places codes that are more densely connected together and codes that are less prevalent in the working space as peripheral—that is a far from the center of the plot as possible, which is one of the properties implemented by GRATIS. The closeness of codes based on prevalence of co-occurrence may also be represented in a dendrogram and also in a proximity plot. These visualizations are possible given that a version of Table 2, as shown in this study, may be transformed into a matrix, and with this matrix, traditional classification and correlation approaches may be implemented. Indeed, in Table 3, we discuss this resulting transformation, as implemented by GRATIS. Finally, once more note that in the QDA Miner Link Analysis, no temporal information is considered when creating these visuals.

Table 3.

Aggregated Matrix Representation of Participants’ Contributions.

	Utility_Applicability	Research_&_Dissertation	Skillset_Building
Actor A	1	1	0
Actor B	2	1	1
Actor C	1	0	2

Network Analysis of Qualitative Data (NAQD)

A special mention in these visualization tools corresponds to network analysis of qualitative data (NAQD). This is a special mention because the software that implements NAQD (see González Canché, 2022d, 2019) was designed to (a) highlight the structure embedded in coded textual data by plotting all the resulting connections and (b) statistically test via quadratic assignment procedures (under the family of random permutation tests, see Rubin & González Canché, 2019) whether certain groups are more concerned about certain topics than others. Like GRATIS, NAQD has not integrated text coding or labeling capabilities, but researchers may rely on machine learning procedures to code these texts free of charge (see González Canché, 2022a, 2022b, 2023b, and 2023c) or may rely on a Microsoft Word Macro that will serve as CAQDAS to label these texts in a similar fashion as NVivo or Atlas.ti, for example. ¹⁴ Once those labels are created, qualitatively or quantitatively via machine driven text classification, researchers may apply NAQD—and GRATIS, as further discuss below.

This brief review of currently available software that employs network analyses principles to visualize relationships, serves to highlight the contribution of GRATIS to these visualization capabilities by incorporating temporal attributes into the resulting maps.

Why to Rely on Network Visualizations?

When we move from participants’ qualitative contributions to data frames, or to matrices, and then to sociograms (or the network representations of matrices), we gain access to a comprehensive vantage point of analysis (Crossley, 2010). Network analyses enable researchers to highlight active or central topics via measures of influence, as well as areas of overlap or consensus, or even areas where disagreements are more prevalent among research participants (similar to what Alexander et al., 2019 referred to as hot zones in the visual replay methodology discussed below). These visualization techniques also allow researchers to identify peripheral conversations or points of discussion (i.e., cold zones). These are the reasons why network analyses have become prevalent in CAQDAS as tools that allow for a comprehensive depiction of the entire web of relationships, thus going beyond the mere quantification and visualization of qualitatively coded content (Crossley, 2010, González Canché, 2019, 2022d).

A common theme observed in our review, and arguably the main limitation of the current application of network analysis of qualitative data, however, is the aggregated nature of the resulting visualizations. The final outputs depict the summation of participants’ contributions (i.e., actor to code relationships) that took place in a given qualitative research setting, while completely omitting the temporal (linear or non-linear) evolution of these contributions and the textual content of each of such contributions. This is an important limitation because, although the chronological evolution of participants’ reasons may provide further insights about their priorities and interests, this information is currently forfeited in available CAQDAS.

To be more specific about how currently available CAQDAS aggregates network analyses, let us observe Table 2. In that table, the chronological information is captured with a column t, where $t=1$ represents the first contribution of each actor as shown in Table 1, $t=2$ , their second contribution, and so on. Although conceptually straightforward, the modeling of these evolving relationships requires dynamic network representations as we discuss below.

Current approaches forego temporal information by simply counting the number of times an actor mentioned a given code. Let us illustrate this using Actor B’s coded essay in Table 2

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{B}\stackrel{2}{{\displaystyle \to }}\mathrm{U}\mathrm{t}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}{\mathrm{y}}_{\_}\mathrm{A}\mathrm{p}\mathrm{p}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y},$

(1a)

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{B}\stackrel{1}{{\displaystyle \to }}\mathrm{S}\mathrm{k}\mathrm{i}\mathrm{l}\mathrm{l}\mathrm{s}\mathrm{e}\mathrm{t}\_\mathrm{B}\mathrm{u}\mathrm{i}\mathrm{l}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{g},$

(1b)

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{B}\stackrel{1}{{\displaystyle \to }}\mathrm{R}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{h}\_\&\_\mathrm{D}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}$

(1c)

where the arrows indicate that Actor B provided those codes. The number on top of the arrow counts how many times Actor B provided those codes throughout her/his essay. In this case, Actor B provided two pieces of text (quotes) classified or coded as Utility_Applicability. The other two codes provided by this actor were Skillset_Building and Research_&_Dissertation with a frequency of one in each instance. This depiction, although useful in summarizing code relevance at the actor level, fails to include temporal and (con)textual components, such as the order in which Actors provided these codes (and the textual content of these codes) throughout their essays. Since this order of emergence in participants’ discourses (and their content) may indicate participants’ priorities, for example, aggregated representations forego the nuances gained by analyzing the chronological evolution of information and processes, which inherently limit the extent of our understandings, even though that information exists in all coded textual data and can be retrieved and mapped.

In continuing with our discussion, note that when all the connections represented in Table 2 are taken together and aggregated, they may be transformed into a weighted and directed matrix representation contained in Table 3. Table 3 is a weighted and directional matrix because the cells may take values greater than 1, with higher values indicating a higher frequency/relevance of a given code for a given participant in the network. The directionality indicates that actors (in the rows of the matrix) are sending connections to the columns.

Finally, one can plot these relationships in a sociogram form (plot of connections) to capture a final snapshot of all these relationships as shown in Figure 4. The sociogram shown in Figure 4 enables the observation of actors’ and codes’/labels’ relevance (Crossley, 2010). Actors A and C were placed at the periphery (i.e., corners) of the network, whereas Actor B was at the center of the plot based on her heightened contribution level in this small example. In terms of code relevance, we see that Utility_Applicability was the most central (or at least the most frequent) reason for these participants’ decisions to enroll in this seminar. Again, although these features are important, this depiction alone falls short in capturing time-based processes and in retrieving the meaning (i.e., quote or text as discussed in our coding section above) of each $actor\to code$ relationship as they evolved.

Figure 4.

Aggregated Network Representation.

Moving Beyond Aggregated Depictions

Alexander et al. (2019) introduced visual replay methodology (VRM), a graphic approach to analyze processes “by integrating the accuracy of visually elicited quantitative counts of contributions [codes or labels as discussed above] and the thoughtfulness of qualitative reflections” (Alexander et al., 2019, p. 36). This focus on processes and dynamics allows researchers to observe and analyze the contextualized situations that shape qualitative research findings and conclusions (Alexander et al., 2019).

It is the integration of time into the analytic process that allows researchers relying on GRATIS to address questions of “how did the contributions of participants evolve?” and “what codes or labels did they prioritize in the evolution of their discourses, speeches, essays, or social media posts?” By being able to visualize the evolution of information, we can also observe the clustering of topics/issues/subject areas or “hot zones” (Alexander, et al., 2019), which capture trends around which the majority of participants’ contributions converged. Similarly, we can also capture “cold regions,” where the opposite was true—that is, by observing contributions that were provided by a few or one participant in a given time.

In addition to enabling the observation of the evolution of information or knowledge as an output to be analyzed by researchers, the insights gained with GRATIS via replay and recording of time-evolving visual textual data may become interactively intersubjective (Alexander et al., p. 46). That is, researchers may offer participants the opportunity to interpret whether the researcher or research team accurately captured their individual insights. Moreover, these participants may also be offered the opportunity to observe how their insights collectively compare to/with those provided by other research participants—that is, interactive reviewability (Alexander et al., 2019). This process may help both empower research participants and assess the quality of the analyses and perhaps more importantly, the transparency and accuracy of researchers’ interpretations.

Procedurally, the implementation of GRATIS requires the iterative creation of multiple matrices, as opposed to one aggregated matrix, as shown in Table 3. Specifically, this iterative process proceeds as follows: in going back to Table 2, we can observe that the first contributions of Actors A, and B, were Utility_Applicability, whereas for Actor C, it was Skillset_Building. To account for their temporal elements, these relationships take the forms

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{A}\to \mathrm{U}\mathrm{t}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}\_\mathrm{A}\mathrm{p}\mathrm{p}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}{\mathrm{y}}_1$

(2a)

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{B}\to \mathrm{U}\mathrm{t}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}\_\mathrm{A}\mathrm{p}\mathrm{p}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}{\mathrm{y}}_1$

(2b)

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{C}\to \mathrm{S}\mathrm{k}\mathrm{i}\mathrm{l}\mathrm{l}\mathrm{s}\mathrm{e}\mathrm{t}\_\mathrm{B}\mathrm{u}\mathrm{i}\mathrm{l}\mathrm{d}\mathrm{i}\mathrm{n}{\mathrm{g}}_1$

(2c)

In this case, subscript 1 at the end of the code represents the chronological appearance of this code or label in the actor’s discourse. For example, subscript 3 in the next equation

$\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{C}\to \mathrm{S}\mathrm{k}\mathrm{i}\mathrm{l}\mathrm{l}\mathrm{s}\mathrm{e}\mathrm{t}\_\mathrm{B}\mathrm{u}\mathrm{i}\mathrm{l}\mathrm{d}\mathrm{i}\mathrm{n}{\mathrm{g}}_3$

(3)

indicates that for Actor C, Skillset_Building also represented her/his third contribution (hence the value 3 in column t in Table 2). From a dynamic perspective, there will be Nth number of matrices to be plotted as sociograms, where N is the highest number of contributions made by a given participant or participants, if two or more had the same number of contributions. In going back to the matrix representation of these connections throughout time, we will have the following matrix for the first contribution.

Time 1	Utility_Applicability	Skillset_Building	Research_&_Dissertation
Actor A	1	0	0
Actor B	1	0	0
Actor C	0	1	0

The major difference of this matrix with respect to the final aggregated version discussed above (see Table 3) is that this is a binary matrix, where there is only presence or absence of connections at time t. Accordingly, note that, since no one provided information labeled as Research_&_Dissertation at time 1, this code (column) has only zeroes.

The second contribution is represented as follows.

Time 2	Utility_Applicability	Skillset_Building	Research_&_Dissertation
Actor A	0	0	1
Actor B	0	1	0
Actor C	1	0	0

The matrix generated at time 3 shows that, as of this time, Actor A no longer made any contributions to the network.

Time 3	Utility_Applicability	Skillset_Building	Research_&_Dissertation
Actor A	0	0	0
Actor B	1	0	0
Actor C	0	1	0

Finally, by the fourth and last time observed in this small example, only Actor B remained active in the system/network.

Time 4	Utility_Applicability	Skillset_Building	Research_&_Dissertation
Actor A	0	0	0
Actor B	0	0	1
Actor C	0	0	0

It is through these temporally disaggregated matrices that GRATIS enables observing the emergence and decay of these chronological contributions in research findings. Moreover, the reliance on data mining/retrieval and visualization using interactive HTML outputs enables using the quote (i.e., textual content) of each actor to code relationship ¹⁵ as an attribute of that relationship, so that researchers may retrieve this content with a single click on the line connecting actors to their codes at time t. For example, Figure 5 shows the GRATIS output at time 1 resulting from clicking on each of the participants’ connections, which matches our original data. Note that although GRATIS’s main contribution is the incorporation of time, it also automatically generates an aggregated depiction of the network, similar to currently available CAQDAS, given its remarkable summary properties.

Figure 5.

GRATIS Network Representation of First Contribution.

Having discussed how GRATIS contributes to currently available visualization and analytic practices, the next section discusses TIS and network analyses as the conceptual and methodological engines fueling GRATIS. Subsequently, we describe the software application and will discuss how GRATIS may be applied relying on non-linear temporal evolution of discourses.

Temporal Information Systems and Network Modeling

As discussed in the introduction, GRATIS, as an analytic framework, is founded on the notion that observing and analyzing the evolution of trends, dynamics, and changes in our studies through the lens of time (Artale et al., 2007), may uniquely enhance our perceptions of the social world and strengthen our level of understandings of such social events (Adam, 1990; Neale, 2015, 2019). Despite these analytic benefits, most currently available data models and database systems are designed to capture a snapshot of the world, which results in analyses that provide limited and partial views of actual circumstances, thus falling short in capturing the dynamics of the world or how it evolves as time passes (Artale et al., 2007, p. 6). Though these shortcomings can impact both qualitative and qualitative studies, the following discussion focuses on qualitative and mixed methods research.

Temporal Information Systems

Although all qualitative datasets contain hundreds or thousands of temporally contextualized pieces of information (Neale, 2015, 2019) and these temporal elements are fundamental to contextualizing and informing research findings and conclusions, qualitative analyses rarely convey a clear depiction of the temporal or social context when ideas emerged or were exchanged. To formally incorporate time-based information, GRATIS relies on temporal information systems (TIS) (Artale et al., 2007) as its guiding framework.

TIS treats the recording and easy retrieval of knowledge over time as offering the possibility of depicting the emergence, maintenance, and decay of information and ideas as provided by research participants. The first step for capturing these social dynamics consists of systematically recording the evolution of events as they emerged (Chomicki & Toman, 1998; Theodoulidis, et al., 1991). In this respect, a salient attribute of GRATIS as an analytic framework and software is that most, if not all, qualitatively coded or machine learning categorized datasets can be analyzed with GRATIS with minimal or no extra data cleaning. That is, as depicted in our background section, GRATIS simply requires each participant’s contributions to be stored following the chronological emergence of information that they provided, as shown in Table 2 and Figure 2.

Linear and Non-Linear Global Timestamps

Our focus on the chronological emergence of events, allows GRATIS to standardize the evolution of information across all participants under global timestamps (Artale et al., 2007; Jensen et al., 1998) rather than at specific time points that are unique to each recorded event and may obstruct compatibility of simultaneous analyses. For example, if we aim to observe how the evolution of information took place across 40 individual interviews and we rely on actual time points (rather than global timestamps, as proposed by GRATIS), it would be impossible to map the simultaneous generation of information unless the 40 interviews were conducted simultaneously by 40 interviewers in different spaces. ¹⁶ GRATIS overcomes this challenge by assigning global timestamps under a chronological approach. Here events are recorded under the umbrella of “first, second, third” and so on as part of an emergence of contributions (see Table 2 under column t). ¹⁷ In addition to ensuring compatibility, as discussed next, this approach allows for the detection of general trends (or hot zones, as depicted by Alexander et al., 2019), while also allowing for a focus on specific cases or actors, hence meeting the three modeling requirements of temporal information systems: orthogonality, reducibility, and compatibility (Artale et al., 2007, p. 13).

In addition to this linear chronological coding timespan approach, which follows the “top-to-bottom” paragraph order of participants’ interviews or essays, for example (as depicted in Tables 1 and 2), researchers may build a non-linear storytelling timespan. The latter focuses on the time period covered by our participants’ storytelling wherein the analytic focus is placed on their experiences and episodic memories. If, for example, we want to understand the strategies that adjunct faculty members have used to save for retirement, the coding timespan may cover each faculty’s trajectories in the academe, which may range from a few months to decades of seniority and involves the development of non-linear chronologically based themes focused on participants’ experiences (Braun & Clarke, 2006). In the following lines, we elaborate further on this important topic.

Non-Linear Temporal Dimensions in Qualitative Research

Temporal dimensions in qualitative research are not constrained to linearity in the form time 0, time 1, …, time n. Instead, these temporal dimensions depend on the organic evolution of the discourse(s) as expressed by our research participants. Guided by complex systems thinking (Maroulis et al., 2010; Mitchell, 2006), when discourses are operationalized as a network, they become dynamic, non-linear, and flexible. That is, what our participants shared regarding “time X,” may have prompted them to share more information regarding “time $X-n$ ,” or “time $X+n$ .” Here, X is a point in time, and $n$ represents time units before or after such point in time took place. The $n$ time units may be hours, days, years, or decades, for example, before or after a given point in “time X.” For instance, let us assume that a participant shared with us the following:

“I finished college in 1997, who knew that 10 years later, in 2007, I was going to start my graduate coursework in a different country, and that these graduate school years were going to be extremely challenging in many aspects of my life. This experience was in high contrast to my middle school dreams to study abroad. The real world did not match my expectations.”

Here, this participant offered information initially situated in 1997, but then there was a jump in time 10 years into the future (time $X+n$ ), when more information about graduate education was offered. Also note that in this same example, our research participant jumped back to her high school years, which following this no-linear rationale may be expressed as “time $X-n$ .”

As analysts interested in capturing evolution of information and/or events, we could do so by respecting or following participants’ speech as they shared it with us (i.e., verbatim from a discourse generation point of view), or we could try to capture the emergence of their speech following their non-linear storytelling. This non-linear time-coding scheme is more challenging/complex but may result in a perhaps more accurate timeline depiction of their experiences. If a temporally linear depiction suffices, then our examples shown in Tables 1 and 2 and Figure 2, are enough to capture our participants’ discourses evolution. However, if we instead seek to offer an analysis of discourse evolution that follows a more nuanced description of storytellings or events, we must rearrange the labels based on how we perceived that our participants’ stories evolved. Specifically, going back to our example regarding “time X” discourses that prompted “time $X-n$ ” stories, we could then present “time $X-n$ ” information and after exhausting this part of our participant’s discourse, move to presenting coded information pertaining to “time X” and then “time $X+n$ ” as we illustrate in Table 4.

Table 4.

Example of Non-linear Storytelling Evolutions.

ID	Time	Information
Actor 1	X-n, 1	Label of X-n, 1
Actor 1	X-n, 2	Label of X-n, 2
Actor 1	X-n, 3	Label of X-n, 3
…	…	…
Actor 1	X, 1	Label of X, 1
Actor 1	X, 2	Label of X, 2
…	…	…
Actor 1	X + n, 1	Label of Xn, 1
…	…	…
Actor n	X-n	Label of X+n, 1

When this particular re-arrangement shown in Table 4 is conducted, the input or assessment of our participants to our rearrangement becomes more important than ever, for we would have to ask them if our interpretations of their non-linear timelines captured their experiences. Here, the notion of interactive reviewability, where participants may be asked to interact with the resulting evolution of their experiences is even more relevant than in the linear or “top-to-bottom” depictions to validate our analytic decisions. If our proposed resulting timeline is inaccurate from our participants’ perspectives, we (researchers) should then rearrange the labels until they are satisfied with the temporality to be plotted by GRATIS.

With these linear and non-linear temporalities in mind, let us formally discuss the three modeling requirements of TIS: orthogonality, reducibility, and compatibility (Artale et al., 2007).

Orthogonality

Orthogonality (Artale et al., 2007) refers to the independent specification and retrieval of classes, relationships, and attributes. In GRATIS, classes are represented by actors and their qualitatively generated (or machine-learning identified) codes or labels. These classes are related across time as shown in equations (2) and (3) above; and each of these classes may be assigned attributes or characteristics (i.e., personal and non-personal attributes in the case of human actors and network-based attributes in the case of codes, as further explained in the Network Modeling section below). The property of capturing relationships across classes at different points in time, allows GRATIS to meet the reducibility requirement of TIS.

Reducibility

Reducibility (Artale et al., 2007) specifies that we should be able to completely rebuild the temporal evolution of a database (i.e., reproduce, replay), at both the individual and group levels, with each of the snapshots captured in the relationships $acto{r}_i$ and $cod{e}_j$ at time $t$ , thus allowing us to go forward or backward in the evolution or devolution of information. It is this possibility of rebuilding the dataset that we are referring in GRATIS as reproducibility, rather than reducibility. This is so because although snapshots may be taken when analyzing the resulting time-evolving networks, users (which include researchers, research participants, and readers) are not constrained to observing only snapshots, instead they may access the entire evolution of the temporal database being analyzed. In this respect, as described in the software application, GRATIS offers play, forward, rewind functionalities () as well as speed control () to accomplish this re(pro)ducibility requirement of TIS.

GRATIS also allows the depiction of actors’ activities with different “lifespans.” This is naturally captured by the number of codes retrieved from each participant’s discourses. For example, in Table 2, we can see that the essay written by Actor A resulted in two codes, whereas the essays written by Actors B and C resulted in four and three codes, respectively. The temporal representation of the evolution of these actors to codes relationships shows that after the third global timestamp, only Actor B continued to provide information in the system and, in this case, this relationship was classified as “Research_&_Dissertation.” This activity or contribution length is referred to as existence span by Artale et al. (2007).

Compatibility

The third requirement specified by Artale et al. (2007) is compatibility or the capability of accounting for, preserving, and retrieving “at each instance of time” nontemporal elements that may help contextualize the information being provided over time (p. 13). In GRATIS, this condition is met with the inclusion of personal, nonpersonal, and network analysis attributes computed as part of this integrative framework. Specifically, in our presentation of the GRATIS software application, we will describe the strategies that researchers may follow to include personal (e.g., age and ethnicity) and non-personal (e.g., employment status and college degree indicator) attributes of our participants that may help further contextualize their contributions—including the “tone” used when providing information such as emotions, for example. We programmed GRATIS to allow these attributes to remain constant over their existence span or evolve over time. These attributes, an example of which we show in Table 4, can be added to our resulting visualizations with a single click in the GRATIS user interface. In addition to these personal and non-personal attributes, we also provide an influence index that is automatically calculated by GRATIS—that is researchers do not need to add this index as an attribute. Notably, this influence index is measured for both actors and codes and is retrieved via network analyses, as described in the next section. More to the point, note also that the size of the actors and codes will vary based on another measure of network influence (betweenness centrality) that, as discussed below, captures the connecting role that actors and codes have in the system (Wasserman & Faust, 1994). This betweenness centrality is also automatically generated by GRATIS.

Another analytic feature related to compatibility is also depicted in Figure 5, where we show an example of the first contribution of all participants shown in Table 2. In this case, by clicking on the links connecting each participant with their codes, we can observe the relevance of that code for their discourses. For example, Actor C mentioned “Skillset_Building” twice and this code represented 66.7% of her/his contributions. As further described next, these pieces of information were derived from another social network analysis concept referred to as degree centrality.

In sum, GRATIS meets the three modeling requirements of TIS (Artale et al., 2007), and these requirements are met by relying on integrating qualitatively or computer-assisted coded data with the dynamic network analysis methods (Butts et al., 2020) that we present next.

Network Modeling

Network analysis and modeling are particularly useful ways to “deal with complex systems in the real world” (Mitchell, 2006, p. 1199) such as those we are modeling with TIS. In the context of GRATIS, network modeling is particularly useful in helping researchers make sense of multiple moving parts that evolve or devolve over time while meeting the three modeling requirements of TIS outlined above.

Operationally, network thinking emphasizes relations among specific events, including connections among individuals and their actions or among broader categories, labels, or codes, as defined by researchers (Maroulis et al., 2010; Maxwell, 2019). Formally, networks are a collection of potentially interactive units (Kolaczyk & Csárdi, 2014; Mitchell, 2006; Wasserman & Faust, 1994). These units are typically referred to as nodes or vertices (e.g., actors, participants, or entities that participants may interact with or be ascribed to), and the connections resulting from their interactions are referred to as edges or links (Wasserman & Faust, 1994). When the units configuring a given network are of the same class and hierarchy (e.g., students interacting with other students) they form a one-mode network. When the units configuring the network are different (e.g., professors ascribed to their employing universities or ascribed to a particular set of beliefs) they form two-mode networks.

GRATIS relies on two-mode networks given our focus on the analysis of the evolution of codes (information, or first mode) provided by our research participants (the units of analysis, or second mode). Following Crossley (2010), although network analyses can be focused on individuals within the network, we can also observe groups of participants who clustered around certain topics or viewpoints during their interviews (referred to as “hot zones” above, see Alexander et al., 2019).

Highlighting Structure

GRATIS follows graph theory (Biggs et al., 1986), dynamic temporal network analyses principles (Butts et al., 2020), and global linear and non-linear timestamps (Artale et al., 2007; Jensen et al., 1998), to capture the time-based emergence of information simultaneously, even though each participant provided these pieces of information individually, at different times and places, and with different time or existence spans. The integration of these three methodological views enables the simultaneous depiction of all contributions taking place in the same global timestamp across all participants regardless of our temporal linear or non-linear decisions. Specifically, as depicted in Figure 5, all first contributions of the participants are shown concurrently at time 1 (i.e., the first global timestamp), regardless of both: their actual location in the dataset (i.e., rows 1, 3, and 7 for Actors A, B, and C as illustrated in Table 2), the actual time and place when/where they wrote these essays, and their “storytelling timespans.”

Contextualization

As briefly described above, GRATIS allows the addition of attributes at the node (i.e., actor or code) and edge (links) levels. At the node level, we can add personal and non-personal attributes that may remain constant over the entirety of the data collection time or that may have changed during this same timeframe. For example, when visualizing coded interviews, coders may add a “tone” attribute (e.g., to depict whether the actor was surprised, angry, or sad when providing us with specific information, more on this below) to further contextualize the message of the code over and above its textual content. In addition to “tones,” we may also capture participants’ personal and non-personal attributes, all of which were programmed in GRATIS to change or not depending on our data sources and analytic decisions. For example, let us assume that we embarked in a multiyear longitudinal study, wherein we may have observed that during some parts of their storytelling some actors were married, but at some points they changed their marital status to separated or divorced. Similarly, we could have also seen that a participant did not have a college degree (time $X-n$ in Table 4, for example), but that, at some point in time, such a participant mentioned to be enrolled in graduate school (time $X+n$ also in Table 4).

Procedurally, these personal and non-personal attributes can be captured with two extra columns in our actor→code dataset as presented in Table 5 (further described in the Data Formats section below) and added to our GRATIS plots with a single click, as shown in Figures 4 and 5. Note, however, that GRATIS does not require long timespans to capture changes in respondents’ contextualizing characteristics, which is exemplified with the “tone” attribute in Table 5 that can be added to each response in a qualitative interview.

Table 5.

Example of Attributes.

Provider	Code	Quote	Personal Attributes	Non-Personal Attributes
Actor A	Code J	Text of code J	Married: No <br>	Employed: Yes <br>
			Sex: Woman <br>	College Degree: Yes
			Ethnicity: “LatinX”	Tone: Calmed
…	…	…	…	…
Actor B	Code F	Text of code F	Married: Yes <br>	Employed: Yes <br>
			Sex: Woman <br>	College Degree: Yes <br>
			Ethnicity: “Black”	Tone: Upset
…	…	…	…	…
Actor C	Code K	Text of code K	Married: No <br>	Employed: Yes <br>
			Sex: Woman <br>	College Degree: No <br>
			Ethnicity: “White”	Tone: Calmed
…	…	…	…	…
Actor C	Code L	Text of code L	Married: Yes <br>	Employed: Yes <br>
			Sex: Woman <br>	College Degree: Yes <br>
			Ethnicity: “White”	Tone: Sad

See complete dataset used in the examples shown in Figures 4 and 5, and A1 6 here https://cutt.ly/N846wVl.

Finally, GRATIS was programmed to automatically compute both the actors’ and the codes’ attributes based on network analysis measures as described in the “Detecting Actors and Codes Relevance” subsection below.

With respect to attributes of the connections, in addition to the “Edge weight,” “Pct Edge weight,” described below, and text content or “quote” described above, the color of the connection shows whether the code just emerged or was the same as the one observed in the previous time. For instance, in Figure 5, the green lines indicate that these connections just emerged as the first contribution across all participants. However, when the visualization is advanced to the next event (see Figure 6), these lines will turn blue, indicating they were present in the previous time, and the new connections will be represented in green. If a participant offered information that was coded under the same code shown at time 1, that participant will have only one connecting line, and it will be dark blue. But if that participant instead offered information that was coded with a different label, that participant will have two connections, with the new one shown in green and the previous one shown in blue. As depicted in our user interface section, by default, GRATIS shows the two most recent contributions over time, but the interface can be adjusted to show up to the 10 most recent contributions ¹⁸ (). The recommended selection of two contributions is particularly useful with large systems or networks, where network visibility may be compromised given the abundance of actors and codes. Moreover, the observation of the two most recent connections enables users to understand what the previous most recent contribution of each actor was up to time $t$ , thus enhancing the contextualization of the evolution of information.

Figure 6.

Time 2 Events.

Detecting Actors and Codes Relevance

An important contribution of GRATIS to detect structure in our coded/labeled dataset, consists of its automatic detection of actors and codes relevance/influence based on social network analysis measures (Freeman, 1978). Specifically, GRATIS computes three measures of centrality that capture movement and structure in the network through actors’ and codes’ positions and paths or connections (Borgatti, 2006). These three centrality measures are degree, betweenness, and eigenvector, as we described next.

Degree Centrality

This form of centrality consists of measuring the number and frequency of connections that a given actor has in the network (Borgatti, 2006). In terms of number of connections, GRATIS computes degree centrality by counting the number of times that an actor provided information that was eventually classified as a code—similar to the aggregated connections depicted in equations (1a), (1b), and (1c) discussed in the background section. That means that the information “No. of contributions” displayed when clicking on an actor or code in the HTML output will show the total number of codes provided by that actor, and the number of times a given code appeared in the network. Specifically, in going back to Figure 5, the information displayed for Actor C indicates that this actor contributed a total of three codes, two of which were classified as “Skillset_Building.” Similarly, when clicking on a code in this HTML output, we can see the number of times a given code was contributed across time by all actors. In the case of the code “Skillset_Building,” the number of times that textual information across all essays was labeled using this code was 3, irrespective of who provided this information. This implies that, degree centrality, as a counting device at the code level per se, does not reflect the relevance of that code to a given actor in particular. To address this limitation, and as noted above, we programmed GRATIS to also include code relevance or frequency per actor by clicking on the line(s) connecting each actor with her/his codes. In the case of Actor C in Figure 5, the line connecting this actor with the code “Skillset_Building” reflects that this participant provided information labeled under this code with a frequency of two times (under “Edge weight”) and that this code represented 66.7% of her/his total number of contributions (under “Pct Edge weight”).

Before moving to the next centrality measure, let us note that although the actual content of the code (i.e., its textual information contained in the column called “Quote/Text Content” in Table 2) changes at each point in time in the GRATIS plot, these degree centrality measures do not—thus meeting the compatibility requirement of TIS (Artale et al., 2007). Rather, these degree centrality measures are individual estimates of their aggregate values throughout each participant’s contributions to the network. These estimates then allow us to further contextualize the relevance of each code for each actor when clicking on their connecting line, and the relevance of the actor or code for the network, when clicking on that actor or code.

Betweenness Centrality

This measure goes beyond frequency counts. Instead, it detects the extent to which a given actor or code falls in between other actors or codes (Wasserman & Faust, 1994) thus effectively serving as a communication bridge in the system to ease the flow of information. In other words, betweenness centrality highlights both the participants who served as bridges of information and the codes that connected the individuals in this network. GRATIS uses betweenness centrality as an attribute to add size to each actor or code. To the extent that the size of a node (actor or code) grows at a given point in time, this indicates that such an actor or code is serving as a bridge of information. Accordingly, the software has standardized this measure to change as time progresses. Based on this dynamic change, researchers can observe which actors and codes were the most influential by serving as bridges of information at each point in time. Going back to Figure 5, note that the biggest (in size) code at time 1 was “Utility_Applicability” for it connected two of the three actors in the system (Actors A and B). At the second event (see Figure 6), this code connected all three participants. Figure 6 also shows the different connections’ colors described earlier.

Eigenvector Centrality

This measure appears as “Influence index” in a dialog box when an actor or code is clicked in the GRATIS HTML output (see Figures 4 and 5). Eigenvector centrality measures the overall influence of a given code or participant as a function of the relevance of its connections. Its influence index is standardized to range between 0 and 1, with 1 indicating the maximum level of influence, and highlights individuals or codes that are central as a function of being connected with individuals and or codes that are actively connected in the network (Borgatti, 2006; Freeman, 1978). Hence, an actor is central in eigenvector if such an actor provided codes or established connections with other actors who were also active or central in the network (influence begets influence).

Given that, mathematically, the eigenvector centrality of a unit $i$ is proportional to the sum of centralities of the units $j$ to whom unit $i$ is connected (Borgatti, 2006; Freeman, 1978), the level of influence of each actor and/or code may serve to further contextualize their contributions and strengthen our understandings. For example, note that Actor B was the most active contributor in the network with four codes in total—Table 2. Two of these codes were labeled as “Utility_Applicability.” With these pieces of information, we can infer that actors connected with the code that was shared the most by the most active actor will increase their relevance or influence in the network. In this virtuous cycle, since Actor B was the most active actor and shared the most shared code, when clicking on this actor, we will note that she has the highest influence index (see Figure 4). Similarly, code “Utility_Applicability” also shows this highest value (Figure 4). In the case of Actor A, she had the lowest value in eigenvector centrality because she had the lowest number of contributions and only one of them was labeled as “Utility_Applicability.” Following eigenvector centrality notions, if both contributions of Actor A were labeled under “Utility_Applicability,” her influence index would have increased as well, even if her total number of contributions remained unchanged.

These TIS concepts and dynamic network analyses methods allow GRATIS to provide free rides and inference support, which according to Shimojima (1996) is the property of being over-specific in representing information, being self-consistent in information content, and generating inferences as a result of following notational rules (see also Green et al., 2006) such as those we presented in equations (1a) to (3) above. From this view, in GRATIS, these free rides and inference support are achieved with the integration of TIS and dynamic network principles.

Procedural Diagram, Coding Approaches, and Data Formats

Figure 1 contains the procedural diagram guiding GRATIS. Overall, we can see that GRATIS follows an exploratory sequential analytic design (Fetters & Freshwater, 2015b) by moving from qualitative to quantitative phases and culminating with their integration, which we describe in the discussion section below. However, as part of these processes, also note that there are some analytic steps that although fundamental for GRATIS are not covered or implemented by our software application. These analytic steps include data collection (e.g., interviews, essays, ethnographic observations, and social media posts), data preparation (e.g., transcriptions, cleaning, and data formatting), data coding (via qualitative or machine-learning-based approaches, as discussed above), and a well-grounded initial understanding and clear conceptual grasp of the meanings of these codes. This latter point means that for the visualization to be meaningful, researchers must first develop a clear understanding of the meaning of these codes —even if as part of the GRATIS output, they can easily retrieve the actual quote associated with each actor and code connection (see Figure 5).

The second set of analytic steps that GRATIS covers are more quantitative in nature but fully rely on the results of the inherently qualitative (or mixed methods via machine learning classification) phase just described. This second set of quantitative analytic steps is based on the application of dynamic network analyses that are informed by TIS’s tenets, as just described.

Edgelist Data Format

As discussed in the background section, GRATIS is flexible to handle text that was coded or classified ¹⁹ following qualitative coding procedures via software like Atlas.ti or NVivo, for example, or based on natural language processing via machine learning and supervised or unsupervised text classification using software like Leximancer, QDA Miner + WordStat add-on, MDCOR or LACOID (González Canché, 2023b; 2023a), for example. This flexibility is based on these resulting coded outputs following an edgelist data format (Wasserman & Faust, 1994).

The data format presented in Table 2 and Figure 2 is referred to as an edgelist or a collection of relationships (Csárdi & Nepusz, 2006; Wasserman & Faust, 1994). These relationships have two minimum components: one column in a data frame containing the provider of information, and another column representing the information being provided (i.e., code/label). In Table 2, these columns are “Provider” and “Code/Label” and in Figure 2 they are called “text” and “code.” GRATIS does not require that the dataset uploaded has these same column names or that this dataset follows the provider and code column order. Instead, researchers should conceptually know that these two columns are the minimal information required to conduct the GRATIS analyses. Once the data are uploaded to the GRATIS user interface, researchers will then select these columns from drop-down menus.

Given that coding software does not render the equivalent of column $t$ in Table 2, GRATIS internally computes this column information with the row order uploaded to the user interface. Accordingly, researchers should be careful in correctly identifying the temporal evolution of codes by participants. In Figure 2, this chronological order is observed with the numbers at the end of each actor’s names, which reflect contribution number based on topic modeling (i.e., the “2” in Actor A.doc2 identifies the second code provided by actor A). ²⁰ With this in mind, GRATIS was programmed to follow the actor→code row order of appearance in a dataset as the chronological evolution of codes by actors, even if these contributions are not consecutive. For example:

Actor	Code	Text
A	X	…
B	Y	…
A	Z	…

Here, although Actor A appears in the first and third rows, GRATIS will identify that her/his first contribution will be X and the second will be Z. In the case of Actor B, there will only be one contribution.

GRATIS aims to add as much context to the evolution of each participant’s contribution as possible. With this goal in mind, GRATIS allows for the inclusion of attribute data at the actor level so that changes in “tone” of delivery (i.e., emotion) are captured, for example. Another example consists of depicting whether potentially important attributes may have changed. For example, if a participant shared a story 10 years prior to the main events studied, and during that timeline (i.e., 10 years before), this participant was single, that marital status can be added to the analyses and updated in the present timeline if that status changed—though it can also be the same across timelines. To demonstrate how to add this information, let us discuss Table 5. ²¹

Table 5 is an expanded version of Table 2. In Table 5, we have the “Provider,” “Code,” and “Quote” columns (column t is not shown but is internally computed by GRATIS). In addition, we added two more columns: “Personal Attributes” and “Non-Personal Attributes.” Notably, with the goal of adding as much context as possible and considering that the GRATIS outputs rely on HTML code, each cell may contain more than one attribute. Specifically, the “<br>” symbol shown in Table 5, although optional, adds a line break to each attribute of interest. That is, to add gender and marital attributes in different lines, we may have the following: “Woman: Yes<br>Married: No” and when displayed this would be result in:

Women: Yes

Married: No

This output is shown in Figure 4. Note that in that same figure, the attributes are shown in bold font, and this format may be added with the commands <b> </b>, with all text in between (i.e., <b>your bold text</b>) being bold in the resulting output. Once more, the use of <br> and <b> </b> are optional but may enhance the clarity of the resulting visualizations. The full HTML formatted input looks like this: <b> Woman:</b> Yes<br><b>Married:</b> No (an example of this data format can be accessed at González Canché, 2023a, which may be used to replicate Figures 4–6).

The text in cursive font shown in Table 5 represents changes for Actor’s C personal and non-personal circumstances—as well as a change in her “tone.” In her first recorded code, she was not married and had no college degree, whereas in her last one, she was married and had a college degree (similarly, her tone changed from calmed to sad). In this example, we can assume that the data collection process had a timespan of several years, or that the participant’s storytelling jumped ahead in time n-number of years, both of which can be handled by GRATIS as depicted throughout this paper.

To close this section, note that although the inclusion of multiple attributes (dynamic or time invariant), along with the depiction of the content of the code, is highly recommended to add more context to the resulting visualizations, both attributes and quotes are optional (as indicated in the GRATIS user interface described next). Indeed, GRATIS requires only two columns (i.e., actor or provider and code/label), to render the output that visualizes the chronological evolution of codes generation and decay.

GRATIS User Interface

The execution of GRATIS requires the following steps as shown in Figure 7:

A. Load data or select a data example

a. By default, GRATIS asks you to load your data but also provides two data examples (i.e., the full set of the coded short essays about the reasons to participate in a data science seminar and a subset of coded interviews on strategies employed by faculty members with short term contracts to participate in savings and retirement plans). To fully replicate Figures 4 to 6, as presented in this study, you can also access that toy dataset at González Canché (2023a), download it and load it to the GRATIS UI.

B. Upload data.

a. To upload a database, search for a comma-separated value (CSV) file stored locally. If you select a data example provided with GRATIS, load that database by selecting “Click to load [essay or interview] data example.”

b. Any uploaded database is then displayed in the GRATIS user interface.

C. REQUIRED information:

a. Select the actor column (or document/essay ID)

b. Select the code/label column.

D. OPTIONAL: (attributes to aid in the contextualization of information evolution, displayed when clicking on an actor, code, or line/connection):

a. Column denoting role (see example datasets ²² ). Indicates specific roles of the actors (e.g., students, faculty, and administrators) or documents (e.g., historical and legal). All codes will display the role “Code” in the visualization—see Figures 4 and 5.

b. Column denoting personal attributes. Actors with the same time variant or invariant personal attributes (i.e., ethnicity, gender, marital status) will have the same color in the network, thus aiding to identify groups. All codes will be orange in the visualization (colors can be changed in the “II Optional Adjustments” tab shown in Figure 7). You cannot select the same column twice for role and or personal or non-personal attributes.

c. Column denoting non-personal attributes. All actors with the same time variant or invariant non-personal attributes (i.e., education level, political affiliation, citizenship status) will show the same shape in the visualization. All codes will be a circle in the visualization (shapes can be changed in the “II Optional Adjustments” tab). You cannot select the same column twice for role and or personal or non-personal attributes.

d. Column containing the content of the code (quote) that changes as time passes. This enables users to retrieve in real time the information provided by actors or collected from documents.

e. You can modify the number of consecutive events to map. As stated above, the recommended number is two, especially if the groups are large. The current event will have a green line, and the immediate previous event will be blue.

f. Finally, you can select the transparency level of the colors to be plotted, with 1 being solid and 0 being transparent. Recommended/default value is 0.8.

E. “Execute GRATIS” on the top panel to obtain two HTML documents.

a. The first document relies on TIS and dynamic networks and depicts the complete evolution of information.

b. The second document is an aggregated network depiction summarizing all the contributions made by each actor or retrieved from every document or essay.

Figure 7.

GRATIS’s User Interface.

These HTML files will be automatically launched in your computer’s default internet browser and can be saved and distributed as standalone files. Their display does not need internet connection. Alternatively, they can be launched manually in the “III Execute GRATIS” tab. Additionally, in this tab, the local directory where those HTML files are stored may be retrieved to share in a webpage or distribute via email. ²³

Software and Source Code Access

As a product of this publication, we are offering full and unrestricted access to the GRATIS software with the goal of democratizing data science by removing all computer and programming requirements to implement the GRATIS methodology. Mac users can download GRATIS here: https://cutt.ly/cb5x1wW; PC users can download the software here: https://cutt.ly/Gb5xVBB. To execute the application, (1) extract the zip folder, (2) open the folder starting with “GRATIS-,” and (3) double click on the file called “GRATIS-Mac or GRATIS-win.” The first time GRATIS is executed, you may be required to select menu “View” and “Force Reload” to generate all paths to this newly extracted folder. If after forcing reload, GRATIS does not display as shown in Figure 7, close the window, execute GRATIS again; if needed, force reload once more. The interface will now load. GRATIS is housed in a secure server location and all executable files are virus-free but may require administrative permission to be deployed -- particularly for MacOs users. Finally, each of these executable files for the Mac and Windows operative systems contain all the source code required to modify and/or expand the application capabilities in the file called “app.R.” ²⁴

Discussion: Integration and Contribution to Mixed Methods Research

GRATIS responds to the integration challenge, wherein the sum of the equally important qualitative and quantitative phases depicted in Figure 1 is more than their individual components (Fetters & Freshwater, 2015a, p. 116). As part of this integration, note that GRATIS does not quantify qualitative data; indeed, the qualitative meanings of these (manually or computer-aid generated) codes are never altered nor should they be lost or overlooked. Indeed, as we show in Figure 1, as part of the qualitative phase and before moving to the quantitative one, we strongly recommend having a clear initial understanding of the meaning of these codes so that, when the GRATIS outputs are deployed (the aggregated and the time-evolving ones), researchers are prepared to continue with their meaning-building journey and comprehensively engage with the vast wealth of information contained in those interactive outputs.

Another integrative feature of GRATIS is its flexibility to model codes generated via “manual” or machine-learning-based text classification procedures; hence, the qualitative phase depicted in Figure 1 may itself be an integrative mixed methods exercise as well (see González Canché, 2019, 2022a, 2022c, 2023b and 2023c).

Interactive Reviewability as Validity or Representativeness Checks

Based on data science, visualization, and text retrieval tools, the HTML time-evolving interactive outputs that GRATIS automatically renders, can be paused at each step as shown in Figures 4–6—and the speed of these visualizations may be controlled as well with a simple click (, wherein higher numbers imply slower evolution of connections). More to the point, when clicking on actors, codes, or their connections in these HTML visuals, their resulting attributes (see Table 5) are displayed to enhance both the story-telling attributes of our studies, as well as the interactive reviewability capabilities of these visuals.

As part of the orthogonality, re(pro)ducibility, and compatibility attributes of TIS, the textual content of each code (i.e., its quote) along with actors’ attributes can be displayed at each step of the process, and snapshots of relevant parts of the evolution of information may be taken for inclusion in final research papers. The selection of these snapshots (see Figures 4–6, for example) represents methodological decisions made by the research team. These decisions are constrained by the fact that for publication purposes, no HTML outputs can yet be embedded in PDFs, which means that researchers should select specific points to highlight their findings and conclusions. The selection of these snapshots poses a challenge associated with “imposing” our analytic choices to the final product (i.e., the final paper to be published). However, for transparency and validity purposes, before making those decisions, we should seek our participants’ inputs and try to validate whether our selection was optimal in capturing their experiences or whether there may be other snapshots that better represent their stories or stands.

Moving beyond our participants’ feedback, we also encourage researchers to, while protecting the identities of their participants, upload the entire HTML GRATIS outputs to websites like GitHub (https://github.com/) so that readers of their final papers may interact with these outputs and possibly corroborate or even contest the conclusions as presented by the research team. This interactive reviewability process may not only strengthen the transparency and validity of our findings, conclusions, and recommendations but also and as or more importantly may even spark other ideas and motivate curiosity among readers interacting with these HTML outputs.

An example of how these outputs may be displayed can be accessed at https://cutt.ly/vbjH1sY. This visual contains the complete evolution of about 40 hours of transcriptions resulting from interviewing 40 faculty members holding unstable faculty appointments (see González Canché, et al., 2021). These transcripts document these faculty members’ strategies to deal with savings and retirements plans. From this perspective, while protecting the identity of our participants, GRATIS also enhances the transparency of the entire analytic process by allowing researchers and readers to interact with the main outputs and potentially create and recreate their own interpretations and analyses based on the evolution of information and continuity of events—that is, interactive reviewability (Alexander et al., 2019).

Limitations

Despite GRATIS’s contributions to mixed methods research, this analytic framework does not yet address communication exchanges among participants in focus or working groups, like the VRM approach presented by Alexander et al. (2019). Nonetheless, expansions of GRATIS’s capabilities may be added to achieve this goal, which remains an area for improvement. Additionally, González Canché (2019 and 2022c) has shown that these analytic approaches are feasible to implement under the umbrella of dynamic network analysis of qualitative data and mapping, organizing, and visualizing interdependent events (MOVIE).

Conclusion

GRATIS contributes to mixed methods research by focusing on temporal data analysis and visualization that integrates qualitative and quantitative approaches to map the contextualized evolution of participants’ contributions. The true value derived from GRATIS can only be attained with the full integration of high quality qualitative and quantitative phases. Therefore, without quality assurance during data collection, preparation, and (qualitative or quantitative) coding, the tools provided by GRATIS will still fall short in accurately depicting the evolution of information that shape our research findings and conclusions.

GRATIS integrates data science, visualization, mining, and retrieval tools into the analysis of qualitatively or computer-assisted coded textual data and does so without requiring any computer or statistical programing skills or programming knowledge from the research team. From this view, GRATIS aims to democratize access to these data science tools. This data science democratization goal is the main reason we also provide access to GRATIS’s source code and documentation (see the subsection called “Software and Source Code Access” above) so that interested researchers may build upon GRATIS’s current capabilities.

In terms of expansions, note that GRATIS was programmed to follow the order of information contained in the rows of the edgelist (Table 2 and Figure 2). We purposefully programmed GRATIS to respect this order so that GRATIS may plot information beyond the linear or the chronological code generation depicted in Table 2 and Figure 2. As indicated above, researchers may instead purposefully seek to assemble a thematic timeline that does not follow start-to-finish or top-to-bottom code emergence in a document or transcript (as we showed in Table 1); rather, researchers may prefer to classify events based on themes (Braun & Clarke, 2006) regardless of their actual location in the analytic documents (e.g., interview transcript, essay, and media posts), or based on their participants’ storytelling or reconstruction of events across time as shown in Table 4. To achieve a non-linear story timeline plot, researchers may carefully rearrange their edgelists’ rows based on this new plotline. Procedurally, researchers may add a column to their coded datasets that captures the order each code may represent in their analyses. After all codes have been generated, researchers may simply sort or order this database (using Microsoft excel, for example) based on this new column, going from 1 to N, where N is the last event being recorded given this non-linear or thematic coding scheme. After this sorting is completed and saved, researchers may upload this sorted database to GRATIS and execute GRATIS as described in this paper. The resulting HTML output will highlight the storytelling evolution of events for, once more, GRATIS will identify this row ordering during the application of TIS.

Regarding other applications of GRATIS, researchers may pursue content analyses of media coverage by specific news outlets, for example. This analysis (as indicated by one of our reviewers) “may show the dynamics of their coverage over time and may help address questions of: How long do crises echo over time? Does coverage really fade away as soon as there is nothing (more) scandalous to report on?” Our reviewer also mentioned “there is certainly a myriad of other content-analysis topics, where dynamics over time is of utmost relevance” and may be analyzed with GRATIS.

Finally, note that although the examples presented relied on short timespans, GRATIS can accommodate multiyear projects following the exact same rationale discussed herein. As suggested by one of our reviewers, our reliance on short-time depictions is strategic for it may serve to combat “the common myth [or assumption] that longitudinal or time-based studies need to stretch over a long period of time.” In this study, we showed how GRATIS may accommodate textual data gathered in a variety of qualitative research settings. We also showed that even with short time frames, we may fully leverage the power of time in our sensemaking processes and render a fully integrative research outcome. GRATIS may also be used to observe how groups of individuals react to the same prompt. Here we could expose them to the same statement, say "January 6th did not happen" and ask them elaborate on this statement. Then, after coding, we can apply GRATIS to map the evolution of their trend of thought, likely seeking for similarities and differences based on certain personal and non-personal attributes. This approach may also be employed in psychological lab experiments.

Considering that deeper understandings may be achieved with the incorporation of time-based attributes that have shaped our participants experiences and discourses, let us paraphrase Alexander et al. (2019) to state that we believe that the time has come to start comprehensively leveraging the relevance of time in the analyses of qualitative evidence, both to add more context and nuance to our understandings, and to open the analytic black box that has traditionally defined qualitative data analysis, at least with respect to the decisions made by researchers to present/select their findings and to offer their conclusions and recommendations. GRATIS offers the possibility of strengthening interactive reviewability, transparency, clarity, and even accountability in qualitative and mixed methods research and does so without posing any financial burden nor computer or statistical programming barrier, thus aiding to democratizing data science in mixed method and qualitative research.