Finding a rhythm: The mediality of researching digital skill as process

Abstract

Our objective for this article is to illustrate the importance of understanding digital skill as process by taking its mediality—interweaving tools, technologies, and media—into consideration. Drawing on 12 case studies with participants performing digital tasks, we use Ingold’s four phases of skill (getting ready, setting out, carrying on, and finishing off) to research and represent the rhythm of digital skill. By using medialities of inscription, scripting, and annotation, we demonstrate how researchers can use mediality to perceive rhythms of digital skill without being physically co-located in the performance. As different medialities enable and constrain the perception and descriptions of digital skill, we develop spotlines as a method that combines different medialities particularly well suited for describing and comparing the temporal order of phases for performing digital skill by rendering each performer’s pace and intensity.

Introduction

How we define, measure, and represent skill is a topic of considerable debate (Ilomäki et al., 2016; Van Deursen et al., 2014; Van Laar et al., 2020). In certain industry circles, skill is touted as a “new currency on the labour market” (Lu, 2019). Responding to such high expectations, public and commercial institutions have created increasingly complex taxonomies (Coursera, 2022; European Commission (EC), 2022; Organisation for Economic Co-operation and Development (OECD), 2016) to quantify the categories of skills needed in this new economy. Such measures risk representing skill as a kind of thing—a static, tangible resource that can be acquired and stored. Defining skill this way runs counter to understanding skill as process (Ingold, 2006, 2011). Skill as process constitutes embodied knowledge brought into being by human enactment. A limitation of this latter approach, however, is its conception of technological mediation as a kind of enframing, making it difficult to research contemporary digital skills as process. Through a close engagement with the work of Tim Ingold, we will explore the process of researching digital skill as process in a way that recognizes its mediality; how performing and perceiving digital skill entails interweaving tools, technologies, and media. Drawing on 12 case studies with 11 participants performing digital tasks, we demonstrate how it is possible to develop a set of methods for researching the rhythms of digital skill as process.

Researching digital skill as process

The exact definition of digital skill is a subject of much debate, constituting an “emergent boundary concept” (Ilomäki et al., 2016) whose meaning shifts depending on its context of use (e.g. policy discourse or pedagogy) as well as from one discipline to the next (e.g. computer science or communication). Scholars emphasize the sheer variety of categories around digital skill: technical, information, communication, collaboration, critical thinking, creativity, and problem-solving (Livingstone et al., 2021; Van Laar et al., 2017, 2020). In this article, we turn to Ingold’s (2006, 2011) anthropological methodology as an example of how to study skill as process, and subsequently, how his approach poses interesting methodological implications and challenges for researching digital skill.

Ingold (2011) defines skill as “any practical operation, carried out by a skilled agent in an environment as part of his or her normal business of life” (p. 195). For Ingold (2011), skill is not contained in the body, but rather is interwoven with capacities of action and perception embodied by the “whole organic being” (p. 5). Embodied, action-centered skills develop through performance, not through a purely intellectual process (Suchman, 2007; Zuboff, 1988). This type of “embodied knowledge” is sensory and “grounded in bodily experiences” (Ellingson, 2008: 244) as well as in how the culture of everyday life impinges on the human experience (Lave, 1988). Thus, becoming skillful can only take place by engaging in a purposeful task in a specific place and time (Ingold, 2011). According to Ingold, skill is not predefined by a procedure—what others have referred to as a plan or program (e.g. Suchman, 2007). It is a processional enactment or performance—an episode of tool use in which “every step is a development of the one before and a preparation for the one following” (Ingold, 2006: 67).

Ingold breaks down the performance of this process into a set of (very British-sounding) phases: getting ready, setting out, carrying on, and finishing off (Ingold, 2006). Getting ready is the starting point of the enactment of any skill, which includes choosing an environment and getting materials. Getting ready includes a series of “loosely connected” mundane tasks related to the “umbrella plan” of a project (Ingold, 2006: 68). Setting out begins when “rehearsal ends and performance begins” (Ingold, 2011: 54)—when a person begins working through a task but has not yet hit a rhythm. This phase transitions from “the umbrella plan to a narrow focus on the initial point of contact between tool and material” (Ingold, 2006: 69). Carrying on is the rhythm that develops when people work “with instruments and materials . . . rather than against them” (Ingold, 2011: 55). Finishing off is an ending point, “an inflection from which the movement is gradually retarded and its amplitude diminished” (Ingold, 2011: 69). Through these phases, skill follows an “operational sequence” that is “processional, rather than successional” (Ingold, 2011: 53). The order of phases is not set in stone, nor is one phase necessarily sharply demarcated from the other. The process moves in a non-linear pathway, flowing organically in and out of phases that are rooted in an environment rather than as a set succession of de-contextualized, discrete steps.

We believe Ingold’s methodological approach can be used to conceptualize digital skill as being “embedded in our practice” instead of “the operation of abstract tools” (Couldry, 2012: 58). The difficulty in applying Ingold’s approach to the study of digital skill is rooted in his distinction between tools and technologies. Technologies are deemed incompatible with skill because they are “an objectification and externalisation” of human productive forces (Ingold, 2000: 321) through automation. Ingold (2011) uses the example of sawing a plank by hand, which entails a rhythmic performance that is distinct from the power saw used to mechanically cut lumber. Rhythm constitutes an essential quality of skill as process that shifts through Ingold’s four phases. Skill is achieved through a holistic, regular motion that combines body, saw, and plank—the pace of repetition—whereas the automated movement of the power saw is out of sync with the actions and perceptions of the human body. Key to adapting Ingold’s methodology to the study of digital skill therefore entails grappling with whether this type of rhythmic performance is possible.

The relationship between rhythm and task is rooted in a long-standing concern for the human relationship to time, place, and technology. Thompson’s (1967) analysis of work-rhythms challenged the technologically deterministic view that an encroaching clock-time was responsible for the industrialization of Great Britain. Lefebvre’s (2004) work on rhythmanalysis illustrated how rhythm exists everywhere there is an “interaction between place, time and an expenditure of energy” (pp. 200–204). Citing Leroi-Gourhan, Ingold (2013) refers to rhythms as “creators of forms” (p. 45). Rhythm pulsates between “ebb and flow, flow and arête, contraction and dilation, growth and decay, condensation and rarefaction, contraction and relaxation, inhalation and exhalation,” making a habit to return, repeat, reiterate, and double-back—“a ritournelle” (Henriques et al., 2014: 4). Rhythm is not a patterned sequence, but a repetition of elementary units defined by “intensive variation, (meta)stabilities,” and activities that imperceptibly reorganize phenomenological qualities (Henriques et al., 2014: 19).

The pace of rhythm assumes a monotonous, regular motion that impacts internal and external measures (Lefebvre, 2004: 78) in ways that are distinct from mechanical repetition. Episodes of tool use include recurrent and occurrent gestures that produce a rhythmic quality “attuned to the multiple rhythms of the environment” (Ingold, 2006: 76). As such, rhythm is implicated in environmental and naturally occurring phenomena that are interwoven through bodies and their environment (Sennett, 2012: 202). Another quality of repetition is intensity, in what Deleuze (1994) describes as “orders of differences” among pressure, temperature, potential, tension, and difference of intensity (p. 222). With skill, the intensity of rhythm is perceptible in what Richard Sennett (2008) describes as engaged concentration and focus when practicing skill. Sennett traces musical performances to show how rhythm is achieved when repetition and anticipation combine in focused performance in much the same way as Ingold’s conception of rhythm in carrying on. Intensity is important for considering other dimensions of work-rhythm such as their standardization (Glennie and Thrift, 1996: 285) to which we will return later. Rhythm is, therefore, essential to performing skill in at least in two ways: (1) its pace is part of the temporal give-and-take between a skilled performer and his or her/their environment; and (2) its intensity helps make skill perceptible to others.

Analyzing rhythm in digital skill is difficult if we maintain Ingold’s conceptual distinction between tool and technology where digital hardware and software would fall in the latter category. There are a number of examples of scholarship that adapt Ingold’s work to study digital technologies, including Moores (2014, 2017), Sumartojo et al. (2016), Pink (2011, 2015), Pink and Leder Mackley (2013), and Lesage (2022). While digital technologies do not produce rhythm, they can enable and constrain its performance (Bucher, 2020; Carmi, 2020; Henriques et al., 2014). For example, Facebook sorts, filters, ranks, and curates content (Bucher, 2020) in ways that amplify or exclude certain rhythms (Carmi, 2020).

We propose investigating how performing digital skill involves a combination of tools, technologies, and media that enable and constrain human perception of environments by recording and/or transmitting traces of skillful agency, part of digital skill’s “conditions of agency” (Ingold, 2021: 115). ¹ Perception entails interweaving tools, technologies, and media rather than focusing on “a single medium or single artifact” (Bruhn, 2016: 14), a process we will refer to as mediality. Mediality emphasizes how tools, technologies, and media do not exist in a pure or essential state but, rather, as medial relations between each other (Bolter and Grusin, 2000; Bruhn, 2016; Grusin, 2010; Muhle, 2019: 133; Straw, 2015).

Introducing mediality into our framework for conceptualizing digital skill has important methodological implications. According to Ingold (2011), storytelling is best suited for conveying skill as process because it describes the shifts in phases (and accompanying rhythms) while allowing for the contingencies of each performance and the specificities of the environment in which the performance takes place. In a research context, collecting accounts of skill relies on an embodied, simultaneous experience of its performance: either by actually performing the task oneself or by being physically co-located in the performance and then documenting these experiences through writing.

Generating written accounts of the experiences of skill makes sense for researchers who are adept at documenting the qualitative details of personal experience and face-to-face encounters—what Geertz (1973) would call a thick description of skill. Storytelling through writing is an essential aspect of the kind of ethnographic work (Gherardi, 2019) that serves as the methodological basis for much research on skill as process (Lave and Wenger, 1990; Patchett, 2016; Van Ittersum, 2014). Written narratives draw on ethnographic traditions in which the researcher’s physical co-location in a situation implies an embodied perception of the performance which is then reproduced through written accounts that convey the situatedness of performing skill while avoiding falling into an abstracted, procedural account. In the case of skill, this kind of co-location allows the researcher to feel the pace and intensity of the rhythm of performance.

The above approach rests on an idealization of immediacy (Bolter and Grusin, 2000) that ignores or minimizes the means of perception afforded by mediality. Traditional ethnographic accounts of performing skill rely on human propinquity between skilled performers and the people and things in their environment, including the researcher(s). If performing digital skill entails the capacity to interweave tools, technology, and media, then imposing a single type of mediality to describe it based on the researcher’s physical co-location in the environment undermines its very nature. Just as performing digital skill is interwoven with mediality, so too must we acknowledge that any storytelling for digital skill entails mediality. The epistemological challenge implied by the above discussion is twofold: (1) can researchers perceive rhythms of digital skill without being physically co-located in the performance? and (2) how do different medialities enable and constrain the perception and descriptions of digital skill?

Researching the shifting medialities of annotating, scripting and inscribing digital skill

Our focus in the following investigation into digital skill will be how we, as researchers, turned to different types of mediality—different processes for interweaving tools, technologies, and media—to perceive and describe the rhythms of performing digital skill.

The fieldwork for our study took place in February 2020 (Lesage et al., 2021) and included an in-person information session, a self-directed search task inscribed with video recording software, and a follow-up interview. Using convenience sampling, we recruited 11 female volunteer participants, ages 18 to 59 years, from a variety of cultural and socio-economic backgrounds. ² Each participant was asked to conduct an online search as part of the process of completing one of three unfamiliar workplace-related tasks: (1) strategic communication (a social media audit), (2) visual communication (modifying an image in Adobe Photoshop), and (3) data analysis (creating a Pivot or Macro Table in Microsoft Excel). These three tasks were chosen to cover a variety of interests and because information about each task could be easily accessed online. Participants were encouraged to use whatever tools, technologies, and media they felt most comfortable with to complete their search, the only caveat being to record the process using a discreet recording software in the background of their device. A total of 12 case studies resulted from the investigation, ³ with recordings that varied in length from 13 minutes to 1 hour 38 minutes. Every research participant completed the task in their everyday lives—selecting their own spaces, working times, and materials. We will focus on one of the typical cases—Kendall’s online search for creating a social media audit using her laptop as well as a pen and notebook—to discuss the three research medialities that we experimented with for this project.

Before giving a more detailed description of each of the three research medialities, here is a brief definition for each: (1) inscriptions are recordings and/or transmissions that are indexical of actions, (2) scripts are symbolic systems (mathematical, alphabetical, theoretical, etc.) used to organize annotations and/or inscriptions, and (3) annotations are purposefully created symbols that record and/or transmit the act by which they were created.

Inscription through video recording

The first phase of the research process involved finding a way for each participant to record themselves performing digital skill. This was achieved by asking participants to produce inscriptions by creating videos with Lookback, a specialized recording application. Unlike most screencast software, Lookback sits discreetly in the background of smartphones, tablets, laptops, and desktop computers, and records the computer screen, audio, and web camera of the participant and their surroundings as one steady video stream. A preliminary analysis of these video recordings led us to conclude that at a certain point participants forgot they were being recorded as they began scratching their faces, making low buzzy noises, or whispering the words they read aloud to themselves.

Asking participants like Kendall to record their own performance of digital skill with Lookback afforded an alternative to researcher co-location that would enable the inscription of their body language in situ, an aspect of social life that has historically been excluded from the analysis (Bruhn, 2016).

We invited participants to use whatever other preferred tools, technologies, and media already at their disposal. For example, Kendall wore earbuds to listen to music on Spotify and as a microphone as well as to mitigate the surrounding noises of the coffee shop, demonstrating how her performance unfolded among people, spaces, technologies, and digital tools. Moores (2014) draws on Ingold’s work to illustrate the practical “knowing” encapsulated by the performance of movement, such as tapping on a keyboard or watching a cursor move on a screen. He distinguished a laptop from Ingold’s tool through the “doubly digital” features of modern media, which require not only the technology (iPads, iPhones, laptops, etc.) but also the operationalization of fingers on the device (Moores, 2014: 204). Kendall combined traditional tools like a notebook and pen with modern technological artifacts like a laptop equipped with Microsoft Office. She navigated search browsers, websites, social media platforms, and software designed with pre-set architectures and affordances.

All participants were invited to choose their preferred environment, whether at home, in the office, on university campus, or other public spaces. Sitting in a coffee shop, Kendall navigated the fluctuating state of the environment, using her own perceptions and actions to perform her search. Her search provided an “occasional (but not definitive) state” of digital tools, technologies, and media, revealing a specific moment in time, a “prism through which it is viewed” (Straw, 2015: 128). How she occupied space as part of completing the task was not dichotomized between physical and digital environments, but rather was a combination of the two. Online spaces like websites and web applications did not offer the ability “to be in a space, but rather to directly or, more often, indirectly act upon that space” (Conatser, 2010: 368).

We treated these inscriptions as “constitutive of information” about performing digital skill and “not merely its transparent presentation” (Drucker, 2020: 34). The important aspect of inscription is its indexicality—ascribing a “direct relationship” to the “original substance” (Latour and Woolgar, 1986: 45). In our case, this meant analyzing the video inscriptions to see whether they were indexical of the rhythms of performance. Viewing the videos after the fact, we could perceive moments when participants achieved a certain rhythm in their performance that suggested they were carrying on with their task. For example, the repetitive motions of Kendall’s hand across her laptop touchpad generated a rhythm on the screen as she searched through her files and webpages. Her movements showed how physical actions were recorded as digital traces that coalesced into a single embodied rhythm.

Using Lookback certainly could be construed as an intrusion into a performance of skill, but one that was arguably less intrusive than standing over Kendall’s shoulder as she completed her task. The software’s status as tool, technology, or medium processually shifted over the course of the performance. Kendall initially used Lookback as a tool to record her session. As she pursued her task, the application became a technology that automated the recording process. Finally, the videos generated by her use of the application served as medium for the researcher.

Our early analysis of these inscriptions addressed the first of our preliminary epistemological challenges; we could perceive rhythms of digital skill without being physically co-located in the performance, thanks to the video inscriptions. The pace and intensity of rhythm were perceptible with these inscriptions because we as researchers could experience the performance through a mediality that was indexical of the temporality of the original performance. Echoing Beaulieu’s (2010) critical examination of co-location for digital ethnography, we would argue that mediality constitutes an essential methodological dimension for researching digital skill.

Scripting through an Excel spreadsheet

Although we could perceive rhythm in the above inscriptions, each video recording lasted the time of performing the task, generating 380 minutes of video footage (in Kendall’s case 21 minutes 8 seconds). These lengthy periods of time made it too difficult and unwieldy to analyze and lacked a narrative description. We, therefore, set out to test various ways to script and annotate each inscription. The advantage of scripts to describe skill is that they are “both immutable and mobile” (Lammes, 2017: 24); they can be easily transcribed or translated. Below is a short example of how we might summarize Kendall’s performance of digital skill as a short script:

Kendall is an undergraduate student in her 20s with long blonde hair and big brown eyes. She sits in a campus coffee shop where people talk and drink as machines beep and people bustle in and out of the busy shop door behind her seat.

Kendall flips open her laptop, sets up Spotify, plugs in earbuds, and takes a notebook and pen from her bag. Tapping on her keyboard she types “online reputation management,” but adjusts her query to “online reputation audit” before pressing “search.” She clicks on searchenginejournal.com to learn about conducting a brand reputation audit and develops a repeated pattern of reading and penning an audit in her notebook.

A young man interrupts her workflow with an extended, unsolicited stare from behind. Kendall turns around at the same time the man walks off—it isn’t clear if she notices him. She plays with her watch, stretches, and continues her work. On Google, she searches “yoga companies” and “yoga brands” before narrowing in on “Alo Yoga” for her social media audit. She scrolls, reads, writes, and repeats—having found her rhythm again. Kendall finishes up by looking at Alo Yoga’s website, stretching, checking Spotify, and signing off.

The above script of Kendall’s performance is written in such a way that it describes digital skill as a narrative that denotes (1) the situatedness of the performance and some of the contingencies that define its unique performance, (2) a processual unfolding through the phases of performing skill, and (3) Kendall achieving a certain perceptible rhythm by engaging in patterned action. Based on these three criteria, we can say that the script is a helpful description of digital skill. However, the account also raises two important challenges. First, while the written narrative does indicate that Kendall achieves, then loses, then re-finds her rhythm as part of the enactment, the account lacks a sense of the pace of this rhythm or of the intensity of each phase and how the ebb and flow of phases relate to each other in the overall performance. For example: How significant is the initial phase of carrying on as a “repeated pattern of reading and penning” compared to the other phases of the enactment including when she was able to re-find her rhythm? Second, the account is written through a “God’s-eye-view” perspective that erases the mediality of the performance which included Kendall recording her own performance. The script could not have been created without the video inscription, yet the written account implies the co-presence of its author.

To address these new challenges and analyze the participant inscriptions in greater detail, we developed combinations of scripts and annotations in Excel. The first step involved creating short written accounts of each successive moment documented in the inscriptions. It took approximately six and half minutes to script every 30 seconds of video into the spreadsheet. The scripts provided a precise translation of time as well as a segmented description of the screen, body, and environment. Each set of scripts started with a case number, case name, case video number, and file name, corresponding to the files the researchers shared on an institutional cloud storage system. We noted moment-by-moment descriptions of the performance of digital skill into cells in a descending chronological order along rows. Each row was initially divided along four columns. The first column indicated a timestamp for every minute, with a yellow bar extending across the row. The second “time” column provided a more granular timestamp. The third and fourth coding column included the “main screen notes” and “second screen notes.” The main screen notes were descriptions of how participants use various applications like YouTube, Instagram, and Excel as well as keyword searches. They also included information related to the participant’s desktop, including the computer’s operating system, software icons, and desktop shortcuts. Figure 1 illustrates the main screen notes, where Kendall reads the Lookback instructions, launches Google, and types “online reputation management” into Google, before revising the keyword search by replacing the word “management” with “audit.” The next field note entry reads

Kendall highlights the first suggested article, which is “Conducting a 5 Minute Brand Reputation Audit.” She expands the question “What is a reputation audit?” from “People also ask.” The answer Kendall is reading says a “reputation audit is a service provided by many public relations firms that helps clients better understand their customers and critics.” She scrolls down. She seems to be moving her cursor to the rough area on the page that she is reading.

Figure 1.

Scripting Kendall’s inscription in Excel.

The “second screen notes” were observations from the webcam frame, including aspects related to the environment, bodily actions, and sounds. Below is an example of a field note entry for Kendall:

Sounds of people talking in the background. Beverage preparation noises. Kendall takes her hands off her neck and puts both elbows on the table, covering her hands over her face and nose.

To script rhythm, we then introduced a new column to the left of the data set to code each script according to their processual phases, using color blockings for each phase (Figure 2). Getting ready (GR) was indicated in a pink color-blocked cell, whereas setting out (SO) was marked in an orange colored-block cell. We encountered difficulty at this stage: it became impossible to distinguish transitions from one phase to another in relation to each discreet scripted moment. The timestamps as discrete units were incommensurate to a processual account, making it difficult to recreate the rhythms originally perceived in the video inscriptions. Reading the Excel scripts led us to conclude that we had “lost” rhythm with this mediality; we could no longer perceive the pace or intensity of performance.

Figure 2.

Coding over code in Excel.

Drawing on the work of Harris (2000), Ingold (2007: 122–125) suggests that scripts as systems for organizing symbols undermine the possibility of reproducing the processuality of embodied performance. In Ingold’s (2007: 131–132) interpretation, Geertz’s thick description is unsuited to narrating what we refer to here as rhythm, being merely a way to script social discourse by “finding the right words to record or convey what has been observed.” Rather than dismissing scripts tout court, we decided to explore alternative approaches that might help us once again find the rhythm in the video inscriptions without relying on a mediality that was indexical of the temporality of the original performance.

Shifting to a script based on Ingold’s four phases

In order to develop a different kind of script, we chose not to rely on a quantitative measure of time for each task and instead used Ingold’s four phases as the basis for organizing the scripting process. Using colored markers, we annotated the performance on paper with a qualitative focus on rhythm and the transitions from one phase of skill to another (Figure 3). We still used timestamps to indicate key moments or transitions, but as a background consideration. We annotated the process with colored square half-brackets and timestamps, keyword searches, and key passage points, which we defined as shifts to various digital tools like Google, Excel, YouTube, Facebook, and Instagram during the performance. We used colored lines to annotate the shifts between phases: getting ready (pink), setting out (orange), carrying on (green), and finishing off (blue).

Figure 3.

Kendall’s marker annotation.

These colored annotations allowed us to describe nuances between phases while leaving clear traces of our own interpretations as researchers. Getting ready involved placing materials at workspaces, opening devices, launching applications, reading task instructions, and taking notes. Kendall got ready as she opened her laptop at a campus coffee shop, opened her browser, and grabbed a notebook and pen out of her backpack.

The flexibility of the annotations enabled us to show transitions from one phase to the next or to nest one phase inside of another. As Figure 1 illustrates, Kendall went through finishing off by reviewing a brand’s website, returning to previously visited websites, and then transitioning back to getting ready by looking over her notes, examining open tabs from the search, and looking at her Spotify playlist. After a short amount of time, Kendall completed the finishing off process by returning to the website, reading her notes, packing up her notebook and pen, and signing off. Rhythm was perceptible once again with the marker annotations, but the representation still lacked intensity and comparability. The next step involved exploring how to create more polished and legible versions of the marker annotations.

Re-annotating digital skill as spotlines

We applied shapes and colors to devise the first digital annotation that we referred to as a footprint—a series of upward and downward half-circles (Figure 4). The colors in the draft marker sheets remained: getting ready (pink), setting out (orange), carrying on (green), and finishing off (blue). The circular shapes moved in an oscillating rhythm through the footprint with additional circles to indicate passage points. Kendall’s footprint, for instance, includes red dots to specify Google searches. The main advantage of this approach is its use of color to show transitions from one phase to the next. The problem was that the half-circles represented time as regular unit intervals. As a result, the re-annotation still worked more like our Excel script, imposing a temporal division that obscured the specific development of rhythm in each performance and muddling any representation of pace or intensity. The clean, modernist aesthetic also left no trace of the researcher’s act of fabrication.

Figure 4.

First digital draft.

Elements from the marker annotations, like color and passage points, were still visible in our first attempts at re-annotation (Figure 5). Re-annotation entailed understanding footprints as forms that were constitutive of information and whose intentional fabrication could combine aspects of scripts, annotations, and inscriptions. The re-annotations were an important transition in the research process: it was the first time that we could make clear comparisons between phases of performing digital skill in one concise image.

Figure 5.

First draft of re-annotations.

Initial attempts at re-annotation produced two options for the shape of the footprint: wavy lines or circular spots. The wavy lines do not adequately describe proportions of time or the intensity between phases as strongly as the circular spots. The transition from getting ready to setting out is easier to identify with spots than it is with lines. The size of the spots varies depending on the length of time a participant is in each phase. In addition, the spots allow phases to bleed together and nest into each other (as is the case when getting ready is embedded within finishing off), describing a recurrent, monotonous pace. As a result, we selected spots for the final version (Figure 6). The spots clearly demarcate the phases and emphasize the processual quality of each performance. The spotline footprint annotations highlight the value of taking a processual, as opposed to successional, approach to researching skill. The processuality is particularly perceptible in Kendall’s movement out of the first phase of carrying on. Kendall carries on until a young man moves behind her, staring at her with an unsolicited gaze. Even though she is turned away from this stranger, she stops working, turns, and looks behind her. By the time she turns around, the young man is gone, and Kendall’s rhythm is lost. The way Kendall’s task is interrupted and how she subsequently works to find a rhythm again accentuate the advantages of researching skill from a processual standpoint.

Figure 6.

A spotline of Kendall’s task.

The footprint spots weave together the rhythms of each participant’s performance while also leaving traces of imprint of the researcher’s process in the analysis of digital skill. The researcher is not objective or value-free and must contend with issues like subjectivity and personal experience (Kassan et al., 2020). To borrow from Sterne (2007, 2019), the researcher is the “instrument” and the “tools of research” (Lookback, paper, markers, Google, Instagram, and Excel) form the “medium” that record the transmission process. The watercolor spots used to re-annotate leave a trace of the researcher’s gesture, reinforcing the subjective dimension of the representation.

The resulting spotline combines three types of mediality: inscription, annotation, and scripting. It draws on elements of inscription to the extent that the form and configuration of spots are based on our perception of video recordings that are indexical of the temporality of the original skilled performance. It draws on scripting to the extent that we developed a system for symbolizing various phases of skillful performance as spots of color that can be represented in varying orders and overlaps (rather than letters or numbers). It draws on annotation by recording the acts by which we as researchers represented the processual phases of skilled performance.

An advantage of the spotline footprints is how they afford a condensed representation of the temporality of rhythm, enabling as a quick read that includes a performance’s repetition, pace, and intensity. In addition, they can be used to compare multiple performances to check for similarities and differences. The idea for comparing spotlines is inspired by Edward Tufte’s (2006) “sparklines” which he defines as

. . . small, high-resolution graphics usually embedded in a full context of words, numbers, images. Sparklines are datawords: data-intense, design-simple, word-sized graphics. (p. 47)

Sparklines combine a set of contextual methods to enable comparison with the “dequantifation” of data (Tufte, 2006: 61). These contextual methods rely on spatial adjacency—stacking the various data sources closely together and thereby “showing vast amounts of data within the eyespan, spatial adjacency assists comparison, search, pattern-finding, exploration, replication, review” (Tufte, 2006: 63). Whereas sparklines afford comparisons of quantitative data through their dequantification, we propose a similar decontextualization of the spatial dimension of skilled performance in order to gain the ability to pursue a qualitative comparison of the temporal paces, orders, and intensities of phases across multiple performances. In addition, spotlines make the rhythm of performing digital skill readable across multiple performances (Figure 7).

Figure 7.

Participant spotlines.

In the video inscriptions used to produce the spotlines, nearly all participants expressed pace and intensity by moving closer and further from the screen, rocking, humming, or mouthing what they read in a particular rhythm (Figure 7). In Abra’s (2) case, the swiping of fingertips and tapping on the smartphone hit a perceptible rhythmic intensity that mirrored the recurrent pattern of the participant moving between Instagram and her notes application. Despite the advantages of inscription, some participants also encountered technical difficulties with Lookback. The video collected from screens or webcams were occasionally pixelated or completely dark. These issues made it difficult to describe certain moments within the searches. In most cases (but not all), these issues occurred when participants watched YouTube videos. This problem presented only a minimal impediment to the research process because the participant’s actions remained perceptible in the video inscriptions. The technical problems also served as a reminder of possible technological impediments that can plague any digitally mediated task. Interviews with participants also mitigated some of these challenges, affording participants with the agency to “fill in the blanks” and explain what happened during technical difficulties.

All participants started with getting ready before transitioning into setting out. For the most part, this early combination of phases receded to the background, allowing other phases unfold in varying orders. Setting out included typing keywords into Google (the preferred search engine of participants), reviewing Google search results, reading webpages, watching YouTube videos, inserting information or data into Google Docs, Word, or a note application. Setting out also involved any component essential to the completion of a task where the rhythm was working against rather than with the tools as we see in carrying on. Finishing off marked the completion of a task (closing browsers, putting materials away, etc.).

Comparing Kendall’s spotline with the others, we see how the spots follow a similar order to those of Abra (2) and Yasmin (4), for example. Each of these cases satisfactorily completed their tasks. In some spotlines, we see extended carrying on phases (Yesfir 7, Olivia 10, Himari 11, Abra 3) where participants really hit their stride and were able to maintain a constant steady pace and intensity while working. Comparing Abra’s (2, 3) two spotlines, we can also see how the same person can go through phases in a different order depending on the task. Yesfir’s (7) and Allison’s (8) spots contain a condensed intensity as they copied-and-pasted material to archive their task. Not all participants were able to progress smoothly through phases of performance. Safiya (5) and Rebecca (6), for example, seem to have a harder time finding and maintaining a rhythm for carrying on. Each experienced an extended setting out phase, suggesting something was stopping them from being able to perform the task. Safiya (5) also exited her task abruptly while still in setting out and just beginning to carry on. In the interview, she explained that she left midway through an image-clipping task because she did not have the required tool—Photoshop—installed on her hand-held device.

One of the interesting findings that appeared when comparing the spotlines was how at least five of the participants engaged in a kind of “mini” setting out phase as part of finishing up (Figure 7: 1, 2, 4, 8, & 9). While this finding requires further investigation, this may have been because of how participants were engaging in an unfamiliar task. Re-engaging in setting out may have served to self-reflect on the performance.

Representing each performance as a spotline may make performing digital skill seem like a lonely and isolated enterprise when compared to accounts of task-oriented communities of potters, carpenters, and weavers (Ingold, 2000; Sennett, 2012; Thompson, 1967). This isolation is not necessarily due to the absence of other humans. What emerges is how the kind of digital skill performed for these tasks was individual and non-standardized (Glennie and Thrift, 1996: 285) in that the pace and intensity of their performances were not disciplined to align with the rhythms of other people around them. Kendall sat in a coffee shop where anyone from a barista to another client sitting at their own laptop worked at their own pace and intensity. While digital skills may be performed in the same location as others, they are not necessarily performed with others. Non-standardization, however, did not stop a passerby from interrupting Kendall’s rhythm. This last observation reaffirms one of the limitations of spotlines: while it is a useful method for visualizing the temporality of phases in skilled performance, it is not suited for visualizing the space in which performance takes place. Spotlines should, therefore, not be used as a standalone method, but as a combination of methods that complement each other through their various medialities.

Conclusion

While we recognize the value of research on the types of digital skills relevant to 21st-century work, we believe that any description or measure of digital skill would greatly benefit from considering its situated performance as process. This article offers a first look at spotlines, a novel research method to study rhythm in the performance of digital skill. While we were constrained by the sample size, the inventive use of audio and video recordings to document the rhythms of digital skill offers a framework that can be replicated and applied to future research on digital skill.

In this article, we set out to illustrate the importance of understanding digital skill as process by taking mediality into consideration. We achieved this goal by combining Ingold’s (2006, 2011) methodological framework for studying the various phases of skilled performance with three types of mediality for research: inscription, scripts, and annotations. The research design presented above can be usefully adapted for a wider application of research in digital media, particularly with respect to the two epistemic challenges we identified above.

With respect to the first challenge, we demonstrated how it is possible to perceive rhythms of digital skill without being physically co-located in the performance. The pace and intensity of rhythm were perceptible through a mediality that was indexical of the temporality of the skilled performance. With respect to the second challenge, we were able to show how different medialities enabled and constrained our perceptions and descriptions of digital skill. While the indexical temporality of the video inscriptions (“Inscription through video recording” section) made it possible to perceive rhythm, this same indexicality made it difficult to narratively describe the performance. As an alternative, we explored different approaches for combining the medialities of scripts and annotations that would be best suited to describing the pace and intensities of skilled performances. The solution we identified, spotlines, allowed us to comparatively investigate the temporal order of phases in performing digital skill. We argue this is a significant finding and more research attention must follow using spotlines as a novel research method to study digital skills.