Stumbling Blocks and Alternative Paths: Reconsidering the Walkthrough Method for Analyzing Apps

Abstract

The walkthrough method was developed as a way to trace an app or platform’s technological mechanisms and cultural references to understand how it guides users. This article explores the method’s enduring strengths and emergent weaknesses regarding technological advances and developments in app studies. It engages with adjacent methods for understanding apps’ intensifying structural and economic complexity, datafication, algorithmic logic, and personalization as well as approaches fostering a feminist ethics of care toward users. Considering these perspectives, the article discusses challenges encountered in teaching the method and applying it to algorithmically driven apps. With TikTok as a central example, examining the walkthrough process demonstrates the method’s incongruence for investigating several aspects of the app, especially its automated personalization. These challenges highlight the need to combine, supplement, or exchange the method with other approaches as part of an expanding and flexible toolkit of methods in app studies.

The walkthrough method for analyzing apps (Light et al., 2018) grew from emerging and established research approaches at the intersection of cultural studies and science and technology studies (STS). Ben Light was already attuned to the guiding role that platforms and software could play in users’ lives, such as through his analysis of Gaydar’s categorization of gay male identity expression (Light, 2007) or attention to the functionalities shaping emergent communities on Squirt, an HTML5 hook-up site configured for mobile use (Light, 2016). Often working through the lens of Actor Network Theory, Light attended to these platforms’ technological mediators and their interventions in social interactions. Collaboration with Jean Burgess more vividly brought to light the role of culture, not only in user practices (Burgess, 2006) but also in the ways that broader cultural contexts shape platform missions and mandates, combining with economic and political pressures that propel platform changes over time (Burgess, 2014). Duguay synthesized both perspectives, hailing from a broad background in the social sciences. Thus, formulating the method required an articulation of how to trace technical mediators and cultural representations within apps, with reference to their surrounding materials, and across their connections within broader digital ecologies.

Numerous workshops, discussions, and applications of the method solidified its definition, steps, and attentional points. This allowed its authors to establish the method’s core use as “a way of engaging directly with an app’s interface to examine its technological mechanisms and embedded cultural references to understand how it guides users and shapes their experiences” (Light et al., 2018, p. 882). This understanding is developed by “establishing the environment of expected use” comprised of an app’s vision, operating model, and governance, paired with a “technical walkthrough” allowing for detailed analysis regarding the app’s interface, features, functions, text, and symbolic representations. By leaving open certain considerations (e.g., theoretical approach, order of procedure), the method provides structure for systematically interrogating apps’ sociocultural influence without constraining researchers to a rigid process. Such rigidity is incongruent with the iterative nature of both qualitative research and technological development within a “permanently beta” (Neff & Stark, 2004) app marketplace of continuous change.

A review of scholarship that has mobilized the method since its debut reveals that it has served a range of purposes. Researchers have used the walkthrough method to examine specific apps, such as the “learn to code” app Grasshopper (Decuypere, 2021); to compare and contrast apps, such as Bivens and Haimson’s (2016) investigation of social media platforms’ gender categories; and even in relation to technologies that confound the conventional conception of apps as limited to mobile or desktop devices, such as Mascheroni and Holloway’s (2019) analysis of the “Internet of toys.” Scholars have also discussed the method, providing insights into ways of building on, supplementing, and adapting it (Møller & Robards, 2019; Nieborg et al., 2020). Their insights respond to shifts in platform and app technologies, digital cultures, and research approaches. Algorithmic functionalities, datafication, personalization, and other technological advancements that are now commonplace within everyday apps largely elude the original method. Simultaneously, platforms and their attendant users have increasingly shifted stances regarding modes of surveillance and profit motives (Srnicek, 2017; Zuboff, 2019), complicating the method’s use in classrooms and with research participants. Digital media research, especially research drawing on feminist approaches, has drawn more attention to the importance of considering user subjectivities and situatedness while prioritizing knowledge co-creation (Luka & Millette, 2018), leading to further questions about the walkthrough’s hands-off approach to app users.

Given that platforms and apps do not remain static, and the emerging field of app studies must not stagnate, this article examines and updates the walkthrough method in light of these changing circumstances. It does so first by highlighting key developments in app technology, digital cultures, and research approaches, identifying shifts in approaches to studying data, algorithms, platform history, and users in relation to the original walkthrough method. This reflection is then paired with lessons learned from teaching the method across contexts, from specialized workshops with early career researchers to mandatory undergraduate courses. Hiccups, feedback, success stories, and tired null findings inform observations of how to bolster the method’s potential as a research tool and conduit for digital literacy. The method’s enduring strengths and emergent weaknesses are then illustrated through a walkthrough of the short video app TikTok. The app’s combination of familiar features with newer, highly personalized and algorithmically driven functionalities demonstrates how other methods become necessary supplements or extensions of the walkthrough while pointing to the need to continue developing digital research methods beyond the current boundaries. We conclude by highlighting next steps for the walkthrough method’s future.

Emerging Directions in Platform and App Studies

The original walkthrough method discusses “apps” and “platforms” interchangeably. It understands apps as stemming from the development of discrete software applications designed to run on operating systems (Pressman, 2005) that often encompass the dual role of platforms, as governed media systems and digital infrastructures (Plantin et al., 2016). However, such an approach overlooks differences among apps as well as platforms’ rapid expansion into ecosystems with considerable technological, organizational, and governing influence over users and markets (van der Vlist, 2022). Scholars have since developed more sophisticated conceptualizations about how platforms are multisided in the different interfaces and functionalities they present to users, advertisers, developers, and other institutional actors (Bucher & Helmond, 2017; Nieborg & Poell, 2018). Discussions of platform ecosystems elaborate that not all apps and platforms are built similarly or serve the same functionalities. Infrastructural platforms, such as Alphabet-Google and Facebook, serve as the backbone for sectoral platforms that address particular niches, such as news, finance, and health (van Dijck et al., 2018). In addition, Van Dijck et al. (2018) identify connector platforms, such as Uber Eats, which do not offer services directly to users, but instead forge relationships between users and service providers. While the walkthrough method made no distinction between these different types of platforms and apps, adjustments may be necessary to capture the nuances of an app and its role within broader digital ecologies. For example, Yang et al. (2022) adapted the walkthrough with a “backend-in” approach that examines what they call meso platforms, or sectoral platforms (Van Dijck et al., 2018), that sit between users and infrastructural platforms. They apply the backend-in approach to an investigation of WeChat Official Accounts, which are developed upon the WeChat App and serve news-based content to users. These perspectives bring necessary attention to differences between apps.

Since data are central to their functioning and business models, apps are increasingly intensifying processes of datafication that constitute the rendering of everyday lives into data. Scholars draw attention to the data cultures arising from these processes, which involve the routinization and institutionalization of data production and cultivation in ways that shape how users respond to data structures (Albury et al., 2017). The stakes of understanding data extraction become higher when considering how data exploitation and surveillance heavily impact individuals embedded in precarious financial and democratic conditions (Milan & Treré, 2019). Furthermore, datafication has become so pervasive that it can be conceptualized as a form of data colonialism (Couldry & Mejias, 2019) in which “social life all over the globe becomes an ‘open’ resource for extraction that is somehow ‘just there’ for capital” (p. 337). Data colonialism not only serves the interests of capitalism but also continues an oppressive history of dispossessing populations of their agency by predetermining their economic and sociopolitical value. These differential effects of data extraction can, more precisely, be thought of as an increasingly prevalent, neocolonialist means through which Eurocentric, patriarchal, heteronormative, White epistemologies are imposed upon other ways of thinking and living (Mumford, 2022).

The walkthrough method can be attuned to data exchanges between users and apps, such as through details recorded in field notes about information required during registration or permissions requested by an app to gather data from one’s device. Inspecting the environment of expected use may reveal how data extraction features in an app’s business model and data-handling practices, which are often identifiable in privacy policies or terms of use. However, the walkthrough fails to capture data processes that are not made explicit to users or corporate stakeholders and is limited in its ability to detect data flows between apps. As such, Weltevrede and Jansen (2019) supplement the walkthrough with tools and techniques that enable network sniffing—identifying the network connections being established by apps—and packet inspection that allows for examining the data sent through a device’s networked connection. While such techniques hailing from the fields of network security and software development can be technically sophisticated, scholars also point to research-based uses of apps or platforms with built-in functionality for detecting the network connections being forged by a device’s apps (Dieter et al., 2019). Incorporating the walkthrough method into an approach that uses such detection tools (e.g., the open-source Exodus platform) alongside consideration of legal and ethical frameworks for data use, Kuntsman et al. (2022) provide a toolkit for evaluating whether, and to what extent, apps uphold or violate principles of data justice. These approaches make processes of data extraction more salient than in the original method.

Data collected from users often input into personalized app features and algorithms that sort, filter, and recommend content. Some elements of personalization have been longstanding; for example, Duguay recalls how, in early applications of the walkthrough method to Tinder, users under 18 years old were unable to search for those over 18 (before the app prohibited use by minors). However, increased data collection has led to fine-grained personalization that extensively affects how facets of users’ identity are constructed within a platform and reflected back to them (Cheney-Lippold, 2017). The algorithmic profiling of users now broadly determines what they see and with whom they are proximate, orienting them among vast swathes of content and masses of users. As such, scholars call for greater attention to the research persona and account configurations used for the walkthrough method (Dieter et al., 2019; Weltevrede & Jansen, 2019), weighing up whether a “clean” account with few data inputs can allow one to study the actual conditions of everyday use when an app’s instantiation is heterogeneous across users.

Depending on the research aim, it could be more effective to generate personas that allow for understanding how particular individual qualities are processed, and responded to, by the app and its algorithms. Albrecht et al. (2019) explore this approach through a series of exercises for understanding the influence of apps in (dis)information campaigns. They move from autoethnographic introspection on the researcher’s part, concerning what it means to feature one’s own account in the walkthrough, to the construction of an information campaigner’s account, imbued with particular behavioral qualities based on marketing tactics, as well as the creation of a fictitious persona, drawing on theater practice to develop its fully fledged history and background. Such accounts serve as personas that may be mobilized differently in relation to various research designs. These scholars also propose that some studies may warrant the researcher embodying a persona over time, emulating browsing habits and other interactions with the platform, and keeping a research diary to track changes, events, and experiences that arise. While akin to algorithm audit (Sandvig et al., 2014) and reverse engineering strategies (Diakopoulos, 2014) for understanding how inputs into platform algorithms generate specific outputs, Albrecht et al.’s (2019) attention to the construction and performativity of a research persona brings ethnographic and autoethnographic elements to the forefront in the walkthrough. It heeds Seaver’s (2017) call for the ethnographic exploration of “algorithms as, rather than in culture” (p. 6), allowing for an understanding of how algorithms are constituted not only through formal, computational functions but also through everyday practices and cultural meaning-making. It also locates the researcher’s subjectivity within the walkthrough’s existing foundations as an ethnography of infrastructure (Star, 1999 as cited in Light et al., 2018, p. 887). These approaches examine algorithmic functionalities that are integral to the walkthrough method’s analysis of an app’s mediator characteristics but remained previously unaddressed in the technical walkthrough.

Calls for iterative explorations add a longitudinal component to the walkthrough method to address its frequent use as a snapshot of an app at one juncture in time. While the method’s authors caution that multiple walkthroughs might be needed to keep pace with app updates (Light et al., 2018), consideration of app history is not directly built into the method. In contrast, Burgess and Baym’s (2020) development of the platform biography presents a way to understand co-evolution among focal platform components (e.g., Twitter’s #, @, and retweet functionalities) and user cultures. While considering the platform’s interface, business model, and other aspects that overlap with the walkthrough method, the platform biography combines analysis of these materials with interviews, users’ expressive content, and public discourse to “map struggles between competing cultures of use” (p. 18) and create a biography—a “life record” (p. 26)—of the platform. However, platforms raise challenges for tracing changes over time, since their software updates are not archivable in the same way as webpages (Rogers, 2019). Returning to the pertinence of platforms’ multisided functionality, Helmond and van der Vlist (2019) address these challenges of platform historiography by moving away from examinations of end-user interfaces and toward tracing how platforms have operated historically for multiple stakeholders. They propose engagement with archived materials for developers (e.g., SDKs, ¹ APIs, ² and their reference documentation) and businesses (e.g., marketing tools; analytics services) that provide a sense of how the platform’s infrastructure and revenue-generation have changed over time. By integrating these approaches into the walkthrough method, consideration of source code and programming boundaries could become integral to establishing a longer view of an app’s vision—its intended uses and users over time—while the technical walkthrough may employ elements of the platform biography to understand which interface features have been hotly contested.

Can Users Be Bracketed?

Methods that include the development of sophisticated research personas and deeper examination of apps over time intensify the walkthrough method’s ethical considerations, warranting attention to users and their wellbeing in relation to research objectives. Although the walkthrough method is designed to engage with the app’s material and contextual aspects, users are inevitably caught up in the research process through the potential for interaction with research accounts or the inclusion of user data in field notes and screenshots. Even indirect interaction, such as when a research account is included in others’ feeds (e.g., on Tinder as a potential date), may disrupt users’ experiences of the app. A turn toward feminist data studies prioritizes reflexivity and an ethics of care that heeds whether the researcher is welcome in a community’s digital environment and among its data, elevates the preservation of privacy, and identifies the researcher’s personal responsibility for protecting those implicated in research (Leurs, 2017; Luka & Millette, 2018). Such principles become tangible when, for example, determining whether and how it may be possible to develop and run the research persona of a (dis)information campaigner in situ without being an unwelcome inhabitant of the platform let alone causing harm or disruption while actively circulating content. When users are treated only as props or examples of platform configurations, researchers risk recreating the extractive principles they critique in relation to platforms. This casting aside of user-specific experiences also frequently gives rise to attempts at understanding platforms outside their sociocultural uses and contexts (Bosch, 2022; Vicari & Kirby, 2022). While Light et al. (2018) propose analyzing social media discourse about a platform or engaging users in participant-led walkthroughs to better understand unexpected user practices, others present adjacent methods for more deeply understanding the “situated knowledges” (Haraway, 1988) specific individuals or aggregations of users (e.g., publics, communities) draw upon in relation to culturally and geographically embedded app practices.

Two such methods involve co-navigating apps with participants. In the “scrollback method,” a user brings the researcher through longitudinal social media traces by scrolling back in one’s feed or profile (Robards & Lincoln, 2017). Drawing from mobilities-oriented ethnographic approaches, the “media go-along” involves the user bringing the researcher through different app features, functions, and screens with attention to the user’s orientation on the app and the embodied role of the mobile device during use and research (Jørgensen, 2016). These methods’ originators, Møller and Robards (2019), reflect on how their approaches, along with the walkthrough method, provide the capacity to investigate mediated mobilities, wherein mobility involves “physical manipulations of media artefacts (such as finger to touchscreen) and imagined but just as real orientation and habitation in digital media environments, interfaces and affordances” (p. 100). They assert that, to attend to mediated mobilities, methodological approaches must consider bodies and affect in relation to media objects and environments, memory and narrative, and the research encounter as it (re)distributes agency across media, participant, and researcher. They point out that what is lost by the walkthrough method’s treatment of users as an “ethical liability” (p. 103) is a perspective on these elements (bodies, affect) beyond the researcher’s experience and interpretation. Therefore, a renewed vision of ethical conduct for the walkthrough method should not only underline concerns over whether user data is intruded upon in collection processes, how it is stored, and how to deidentify users in screenshots and examples as findings are disseminated. It must also consider what might be lost through attempts to bracket ³ users outside the research design and whether this can lead to over-generalizations and universalizing assumptions about how particular apps shape a diversity of users’ lives.

Walkthrough in Action I: Learning Through Teaching

In workshops and teaching environments, the walkthrough method’s authors developed a key exercise for demonstrating the method’s effectiveness: a “comparative app walkthrough” (e.g., Burgess et al., 2015). In this exercise, small groups of learners chose two apps from the same genre (e.g., music, social networking, health) and conducted a walkthrough on each app with attention to contrasting cultural representations and technical mediators. Differences between apps made aspects of their vision, business models, governance, and technical design especially salient for further analysis. Although the method’s actual use follows research design principles that stress developing a research question before choosing a method of investigation, the comparative app walkthrough exercise was an inductive first step—an exploratory immersion in the apps—toward considering pertinent research questions for more formal deployment of the walkthrough method. However, issues noted earlier concerning the method’s failure to recognize different kinds of platforms, apps’ demands for user data and interactions with users’ devices, and the crucial role of users in the method raised challenges when Duguay ran this exercise.

In Duguay’s courses and workshops, the instruction to compare findings from two apps that serve a similar purpose has given rise to comparisons between: Facebook and VKontakte, Instagram and Snapchat, Google Maps and Waze Navigation, Spotify and Apple Music, Strava and Nike Fitness Club, and many others. Upon completing the exercise, groups were asked to present comparisons worthy of further analysis. Some unearthed fascinating findings, such as the difference between how Facebook’s account deletion functionality guilts users into reconsidering their choice by displaying photos of friends who they will miss while VKontakte’s deletion is more abrupt with a short dropdown menu of choices and the ability to announce one’s reason for leaving. Such differences led the group to imagine next steps for research, such as an investigation of the Russian app ecology, thinking perhaps there are few other popular options for social media if VKontakte is unphased by users deleting their accounts. Alternatively, further research could involve a Facebook walkthrough focused on identifying the app’s mechanisms for retaining users, including and beyond the deletion screen, contributing to scholarship about the complications of leaving this platform (Karppi, 2018; Light & Cassidy, 2014).

In other instances, groups worked through the exercise quickly and had little to share at the end. Despite the temptation to dismiss this outcome as stemming from group misalignments, distractions, or disinterest, Duguay noticed it was most prominent when students chose to examine connector apps, comparing, for example, Uber with Lyft or DoorDash with Foodora. The most underwhelming report came during a graduate course in which a group compared two apps for ordering pizza. With these apps involving zero interaction among users and limited functionality, the comparative app walkthrough did not spark any further research ideas. However, looking over their shoulders during the exercise, Duguay noticed that although each app was configured for a different company, their interfaces were nearly identical. The apps were infrastructural platforms and companies that wished to reach consumers were their key users. Thus, while the user interfaces for placing pizza orders were straightforward, a walkthrough of these apps from the perspective of a pizza franchise could have revealed more about the infrastructural platform(s) for shaping of this slice of the fast-food industry. Similarly, explorations of connector apps would have been more engaging from the perspectives of individuals offering goods or services, who are more dependent on the platform than end-users, but the creation of an Uber driver or Dasher profile is too elaborate for an in-class activity. Even so, students might have noticed more about these kinds of apps had the method included discussion of their multisided and multilayered functionalities.

Aside from their common use in everyday life, connector and other simple apps attracted learners during walkthrough training due to the reduced investment they required in terms of data and disruption to the personal context in which most apps are used. Thanks to widespread public discourse about data controversies, stemming in part from popular documentaries like The Social Dilemma (Orlowski-Yang, 2020), and whistleblower coverage such as that garnered by Frances Haugen’s Instagram revelations, as well as their courses in media and communication, an increasing number of students have told Duguay they are wary of sharing their information with an app company for the purpose of a class activity. Some groups elected only one individual to download the apps, forfeiting their hands-on experience with the method’s technical walkthrough, and were hesitant to use an existing email account or single sign-on service through Google or Facebook. Making a new email account is often taxing, with several email providers requiring two-factor authentication through a mobile number. Students worried about forgetting to delete the app later, accidentally allowing it to retain and accumulate data over time. If using apps already on their phone, students’ data could be inadvertently brought into group discussions, especially when their personal profiles and connections were included in walkthrough field notes or screenshots.

Furthermore, the method assumes individual access to a mobile device that can be put to research purposes. As personal, mobile devices become intimate due to their focal role in everyday life (Mowlabocus, 2016), asking learners to add an app among their personal repertoire and freely share their screen with group members during the walkthrough may stretch social boundaries that are otherwise firmly in place in the classroom. At a fundamental level, the assumption that personal devices are available for the walkthrough breaks down when learners arrive short on battery life, storage space, or the technical capacity to run certain apps due to the age of their phones. Thus, the method assumes learners and researchers have access to the appropriate technical resources and can build the necessary boundaries between their personal and research use, or are comfortable foregoing these boundaries. Such conditions for applying the method are called into question in relation to those with varying socioeconomic status or vulnerable identities whose personal information may put them at risk if implicated in research or learning contexts.

Finally, interactions with other users posed challenges for learners’ application of the walkthrough method, whether engaging with others or attempting to avoid them. Often in opt-in workshops held at conferences or summer schools, in which scholars were seriously considering using the method in their dissertations or research projects, participants noted that they could not fully assess an app’s functionality without interaction. Certain screens, features, text, and images were only shown to users within exchanges, such as dating apps’ match screens and messaging functionality. Some learners simply innovated on the method, taking heed of the ethical warning to avoid interaction with the app’s existing users by instead having group members connect with them and stage interactions through the app. Yet others refused to engage in some steps of the walkthrough, abstaining from actions within the stage of “everyday use” that may give certain impressions to existing app users, from intentionally posting a photo on Instagram or creating a TikTok video to subtler acts, like joining a Facebook group, which would be announced automatically in users’ feeds. They especially avoided these activities when using a personal account, concerned their actions during an in-class exercise would be incongruent with their longer-term social media presence. These hesitations reflect the method’s inevitable need to reckon with users as part of the research environment, grappling with the lacuna in one’s findings if interactions are avoided altogether or deciding how to navigate interactions ethically in light of the researcher’s subjectivity and relationship to the platform and others.

Walkthrough in Action II: TikTok’s Stumbling Blocks

Launched in 2018, TikTok is a short video platform that saw rapid uptake in 2019 and further accelerated growth in 2020 when youth took to the app to sustain social connection and share entertaining media during the COVID-19 pandemic (Kaye et al., 2022). News outlets report that TikTok accrued billions of users and became the most downloaded app internationally in 2022 (Evans, 2022), with the PEW Research Center finding that 67% of American teens use the app (Vogels et al., 2022). We determined that discussion of this highly popular app would be useful for identifying hurdles in the application of the walkthrough method for two reasons. Given Duguay’s research about LGBTQ+ people’s use of social media and Gold-Apel’s focus on ADHD (attention-deficit/hyperactivity disorder)-related networked publics, we were already aware of public discourse about how TikTok’s algorithmic profiling and datafication made such users feel seen (e.g., Boseley, 2021; Joho, 2022), but algorithms and data flows were not central in the original method. Second, the walkthrough method has featured in emerging TikTok research (e.g., Bhandari & Bimo, 2022; Kaye et al., 2022; Zulli & Zulli, 2020) and while scholars’ choice to use the method demonstrates its utility in investigating the app, their publications focus on findings with little discussion of the method’s limitations. Thus, Gold-Apel conducted a technical walkthrough in May 2022 with the present Apple iOS version of TikTok, following the steps of registration, everyday use, and closure/leaving, supplemented with exploration of the app’s environment of expected use (e.g., examining company materials for creators, developers, and users), with attention to instances when the method became insufficient for fully understanding how the app guides users—the method’s focal purpose. The following three realities of TikTok’s functionality and design presented complications for applying the method in research concerning this app.

The Centrality of the For You Feed

TikTok’s Creator Portal states that the For You feed (colloquially shortened to FYP ⁴ —For You Page—by users) is a “central feature” of the platform, designed to surface an endless stream of videos that are curated to a user’s interests. The walkthrough confirms that TikTok’s design reinforces the FYP’s centripetal force: users are brought to this screen that is designated as “Home” immediately upon opening the app, prior to registration, and they are returned there after registering. Users unfamiliar with TikTok may find it hard to navigate away from the algorithmically curated video feed to explore other functions and features of the app, like private messaging or content creation. In their TikTok walkthrough, Bhandari and Bimo (2022) found that, “Visually, such activities are presented as secondary to the content presented by the algorithm . . . While other platforms obfuscate the fact of algorithmic intervention, TikTok dispenses with this illusion by highlighting the role its algorithms play” (p. 5). Since the FYP is configured as users’ main functionality for encountering content, researchers studying TikTok may ask why certain individuals are seeing videos about particular topics, but the original walkthrough method does not specifically address how to analyze algorithmic functionalities.

Since many algorithms are value-generating mechanisms for commercial entities, information about how they process user inputs is often lacking or ambiguous. TikTok’s documentation provides examples of different factors that its recommendation system processes, which fall into the broad categories of user interactions (exchanges with others on the app), video information (captions and other data associated with videos users create), and device settings including one’s country and device type (TikTok, 2020). To determine which user inputs alter the FYP’s curation, researchers could apply algorithm audit or reverse engineering approaches (Diakopoulos, 2014; Sandvig et al., 2014), rigorously recording the actions they take in app and documenting the algorithm’s outputs. However, the multiple data points TikTok collects, and the opaque ways these are processed in aggregate, place demands on researchers to closely trace TikTok’s data flows and consider elaborate research personas as a means of curating data inputs.

Hyper-Individualization Through Complex Data Flows

TikTok’s design and vision emphasize a personalized experience for all users, which is realized through its approach to collecting and processing user data. The app’s Privacy Policy (TikTok, 2022) outlines the extensive range of data collected, from information about images and audio within users’ video content to data from third-party platforms, cookies, and the user’s device. However, it is difficult to tell how these data are mobilized within everyday use and how they are weighted in the recommendation system. In creating a new TikTok account, the app prompted Gold-Apel to register either through a third-party platform, such as Google or Facebook, or by entering a phone number or email address. Avoiding data sharing functionality between platforms, she registered with an email address newly created for this purpose. She was then required to enter her birthday to ensure compliance with TikTok’s minimum user age of 13 years. Here, the app’s personalization creates a fork in user experiences, as TikTok looks rather different to users under 16, who automatically have their accounts’ visibility set to private and do not have access to the direct messaging function (TikTok, 2021). Similar junctions followed as Gold-Apel was prompted to select interests from a long list of options, such as comedy, daily life, and life hacks, and was encouraged to allow TikTok to access contacts through the device “to help you connect with friends.” While several apps provide such personalization options for manual completion, and the original walkthrough method identifies that the options or menus made available can provide indications about the app’s anticipated users and uses, such data are compiled in conjunction with volumes of automated data collection that is generally imperceptible during the walkthrough.

In the process of exploring TikTok’s everyday use, researchers generate data that shape the content they encounter, further altering the app experience in ways that are hyper-individualized. Advertisers are encouraged to use “behavior targeting” to serve ads to potential consumers based on the behavioral data TikTok collects, which ranges from video viewing completion, liking, commenting, and sharing to viewing others’ profiles and following them (TikTok for Business, n.d.). These data metrics, along with what may be gleaned through image, text, and audio recognition tools applied to user content, means that trying out app features within a research setting continuously alters the researcher’s relationship to the platform and other actors, such as advertisers and creators who may respond to indications of engagement. TikTok’s automatic collection of data concerning the user’s device, Wi-Fi network, and location (Zhao, 2021) constitutes additional inputs that can be challenging to track. It also raises questions about how such data shape the research, considering, for example, how institutional Wi-Fi networks may affect recommendations in contrast to home Wi-Fi networks.

With many manual and automated data inputs, TikTok walkthroughs warrant close attention to the researcher’s account and the persona constructed initially as well as how the app’s treatment of a persona may change while the researcher navigates the app. Zulli and Zulli (2020) show how the method can be altered to work with, rather than against, TikTok’s algorithmic curation. They performed two walkthroughs: one author became a “‘regular user’ [of TikTok] to experience the video-editing process and content tailoring based on active participation” while the other author “avoided specific platform engagement (to the best of her ability) to observe the general platform design, user and platform patterns, and activity flows” (p. 5). In this sense, multiple and varied personas can be created and comparatively implemented in iterative walkthroughs to determine the influence of particular data points on TikTok’s algorithmic recommendation system. However, it is necessary to recognize the limitation that even a relatively empty account, intended to act as a control or baseline, is unlikely to be a completely neutral research tool due to the app’s automatic detection of user data. Alternatively, TikTok’s hyper-individualization is ripe for the use of fully fledged personas, such as those developed through Albrecht et al.’s (2019) performance studies approach, which allow for an ethnographic or autoethnographic understanding of how the app interacts with archetypical accounts, profiling their likely choices and actions. In these ways, researchers may benefit from leaning into the app’s hyper-individualization and personalization by intentionally configuring accounts to work with TikTok’s algorithmic profiling as a means of accessing particular user networks or content flows.

Ephemeral Interface Instantiations

TikTok’s rapid uptake has inspired many changes to its user interface over time. While the app’s newsroom webpage and blog for advertisers reflect near-constant shifts in tools for creators and businesses alike, the technical walkthrough—as a snapshot of an app at a particular time—fails to capture this dynamism. As Gold-Apel completed the technical walkthrough, TikTok released an update that replaced the Discover tab in the bottom menu with a Friends tab. This change drastically altered how users searched for content beyond the FYP: rather than navigating to the Discover tab to see trending hashtags and video themes with content by strangers, centering the Friends tab encouraged users to follow content by those already known to them. While Gold-Apel recorded this change in field notes during the technical walkthrough, its impact could not be judged within a single iteration of the method.

Three months following the menu change, the authors found that the Discover tab had returned and the Friends button was gone. Following platform historiography approaches (Helmond & Van der Vlist, 2019), the authors checked advertiser and developer resources but uncovered no indication of when exactly the Discover tab was reinstated. Instead, we found traces of information regarding the feature’s reception and broader context outside the app and its company-produced materials. For example, tech industry coverage from May 2022 when the change first took place deliberated the implications of the Friends tab for users, creators, and advertisers (e.g., Lin, 2022). TikTok’s tweet on 5 May announcing the change (TikTokComms, 2022) was met with a slew of unhappy replies, some of which reprimanded the platform for trying to be like Instagram or other apps, with the lasting outcry indicating that the Friends tab remained visible for some users until June. This coverage and outcry accompanying a specific app change indicate that methods considering an app over a longer duration and taking into account struggles among platforms and their stakeholders, such as the platform biography (Burgess & Baym, 2020), can shed further light on evolutions in TikTok’s interface, especially those met with controversy. Given that a TikTok support page describing the Friends tab was still live in November 2022, methods that consider different users’ interface configurations (e.g., depending on location, age, privacy jurisdiction) may also be warranted in investigating TikTok over the long-term, given the reality that few apps function universally across users.

Conclusion: Toward Multidimensional App Analysis

Inspired by Duguay’s teaching experiences and Gold-Apel’s TikTok research, this article has explored the walkthrough method’s enduring relevance and emergent shortcomings in terms of unfolding technological and research developments. The original method establishes an understanding of how one might approach platforms and apps from a combined STS and cultural studies perspective, considers how such technologies and their cultural references configure users, and specifies standards for ethical conduct that aim to avoid disruptions to users. However, the intensification of apps’ data extraction, algorithmic profiling, update cycles, as well as connections forged among users, advertisers, and other apps and devices within an increasingly complex platform ecology warrant consideration of supplemental and alternative methods. Scholars have proposed approaches to understanding different kinds of platforms, tracing data flows, auditing algorithms, attending to research choices and personas, and building app historiographies and biographies while applying feminist lenses to examine whether users can ever truly be bracketed from research. Such insights are pivotal for understanding the challenges experienced by learners when applying the method, such as the boredom that arises from treating all platforms identically or the urgency of considering user interactions. They also alleviate the method’s limitations or constraints when applied to algorithmically driven apps, such as TikTok, providing new approaches to tracing data inputs, constructing research personas, and making choices that can tell researchers about the app’s datafication, algorithmic curation, and interface changes over time.

Given our close evaluation of this method in relation to advances in app studies, it is clear that the walkthrough method should not be discarded but can instead serve as a springboard for deeper and more extensive app analysis. Its key phases of establishing the environment of expected use and conducting the technical walkthrough provide the researcher with a foundational understanding of an app’s cultural context and technological functionality. However, the method’s minimal engagement with data flows, algorithmic processes, users, and personalized functionalities leaves gaps, depending on whether these elements are of importance in a particular study. There is no sense in trying to subsume all approaches to the study of apps under one comprehensive method, as it leads to a burdensome process for those looking to apply a coherent method for app analysis. However, conforming to the original method’s scope may also foreclose new methodological innovations, which will surely continue to develop as app technologies change. Rather, we have illustrated the utility in being able to supplement, extend, and interchange the walkthrough method with other approaches across platform and app studies. Together, these and emergent methods can be viewed as an essential and expanding toolkit for examining a range of apps. Therefore, we recommend increased dialogue and experimentation among those whose research concerns these technologies, which will enable researchers to apply the most appropriate methods for their specific questions and projects as well as for interrogating the particular sociotechnical configurations of the apps they study.