Sounds Convincing: Modes of Listening and Sonic Skills in Knowledge Making

Abstract

This article investigates the role of listening in the knowledge making practices of Western scientists, engineers, and physicians from the 1920s onwards. It does so by offering a two-dimensional typology of the modes of listening that they employ. Distinguishing between two dimensions allows us to make sense both of the purpose and of the ways in which scientists, engineers, and physicians have listened to their objects of study; and it also allows us to appreciate the importance of shifting between modes of listening. At the same time, we argue, understanding the role of sound in knowledge making cannot be limited to the study of listening alone; rather, we have to pay attention to how listening is embedded in broader sonic skills — including the handling of tools for the making, recording, storing, and retrieving of sounds.

Well and sorrowfully do we know the listener who is no listener at all, who passively sits through a concert, intellectually contributing nothing; waiting, like a cabbage or a stone, for something to happen to him. He hears without listening.

(Gibling 1917, 386)

Sophie Gibling's essay from 1917, entitled ‘Types of Musical Listening’, is a fine example of the kind of typology that held much sway with cultural commentators and musical critics in the early twentieth century. After distinguishing several types of imperfect listeners, her essay culminates in a description of the ideal listener. The ideal listener prepares himself for concerts emotionally and intellectually, listens past imperfections in specific performances to appreciate the beauty of the composition, and is ready to merge completely with the music, becoming a ‘purely abstract spirit’ (389). Listening well, to Gibling, is a matter of the ‘quality of a man's personality’ (388).

In the century following the publication of Gibling's essay in the Musical Quarterly, many more typologies of ‘modes of listening’ have been published, spanning fields as diverse as cultural studies, musicology, media studies, communication studies, and psychoacoustics. Like Gibling, some authors display strong preferences for particular modes, while others question the value of normative judgements (Stockfelt 1997, Subotnik 1991). Alongside those concerned with musical listening, authors have offered taxonomies of listening for domains such as radio broadcasting (Douglas 1999, Goodman 2010), film sound (Chion 2005 [1990]), or everyday environments (Truax 2001 [1984]).

This article extends the discussion of taxonomies of listening to yet another set of empirical domains: science, medicine, and engineering. Distinguishing between modes of listening, we argue, is helpful to understand how Western scientists, doctors, engineers, and mechanics have created and justified knowledge claims by listening to the sounds of bodies, machines, and other objects of research. It thus specifies and strengthens earlier claims about the role of hearing, as one of the non-visual senses, in the sciences (see below). In order to do so, we develop a two-dimensional taxonomy of their listening practices — one which takes into account both the purposes of listening and the ways of doing so. A particular strength of this taxonomy is that it allows us to show how practitioners shift between different modes — an ability which is at least as important for knowledge making as the competence in using any given specific mode of listening. At the same time, the focus on listening should not be isolated from other skills. Rather, we claim that we substantially deepen our understanding of knowledge dynamics by showing how listening modes inform the use of sonic skills in processes of knowledge making.

Indeed, when we use the word ‘mode’, we do not intend to focus exclusively on the cognitivist dimension. We understand the modes of listening as being linked to particular bodily practices and embedded in a broader set of sonic skills. Sonic skills, in our approach, include not only listening skills, but also the techniques doctors, engineers, and scientists need for what they consider an effective use of their listening and recording equipment. Such skills may entail the proper positioning of a stethoscope on a patient's body, the efficient use of a magnetic tape recorder in bird sound recording, or simply archiving sound. While Jonathan Sterne (2003) has coined the notion of ‘audile techniques’ to include listening, the bodily postures and the mediation involved in the use of listening instruments, we refer to sonic skills in order to discuss making, recording, storing, and retrieving sound in addition to listening to sound.

The goal of our article is thus to show how listening modes, and the ways in which they feed into sonic skills, clarify processes of knowledge production in science, engineering, and medicine. It is structured in four main parts. In the first section, we introduce our case studies and methodological approach to studying the listening practices of scientists, doctors, engineers and mechanics. We then explain the origins of our typology of listening modes, which has taken inspiration both from existing scholarly work in sound studies and from actors’ categories. In two subsequent sections, we outline our typology in more detail, paying attention to its two dimensions (purposes and ways, or: the why and the how). In the final two sections, we outline how an analysis of listening modes can usefully be integrated into a study of broader sonic skills, and substantiate how both notions — listening modes and sonic skills — are relevant for understanding processes of knowledge production.

Research context and case studies

Empirically, our article draws upon research done in the context of the project Sonic Skills: Sound and Listening in the Development of Science, Technology and Medicine, 1920-now (Bijsterveld 2009). This project set out to investigate the contested status of listening in Western science. The prevalence of visual metaphors for knowing (‘I see’) (Tyler 1984) and of graphic representations (Pauwels 2006, Wise 2006) have been taken as evidence for the existence of a visual culture of science. At the same time, recent scholarship has indicated that, while the sense of vision is undeniably important, other senses also play a role in the practices of scientists (Mody 2005, Kursell 2008, Burri et al. 2011), physicians (Bynum and Porter 1993, Lachmund 1999, Rice 2008), engineers and mechanics (Borg 2007, Orr 1996), even if they rarely find mention in official research publications.

Against this background, the Sonic Skills project has explored the role of listening practices in the knowledge-making of scientists, physicians and engineers from the 1920s onwards. It has asked how and with the help of what tools scientists, engineers, and physicians have employed their ears in making sense of what they studied; how these listening practices have generated new knowledge; and under which conditions sonic skills have been accepted or rejected as ‘objective’ entries of inquiry. By studying the 1920s and after, the project has taken the rising use of sound recording technologies in the sciences into account.

The project has studied these sonic skills in a number of different empirical domains. These domains, however, are not disciplines such as audiology or acoustics, in which sound is foregrounded as an object of study, but rather, ones in which sound is used as an entrance to knowledge acquisition. These domains include car mechanics and engineers listening to engines (Krebs 2012a, 2012b, 2012c); doctors listening to patients’ bodies and hospital equipment (Van Drie 2013) and medical students learning to listen (Harris and Van Drie forthcoming); ornithologists in the field and the laboratory listening to birds (Bruyninckx 2012, 2013); and sonification researchers developing and listening to auditory displays as alternatives or additions to scientific visualizations (Supper 2012a, 2012b, 2014). The project is thus interdisciplinary in its scope and subject matter, as it compares listening practices and other sonic skills in different disciplines. But it is also interdisciplinary in its approach, as it combines historical and ethnographic perspectives.

Geographically, the cases span Western Europe, the United States and Australia. They cover different phases in research and design, ranging from tinkering with technology on shop floors and trial-and-error in laboratories, to recording natural phenomena and the display of data. This article presents a first effort to draw the results of the different case studies together into a coherent perspective on the listening practices of scientists, physicians, and engineers.

Existing taxonomies of modes of listening: Analytic and actors’ categories

Thinking about listening modes has a long tradition both within the academic field of sound studies, and among practitioners who work with sound. In this section, we want to discuss both the scholarly and the practical inspirations for our own typology of listening modes.

Perhaps the most well-known typology of modes of listening — although it may be more accurately characterized as a typology of listeners — is the one developed by Theodor Adorno (1976 [1962]), in which he distinguishes figures such as experts, good listeners, culture consumers, emotional listeners and entertainment listeners. Adorno makes no secret of his preference for structural listening, nor of his contempt for practices such as entertainment listening. With this preference, Adorno's typology echoes the concerns of many music theorists and critics since the middle of the nineteenth century, beginning with the Viennese music critic Eduard Hanslick, who provided an opinionated taxonomy to distinguish ‘poor listening practices from the true method of listening, aesthetic listening’ (Hui 2013, 34). Similar taxonomies accumulated in the course of the nineteenth and early twentieth century, usually taking attentive, absorbed listening as the gold standard. ¹ Anxieties about proper listening modes were not limited to the world of classical music, but also took hold in the domain of radio broadcasting. Historians of broadcasting, for instance, have traced public debates about proper modes of listening to the radio back to the 1930s, when many commentators warned of the social, political and psychological dangers of distracted listening (Goodman 2010), and tried to persuade ‘the listener to “incline their ear” not only in the right direction (towards beautiful, honest and reputable things), but also in the right way (selectively, attentively and with appropriate discrimination)’ (Lacey 2013, 183).

In the course of the twentieth century, however, these typologies with their outspoken normative preferences increasingly came under fire or were qualified by being put in historical perspective. Fervent critiques of Adorno's categorization scheme and his advocacy for structural listening have been expressed within musicology (Subotnik 1991, 1996, Stockfelt 1997), and have inspired a slew of work proposing relativistic and postmodern alternatives (Dell'Antonio 2004). The music theorist Ola Stockfelt (1997), for instance, argues that different modes of listening are appropriate for and indeed demanded by different genres — with Adorno's preferred mode, structural listening, being adequate only for a very specific type of Western art music. Modes of listening should therefore be judged in terms of adequacy to a genre: adopting an adequate mode means being able ‘to listen for what is relevant to the genre’ (Stockfelt 1997, 137). With this, Stockfelt moves away from treating modes of listening as personal characteristics (linked to particular character traits or socioeconomic factors), and instead treats them as part of a repertoire from which individuals can choose. Shifting between different modes is not only possible, but common.

This possibility of shifting between modes of listening (often akin to shifting between levels of attention) has been an important element of many recent typologies. For instance, in his book on Acoustic Communication, Barry Truax (2001 [1984]) distinguishes three modes of listening, each of them characterized by a different level of attention. The first kind, background listening, is entirely passive and refers to listening that is not directed at achieving any practical purpose. By comparison, listening-in-readiness (for instance the practice of recognizing a vehicle by its sound) is more active, while listening-in-search (as exemplified by a ship captain whistling and using the echo for the purposes of orientation) is more active still. Truax’ research is embedded in a normative concern about noise pollution in modern society, as he worries that the skills of listening-in-readiness and listening-in-search are rapidly dwindling due to the noise of modern technology.

In her book Listening In, Susan J. Douglas (1999) offers an ‘archaeology of radio listening’ that goes well beyond a distinction of concentrated and distracted listening practices. Among the many listening modes she mentions are practices such as linguistic, musical, informational, exploratory, story, advertisement or fidelity listening (1999, 33–35). Some of Douglas’ categories describe listening in terms of what someone is listening to (e.g. stories or ads), some in terms of what they listen for (e.g. information or sound quality), and some in terms of how they listen cognitively (e.g. dimensional listening, in which the listener imagines spaces, or associational listening, in which networks of memories are triggered through sound).

In this article, we follow in Douglas’ footsteps as we develop a typology that operates on more than one dimension, but we do so in a way that explicitly discusses how these dimensions relate to each other. Like Douglas and other scholars, we start from the idea that listeners dispose of a repertoire of listening modes between which they can shift. Yet we go one step further in arguing that shifting between different modes of listening is not only possible and common, but that the very capability of shifting is itself an essential skill in the knowledge-making practices of many scientists, engineers, and doctors.

Before delving into the details of our own two-dimensional typology of listening modes in the sciences, however, we want to briefly acknowledge that it is inspired not only by scholarly work in sound studies, but also by the discourses of the actors that we studied. That is to say, some of our categories are based on actors’ categories. For instance, the distinction between ‘monitory listening’ (listening to monitor whether everything is working well) and ‘diagnostic listening’ (listening to diagnose the specific source or cause of a problem), which we will explore in more detail in the next section, is present in the discourse of car mechanics themselves. It played an important role in the process of the formalization and professionalization of the trade of German car mechanics in the 1930s, as discussed by Stefan Krebs (2012a, 2012b, 2012c). In order to convince car drivers to entrust their faulty cars to the new profession of certified car mechanics, the mechanics needed to ensure that car drivers would trust their own ears enough to know when to bring their car to the garage, but not enough to try to fix it themselves. Once it was established that there was a problem, the task of diagnosing and fixing the specific problem would be left to the mechanics. In their effort to gain exclusive jurisdiction over the ability to repair cars, the mechanics delineated their skill of professional diagnostic listening from the monitory listening skills of drivers. This distinction between two listening modes thus helped the mechanics in their quest for cultural authority.

Explicit references to taxonomies of listening modes are even more widespread in the scientific community dedicated to sonification research, which aims to systematically explore the use of sound to represent data and convey information. Indeed, the distinction of different modes of listening has been a rather stable feature of the sonification literature, from some of the founding texts of the community (Gaver 1989, Williams 1994) to recent contributions (Vickers 2012, Grond and Hermann 2014). An example is William Gaver's (1989) work on the SonicFinder, an early attempt to use auditory displays as part of a computer interface, and thus a forerunner of now-ubiquitous sounds such as those informing the user about new e-mails, or those accompanying the action of moving a file into a digital trashcan. It builds upon the fundamental distinction between two modes of listening: everyday listening and musical listening. The former is directed at identifying the sources of a sound, while the latter focuses on its formal characteristics, such as pitch or timbre. Gaver's displays are designed to exploit the abilities of everyday listening in particular: ‘If sounds are to be used in the interface, they should be used much as they are in our everyday lives. […] We do not hear the pitch of closing doors; instead we are more likely to hear their size, the materials from which they are made, and the force used to shut them’ (Gaver 1989, 72f.) This distinction still resonates in sonification research today, and — although many sonifications do demand a certain amount of musical listening skills, as pitch is a widely used parameter in sonification designs — so does the emphasis on everyday listening skills (Hermann 2011). Indeed, since the sonification community struggles with the fact that many potential end-users of sonification (specialists in the scientific domains from which data are translated into sound) are reluctant to use sonification because they distrust their own ears, ² references to everyday listening can reduce the fear of listening, and thus potentially help to convince people of the usefulness of sonification.

Similarly, scholars in the sonification community frequently make a distinction between synthetic listening (which involves listening to sounds holistically) and analytic listening (which involves focusing on specific elements of the sound), which in turn has been appropriated from literature in auditory perception research (Hermann 2002, Walker and Nees 2011, Williams 1994, Worrall 2009). This distinction also turned out to be useful for our own typology, which takes up the categories of analytic and synthetic listening — along with an additional category of interactive listening — as two fundamentally different ways of listening.

Our typology thus makes use of existing categories employed by the actors that we study, but only as partial inspiration, together with secondary literature. ³ The most important contribution of our typology over the existing actors’ categories, however, is the fact that it looks at different modes of listening in two dimensions, and puts them in a broader context of the sonic skills that are involved in knowledge production.

Purposes of listening: Why scientists, engineers, and physicians listen

Our proposed typology operates on two dimensions, taking into account both the purposes for which scientists, engineers, and physicians listen, and the ways in which they do so. By distinguishing three modes on each of the two dimensions, we end up with nine possible combinations in total. In this section, we distinguish between three purposes of listening: monitory, diagnostic, and exploratory. Subsequently, we will explore three different ways of listening: synthetic, analytic, and interactive.

Monitory listening refers to checking for possible malfunctions — for instance, when car drivers pay attention to ‘the rhythmic and silent run of the engine’ and ‘the regular humming of the gearbox or chain drive’ (Küster 1919, quoted in Bijsterveld and Krebs 2013, 20). Monitory listening is also employed in the scientific laboratory and field by researchers checking the proper running of their equipment (Bruyninckx 2013, Mody 2005), and in the hospital by doctors and nurses monitoring the vital signs of patients. Monitory listening usually accompanies other tasks and activities, often unrelated to sound — whether those are driving a car, operating a microscope, or performing surgery on a patient. The fact that the sound can be perceived in the background while focusing on other tasks, but that sudden and unexpected changes in the sound nonetheless immediately draw the attention of the listener, is of great benefit here. Similarly, the ability to ‘monitor multiple processes simultaneously’ (Dayé and de Campo 2006, 350) has been argued to be an advantage of sonification over graphic displays. Consequently, sonifications developed specifically for the purposes of monitoring have been an important area of sonification research in recent years, ranging from applications developed for the medical field to those that monitor web server activity. Indeed, it has been claimed that the monitoring of information in the background, while paying attention to another task, is where sonification and auditory display really ‘come into their own’ (Vickers 2011, 456).

While monitory listening is concerned with establishing whether something is wrong, diagnostic listening is about pinpointing what is wrong precisely. The quintessential example of diagnostic listening is that of physicians using their stethoscope during physical examinations to distinguish the ‘normal’ sounds of a healthy body from the ‘abnormal’ ones of a sick one, and to diagnose specific diseases based on those sounds. This skill is not limited to the medical field, but practitioners from other domains frequently reference the listening practices of doctors to explain their own: ‘If the physician cannot make his diagnosis by the appearance of the patient, he will take his stethoscope and listen to the patient's body. This is how you ought to proceed with the car engine as well’ (Hessler 1926, quoted in Krebs 2012c, 83) In ornithology, the skill of diagnostic listening is essential for the correct identification of species, but also for ensuring sufficient recording quality. In sonification, too, diagnostic listening has an important role in quality control, as errors in sonification design are often picked up by listening.

Exploratory listening refers to listening out for new phenomena. The notion was developed by Douglas (1999) for the practice of amateur radio hams trying to discover distant stations, but also plays a role in the listening practices of scientists. Narratives of field observation in ornithology, for instance, often feature ornithologists letting themselves be guided through the woods by ear, always listening out for rare, exotic or appealing bird songs, such as in this account by the naturalist J. Schafer: ‘While going through a thicket of hazel brush, briars and vines, a bird was heard singing so softly that it was some time before I could locate the exact place where the song came from. After listening a short time I recognized the song to be that of a Catbird, but to make sure of the identity of the singer, it was driven from its hiding place’ (Schafer 1916, quoted in Bruyninckx 2013, 36).

Although sonification is usually a more mundane activity, taking place with headphones in front of a computer screen, the exploratory listening of sonification researchers too can get entangled with romantic narratives of adventurous scientists making chance discoveries thanks to their dedicated attention to their sonic environment. See, for instance, this description of the solar wind sonifications by Robert Alexander:

Alexander typically compresses 44,100 data points into a second of sound, the sampling rate of a compact disc.

Then, he puts on his headphones.

On that particular day he found a hum everywhere in the data. ‘I thought I was hearing noise’, he recalls.

But it was more than that. The hum had a frequency of 137.5 hertz which would correspond to about 26 days in the original data. That would be the time taken for a particular feature on the sun to swing back around. In other words, he could lock on a feature and listen in.

Alexander realized what he was hearing and messaged a colleague. ‘The frequency I'm listening to is the rotation speed of the sun. I don't think anyone's ever done this.’ (Markendaya 2012)

In a recent video documentary for Vice Magazine, Alexander explains the exploratory nature of his listening: ‘I was digging through, you know, 20 or 30 different data parameters and listening to them all, and I realized that if I listened to carbon, that I could hear a very strong harmonic presence’. ⁴ In the astrophysical research group he was working with, carbon had until that point not been mentioned as being relevant to the study of solar wind; instead, different types of solar wind had been distinguished through measurements of oxygen charge states. It was through listening to different sonic realizations of the same dataset that Alexander's research group became aware of the potential of carbon as a more reliable indicator for solar wind activity. The results were written up, submitted to and published in The Astrophysical Journal — with a brief mention of the sonification process that had led to the discovery, but with none of the romantic flourishes of a lone researcher making a chance discovery when donning his headphones (Landi et al. 2012).

Ways of listening: How scientists, engineers, and physicians listen

The three modes of listening discussed so far were concerned with the purposes for which scientists, engineers, and physicians listen; in the following, we want to introduce three additional modes, which describe the ways in which they do so: synthetically, analytically, or interactively. These three modes are not mutually exclusive from the ones introduced in the previous section; rather, any given listening practice of a scientist, engineer or physician can always be characterized both in terms of its purpose, and in terms of its way. A car driver listening to the roar of the engine while driving, for instance, engages in both monitory and synthetic listening.

The term synthetic listening comes from literature in auditory perception research, and has become a mainstay of sonification literature. Its meaning is usually defined in opposition to another category in our typology, analytic listening. For instance, in the first book publication on sonification, the terms were defined as follows:

Synthetic perception takes place when the information presented is interpreted as generally as possible; for example, hearing a room full of voices or listening to the overall effect of a piece of music. Analytic perception takes place when the information is used to identify the components of the scene to finer levels; for instance, listening to a particular utterance in the crowded room or tracking one instrument in an orchestral piece or identifying the components of a particular musical chord. (Williams 1994, 98)

This definition of synthetic listening and analytic listening is also captured in Albert Bregman's (1994) influential work on auditory perception, and still resonates in sonification discourse today (Hermann 2002, Worrall 2009, Walker and Nees 2011). For sonification, both synthetic and analytic listening play a role: both the ability to perceive complex auditory events on the whole, and that of breaking it down into its component pieces and singling out particular streams of sound for attention — and indeed, the capability of switching between these different modes is considered an important asset for the use of sonifications. The experience of attending a concert is often used as an example both for the ability to perceive a piece of music as a whole, and for analytically attending to specific instruments within the piece: ‘For example, in a concert hall we can hear a symphony orchestra as a whole. We can also tune in our focus and attend to individual musical instruments or even the couple who is whispering in the next row’ (Hermann et al. 2011, 3).

When medical students learn to use their stethoscopes, they primarily learn the skills of analytic listening: navigating an initially confusing world of sound by differentiating the sounds of the patients’ bodies from the sound produced by the tool itself and the sound of their own body. However, the skill of synthetic listening is equally important for the practices of scientists and engineers. In many cases, successful use of sonic skills involves the combination of both analytic and synthetic listening at different stages of the process of knowledge-production. For instance, the quick identification of a bird in the field often involves synthetic listening, as ornithologists listen for general features, recognizing the bird ‘more by the quality or style, or both, of its utterance than by the number and succession of its notes’ (Summers 1916, 79). However, once such a quick identification through synthetic listening has been made, the ornithologists may listen analytically to rule out confusion with similar-sounding species, or to notate specific elements of the sound.

Although synthetic and analytic listening are usually defined as opposites, they have one important aspect in common: both modes assume that the sound source itself is stable or unfolds according to its own dynamic rules. In many instances where scientists, engineers, and physicians listen, however, they actually intervene into the sounds while listening. We therefore distinguish an additional way of listening, that of interactive listening. If synthetic listening means hearing the whole orchestra, and analytic listening means focusing on a particular stream of sound (e.g, the second oboe), interactive listening means that the listener decides to replace the second oboe by a didgeridoo halfway through in order to better grasp the dynamics of the piece. Scientists, physicians, and engineers often engage in such interactive listening in order to find out more about their subjects. Ornithologists, for instance, may interact with the birds that they study by deliberately exposing them to specific sounds — such as recordings of birdsongs or traffic noise — in order to elicit a response (Bruyninckx 2013, 94f.). Car mechanics, too, often engage in interactive listening, for instance when listening for changes in the sound of the engine while changing gears; and so do car drivers, when they pay attention to the sounds of the car in deciding when to shift gears. Interactive listening to car engines can thus serve both monitory (for drivers) and diagnostic (for mechanics) purposes.

Interactive listening is also common in sonification research, where it is used mainly for diagnostic or exploratory purposes. ⁵ As our ethnographic research has shown, diagnostic interactive listening is especially common during the design process: errors in the sonification design (or even in the underlying dataset) often express themselves as discrepancies between expected and actual sounds, and can be corrected by alternately adjusting settings and listening to the results until the expectations and the outcome are aligned with each other. Exploratory interactive listening has also become increasingly popular in sonification research; the growth of a whole subfield dedicated to ‘interactive sonification’ is testament to this trend (Hermann and Hunt 2005, 2011). In interactive sonifications, users can ‘change selections quickly and easily to gain multiple auditory viewpoints’ (Flowers 2005, 4), which are intended to give a better understanding of the data, especially for exploratory tasks.

At a first glance, the categories of synthetic, analytic, and interactive listening may seem similar to Barry Truax’ (2001 [1984]) distinction of background listening, listening-in-readiness and listening-in-search. For instance, Truax’ example of listening-in-search, in which a ship captain whistles and uses the resulting echo for navigation purposes, could be considered an instance of interactive listening. However, in Truax’ categories, these three modes are distinguished by different degrees of active attention paid by the listener. In our scheme, it is not a matter of different degrees, but rather of different kinds of attention. And although the categories of synthetic and analytic listening have emerged from psychoacoustic research, our scheme does not assume that these listening skills are limited to the mind only. Rather, they go hand in hand with particular bodily strategies — think, for example, of a doctor percussing a patient's chest, an ornithologist cupping his hands around his ears, ‘rotating slowly like an aural CCTV camera’ (Lorimer 2008, 391) while listening out for a particular bird, or a sonification researcher convulsing in pain when enduring unpredictable and piercing sounds in an attempt to identify errors in the sound-generating computer code.

Distinguishing modes of listening on two dimensions can give us a multi-layered and nuanced appreciation of the listening practices involved in scientific research, medical work and engineering. Looking at only one dimension in isolation would give us a very partial understanding of these practices. For instance, when looking only at why scientists, engineers, and physicians listen, we may overlook the specific bodily and cognitive skills that this entails. There is not one single technique for monitory listening — listening for the purposes of monitoring may involve either focusing on general patterns of sound (as car drivers do when listening out for the auditory feedback of their car engines and surroundings), or focusing on particular elements of that sound (as physicians do on their daily ward round when checking whether specific symptoms that were discovered during yesterday's diagnosis cleared up), or even interacting with the source of that sound (as ornithologists do when playing a sound recording to a bird to elicit a reaction). Looking at the dimension of how we listen in isolation without taking into account the purposes, on the other hand, risks losing sight of why these sonic skills matter in the first place — the listening modes could then be taken as ends in themselves, rather than analytical tools telling us something about how scientists, engineers, and physicians use their bodies and senses for particular ends that in and of themselves may not have anything to do with sound.

Sonic skills: Virtuosity in shifting modes and handling tools

Although the graphic representation as a table of listening modes (see Table 1) might at a first glance give an impression of stagnancy and rigidness, its real strength lies in providing a stable reference point for the dynamic listening practices that we found. The professional status of some practitioners — such as doctors or car mechanics — is intimately intertwined with their recognition as expert diagnostic listeners. Yet as we argue in this section, it is often the ability to shift between different modes of listening (rather than the specialization in one specific mode) that expresses the virtuosity of their sonic skills, and helps them to underpin their knowledge claims. Furthermore, we argue, this ability to shift is intimately linked to the handling of tools and to broader sonic skills that go beyond the modes of listening themselves.

TabLE 1

OVERVIEW OF LISTENING MODES

why	how	Synthetic listening	Analytic listening	Interactive listening
Monitory listening		Listening to overall features of sound for the purposes of monitoring.	Attending to specific characteristics of sound for the purposes of monitoring.	Interacting with a sound source for the purposes of monitoring.
Diagnostic listening		Using a (quick) overall impression of a sound for the purposes of diagnosis.	Attending to specific characteristics of a sound for the purposes of diagnosis.	Interacting with a sound source for the purposes of diagnosis.
Exploratory listening		Listening out for general impressions for the purposes of exploration.	Attending to specific features of sound for the purposes of exploration.	Interacting with the sources of a sound for the purposes of exploration.

In the everyday knowledge practices of the experts we studied, the different listening modes often build upon each other. For instance, it is important for the successful work of car mechanics — for which diagnostic listening is essential — that car drivers recognize the need to bring their car to the garage, for which monitory listening is crucial. In some instances, such a shift in the purpose of listening goes hand in hand with a shift in the way of listening. For instance, the solar wind sonifications (described above) were, at least at first, an example of synthetic exploratory listening, as Robert Alexander somewhat randomly listened for general patterns in the data in the hope that something of interest would jump out at him. Once he noticed harmonic presences in the charge states of carbon, however, a shift seemed to occur towards listening to a particular element to diagnose the dynamics of solar wind activity. Doing so also involved focusing his attention on one specific aspect of the sound. In other words, a shift occurred not only in the purpose of listening (from exploratory to diagnostic), but also in the way of listening (from synthetic to analytic).

In the last two examples, diagnostic listening followed monitory or exploratory listening, while analytic listening followed synthetic listening. It might be tempting to conclude from this that the modes of listening always occur in a fixed order, with diagnostic and analytic listening as the natural culmination and end-point. This is not the case, however. An example from ethnographic research in the hospital, conducted by Anna Harris, demonstrates that monitory listening does not always precede diagnostic listening. During the initial examination after a patient is admitted to a hospital, doctors usually engage in diagnostic listening. But when performing subsequent check-ups during their daily rounds through the hospital, they are more likely to perform monitory listening, as they check whether specific symptoms detected during earlier diagnoses persist or have cleared up.

In fact, the different modes often occur in a constant back-and-forth shift, wherein the listener repeatedly zooms in and out. Many sonification designs — especially those made for the purposes of data exploration — are deliberately built to facilitate rapid shifts between different modes of attention. In Thomas Hermann's (2002) dissertation on sonifications for exploratory data analysis, for instance, the listener is described as engaged in different modes of listening, and many of the proposed sonification designs feature multiple streams of data, which can be listened to simultaneously or separately. It thus seems to be designed with a listener in mind who may at times synthetically listen to several streams of sound simultaneously, and at other times analytically zoom in on particularly promising specific sound streams. This is interjected, of course, with the occasional intervention into the sound source itself. Constantly shifting back and forth between analytic, synthetic and interactive modes of listening is thus facilitated and intended by the sonification design.

This example flags the connection between the ability to shift between modes of listening and the availability of particular tools. The introduction of a novel type of stethoscope in German automotive engineering during the interwar period would be another, earlier example of how tools can effectively enable the process of mode-shifting. While listening rods and traditional stethoscopes had been used to focus on one component of the car engine, this new stethoscope had two sensors. These sensors, the Tektoskop and the Tektophon, enabled mechanics to listen to two differently located components of the engine simultaneously and make a detailed comparison of the sounds they heard (Anonymous 1929b, quoted in Bijsterveld, Cleophas, Krebs and Mom, 2014, 80). Such a construction afforded shifting between synthetic and analytic listening, as it brought together distinct sounds in one listening ‘frame’, while keeping the option of alternation in auditory focus alive.

Similarly important for the skill of mode-shifting has been the rise of sound recording technologies in ornithology. The phonograph, sound camera, gramophone and tape recorder enabled ornithologists to record the sounds of birds in ways that many of these scientists considered more accurate, less dependent on individual listening capacities and therefore more ‘objective’ than the notation of sound in traditional onomatopoeic terms, musical staff or graphs (Bruyninckx 2013, 63). In addition, these tools allowed them to repeat their listening exercises as often as they wanted, or even to slow down the speed of the recording, thus affording precision in notation after the field trip (Bruyninckx 2013, 49). This enabled cycles of analytic listening, as repeated listening allowed ornithologists to focus on different components of bird sound across different listening sessions.

The slowing down of gramophones was useful not just for ornithologists engaging in the manual notation of bird sound, but even for those favouring automated visualizations. For instance, the ornithologist William Thorpe, a strong advocate of the spectrograph (an instrument that visualizes the frequency and amplitude of sound over time), acknowledged that the analysis of spectrograms was best accompanied by listening to the sound recordings, and in particular, by doing so at reduced speed. Slowing down gramophones from 78 to 28 cycles, according to Thorpe, made the complexities and varieties of bird sound such as high frequencies and rapid sequences more easily accessible to the human listener and allowed focusing on different components of the sound than those prevalent with 78 cycles (Thorpe 1958). Thorpe also stressed, however, that recordings played back at such reduced speed would ‘at first hearing have no apparent resemblance to the original’ (Thorpe 1958, 542). Despite the analytic value of listening at reduced speed, then, synthetic listening was often best accomplished at the original speed. An important function of the model of ‘infinitely variable speed turntable’ favoured by Thorpe (1958, 542) is precisely that it enabled scientists to quickly and easily switch not just between different speeds of playback, but also between different modes of listening. The availability and use of particular recording and playback tools thus affected the options for modes of listening, and therefore the character of mode-shifting, which in turn fed into the knowledge claims formulated.

Tools can open up particular modes of listening and particular means of shifting between modes, but they may also enhance the epistemological status of listening practices in science, medicine, and engineering. As Tom Rice (2008, 2010) and Melissa van Drie (2013) have underlined, the stethoscope is an important signifier, a visual icon, of the expertise and jurisdiction of the doctor. Even as the practical importance of auscultation for medical diagnosis has declined over the years, the stethoscope has kept its symbolic function. Indeed, its symbolic sway reaches beyond the confines of the medical field, as other professional groups have also appealed to the symbolic authority of the stethoscope. For instance, in the automotive industry, engineers and mechanics are often portrayed in white coats and using a stethoscope on a car engine, thus explicitly alluding to the image of a doctor using a stethoscope to examine a patient (Krebs and Van Drie 2014). Tools may thus function as symbolic capital underlining the epistemological authority of their users.

It is important to note that the sonic skills involved in knowledge-making practices are not a matter of listening alone. The examples of ornithology and sonification used in this article have already hinted at the fact that both the recording and the design of sounds are equally important elements of sonic skills. Similarly important are the ability to reproduce sounds through physical mimicry (Harris and Van Drie forthcoming), or to store, retrieve, and circulate sound recordings (Bruyninckx 2013). This, too, can be a matter of epistemological authority. Many sonification researchers, for instance, regard the fact that ‘the traditional carrier of the symbolic knowledge generated by science, paper, hardly begins to meet the requirements of communicating sound’ (Dayé and de Campo 2006, 360) as a major stumbling block for the scientific acceptance of sonification. While sound has traditionally been difficult to circulate and integrate with written text — and thus to function as what Bruno Latour (1990) calls a scientific ‘inscription’ — the development of digital media that allow for an easier integration of text and sounds may help sound recordings to (partially) catch up with graphical images when it comes to exerting scientific authority (see Supper 2012b, 2015). Here, too, tools and sonic skills are intimately intertwined.

The missing third dimension: Listening to what?

In this article, we have proposed a two-dimensional typology of listening modes, as well as some reflections on how these listening modes — and in particular the ability to shift between different modes of listening — link up to other sonic skills in knowledge-making practices. However, a third dimension of listening has mostly been taken for granted in our analysis: the dimension of what it is that scientists, engineers, and physicians are listening to. Sidestepping this dimension in our typology was a deliberate decision: while the other two dimensions allowed us to come up with a finite number of categories that would nonetheless exhaustively describe the listening practices of scientists, engineers and physicians, the question of what it is that they are listening to opens up an infinite number of possible answers. It therefore defies categorization. Nonetheless, that does not mean that the subject matter that listeners lend their ear to is irrelevant.

In fact, it matters a great deal what scientists and other practitioners listen to, and it would therefore be a grave mistake to disregard the subject matter when discussing listening modes and sonic skills. Indeed, in this final section of our article, we want to use an example from ornithology to argue that subject matter and sonic skills are intimately intertwined: the sonic skills involved in listening to, recording, storing or retrieving sound co-define the conception of the objects under study.

In the 1930s, British and American ornithologists struggled immensely with the technical and logistic complexities of making sound recordings of birds in field settings. While Cornell University student Albert Brand and his colleagues used a large, sensitive sound camera and testing equipment to capture bird sound, amateur birdwatcher Ludwig Koch and his British companions — ornithologists and recording engineers working for the BBC — worked with a phonograph recorder and wax discs. What they had in common, however, was that they had to move heavily loaded vans around to do their recording work. Not only did this affect where they could make recordings — accessible by road, for instance — it also influenced what they recorded. Although the sounds of nature had their primary interest, their microphones also picked up the sounds of modern civilization, as they often discovered after the fact and with great disappointment (Bruyninckx 2013, 64 ff).

Each of the two groups came up with their own technical solutions to these challenges. Choices regarding the type and positioning of microphones, for instance, had implications not just for how they made their recordings, but also for their conceptions of birdsong and their research findings. The British group carefully installed sets of microphones around the space in which a particular bird was expected to produce its song, and would adjust the sound level of each microphone afterwards, in the editing process. With a little luck, this resulted in recordings that captured both the bird's song and its environmental sounds, giving the recording an atmospheric touch — even though bird song was foregrounded. Should the bird fly away, however, the whole set-up would have to be re-installed (Bruyninckx 2013, 71 ff).

Unlike the British group, who worked on BBC nature films that were meant to educate and entertain a wide audience, the ornithologists at Cornell were primarily interested in establishing a firm scientific reputation. This may partially explain why they adopted a different approach to recording, making use of a parabolic reflector surrounding a microphone.

The principle was such that the surface of the parabola reflected sound waves to a dynamic microphone at its focal point. Focusing the sound waves like this drastically increased the input to the recording equipment and concentrated it to at least twenty decibels louder than the sounds not caught by its narrow shape, which amounted to an amplification of about fifteen times (Sellar 1976, quoted in Bruyninckx 2013, 73).

This reflector enabled the Cornell ornithologists to pick up bird sound from a considerable distance, making the exact position of the microphones less important and bringing less accessible parts of nature within easier reach. At the same time, microphones with parabolic reflectors staged a ‘sterile sound’, creating a close-up’ of ‘instances of acoustic behaviour’. This, argues Bruyninckx, did not only create a form of sound that was half-way between the laboratory and the field, but also ‘a still-life motif in a clearly demarcated acoustic landscape’ (Bruyninckx 2013, 55, 76, 79).

It is only since the beginning of this century that ornithologists began to realize that their focus on bird sound proper had come with a price. Recent research has signalled that a particular bird species sings at a higher frequency when living in urban areas than the same species living in rural surroundings (Bruyninckx 2013, 151). For a long time, ornithologists had simply missed this possibility as a consequence of their preference for clean sound. They had treated environmental noise as disturbance rather than as informant. As their approach had been both enabled and constrained by their tools, their carefully crafted sonic skills had indeed affected their knowledge claims.

Conclusions

We have argued that we need both the notion of listening modes and the notion of sonic skills to understand how sound has been used as an inroad to knowledge making in science, medicine, and engineering. We started our analysis by presenting a typology of listening practices. Certainly, we have not been the first to present such a typology. Typologies of listening abound in scholarly work on sound. Building on such literature and on our own case studies of Western scientists, doctors, mechanics, and engineers, we have distinguished between six modes of listening, operating on two dimensions. While monitory, diagnostic, and exploratory listening refer to different purposes of listening in the sciences, analytic, synthetic, and interactive listening express particular ways of listening. We have also stressed, however, that an important skill in the work of scientists, engineers, and doctors is not just the ability to engage in any given one of these modes, but especially that of shifting between modes. Tools and instruments, such as multi-channel stethoscopes, tape recorders and sonification software, enable particular forms of listening and mode-shifting.

We also flagged that sonic skills are not limited to listening skills. Although our notion of sonic skills encompasses the skills experts need to employ such listening modes, they also encompass the ability to design, record, store, mimic and retrieve sound. All of these sonic skills are linked to the handling of specific instruments. Virtuosity in sonic skills thus means not just the ability to use our ears, but also that to handle various tools and instruments. These tools often play not just a practical, but also a symbolic role in the knowledge practices of scientists, engineers, and doctors: they can function to symbolically enhance the epistemological status of listening.

Sonic skills have repercussions for the knowledge claims that can be made in science, engineering, and medicine. The decision to employ a particular technique in the recording of sound, for instance, is not an innocent matter: it can affect the substance of knowledge claims that are made, and the conceptions of the objects under study. In order to understand the knowledge practices of scientists, engineers and physicians, it thus pays to consider the listening modes and sonic skills that are involved in their production. Doing so deepens our insights in the role of sound and listening in the sciences, and might also inspire research into the contribution of other non-visual senses in knowledge making.

As for sound and listening, there is more to unravel than we have done so far. We have, for instance, only just started to explore the relevance of immediacy in the sonic skills used for ‘life-saving’ actions concerning patients, engines, and expensive laboratory machines, and thus for the status of such sonic skills in science, medicine, and engineering. We are also interested in the continuous re-opening of debates about the status of sensory information with the introduction of new, and epistemologically still unstable tools in science that make the invisible visible, the inaudible audible or that translate data from one sensory mode into another, as was the case for the spectrograph. There is thus still much to be learned about the conditions under which sonic skills are accepted or contested in knowledge making practices. Without an understanding of modes of listening and their relation with tools and the skills to handle these tools, however, we cannot tackle such issues.

Notes on contributors

Alexandra Supper is an assistant professor in the Department of Technology and Society Studies, and academic coordinator of the Graduate School of Arts and Social Sciences, both at Maastricht University. Previously, she was visiting professor of the Science-Technology Society at the University of Vienna. She holds a PhD in science and technology studies from Maastricht University, and an MA in sociology from the University of Vienna. Her research has been published, among others, in the journal Social Studies of Science and in the Oxford Handbook of Sound Studies.

Karin Bijsterveld is an historian and professor of Science, Technology, and Modern Culture at Maastricht University. She is author of Mechanical Sound: Technology, Culture, and Public Problems of Noise in the Twentieth Century (MIT Press 2008), co-editor of the Oxford Handbook of Sound Studies (Oxford University Press 2012, with Trevor Pinch) and co-editor of Sound Souvenirs: Audio Technologies, Memory and Cultural Practices (Amsterdam University Press 2009, with José van Dijck). Her most recent book is the monograph Sound and Safe: A History of Listening Behind the Wheel (Oxford University Press 2014, with Eefje Cleophas, Stefan Krebs and Gijs Mom). See also: http://fasos-research.nl/sonic-skills/.