Listening Is Making Sense

(essay for the field notes #2 by the German label Gruenrekorder)

Speech, music and noise all reach our ears in the form of sound. In spite of this, we have grown used to considering them as separate spheres. Overcoming this separation results in a kind of listening that integrates sensual and cognitive perception: listening to the city as though it was music, to music as though it was speech, and to speech as though it was some kind of noise. These different kinds of sound each fulfil two functions: they transmit information and they have aesthetic value, registered by our senses as an experience ranging from painful to pleasurable. In the following, I would like to try to approach the phenomenon of sound from various angles, concluding with a look at field recording as an artistic strategy in which this integrated kind of listening plays a special role. But I will begin by looking for a definition of sound, an issue in which the question of a sound’s location is particularly important.

Sound as event

What is sound? Acoustic science gives the answer that a sound is triggered by the movement or vibration of a sound source, creating longitudinal pressure waves in a surrounding medium that spread in a wave-like motion. This medium is usually air, but it can also be water, helium, metal or any other sound conductor. Such an understanding of the phenomenon of sound depends crucially on the medium and on the prevailing conditions of pressure and temperature. The sound itself – its volume, pitch and timbre – is determined by these external factors. As land creatures, we humans have become used to perceiving sound via the air, usually somewhere close to sea level at a moderate temperature. But this is actually just one of many ways sound can travel. Due to the importance of this link between a sound and its surrounding medium in judging that sound, many have come to accept the wave view, according to which sound is equated with sound waves and their behaviour in a medium. This permits the explanation of many acoustic phenomena – the Doppler effect, sound cancellations, echoes – that derive from the movement of sound sources or the interference and reflection of sound waves. According to this theory, the answer to the question of where exactly a sound is located is: in the medium.

In 1999, Robert Pasnau contradicted this medium theory, pointing out that in our everyday perception, we continue to identify sounds with their source. We localize sounds where they originate and we do not think of them as moving through a medium or being dispositions of the medium itself. If our senses are not constantly deceiving us, then the wave view must be false. Pasnau argued that sound is actually a property of the object, a quality as peculiar to the sounding body as its colour. At first glance, this seems to make sense, as key acoustic properties of a sound are determined by the physical properties of the object that produces it. Consequently, sound is identical with the vibration of a sounding body and is a characteristic of this object. Following this view, the location of a sound is always the sound source and not the medium.

Here too, however, objections can be formulated that cast doubt on this property view. The argument that we perceive sounds in a localized way only applies to the medium of air; due to the higher speed of sound in water, divers, for example, can no longer pinpoint sounds in space, they appear to be coming from all around, from all directions. If we base our thinking about sound on our own mode of perception, our conclusions will be subject to our own preferences in the choice of medium, thus overlooking the fact that sound behaves differently in different mediums. This applies above all to cases where there is no medium, as in a vacuum, in which it is widely accepted that no sound can be created. The property theory must fail, since it claims that sounds are a property of sound sources, thus also existing in a vacuum. As Casey O’Callaghan (in "The Argument from Vacuums”) points out, not only is sound in a vacuum impossible to perceive, and thus not provable by experiments, but there is also a fundamental absence of the conditions for the existence of sounds and the development of acoustic properties.

Where is the sound if not in the medium and not in the sound source? So far, we know that the sound properties of an object depend on the specific properties of the surrounding medium: when the medium changes, the sound changes with it. Rather than conceiving of sound purely as a property of sounding bodies, we must always think of it in relation to the surrounding medium. In his book sounds and several essays (see here and here), Casey O’Callaghan takes a third approach to describing sound in terms of cognitive theory. His event view is based on the assumption that sound waves transmit information about sounds, but are not identical with sounds. Instead, sound waves constitute a stimulus to audition and are the result of events that occur when objects and sounding bodies interact with a surrounding medium. For O’Callaghan, sounds are “particular individuals” which have a duration and which differ from other sounds by specific sound properties. This idea of sounds as events corresponds to what we know about our auditory perception, about hearing. (A good overview of the various theories on sound can be found on the Stanford Encyclopaedia of Philosophy website)

Sound as an auditory object

We constantly scan our surroundings for acoustic information. Our ears cannot be shut, our sense of hearing is always on receive. All that we hear is initially shapeless noise – flat, without spatial depth, not localised. From the day we are born, we learn to single out individual sounds from this acoustic chaos and to pinpoint them in space. Soon, we are able to identify specific sounds with specific objects and to recognize them. To achieve this, our brains refer to mental representations with which what we hear is constantly compared. In the historical development of our species, this ability to deduce a sound’s source in a matter of seconds and to react accordingly was of vital importance, a matter of life and death.

A distinction is generally made between hearing and listening: hearing is attributed a passive role, while listening involves the active processing of acoustic stimuli with the help of cognitive operations. Another model might distinguish between acoustic and auditory processing of information. Acoustic perception is the first stage in processing sound signals, during which signals are registered and roughly categorised, but not subjected to any further analysis. Even at this stage, however, meaning is attributed to stimuli and particular behaviour can be triggered. The auditory perception of the second stage is where a more differentiated analysis of the acoustic message takes place, bringing a large number of processes to bear: identification of sounds, segregation of the sound mix, cognitive processing, checking against long- and short-term memory, interpretation, evaluation, and emotional reactions to what has been heard. Albert Bregman calls this process of perception in two stages auditory scene analysis.

How is it that our sense of hearing is capable of reconstructing the surrounding reality purely on the basis of information derived from overlapping pressure fluctuations in the air? To clarify what the auditory system does, Bregman makes a comparison with seeing: It is as if what is taking place on a lake – boats passing, people jumping into the water, etc. – were to be measured with the help of two channels dug up from the shoreline and which fill with water from the lake. A handkerchief fixed across each channel would be moved by the waves that reach the shore and flow up the channel. On the basis of nothing but the movement of these handkerchiefs, the task would be to ascertain the number of boats passing, their distance and direction, whether someone is jumping into the water, etc.—all without looking at the lake. The auditory system is capable of this.

Let us imagine our acoustic environment as a neural spectrogram. The aim of auditory scene analysis is to create a separate description of every object in this environment. According to Bregman, sounds are the result of “happenings,” and he calls the mental representation of such a happening an “auditory stream”. This stream can consist of several sounds perceived by the listener as a single event (someone talking, someone playing the piano, a dog barking, etc.) or it can consist of a single sound. Perception is a permanent process of constructing mental representations, and an auditory stream is one stage in this process.

An auditory scene – i.e. the diffuse sound mix that reaches our ears in everyday life – consists of such auditory streams, overlapping in time and space. Out of this, our perceptual apparatus constructs auditory objects by grouping incoming sound waves and attributing them to sound sources. The brain performs these distinctions on the basis of sound properties such as pitch, timbre and loudness. During this process, our ear appears to be occupied above all with the localization of sound sources. It is possible to correctly perceive an auditory object, but not to be able to correctly localise its source. The auditory object is not identical with the object by which the sound is generated (this is the reason for many acoustic illusions, e.g. the ventriloquist’s effect and the McGurk effect).

In some cases, however, the auditory objects we perceive contain sounds that do not reach our ear as physical sound waves at all – as highlighted by the case of acoustic illusions. One way our brain constructs auditory objects is by assessing their overtone content. In certain cases, this may suggest a fundamental tone that is not actually present. Our brain creates this tone by deduction from the overtone spectrum. (Such tones are called “missing fundamentals”. On the telephone, we hear voices with more deep tones than are actually transmitted by the receiver. In sound technology, similar effects are used to suggest a larger bass range in small loudspeakers.) A related effect associated with combination tones (also known as the Tartini effect) is used by organ builders to create deep tones not otherwise possible in a limited space due to the excessive size of the pipes required. Two organ pipes generate a combination tone that results from the difference in frequency between the two source tones (Discussed in detail in Matthew Nudd’s essay “Auditory Perception and Sounds”). The pipe-organ illusion shows that there is a difference between genuine perception and auditory experience; the latter also includes the above-mentioned acoustic illusions as well as tinnitus and other acoustic hallucinations, not to mention psychological factors and neuronal disorders.

Our senses are active information processing systems that allow us, from birth, to draw conclusions about the outside world even without cognitive processing operations. The capacity for auditory scene analysis seems to be innate and forms the basis for all other associated cognitive processing operations. The path from hearing to listening, from acoustic to auditory information processing, begins when we assign meanings to noises and establish semantic connections between them. The perception and processing of sounds is the first step on the path to linguistic competence and musicality. “Listening is making sense.”

Sound as sign

When a baby perceives the voice of its mother, it not only associates this voice with its location and particular timbre in order to recognize it among many other voices, but it also assigns to it the meaning of protection, security and source of food. The sound of the mother’s voice is transformed into a sign for an entire range of emotional and physical needs. To this are added other sounds, which are linked to other semantic content. In this process, auditory scene analysis helps to highlight important signals against the background of disturbing noise and to tell speech apart from music and other sounds. From the noises of spoken language, the child learns to separate out specific sound signs, to recognize them, and to assign meaning to them.

The association of a sound sign or word with a specific meaning in language is arbitrary; any other sign would do just as well. In the case of non-linguistic sounds, on the contrary, there is a firm link to the sound source; a sound or noise always refers precisely to the event that triggered it. The sound of a water drop is always a sign of the event that we associate with this sound: a drop of water falls and makes a sound when it lands. Higher-order meanings only arise when the simple meaning of an auditory object is enriched by an additional context, as in the case of signals with a communicative function anchored in everyday life. Sounds can also acquire symbolic meaning via religious or ritual practices; the cultural and social context inscribes itself into the meaning of the sound sign in the long term and determines its reception independently of its specific auditory form.

But sounds also acquire meaning based on their spatial distance from the receiver. One way of describing this spatial distance is by distinguishing between figure, ground and field. (This follows the terminology of Murray Schafer; one could also use other terms like “immediate, support, background” or “foreground, mid-ground, background”. For natural environments, Chris Watson used the terms “species, habitat, atmosphere”.) This denotes not only a distinction between sounds that are near, at medium distance and far away, but also the relationship between the listener and the sound source. Figure stands for the most important sound or group of sounds in a given setting which a listener relates to or interacts with. This may be an interlocutor, the melody of a piece of music, or a sound that stands out from the everyday soundscape. Although ground sounds still belong to the listener’s social context, they are often perceived in passing, only registered when they suddenly disappear. And finally, field represents a broader cultural context in which figure and ground are positioned. Taken together, these three levels constitute an acoustic frame of reference within which sounds are assigned varying degrees of importance. The sounds on the levels of “figure” are signals that we listen to, whereas the sounds on the levels of “ground” and “field” are heard but not accorded any great importance.

This model of an acoustic frame of reference describes not only the reality of our everyday world, but also the virtual locations of sounds created for the listener in various media and art forms. The perceived distance between the listener and these media sounds has no need to reflect reality; here, the levels of figure, ground and field are shifted and switched in order to convey the desired message. Since the beginning of sound recording, sound has become malleable material in the hands of artists and media producers who use the positioning of microphones and technical manipulation to detach sounds from their original context and combine them into new structures. Art forms that emerged as a direct result of the possibility of storing and processing sounds include Musique Concrète, electronic music, sound design in films, radio plays and the related genre of acoustic art, to name just a few. Finally, as mentioned above, I want to take a look at a genre that is characterized by a purist approach to sound, which often crosses the borders between language, music and noise, and which is concerned above all with situating sound in space: field recording. (This term was chosen in the 1930s for the documentation of unknown musical cultures and scientifically interesting animal sounds “in the field”. Initially, the emphasis was on the archiving of recorded documents. Only later did field recordings also become interesting to sound artists. Of central importance in this context is Murray Schafer with his World Soundscape Project that took nature as the subject of “soundscape compositions”.)

Sound and field

As a technical term, ‘field recording’ stands primarily for recordings that are made outside the studio. Today, it is associated above all with artistic strategies that draw our attention to sounds within a human or natural environment but without taking them out of their context. (There is also the term “phonography” which differs from “field recording” in its interest in leaving the original recordings unedited and – by analogy to photography – seeing them as direct, unaltered reproduction of an acoustic reality. The extent to which a recording is manipulated by the act of choosing and “framing” a specific segment remains a controversial issue. Whatever term we use, the main point here is the artistic intention.) Rather than isolating them from the outside world by bringing them into a studio, the sound artist goes to where the sounds occur. These sounds are usually unpredictable and uncontrollable, but above all they cannot be separated from their acoustic setting. This sound field is the event horizon within which sound artists move and from which they make their own selection by their choice of technical means and recording perspective. In principle, the practice of field recording is nothing more than a type of auditory scene analysis. The headphones amplify this process by enlarging the audible environment and making details stand out that would otherwise be missed (in the above-mentioned model: attention can be shifted from figure to ground or field). The sound artist gives listeners access to a personal interpretation of the environment, letting them merge with his/her perspective.

Field recordings focus attention on sound itself – as material, as the actual object of hearing. The texture and grain, the tactile quality of the found sound events becomes the artistic material. The intention of most artists working with field recordings will be to draw the listener’s attention not to a sound’s source, but to the peculiar qualities of the sound itself. It is not about making faithful acoustic portrayals of events or soundscapes for documentary purposes; instead, the sounds are listened to for their own sake, as sounds. Ideally, the referential quality of the sound, the direct link to its source object, dissolves and becomes, in our perception, an autonomous auditory object. But this doesn’t happen on its own: it requires a number of artistic interventions described in the following using the terms perspective, context and composition. These interrelated approaches have a shaping influence on outstanding works in field recording.

• Perspective
The selection of audio equipment and the positioning of the microphones in the field have such an influence on sound recordings that they constitute an artistic statement in their own right. In recent years, field recordings have made an important contribution to sharpening our awareness of sounds other than those transmitted by the medium of air. Examples include recordings with contact microphones (e.g. on wires, bridges and other resonating bodies) and the use of underwater microphones. Such approaches highlight the above-mentioned dependency of our sense of sound on the surrounding medium (rather than the recording medium). But recordings made with conventional microphones can also capture special sound properties if they are placed in unusual locations. One example would be recordings made inside bottles or pipes, allowing an appreciation of their specific resonance. The microphones do not have to remain in a fixed position and moving them within the sound environment can also be used to create a distinctive perspective. Many sound artists work with hidden microphones to avoid provoking an unwanted reaction.

• Context
Field recording means that environmental sounds are conserved and played back at another time and place, separating them from their original context. But this is true of all recordings and is thus not a distinctive feature. Working with context means departing from a linear model of communication (the artist sends an acoustic message that is deciphered by the recipient) and questioning, commenting and reflecting on the listener’s relationship to his/her social and natural environment (Barry Truax writes about this in his essay “Sound in Context”). In such an approach, the listener’s involvement in his/her acoustic environment becomes part of the artist’s practice. One such shifting of context involves playing back sounds in an environment and subjecting them to further processing. Jacob Kierkegaard made recordings in abandoned houses near Chernobyl and then played them back in the same rooms over and over, in the style of Alvin Lucier’s I’m sitting in a room, until the resonance of the space emerged. The fact that Kierkegaard made these recordings at a location that was quite literally “charged” plays an important role in the reception of the work. Another way of creating new contexts is seen in buildings by Francisco Lopez and air.ratio by Eric La Casa, works that deal exclusively with the noises made by ventilation systems, publishing them in almost lexical form on CD. In the context of a conventional recording situation, the sounds presented here would be unwanted background noise; the change of perspective and the way they are arranged on the recording medium transform them into an object for acoustic study.

• Composition
There is a sliding scale of compositional interventions in field recording, ranging from “neutral” use of ambient sounds through to simulations of acoustic environments that are created by montage and studio processing. The larger the compositional element, the closer field recording comes to other areas of electroacoustic music. Field recording is a technical procedure which is open to all sides and which can be integrated into other musical genres. Perhaps the greatest similarity between the sound artists working with field recording is their tendency to let themselves be guided by the found sounds, rather than approaching the compositional task with preconceived expectations. The artist composes and is composed at the same time. Field recording does not stand for a chain of sounds that release a single, one-way message from the god-like artist-author; it is a multi-dimensional space, a fabric of sound quotations that draws on the sound repertoire of the world.

With these artistic strategies, it is possible to make sounds and their meanings seem ambiguous and to leave the listener unsure of their origins. The formerly stable link between a sound sign and its source, the sound’s clearly referential character, is eroded. This calls for active listeners who do not wish to have their expectations confirmed, but who bring with them a certain curiosity and openness to the world. The vagueness of the recordings makes space for subjective interpretations, challenging the listener’s familiar auditory experience and knowledge of the world. What we hear may not always make any actual sense, but we do become aware of a fundamental requirement of our sense of hearing: we have no choice but to constantly construct meaning.




Bibliography:

Jean-Francois Augoyard and Henry Torgue: Sonic Experience, A Guide To Everyday Sounds, McGill-Queens’s University Press, 2005

Barry Blesser and Linda-Ruth Salter: Spaces Speak, Are You Listening?, Cambridge MA, 2006

Albert Bregman: Auditory Scene Analysis, MIT Press, 1990

Casey O’Callaghan: Sounds, Oxford University Press, 2007

Barbara Flückiger: Sound Design, Schüren, 2001

Stephen Handel: Listening, An Introduction to the Perception of Auditory Events, MIT Press, 1989

Leigh Landy: Understanding The Art of Sound Organization, MIT Press Cambridge, 2007

Theo van Leeuwen: Speech, Music, Sound, Palgrave MacMillan, 1999

Jean-Jacques Nattiez: Music and Discourse, Princeton University Press, 1987

David Rothenberg und Marta Ulvaeus: The Book of Music & Nature, Wesleyan University Press, 2001

Oliver Sacks: Musicophilia, Knopf, 2007

Götz Schmedes: Medientext Hörspiel, Waxmann, 2002

Daniel Schmicking: Hören und Klang, Königshausen u. Neumann, 2003

Barry Truax: Acoustic Communication, second ed., Ablex Publishing, 2001