Observational Listening: Methods and Criticism

Whenever connoisseurship is found operating within science or technology we may assume that it persists only because it has not been possible to replace it by a measurable grading. — Michael Polanyi, Personal Knowledge: Towards a Post-Critical Philosophy

Observational listening is an integral part of every facet of audio. From the recording engineer who selects microphones based on their sound to the consumer choosing loudspeakers in a showroom, the observational listening experience plays a powerful role in music recording and reproduction. At every stage in the recording process, countless judgments are made about the quality of the perceived sound. (6)

No one doubts the necessity or utility of observational listening. Yet judgments of sound quality expressed in published product reviews are criticized as capricious, fantasy-inspired, biased, a result of the placebo effect, or influenced by external variables, while value judgments made at every other point in the recording/reproduction chain are accepted without question. Audio equipment reviewers are singled out for attack by objectivists—the “journalistic elite� (7)—even though reviewers employ the identical observational listening methodology as the recording engineer selecting microphones or the violin maker evaluating his creation.

Contrary to the objectivists’ misconceptions, much observational listening as practiced by magazine reviewers is conducted under carefully controlled conditions—more controlled, in fact, than the conditions present during many other stages of the music recording and reproduction processes. Levels between components are matched to within 0.1dB or less. Linear differences, such as whether the product is polarity-inverting or not, are accounted for. Listening sessions are conducted daily for weeks or months before the review is written. A wide range of familiar source material is used over long periods of time and over a variety of equipment, precluding the possibility of ascribing a particular characteristic to a component that is actually a characteristic of the recording.

I cannot speak for the profession as a whole, but my listening (and that of my colleagues who write for the same publication) is anything but casual. My listening room was purpose-built from the ground up solely for music listening. Many of my colleagues are active recording engineers and use their own recordings in evaluating equipment. Some are musicians, daily exposed to the ultimate reference of live, unamplified instruments in an acoustic space. All of us became reviewers because of our lifelong dedications to music and music-reproduction technology. We take our responsibility to readers—and to audio truth—seriously; our attitude is the antithesis of caprice or whim. Indeed, the content of this paper exemplifies an approach to observational listening that is anything but cursory, casual, or superficial.

The objectivists dismiss observational listening by claiming the listening setup is sloppy, haphazard, casual, and that the reviewer has no regard for the sonic differences imposed by easily explainable measurable differences. According to this argument, the differences heard among components are nothing more than differences in level or frequency response—objective differences to which reviewers are supposedly oblivious. Further, observational listeners are often characterized as technical know-nothings operating from platforms of ignorance. (8) That may sometimes be the case, but the technically competent and conscientious observational listener should not be condemned by association. Just as there are varying levels of competence in any field, an entire philosophical position—the validity of the listening experience—cannot be summarily dismissed because some of its practitioners fail to uphold the highest standards.

Observational listening impressions reported in product reviews are also attacked because of alleged reviewer bias. But in my own work (and that of my colleagues), the positive or negative tone of a review is based solely on the component’s sonic performance. The product’s brand name, size, cost, reputation, faceplate thickness, and whether the manufacturer buys advertising play no role in the review’s outcome. Although an expensive product from a reputable manufacturer will be expected to sound better than an inexpensive product, any such preconceptions vanish when the products reproduce music. The listening experience is the sole criterion by which a product is judged. Other factors, such as build quality, value, user interface, and compatibility with other products play minor roles, but it is the product’s ability to convey the music’s meaning that determines the positive or negative tone of the review. It is not unusual for an expensive product from a company with an excellent reputation to receive a negative review, or for a low-priced product to receive a favorable assessment.

Further, there is a high degree of correlation between the descriptions of a particular product’s sonic character published by different magazines around the world. Reading several reviews of a product—reviews sometimes published simultaneously—often reveals a clear consensus about the component’s specific sonic signature. This evidence that observational listening as practiced by magazine reviewers reveals the unique musical characteristics of individual audio components is dismissed by objectivists who allege premeditated fraud. The resort to conspiracy theories is always the sign of a weak argument.

The entire purpose of observational listening is to discover and report to the magazine’s readers how the component affects the musical experience. The magazine’s continuing success is predicated on the accuracy of the sonic descriptions and the judgments of the products as determined by the world at large. The growth of high-end magazines—and their increasing commercial success in a marketplace long dominated by publications reflecting an objectivist philosophy—reflects the concurrence between reviewers’ and readers’ observations and experiences.

Next: Listening vs. Measurement

Pages: 1 2 3 4 5 6 7 8 9 10 11 12