1

Science, Music, and Cognitivism

1.1 Mechanical Philosophy, Mathematics, and Music

Natural philosophy, developed by such giants as Copernicus, Galileo, Boyle, Newton, and Descartes, reigned during the scientific revolution from 1543 (the year of Copernicus’s publication of On the Revolutions of the Heavenly Spheres) to 1687 (the year of Newton’s publication of Mathematical Principles of Natural Philosophy) (Shapin, 1996). Shapin notes that the natural philosophy that emerged during the scientific revolution could also be called mechanical philosophy because it recognized that a variety of machines could be created that appeared to be purposeful, intentional, or sentient. Early natural philosophers were inspired by the properties of clocks because during the scientific revolution a variety of clockwork automata were created (Wood, 2002).

Viewing the world as a clock, natural philosophy embraced mathematics for describing and explaining nature’s workings (Shapin, 1996). Mathematics played a key role in the theories of Galileo, Bacon, and Boyle; Newton’s discoveries revealed that physical laws expressible in mathematical form govern the universe. Music played a key role in the pursuit of the mathematical explanation of nature. Kepler sought musical harmony in the motions of the planets (Stephenson, 1994). Lagrange used musical sound to link properties of his calculus to the physical world (Dhombres, 2002). As a student, Newton explored various mathematical means for dividing the octave into smaller musical intervals (Isacoff, 2001). Many natural philosophers believed that music provided evidence of the mathematical perfection of the natural world.

Of course the relation between mathematics and music originated long before natural philosophy. Around 500 BC, Pythagoras linked perceived pitch to the frequency at which a string vibrated. The Pythagoreans also determined that the most consonant musical intervals are ratios of string lengths that involved simple whole integers: 1/1 for unison, 2/1 for the octave, 3/2 for the perfect fifth, and 4/3 for the perfect fourth. In contrast, the ratio for a very dissonant interval, the tritone, is 45/32. Pythagorean geometry led to the discovery of irrational numbers, but irrational numbers are also found in music, and are related to dissonant musical intervals (Pesic, 2010).

Pythagorean notions of consonance led to tuning discrepancies (Donahue, 2005). If one starts at one pitch, and moves to a note that is seven Pythagorean octaves higher, one does not reach exactly the same note that is produced by moving 12 Pythagorean perfect fifths higher. The two final notes differ by the so-called Pythagorean comma (which equals 24 cents, where 100 cents = 1 semitone). Thus, one cannot have a perfect musical tuning system that includes perfect ratios for both octaves and fifths. This taunted those who believed in the mathematical perfection of music or nature. If consonant ratios are mathematically perfect, then why can they not be used to tune instruments whose notes spanned multiple octaves? At the start of the scientific revolution, many scholars, motivated by this problem, attempted to develop alternative approaches to tuning (Cohen, 1984).

1.2 Mechanical Philosophy and Tuning

1.2.1 Tempered Scales

The new approaches to tuning that emerged during the scientific revolution were motivated by several profound changes in music (Cohen, 1984). First, music became polyphonic: more than one voice or instrument simultaneously performed different parts. As a result, musical harmony became central to music, and the consonant combination of multiple musical parts required a proper tuning system. Second, new intervals (thirds and sixths) became accepted as being consonant. The English composer John Dunstable popularized these intervals in the early 15th century. However, Pythagorean tuning ignores these intervals. New approaches to tuning had to ensure the consonance of these new intervals. Third, fixed intonation instruments—instruments with notes tuned to specific pitches that cannot be altered during performance, like the modern piano—were more central to music. In addition, the notes of these instruments ranged over several octaves.

All of these developments created a need for a practical solution to the Pythagorean tuning discrepancy. In general, mechanical philosophers addressed this problem by developing new methods for dividing the octave into smaller intervals to define sets of available musical notes: they invented tempered scales. The primary goal of a tempered scale is to remove the Pythagorean comma (Donahue, 2005). This is accomplished by ensuring that, from some starting note with a frequency f, the note an octave higher has a frequency of 2f. In other words, the octave has primacy. Then, some other notes are added; these notes conform to the Pythagorean interval ratios. Finally, the remaining notes are included. These notes involve intervals whose ratios necessarily depart from the Pythagorean ideals. In other words, a tempered scale is a compromise: it ensures that some intervals conform to the Pythagorean ratios but deforms other intervals to eliminate the Pythagorean comma.

One example of a tempered scale is called just intonation; it was explored by mathematicians of the early 17th century (Barbour, 1972). Just intonation produces consonant harmonies involving fifths and thirds, but at the same time can produce very dissonant harmonies for other musical intervals. Another example of a tempered scale is called mean-tone temperament, which was invented by Pietro Aron in the early 16th century (Barbour, 1972). Mean-tone temperament distorts perfect fifths. This tuning favours thirds over fifths, which is the opposite of what is found in Pythagorean tuning (Donahue, 2005).

One problem with tempered scales is that they are defined with respect to a particular musical key (i.e., a particular starting frequency f). This means that the notes that define a scale in one musical key differ from those that define a scale in another. To perform the same piece in a different musical key (i.e., to musically transpose the composition), the instrument must be re-tuned. Another tempered scale, equal temperament, offered a solution to this problem.

Equal temperament, first mentioned by French philosopher, theologian, and mathematician Marin Mersenne in 1636, divides the octave into 12 equal segments, each representing the musical interval of a semitone. Equal temperament solves the problem of musical transposition because one can move a composition from key to key without having to re-tune an instrument. As a result, it became an ideal tuning for keyboard instruments (Isacoff, 2001, 2011).

However, equal temperament brought with it a new set of problems. By dividing the octave into 12 equal segments, it introduces irrational interval ratios. Consider some base pitch with frequency f. The pitch an octave higher has a frequency of 2f, or, to make an explicit link to the mathematics of equal temperament, a frequency of 212/12f. In equal temperament the tone that is a semitone higher than f will have the frequency 21/12f, the tone two semitones higher than f will have the frequency 22/12f, and so on. The appearance of irrational ratios—defined as 2 raised to some x/12 power—had two negative consequences. First, calculating the desired frequencies—a requirement for actually tuning an instrument to equal temperament—was difficult. A number of specialized tools and methods had to be invented in the 18th century to deal with this problem (Scimemi, 2002). Equal temperament existed as a theoretical notion for a long time before it became practical to use it to tune instruments in the late 19th century. Second, the presence of irrational ratios meant that many musical intervals deviated enough from the Pythagorean ideals to sound less consonant. Indeed, the distorted ratios found in any tempered scale challenge the Pythagorean notion of consonance, as does the new musical aesthetic that considers intervals like thirds to be consonant.

This latter issue raises a theoretical problem concerning music that was also a concern of mechanical philosophers. Cohen (1984) calls it the problem of consonance. While the Pythagoreans identified particular frequency ratios as defining consonant musical intervals, they had no account of why this was so. The scientific revolution developed a very popular answer to this question that related consonance to the physical properties of sound. This answer is known as coincidence theory.

1.2.2 Coincidence Theory

In 1558, Gioseffo Zarlino explained the consonance of certain Pythagorean ratios with what is known as his scenario (Cohen, 1984). According to Zarlino, the consonant Pythagorean ratios all involve the first six integers; the set of integers from 1 to 6 defined the scenario. Zarlino proposed that any ratio of numbers that belonged to the scenario would be consonant, and supported this proposal with a variety of mystical arguments about why the members of the scenario were perfect numbers. In spite of this appeal to mysticism, coincidence theory was the most popular theory of consonance that emerged during this period (Cohen, 1984). It was then adopted, extended, and popularized by many leading figures of the scientific revolution including Galileo, Mersenne, Descartes, and Euler.

Coincidence theory attempts to establish the physical basis of consonance (Cohen, 1984). It begins with the observation that a plucked string produces sound by striking or percussing the surrounding air. We hear sound when these percussions reach our ears. Coincidence theory then recognizes that strings vibrating at different frequencies generate percussions at different rates. In some instances, the percussions generated by two different sound sources will reach the ears at the same time; these coincident percussions will be pleasing to the ear, or consonant. When the coincidence of percussions diminishes, so too will consonance. In other words, coincidence theory linked the purely mathematical view of consonance developed by the Pythagoreans to physical properties of sound vibrations, as they were understood in the 16th and 17th centuries.

1.3 Psychophysics of Music

The study of music during the scientific revolution produced new approaches to tuning and developed a theory that attempted to explain consonance in terms of coincident patterns of vibrations (Cohen, 1984). From this perspective, musical properties were physical properties of the world that could be studied scientifically and described mathematically. For instance, consonance was related to physical vibrations, and was not a construction of the mind.

1.3.1 Psychophysics

The systematic psychological study of consonance did not begin until the latter half of the 19th century; the golden age of this research occurred between 1840 and 1910 (Hui, 2013). It was during this period that psychophysical explorations of music began.

Psychophysics was invented by Gustav Fechner, who began by searching for relationships between physical properties of stimuli and the properties of the experiences that they produced (Fechner, 1966/1860). The psychophysical study of music began with attempts to identify universal mathematical laws that related physical properties of sound and music to mental properties of musical experience (Hui, 2013).

To Hui (2013), the psychophysical study of sound sensation is in turn related to an understanding of musical aesthetics. Her general argument is that the psychophysical study of music involved a constant tension between universal psychophysical laws and individual aesthetic responses, a conflict that could only be resolved by acknowledging that psychological processes contribute to the experience of music. Crucially, this meant that the physical properties of sound were not the only determinants of such phenomena as consonance.

At the same time, new theories of tuning became practical realities. When psychophysicists began to study music, equal temperament was rising in popularity (Isacoff, 2001). In short, psychophysicists studied music at a time when radically new notions of consonance and dissonance were emerging. How could one reconcile a natural science of music or consonance with the many changes in music and musical preference arising in the latter half of the 19th century?

1.3.2 On the Sensations of Tone

One of the most influential accounts of musical psychophysics (Hiebert, 2014) was Hermann Helmholtz’s book On the Sensations of Tone as a Physiological Basis for the Theory of Music (Helmholtz & Ellis, 1863/1954). Helmholtz wove three different threads together. One concerns the physics of sound, musical sound in particular, and includes details about various devices for sound production and measurement. A second concerns the physiology of hearing, and includes a detailed account of the structure and function of the cochlea and the basilar membrane. A third concerns the implications of the properties of sound, and the physiology of hearing, for the perception of music, and provides detailed discussions of different tunings and musical aesthetics.

The core idea that Helmholtz uses is a reinvention of coincidence theory. Coincidence theory focused on the relationship between two pure tones. Helmholtz recognized that in almost every case a musical instrument will not generate a pure sine wave with frequency f. Sympathetic vibrations add to this fundamental frequency additional sine waves at various frequencies defined by the octave (e.g., 2f, 3f, 4f, and so on). These additional frequencies, called partials or harmonics, occur at weaker intensities than the fundamental, and tend to be weaker and weaker as the partial frequency becomes higher and higher. Helmholtz’s theory of consonance emphasized harmonic interference.

When two musical sounds occur together, not only do their fundamental frequencies interact but their partials interact as well. For Helmholtz, interference patterns among all of the frequencies that are present cause the consonance of combined musical tones. Interference between partials would result in beats that produced an effect of roughness; consonant tones had little roughness; dissonant tones had more roughness because of more interference among partial frequencies. Importantly, these interference patterns affected the perception of music because they were detectable by the physiological mechanisms of hearing (Hui, 2013).

Helmholtz provided a detailed analysis of beat patterns related to a variety of musical intervals, and used this analysis to make fine-grained predictions about consonance. His quantitative account of consonance coincides with a variety of experimental studies of consonance conducted in the 20th century (Krumhansl, 1990a). Helmholtz used his new theory of consonance to inform musical aesthetics and to defend his personal views on the beautiful in music. Helmholtz was a champion of just intonation (Hiebert, 2014; Hui, 2013) and was highly critical of the rising popularity of equal temperament. He believed that just intonation produced music that was far more consonant than was possible in equal temperament. Hui points out that Helmholtz makes the physiological argument that just intonation produces music more consistent with the physiology of hearing than is produced using equal temperament. However, Hui also notes that Helmholtz recognized that there were individual differences in musical aesthetics. How was this to be reconciled with his detailed psychophysical theory?

Helmholtz argued that while his psychophysical account of the sensations of tone could inform the elementary rules of musical composition, these rules of composition are not natural laws. Different listeners were capable of enduring (and appreciating!) different degrees of roughness. Individual differences in musical tastes were a function of experience, culture, and education. One’s musical knowledge, culture, or experience could influence the aesthetic experience of consonance and dissonance.

1.3.3 Individual Contributions

Helmholtz’s influential studies suggest that musical perception is only based in part on universal laws involving the physics of sound and the physiology of hearing. Music perception is also affected by aesthetic considerations that stem from an individual’s own culture, experience, and expertise. This position coincided with the tenets of musical Romanticism, which arose during the same era as musical psychophysics. Musical Romanticism emphasized individuality. That is, in Romanticism individual composers aimed to communicate their personal emotions and imaginations; Romanticism also heralded the virtuoso instrumentalist (Longyear, 1988). Informed by the theories of music critics like Eduard Hanslick (Hanslick, 1854/1957), audiences began to carefully listen to music—not just to enjoy it but also to understand it (Hui, 2013). Such intellectual listening depends heavily upon an individual’s musical tastes and knowledge.

One consequence of psychophysicists accepting that an individual’s mind or knowledge greatly affects musical perception is the need to discover the psychological laws that govern these influences. This research goal is central to the cognitive study of music that began in the middle of the 20th century. The cognitive approach recognized that there could exist abstract laws governing perception and thinking, but these laws are in turn linked to physical mechanisms. In the next section, we briefly introduce this cognitive approach and then describe the kinds of theories it produced to account for musical perception.

1.4 From Rationalism to Classical Cognitive Science

1.4.1 Rationalism

Mechanical philosophy was not only interested in explaining the natural world but also in explaining the mind, as exemplified in the 17th century philosophy of René Descartes (Descartes, 1637/2006, 1641/1996). To establish a rigorous philosophy, Descartes adopted the mathematical approach of derivation from axioms. The basis of his philosophy was the axiom that he was a thinking thing: “A thing that doubts, understands, affirms, denies, is willing, is unwilling, and also imagines and has sensory perceptions” (p. 19). Cartesian philosophy proceeded by using this axiom to prove the existence of a perfect God who would not deceive, and then to establish the existence of an imperfectly sensed external world. As this philosophy derives new truths from axioms, we know it as rationalism. Cartesian philosophy has continued to inspire the study of mind centuries after its invention. In particular, it provides the philosophical foundations for modern cognitivism.

1.4.2 Cognitivism

For most of the first half of the 20th century, experimental psychology was dominated by an approach called behaviourism (Boring, 1950; Leahey, 1987). Behaviourism focused on investigating mechanistic links between observables—observable environmental stimuli and observable responses to these stimuli from organisms. Behaviourists strove to exclude elements that could not be directly observed (such as internal mental states) from their theories.

The 20th century invention of the digital computer made possible a new approach to studying psychology—cognitivism. Cognitivism reacted against behaviourism. In explaining the operations of a digital computer, one typically appeals to internal (and directly unobservable) states: the information represented inside the computer’s memory (the symbols) and the operations used to manipulate this internal information (the rules). Cognitive psychology explored the hypothesis that thinking was identical to the operations of a digital computer. That is, cognitivism hypothesized that cognition is literally the rule-governed manipulation of mentally represented information.

Inspired by the digital computer, what is now known as classical cognitive science is a modern descendant of Cartesian philosophy (Dawson, 1998, 2013; Dawson, Dupuis, & Wilson, 2010a). In viewing thought as analogous to the rule-governed operations of a digital computer, classical cognitive science is a modern variant of viewing thinking as analogous to axiomatic derivations. In accordance with mechanistic philosophy’s affinity for mathematics, and with Descartes’s axiomatic philosophy, classical cognitive science often uses mathematics or logic to investigate cognitive phenomena (Dawson, 1998).

Nineteenth-century psychophysics is also a precursor to modern cognitivism. Helmholtz argued that hearing served to build internal representations of the world. That is, the result of audition is to produce “mental images of determinate external objects, that is, in perceptions” (Helmholtz & Ellis, 1863/1954, p. 4). Behaviourism, in contrast, attempted to expunge mentalistic terms from its vocabulary (Watson, 1913). Behaviourism rejected using mentalistic terms like “unconscious inference” or “mental image”—terms that were central to Helmholtz.

1.5 Musical Cognitivism

1.5.1 Active Musical Processing

It has been argued that the study of music offers the strongest challenge against psychological behaviourism (Serafine, 1988). In Serafine’s view, the creation and appreciation of new compositions, or of new musical styles, is unexplainable in terms of behavioural reinforcement. It instead demands a representational theory. For this reason, behaviourist accounts of the psychology of music are nonexistent. When behaviourism dominated psychology, musical psychologists were not as interested in explaining musical perception as they were in designing tests to measure individual differences in musical ability (Larson, 1930; Seashore, 1915, 1936, 1938/1967; Stanton & Seashore, 1935).

What is a fundamental difference between cognitive theories that appeal to mental representations and those, such as behaviourism, which do not? According to the psychology texts that emerged after the cognitive revolution in the latter half of the 20th century (Lindsay & Norman, 1972; Neisser, 1967; Reynolds & Flagg, 1977), the key difference is active processing. Cognitivists argued that behaviourism treated organisms as passive responders to the environment (Bertalanffy, 1967, 1969). In contrast, cognitivism assumed “a constantly active organism that searches, filters, selectively acts on, reorganizes and creates information” (Reynolds & Flagg, 1977, p. 11, their italics). For some, cognitivism was revolutionary not because it proposed representations but because it proposed active information processing.

According to classical cognitivism, organisms are active processors that first receive information from the environment, and then organize this information into representations of the world that can be used to think and plan, and finally perform some action on the world based upon this thinking and planning (Dawson, 2013). In other words, our experience of the world results from combining the information that we receive about the world with our existing beliefs, desires, and knowledge. Thus, modern cognitivism not only embraces the individual differences revealed by the psychophysical study of music (Hui, 2013), but also attempts to develop scientific theories of an individual’s cognitive contributions to experience.

With the invention of new information-processing technology and new experimental methods, cognitivism went far beyond the psychophysical tradition of the 19th century. Modern cognitivists were not content to merely appeal to vague notions like unconscious inference. Instead, they were obligated to determine the nature of mental representations, as well as the kinds of rules used to manipulate them (Chomsky, 1965; Dutton & Starbuck, 1971; Feigenbaum & Feldman, 1995; Newell & Simon, 1972; Winston, 1975).

Unsurprisingly, the cognitive revolution has produced many important representational theories of music cognition (Cook, 1999; Deliège & Sloboda, 1997; Deutsch, 1982, 1999, 2013; Francès, 1988; Howell, Cross, & West, 1985; Krumhansl, 1990a; Lerdahl, 2001; Lerdahl & Jackendoff, 1983; Sloboda, 1985; Snyder, 2000; Temperley, 2001). In general, cognitive theories of music propose that musical perception involves organizing musical stimuli using existing mental representations. Musical cognitivists aim to explain the nature of musical representations, as well as the processes that manipulate these representations. The remainder of this section provides two examples to illustrate how classical cognitive scientists study music. We will see that these two topics will be important in later chapters when we begin to examine the musical regularities that artificial neural networks exploit.

1.5.2 The Tonal Hierarchy

Carol Krumhansl’s investigations of the cognitive foundations of musical pitch provide a prototypical example of the representational approach to music cognition (Krumhansl, 1990a). Her research goal concisely defines classical cognitive science’s perspective on music cognition: “to describe the human capacity for internalizing the structured sound materials of music by characterizing the nature of internal processes and representations” (Krumhansl, 1990a, p. 6).

To accomplish this goal, Krumhansl (1990a) makes a number of design decisions to choose from the vast possibilities available in terms of the musical stimuli to use, the musical responses to observe, and the choice of subjects to study, not to mention the description, analysis, and modelling of experimental results. After considering these possibilities, Krumhansl decides to explore cognitive representations of the pitch-classes used to define Western tonal-harmonic music.

Several factors guide this decision. First, even though musical pitch is related to a continuous physical property (sine wave frequency) and is therefore in principle infinitely variable, Western tonal music is experienced as involving only 12 different pitch-classes (Révész, 1954). This leads to the principle of octave equivalence (Forte, 1973; Patel, 2008; Roig-Francolí, 2008; Straus, 2005), which assigns pitches separated by multiples of an octave to the same pitch-class.

Second, even though the set of pitch-classes is very small, it serves as the foundation of Western tonal music. One can use combinations of pitch-classes to define complex musical objects. For instance, by using pairs of pitch-classes one can define musical intervals. By combining two or more musical intervals (e.g., by using three or four pitch-classes) one can define chords. A combination of seven pitch-classes defines a musical key that is built around a tonic note (one of the pitch-classes) and is associated with a particular sonority called its mode (which in Western music is typically either major or minor). In short, an understanding of the cognition of pitch should provide the foundation for an understanding of the cognition of more complex musical stimuli.

Third, while one constructs complex musical stimuli by combining pitch-classes together, there are powerful constraints on such combinations. Not every pitch-class combination in music is equally likely. In particular, consonant combinations are far more likely than dissonant combinations. Furthermore, establishing a particular tonality—a critical characteristic of Western music—requires that a subset of pitch-classes is more likely to be present in a composition, while at the same time the remaining pitch-classes are less likely to occur. In addition, if one only considers the to-be-included pitch-classes then some are more likely to occur than others are. In short, the cognition of Western tonal music attends to various relationships between pitch-classes, and one can explore these relationships by studying reactions to a very manageable stimulus set (i.e., the 12 pitch-classes).

Krumhansl’s basic method for studying the cognition of musical pitch is called the probe tone method (Krumhansl & Shepard, 1979). The probe tone method involves a series of musical trials. On any given trial a musical context is established (e.g., by playing part of a musical scale, a chord, or some other musical stimulus). Then a single probe note—one of the pitch-classes—is played. A subject’s task is to rate how well the probe note is related to the musical context. Typically, subjects make this rating on a seven-point scale where a rating of one indicates a very bad relationship, a rating of four indicates a moderate relationship, and a rating of seven indicates a very good relationship. Variations of this general paradigm permit subjects to make judgments about more complex musical stimuli (Krumhansl, 1990a).

The probe tone method reveals systematic relationships between pitch-classes within a particular musical context (e.g., a specific musical key). After establishing a musical key, the pitch-class given the highest rating of relationship to the context is the tonic of the key. For example, if the musical context establishes a key of A major, then the pitch-class A receives the highest rating, as is illustrated in Figure 1-1.

Figure 1-1 An illustration of “relatedness ratings” for each pitch-class in the context of the musical key A major.

The tones that receive the next highest ratings are those that belong in the third or fifth positions of the musical scale that defines the musical key. For the key of A major, these are the pitch-classes C♯ and E (see Figure 1-1). Receiving yet lower ratings are the remaining four pitch-classes that belong in the key’s musical scale. For the key of A major, these are the pitch-classes B, D, F♯, and G♯. The pitch-classes not found in the key’s scale elicit the lowest ratings. For the key of A major, these are the pitch-classes A♯, C, D♯, F, and G (see Figure 1-1).

The set of relationships between the 12 pitch-classes and a particular musical key is called the tonal hierarchy (Krumhansl, 1990a). If one changes the musical key, then one finds the same general pattern of ratings, but they are associated with different pitch-classes. For example, in the key of A major C♯ receives a very high rating, while C receives a very low rating (see Figure 1-1). However, in the key of C major C♯ receives the lowest rating while C receives the highest. In other words, each musical key is associated with its own tonal hierarchy (i.e., with a specific set of pitch-classes receiving high, moderate, low, and lowest ratings).

Krumhansl (1990a) has used the tonal hierarchies associated with different musical keys to explore higher-order relationships among musical keys. If two different keys are similar to one another, then their tonal hierarchies should be similar as well. In one study, the correlations between tonal hierarchies were calculated for every possible pair of the 12 different major and 12 different minor musical keys (Krumhansl & Kessler, 1982). Then a graphic representation of the similarity relationships between musical keys was determined by using multidimensional scaling to convert the correlations into a multidimensional map. In this map, a different point represents each key; points representing similar musical keys are nearer to one another in the map. Krumhansl and Kessler discovered that a four-dimensional map provided the best representation of the similarity data. This map arranged the keys in a spiral that wrapped itself around a toroidal surface.

The spiral arrangement of notes around the torus reflects elegant spatial relationships among tonic notes related to a standard device in music theory called the circle of fifths (Krumhansl, 1990a; Krumhansl & Kessler, 1982). The circle of fifths arranges the 12 pitch-classes around a circle; pitch-classes that are adjacent in the circle are a musical interval of a perfect fifth (seven semitones) apart. In the map discovered by Krumhansl and Kessler, the nearest neighbours for any key along the torus were its neighbouring keys in the circle of fifths. For instance, the nearest neighbours along the torus to the point representing the key of A major were the points for the keys of E major and D major. These two pitch-classes are on either side of A in the circle of fifths.

Krumhansl (1990a) summarizes a great deal of evidence in support of the tonal hierarchy as well as other organizational principles for music cognition. The tonal hierarchy is not a musical property per se, but is instead a psychologically imposed organization of musical elements. “The experience of music goes beyond registering the acoustic parameters of tone frequency, amplitude, duration, and timbre. Presumably, these are recoded, organized, and stored in memory in a form different from sensory codes” (Krumhansl, 1990, p. 281).

From Krumhansl’s (1990a) classical perspective, music cognition is a dynamic process in which mental representations relate musical sounds to one another and establish hierarchical relationships involving pitches, musical keys, intervals, and chords. The melodic, harmonic, and tonal roles of musical events are being continuously interpreted, organized, and structured. Importantly, these cognitive processes reflect the contributions of the individual to his or her own musical perceptions, contributions that were of great concern to psychophysical researchers (Hui, 2013) but which they did not explain in detail.

1.5.3 The Tritone Paradox

Krumhansl’s (1990a) tonal hierarchy provides one prototypical example of how mental representations can explain musical cognition. The musical illusion called the tritone paradox provides another example. Below I will introduce the tones used to create this illusion, describe the illusion itself, and then provide a representational explanation of why the tritone paradox occurs.

The arrangement of the keys on a piano reveals two different properties. First, pitch ascends linearly from left to right along the keyboard. That is, after you play a note by pressing one of the piano keys, if you press the key immediately to its left, you will play a note that is a semitone lower; if you press the key immediately to its right, you will play a note that is a semitone higher. Second, pitch-class (the name of the note associated with any piano key) is arranged circularly. This is because the relationships between note names repeat themselves along the keyboard. For instance, find any key on the piano that plays an A. Its nearest neighbour to the left will be a key that plays a G♯; its nearest neighbour to the right will be a key that plays an A♯.

If you wished to use a single graph to depict these two relationships, then you would likely use a three-dimensional spiral like the one illustrated in Figure 1-2 (Deutsch, 2010). The vertical dimension of the spiral represents pitch. As you move upward from the bottom of this spiral toward the top, pitch gets higher. As a result, for any note in this spiral the note on one side will be higher in height (representing a pitch that is a semitone higher), while the note on the other side will be lower (representing a pitch that is a semitone lower).

In contrast, horizontal position around the spiral represents pitch-class relationships. No matter where you are in the spiral, one pitch-class will have the same neighbours. For instance, an A♯ will always have an A on its one side and a B on its other along the helix. Indeed, position in the horizontal dimension around the spiral is a reliable indicator of pitch-class. Imagine that the spiral in Figure 1-2 wraps itself around a cylinder. Every instance of one pitch-class is vertically aligned at the edge of this cylinder. For instance, the three different instances of A in Figure 1-3 are all stacked in the same column along the edge of an imaginary cylinder. In other words, position around the horizontal “circular dimension” of the Figure 1-2 spiral encodes pitch-class.

Figure 1-2 A three-dimensional spiral can simultaneously capture the linear arrangement of pitch and the circular arrangement of pitch-class on a piano keyboard.

Psychologist Roger Shepard reasoned that if he could eliminate the pitch dimension of music (e.g., if one were to compress the Figure 1-2 spiral to remove its height, so that all note names would fall around a single circle), then he could create an interesting musical illusion (Shepard, 1964). He used computer-generated sounds, now called Shepard tones, to test this hypothesis. A Shepard tone is a musical sound whose components all belong to the same pitch-class. For instance, a Shepard tone for the pitch-class A may be built from seven different pitches of A (e.g., A1, A2, A3, A4, A5, A6, and A7; where A4 is the A above middle C on the piano, A5 is the A that is an octave higher than A4, and A3 is the A that is an octave lower than A4). However, the loudness of each Shepard tone component decreases rapidly, moving in either direction away from the Shepard tone’s central component. If one constructs a tone in the manner described above, the result is a sound that, when heard, specifies a particular pitch-class but does not clearly indicate the octave of the note being heard.

Shepard (1964) found that the Shepard tones produced a marked illusion of musical circularity. When Shepard presented 12 tones (one for each pitch-class) in sequence (e.g., A, A♯, B, C, D, D♯, E, F, F♯, G, and G♯), and then continued to repeat the sequence, listeners reported a definite linear progression of pitch. That is, subjects could hear successive notes increasing in pitch, with the current sound being a higher pitch than the one that preceded it. However, the experience puzzled the listeners. Some wondered why the pitch was increasing but did not really seem to get higher. “Some subjects were astonished to learn that the sequence was cyclic rather than monotonic and that in fact it repeatedly returned to precisely the tone with which it had begun” (Shepard, 1964, p. 2349). Shepard himself compared these auditory effects to the circular staircase illusions in the artwork of Maurits Escher.

Diana Deutsch used Shepard tones to discover another musical illusion called the tritone paradox (Deutsch, 1986, 1987, 1991). In this illusion, listeners hear pairs of Shepard tones and must judge whether the first or the second member of each pair has the higher pitch. In Deutsch’s paradigm, the pitch-classes in each pair are separated by a musical interval called a tritone. If two pitches are a tritone apart, then they are six semitones apart, or separated by exactly half an octave. For example, A and D♯ are separated by a tritone, as are B and F.

Interestingly, the tritone paradox emerges by comparing the judgments of different subjects. For instance, one subject might judge that an A has a higher pitch than a D♯. However, a different subject might make exactly the opposite judgment from the same stimulus. “This demonstration is particularly striking when played to a group of professional musicians, who are quite certain of their own judgments and yet recognize that others are obtaining entirely different percepts” (Deutsch, 2010, p. 13).

How can one explain the tritone paradox? Deutsch (2010) proposes an elegant theory consistent with classical cognitive science. She suggests that pitch-class is represented by arranging pitch-classes around a circle that makes explicit the neighbourhood relations between pitch-classes. She further proposes that some notion of pitch height is also encoded in this representation. As a result, pitch-classes that fall on one side of this circular representation are judged higher in pitch than pitch-classes that fall on the other side of the circle.

This is illustrated in Figure 1-3. The top part of this figure arranges pitch-classes around a circle of minor seconds, so that neighbouring pitch-classes are a semitone apart. (This is identical to the neighbourhood relationship between note names found on a piano keyboard.) Pitch-classes joined by a diagonal through this circular arrangement are a tritone apart (e.g., C and F♯). In addition, a dashed line divides this representation into two, so that pitch-classes that fall above the dashed line are judged to have a higher pitch than pitch-classes that fall below the dashed line. The top part of Figure 1-3 illustrates a representation in which causes C to be judged to have a higher pitch than F♯.

However, Deutsch points out that there is no reason for there not to be individual differences in the orientation of this circle of pitch-classes. For instance, the bottom part of Figure 1-3 provides the same circular representation of pitch-classes, but it has been rotated 180° around the centre when compared to the top circle in the figure. As a result, different pitch-classes fall above the dashed line. A listener using the bottom representation of pitch-class would judge F♯ higher than C, which is opposite to the judgment obtained using the upper representation in the figure.

Figure 1-3 Using circles of minor seconds to explain the tritone paradox.

The tritone paradox, and Deutsch’s explanation of this effect, provides a prototypical illustration of the classical approach to musical cognition. First, the physical properties of the musical stimulus are not sufficient to explain how it is experienced. For example, in the tritone paradox two different listeners can be presented exactly the same stimulus but have opposite experiences of it. As a result, an account of this phenomenon requires an appeal to internal processing of the external stimulus that produces the listener’s final experience.

Second, the account of the tritone paradox depends on the properties of particular kinds of mental representations. When a listener hears a particular tone, the tone is mapped onto the particular location of a circle of pitch-classes like those illustrated in Figure 1-3. However, this circle of pitch-classes makes explicit other properties as well—in particular, the relative heights of different pitch-classes.

It is this additional information from the representation that results in the listener hearing a specific stimulus as having a lower pitch than the other has. Individual differences in a general property of this kind of representation (e.g., its orientation) explain individual differences in musical experiences. As was the case with Krumhansl’s (1990a) tonal hierarchy, Deutch’s explanation of the tritone paradox appeals to regularities governing representations but can also explain individual differences in these representations. Again, musical cognitivism’s emphasis on active information processing embraces and explains the individual differences in perception that emerged from the psychophysical study of music (Hui, 2013).

1.6 Summary

The purpose of this chapter was to provide a historical overview of the science of music and to use this overview as a context for modern cognitivism. The chapter started with the perspective on music that began with the ancient Greeks, and which then flourished in the natural philosophy of the Enlightenment. According to this perspective, music was an attribute consistent with the perfect structure of nature, and all of its properties could be elegantly explained using mathematics. However, as natural philosophy matured, it made discoveries that challenged this view of music. In particular, the mathematical perfection of music was not reflected in various theories of consonance or of tuning. In the 19th century, other factors affected the account of music that mechanical philosophy held. In particular, changes in musical aesthetics as part of the rise of Romanticism, combined with discoveries of the new musical psychophysics, contradicted the view that music could be explained purely in terms of its mathematical or physical properties. In the latter half of the 19th century, it became clear that explanations of musical experience required appeals to the beliefs and cultures of individual listeners in addition to the physical properties of sound. Music was not merely physical: it was psychophysical.

The rise of cognitivism in the 20th century continued the tradition of explaining the perception and experience of music by appealing in part to psychological contributions of the listener. Classical cognitivism’s approach to studying music represents a modern blending of the various traditions introduced in the current chapter. Like mechanical philosophy, musical cognitivism explains musical perception by appealing to formal or mathematical constructs. This is evident in its appeal to the rule-governed manipulation of mental representations. Like early musical psychophysics, musical cognitivism recognizes that perception of music depends on the interaction between the physical properties of acoustic stimuli and the organizational role of mental representations. This permits the study of musical cognition to appeal to scientific laws but also to be sensitive to individual differences. In short, musical cognitivism is an approach that is deeply rooted in natural philosophy but at the same time inspired by modern developments like the digital computer. In particular, musical cognitivism is primarily interested in identifying the nature of mental representations of music and the nature of the rules or operations that manipulate these representations in order to produce our musical experience.

However, the classical approach, inspired by the digital computer, is not the only school of thought in modern cognitivism (Dawson, 2013). Artificial neural networks form the basis of another, called connectionist cognitive science, which challenges some of the core assumptions of the classical approach. In addition, there is a growing interest in connectionist cognitive science in using artificial neural networks to capture some informal aspects of musical cognition, informal properties hypothesized to be beyond the reach of classical musical cognitivism. The purpose of this book is to explore some aspects of connectionist musical cognitivism; the next chapter provides an introduction to this general approach, as well as to the methodology employed in later chapters.